This paper considers convex games involving multiple agents that aim to minimize their own cost functions using locally available information. A common assumption in the study of such games is that the agents are symmetric, meaning that they have access to the same type of information. Here we lift this assumption, which is often violated in practice, and instead consider asymmetric agents; specifically, we assume some agents have access to first-order gradient information and others have access to the zeroth-order oracles (cost function evaluations). We propose an asymmetric learning algorithm that combines the agent information mechanisms. We analyze the regret and Nash equilibrium convergence of this algorithm for convex and strongly monotone games, respectively. Specifically, we show that our algorithm always performs between pure first- and zeroth-order methods, and can match the performance of these two extremes by adjusting the number of agents with access to zeroth-order oracles. Therefore, our algorithm incorporates the pure first- and zeroth-order methods as special cases. We provide numerical experiments on a market problem for both deterministic and risk-averse games to demonstrate the performance of the proposed algorithm.

In light of the inherently complex and dynamic nature of real-world environments, incorporating risk measures is crucial for the robustness evaluation of deep learning models. In this work, we propose a Risk-Averse Certification framework for Bayesian neural networks called RAC-BNN. Our method leverages sampling and optimisation to compute a probabilistically sound approximation of the output set of a BNN, represented using a set of template polytopes. To enhance risk-aware robustness evaluation, we integrate a coherent distortion risk measure–Conditional Value at Risk (CVaR)–into the certification framework, providing probabilistic guarantees based on empirical distributions obtained through sampling. We validate RAC-BNN on a range of regression and classification benchmarks and compare its performance with a state-of-the-art method. The results show that RAC-BNN effectively quantifies robustness under worst-performing risky scenarios, and achieves tighter certified bounds and higher efficiency in complex tasks.

Considering non-stationary environments in online optimization enables decision-makers to effectively adapt to changes and improve their performance over time. In such cases, it is favorable to adopt a strategy that minimizes the negative impact of change to avoid potentially risky situations. In this paper, we investigate risk-averse online optimization where the distribution of random costs changes over time. The Conditional Value at Risk (CVaR) is employed as risk measure. Due to the difficulty of obtaining the exact CVaR gradient, we employ a zeroth-order approach that queries the cost values multiple times per iteration and estimates the CVaR gradient from these samples. In regret analysis, the varying distributions are captured by a novel variation metric based on the Wasserstein distance. Given that the distribution variation is sublinear in the iteration horizon, we show that the developed learning algorithm achieves sublinear dynamic regret with high probability for both convex and strongly convex functions. Moreover, theoretical results suggest that dynamic regret bounds decrease with increasing sampling numbers until they reach a specific limit. Finally, we provide numerical experiments of dynamic pricing in a parking lot to illustrate the efficacy of the designed algorithm.

In this article, we deal with stochastic optimization problems where the data distributions change in response to the decision variables. Traditionally, the study of optimization problems with decision-dependent distributions has assumed either the absence of constraints or fixed constraints. This work considers a more general setting where the constraints can also dynamically adjust in response to changes in the decision variables. Specifically, we consider linear constraints and analyze the effect of decision-dependent distributions in both the objective function and constraints. First, we establish a sufficient condition for the existence of a constrained equilibrium point, at which the distributions remain invariant under retraining. Moreover, we propose and analyze two algorithms: repeated constrained optimization and repeated dual ascent. For each algorithm, we provide sufficient conditions for convergence to the constrained equilibrium point. Furthermore, we explore the relationship between the equilibrium point and the optimal point for the constrained decision-dependent optimization problem. Notably, our results encompass previous findings as special cases when the constraints remain fixed. To show the effectiveness of our theoretical analysis, we provide numerical experiments on both a market problem and a dynamic pricing problem for parking based on real-world data.

In the era of data-driven decision-making, real-world applications often face uncertainties arising from noise, environmental shifts, and adversarial perturbations. These challenges can degrade model performance, lead to poor decisions, and introduce unforeseen risks. This thesis tackles these issues by developing robust decision-making frameworks for optimization, control, and games, with a particular focus on distributional robustness and risk-averse learning under uncertain data distributions. It consists of four parts.

In the first part, we consider outlier-robust distributionally robust optimization (DRO) problems, where the data distributions are subject to Wasserstein perturbations and outlier contamination. We propose a novel DRO framework leveraging a distance inspired by Unbalanced Optimal Transport (UOT). This UOT-based distance incorporates a soft penalization term in place of traditional hard constraints, enabling the construction of ambiguity sets that are more robust to outliers. Under appropriate smoothness conditions, we establish the strong duality of the proposed DRO formulation. Additionally, we present a computationally efficient Lagrangian penalty formulation and demonstrate that strong duality also holds. We provide empirical results that demonstrate that our method offers improved robustness to outliers and is computationally less demanding.

In the second part, we focus on the decision-dependent optimization problems, where the data distributions react in response to decisions, affecting both the objective function and linear constraints. We establish a sufficient condition for the existence of a constrained equilibrium point, at which the distributions remain invariant under retraining. We propose dual ascent and projected gradient descent algorithms, each with theoretical convergence guarantees, operating in the dual and primal spaces, respectively. Furthermore, we explore the relationship between the equilibrium point and the optimal point for the constrained decision-dependent optimization problem.

In the third part, we study risk-averse learning in online convex games using conditional value at risk (CVaR) as the risk measure. For the zeroth-order feedback setting where agents access only cost values of their selected actions, we propose risk-averse learning algorithms with sample reuse and variance reduction. For the first-order feedback setting where agents obtain gradient information, we develop a first-order risk-averse leaning algorithm based on value at risk estimates. Despite the bias in CVaR gradient estimates, we establish high-probability convergence guarantees for all proposed algorithms.

In the final part, we explore distributional reinforcement learning (DRL) in linear quadratic regulator (LQR) problems. A key challenge in DRL is the design of the distribution representation for policy evaluation. For discounted LQR control, we derive a closed-form expression for the random return and analyze its properties, including variance bounds, sensitivity, and finite approximation error. For unknown models, we introduce a model-free method to estimate the return distribution with sample complexity guarantees. We also extend these results to partially observable linear systems. Using the learned return distribution, we propose a zeroth-order policy gradient algorithm for risk-averse LQR using CVaR as the risk measure.

I en era av datadrivet beslutsfattande ställs verkliga tillämpningar ofta inför osäkerheter som uppstår från brus, miljöförändringar och adversariala störningar. Dessa utmaningar kan försämra modellens prestanda, leda till dåliga beslut och introducera oförutsedda risker. Denna avhandling hanterar dessa frågor genom att utveckla robusta beslutsramverk för optimering, styrning och spel, med särskilt fokus på distributionell robusthet och riskavert inlärning under osäkra datadistributioner. Den består av fyra delar.

I den första delen undersöker vi outlier-robusta problem inom distributionell robust optimering (DRO), där datadistributionerna är utsatta för störningar i form av Wasserstein-perturbationer och outlier-kontaminering. Vi föreslår ett nytt DRO-ramverk som utnyttjar ett avstånd inspirerat av Obalanserad Optimal Transport (UOT). Detta UOT-baserade avstånd inför en mjuk penaliseringskomponent istället för traditionella hårda begränsningar, vilket möjliggör konstruktionen av tvetydighetsmängder som är mer robusta mot outliers. Under lämpliga jämnhetsvillkor fastställer vi stark dualitet för den föreslagna DRO-formuleringen. Dessutom presenterar vi en beräkningsmässigt effektiv formulering med Lagrangestraff och visar att stark dualitet även gäller här. Vi presenterar empiriska resultat som visar att vår metod erbjuder förbättrad robusthet mot outliers och är beräkningsmässigt mindre krävande.

I den andra delen fokuserar vi på beslutberoende optimeringsproblem, där datadistributionerna förändras som svar på besluten och påverkar både målfunktionen och linjära begränsningar. Vi fastställer ett tillräckligt villkor för existensen av en begränsad jämviktspunkt, där distributionerna förblir oförändrade vid omträning. Vi föreslår dual ascent- och projicerade gradientnedstigningsalgoritmer, båda med teoretiska konvergensgarantier, som arbetar i respektive duala och primala rum. Vidare undersöker vi sambandet mellan jämviktspunkten och optimalpunkten för det beslutberoende optimeringsproblemet med begränsningar.

I den tredje delen studerar vi riskavert inlärning i online konvexa spel genom att använda Conditional Value at Risk (CVaR) som riskmått. För feedbackinställningen med nollte ordningen, där agenter endast har tillgång till kostnadsvärden för sina valda handlingar, föreslår vi riskaverta inlärningsalgoritmer med provåteranvändning och variansreduktion. För feedbackinställningen med första ordningen, där agenter får gradientinformation, utvecklar vi en riskavert inlärningsalgoritm baserad på Value at Risk-estimat. Trots bias i gradientestimat för CVaR, fastställer vi konvergensgarantier med hög sannolikhet för alla föreslagna algoritmer.

I den sista delen utforskar vi distributionsförstärkt förstärkningsinlärning (DRL) i problem med linjära kvadratiska regulatorer (LQR). En nyckelutmaning i DRL är utformningen av representationen för returdistributionen vid policyutvärdering. För diskonterad LQR-kontroll härleder vi ett slutet uttryck för den stokastiska returen och analyserar dess egenskaper, inklusive variationsgränser, känslighet och fel vid ändlig approximation. För okända modeller introducerar vi en modellfri metod för att uppskatta returdistributionen med garantier för provkomplexitet. Vi utvidgar också dessa resultat till partiellt observerbara linjära system. Med hjälp av den inlärda returdistributionen föreslår vi en policy-gradientalgoritm av nollte ordningen för riskavers LQR med CVaR som riskmått.

Distributionally Robust Optimization (DRO) accounts for uncertainty in data distributions by optimizing the model performance against the worst possible distribution within an ambiguity set. In this paper, we propose a DRO framework that relieson a new distance inspired by Unbalanced Optimal Transport (UOT). The proposed UOT distance employs a soft penalization term instead of hard constraints, enabling the construction of an ambiguity set that is more resilient to outliers. Under smoothness conditions, we establish strong duality of the proposed DRO problem. Moreover, we introduce a computationally efficient Lagrangian penalty formulation for which we show that strong duality also holds. Finally, we provide empirical results that demonstrate that our method offers improved robustness to outliers and is computationally less demanding for regression and classification tasks

Distributional reinforcement learning (DRL) enhances the understanding of the effects of the randomness in the environment by letting agents learn the distribution of a random return, rather than its expected value as in standard reinforcement learning. Meanwhile, a challenge in DRL is that the policy evaluation typically relies on the representation of the return distribution, which needs to be carefully designed. In this paper, we address this challenge for the special class of DRL problems that rely on a discounted linear quadratic regulator (LQR), which we call distributional LQR. Specifically, we provide a closed-form expression for the distribution of the random return, which is applicable for all types of exogenous disturbance as long as it is independent and identically distributed (i.i.d.). We show that the variance of the random return is bounded if the fourth moment of the exogenous disturbance is bounded. Furthermore, we investigate the sensitivity of the return distribution to model perturbations. While the proposed exact return distribution consists of infinitely many random variables, we show that this distribution can be well approximated by a finite number of random variables. The associated approximation error can be analytically bounded under mild assumptions. When the model is unknown, we propose a model-free approach for estimating the return distribution, supported by sample complexity guarantees. Finally, we extend our approach to partially observable linear systems. Numerical experiments are provided to illustrate the theoretical results.

This paper considers risk-averse learning in convex games involving multiple agents that aim to minimize their individual risk of incurring significantly high costs. Specifically, the agents adopt the conditional value at risk (CVaR) as a risk measure with possibly different risk levels. To solve this problem, we propose a first-order risk-averse leaning algorithm, in which the CVaR gradient estimate depends on an estimate of the Value at Risk (VaR) value combined with the gradient of the stochastic cost function. Although estimation of the CVaR gradients using finitely many samples is generally biased, we show that the accumulated error of the CVaR gradient estimates is bounded with high probability. Moreover, assuming that the risk-averse game is strongly monotone, we show that the proposed algorithm converges to the risk-averse Nash equilibrium. We present numerical experiments on a Cournot game example to illustrate the performance of the proposed method.

Distributionally Robust Optimization (DRO) accounts for uncertainty in data distributions by optimizing the model performance against the worst possible distribution within an ambiguity set. In this paper, we propose a DRO framework that relies on a new distance inspired by Unbalanced Optimal Transport (UOT). The proposed UOT distance employs a soft penalization term instead of hard constraints, enabling the construction of an ambiguity set that is more resilient to outliers. Under smoothness conditions, we establish strong duality of the proposed DRO problem. Moreover, we introduce a computationally efficient Lagrangian penalty formulation for which we show that strong duality also holds. Finally, we provide empirical results that demonstrate that our method offers improved robustness to outliers and is computationally less demanding for regression and classification tasks.

This paper studies a class of strongly monotone games involving non-cooperative agents that optimize their own time-varying cost functions. We assume that the agents can observe other agents' historical actions and choose actions that best respond to other agents' previous actions; we call this a best response scheme. We start by analyzing the convergence rate of this best response scheme for standard time-invariant games. Specifically, we provide a sufficient condition on the strong monotonicity parameter of the time-invariant games under which the proposed best response algorithm achieves exponential convergence to the static Nash equilibrium. We further illustrate that this best response algorithm may oscillate when the proposed sufficient condition fails to hold, which indicates that this condition is tight. Next, we analyze this best response algorithm for time-varying games where the cost functions of each agent change over time. Under similar conditions as for time-invariant games, we show that the proposed best response algorithm stays asymptotically close to the evolving equilibrium. We do so by analyzing both the equilibrium tracking error and the dynamic regret. Numerical experiments on economic market problems are presented to validate our analysis.