In the era of data-driven decision-making, real-world applications often face uncertainties arising from noise, environmental shifts, and adversarial perturbations. These challenges can degrade model performance, lead to poor decisions, and introduce unforeseen risks. This thesis tackles these issues by developing robust decision-making frameworks for optimization, control, and games, with a particular focus on distributional robustness and risk-averse learning under uncertain data distributions. It consists of four parts.

In the first part, we consider outlier-robust distributionally robust optimization (DRO) problems, where the data distributions are subject to Wasserstein perturbations and outlier contamination. We propose a novel DRO framework leveraging a distance inspired by Unbalanced Optimal Transport (UOT). This UOT-based distance incorporates a soft penalization term in place of traditional hard constraints, enabling the construction of ambiguity sets that are more robust to outliers. Under appropriate smoothness conditions, we establish the strong duality of the proposed DRO formulation. Additionally, we present a computationally efficient Lagrangian penalty formulation and demonstrate that strong duality also holds. We provide empirical results that demonstrate that our method offers improved robustness to outliers and is computationally less demanding.

In the second part, we focus on the decision-dependent optimization problems, where the data distributions react in response to decisions, affecting both the objective function and linear constraints. We establish a sufficient condition for the existence of a constrained equilibrium point, at which the distributions remain invariant under retraining. We propose dual ascent and projected gradient descent algorithms, each with theoretical convergence guarantees, operating in the dual and primal spaces, respectively. Furthermore, we explore the relationship between the equilibrium point and the optimal point for the constrained decision-dependent optimization problem.

In the third part, we study risk-averse learning in online convex games using conditional value at risk (CVaR) as the risk measure. For the zeroth-order feedback setting where agents access only cost values of their selected actions, we propose risk-averse learning algorithms with sample reuse and variance reduction. For the first-order feedback setting where agents obtain gradient information, we develop a first-order risk-averse leaning algorithm based on value at risk estimates. Despite the bias in CVaR gradient estimates, we establish high-probability convergence guarantees for all proposed algorithms.

In the final part, we explore distributional reinforcement learning (DRL) in linear quadratic regulator (LQR) problems. A key challenge in DRL is the design of the distribution representation for policy evaluation. For discounted LQR control, we derive a closed-form expression for the random return and analyze its properties, including variance bounds, sensitivity, and finite approximation error. For unknown models, we introduce a model-free method to estimate the return distribution with sample complexity guarantees. We also extend these results to partially observable linear systems. Using the learned return distribution, we propose a zeroth-order policy gradient algorithm for risk-averse LQR using CVaR as the risk measure.

I en era av datadrivet beslutsfattande ställs verkliga tillämpningar ofta inför osäkerheter som uppstår från brus, miljöförändringar och adversariala störningar. Dessa utmaningar kan försämra modellens prestanda, leda till dåliga beslut och introducera oförutsedda risker. Denna avhandling hanterar dessa frågor genom att utveckla robusta beslutsramverk för optimering, styrning och spel, med särskilt fokus på distributionell robusthet och riskavert inlärning under osäkra datadistributioner. Den består av fyra delar.

I den första delen undersöker vi outlier-robusta problem inom distributionell robust optimering (DRO), där datadistributionerna är utsatta för störningar i form av Wasserstein-perturbationer och outlier-kontaminering. Vi föreslår ett nytt DRO-ramverk som utnyttjar ett avstånd inspirerat av Obalanserad Optimal Transport (UOT). Detta UOT-baserade avstånd inför en mjuk penaliseringskomponent istället för traditionella hårda begränsningar, vilket möjliggör konstruktionen av tvetydighetsmängder som är mer robusta mot outliers. Under lämpliga jämnhetsvillkor fastställer vi stark dualitet för den föreslagna DRO-formuleringen. Dessutom presenterar vi en beräkningsmässigt effektiv formulering med Lagrangestraff och visar att stark dualitet även gäller här. Vi presenterar empiriska resultat som visar att vår metod erbjuder förbättrad robusthet mot outliers och är beräkningsmässigt mindre krävande.

I den andra delen fokuserar vi på beslutberoende optimeringsproblem, där datadistributionerna förändras som svar på besluten och påverkar både målfunktionen och linjära begränsningar. Vi fastställer ett tillräckligt villkor för existensen av en begränsad jämviktspunkt, där distributionerna förblir oförändrade vid omträning. Vi föreslår dual ascent- och projicerade gradientnedstigningsalgoritmer, båda med teoretiska konvergensgarantier, som arbetar i respektive duala och primala rum. Vidare undersöker vi sambandet mellan jämviktspunkten och optimalpunkten för det beslutberoende optimeringsproblemet med begränsningar.

I den tredje delen studerar vi riskavert inlärning i online konvexa spel genom att använda Conditional Value at Risk (CVaR) som riskmått. För feedbackinställningen med nollte ordningen, där agenter endast har tillgång till kostnadsvärden för sina valda handlingar, föreslår vi riskaverta inlärningsalgoritmer med provåteranvändning och variansreduktion. För feedbackinställningen med första ordningen, där agenter får gradientinformation, utvecklar vi en riskavert inlärningsalgoritm baserad på Value at Risk-estimat. Trots bias i gradientestimat för CVaR, fastställer vi konvergensgarantier med hög sannolikhet för alla föreslagna algoritmer.

I den sista delen utforskar vi distributionsförstärkt förstärkningsinlärning (DRL) i problem med linjära kvadratiska regulatorer (LQR). En nyckelutmaning i DRL är utformningen av representationen för returdistributionen vid policyutvärdering. För diskonterad LQR-kontroll härleder vi ett slutet uttryck för den stokastiska returen och analyserar dess egenskaper, inklusive variationsgränser, känslighet och fel vid ändlig approximation. För okända modeller introducerar vi en modellfri metod för att uppskatta returdistributionen med garantier för provkomplexitet. Vi utvidgar också dessa resultat till partiellt observerbara linjära system. Med hjälp av den inlärda returdistributionen föreslår vi en policy-gradientalgoritm av nollte ordningen för riskavers LQR med CVaR som riskmått.

Distributionally Robust Optimization (DRO) accounts for uncertainty in data distributions by optimizing the model performance against the worst possible distribution within an ambiguity set. In this paper, we propose a DRO framework that relieson a new distance inspired by Unbalanced Optimal Transport (UOT). The proposed UOT distance employs a soft penalization term instead of hard constraints, enabling the construction of an ambiguity set that is more resilient to outliers. Under smoothness conditions, we establish strong duality of the proposed DRO problem. Moreover, we introduce a computationally efficient Lagrangian penalty formulation for which we show that strong duality also holds. Finally, we provide empirical results that demonstrate that our method offers improved robustness to outliers and is computationally less demanding for regression and classification tasks

This paper considers risk-averse learning in convex games involving multiple agents that aim to minimize their individual risk of incurring significantly high costs. Specifically, the agents adopt the conditional value at risk (CVaR) as a risk measure with possibly different risk levels. To solve this problem, we propose a first-order risk-averse leaning algorithm, in which the CVaR gradient estimate depends on an estimate of the Value at Risk (VaR) value combined with the gradient of the stochastic cost function. Although estimation of the CVaR gradients using finitely many samples is generally biased, we show that the accumulated error of the CVaR gradient estimates is bounded with high probability. Moreover, assuming that the risk-averse game is strongly monotone, we show that the proposed algorithm converges to the risk-averse Nash equilibrium. We present numerical experiments on a Cournot game example to illustrate the performance of the proposed method.

Distributionally Robust Optimization (DRO) accounts for uncertainty in data distributions by optimizing the model performance against the worst possible distribution within an ambiguity set. In this paper, we propose a DRO framework that relies on a new distance inspired by Unbalanced Optimal Transport (UOT). The proposed UOT distance employs a soft penalization term instead of hard constraints, enabling the construction of an ambiguity set that is more resilient to outliers. Under smoothness conditions, we establish strong duality of the proposed DRO problem. Moreover, we introduce a computationally efficient Lagrangian penalty formulation for which we show that strong duality also holds. Finally, we provide empirical results that demonstrate that our method offers improved robustness to outliers and is computationally less demanding for regression and classification tasks.

This paper studies a class of strongly monotone games involving non-cooperative agents that optimize their own time-varying cost functions. We assume that the agents can observe other agents' historical actions and choose actions that best respond to other agents' previous actions; we call this a best response scheme. We start by analyzing the convergence rate of this best response scheme for standard time-invariant games. Specifically, we provide a sufficient condition on the strong monotonicity parameter of the time-invariant games under which the proposed best response algorithm achieves exponential convergence to the static Nash equilibrium. We further illustrate that this best response algorithm may oscillate when the proposed sufficient condition fails to hold, which indicates that this condition is tight. Next, we analyze this best response algorithm for time-varying games where the cost functions of each agent change over time. Under similar conditions as for time-invariant games, we show that the proposed best response algorithm stays asymptotically close to the evolving equilibrium. We do so by analyzing both the equilibrium tracking error and the dynamic regret. Numerical experiments on economic market problems are presented to validate our analysis.

Distributional reinforcement learning (DRL) enhances the understanding of the effects of the randomness in the environment by letting agents learn the distribution of a random return, rather than its expected value as in standard RL. At the same time, a main challenge in DRL is that policy evaluation in DRL typically relies on the representation of the return distribution, which needs to be carefully designed. In this paper, we address this challenge for a special class of DRL problems that rely on discounted linear quadratic regulator (LQR) for control, advocating for a new distributional approach to LQR, which we call distributional LQR. Specifically, we provide a closed-form expression of the distribution of the random return which, remarkably, is applicable to all exogenous disturbances on the dynamics, as long as they are independent and identically distributed (i.i.d.). While the proposed exact return distribution consists of infinitely many random variables, we show that this distribution can be approximated by a finite number of random variables, and the associated approximation error can be analytically bounded under mild assumptions. Using the approximate return distribution, we propose a zeroth-order policy gradient algorithm for risk-averse LQR using the Conditional Value at Risk (CVaR) as a measure of risk. Numerical experiments are provided to illustrate our theoretical results.

We consider risk-averse learning in repeated unknown games where the goal of the agents is to minimize their individual risk of incurring significantly high cost. Specifically, the agents use the conditional value at risk (CVaR) as a risk measure and rely on bandit feedback in the form of the cost values of the selected actions at every episode to estimate their CVaR values and update their actions. A major challenge in using bandit feedback to estimate CVaR is that the agents can only access their own cost values, which, however, depend on the actions of all agents. To address this challenge, we propose a new risk-averse learning algorithm with momentum that utilizes the full historical information on the cost values. We show that this algorithm achieves sub-linear regret and matches the best known algorithms in the literature. We provide numerical experiments for a Cournot game that show that our method outperforms existing methods.

This paper investigates the output-based event-triggered communication and control for linear multi-agent consensus under a directed graph based on a co-design method. The communication among the agents, as well as the controller updates, are determined by some new event-triggering mechanisms, in order to reduce the use of the network resources. To simultaneously guarantee the significant properties including the asymptotic consensus, and strong Zeno-freeness (strictly positive inter-event times), a novel dis-tributed dynamic event-triggered protocol is proposed. Unlike the most-existing emulation-based approaches in which the control gain is previously decided, a systematic co-design procedure is proposed to design the controller gain, the observer gain, and the event-triggering mechanisms altogether in terms of solving a linear matrix inequality opti-mization problem. Based on the resulting hybrid system framework, a hybrid model is established for the distributed closed-loop system and the asymptotic consensus is achieved. Finally, a numerical example is presented to verify our systematic design methodology.

We consider an online stochastic game with risk-averse agents whose goal is to learn optimal decisions that minimize the risk of incurring significantly high costs. Specifically, we use the Conditional Value at Risk (CVaR) as a risk measure that the agents can estimate using bandit feedback in the form of the cost values of only their selected actions. Since the distributions of the cost functions depend on the actions of all agents that are generally unobservable, they are themselves unknown and, therefore, the CVaR values of the costs are difficult to compute. To address this challenge, we propose a new online risk-averse learning algorithm that relies on one-point zeroth-order estimation of the CVaR gradients computed using CVaR values that are estimated by appropriately sampling the cost functions. We show that this algorithm achieves sub-linear regret with high probability. We also propose two variants of this algorithm that improve performance. The first variant relies on a new sampling strategy that uses samples from the previous iteration to improve the estimation accuracy of the CVaR values. The second variant employs residual feedback that uses CVaR values from the previous iteration to reduce the variance of the CVaR gradient estimates. We theoretically analyze the convergence properties of these variants and illustrate their performance on an online market problem that we model as a Cournot game.