kth.sePublications
Change search
Refine search result
1 - 12 of 12
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Rows per page
  • 5
  • 10
  • 20
  • 50
  • 100
  • 250
Sort
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
  • Standard (Relevance)
  • Author A-Ö
  • Author Ö-A
  • Title A-Ö
  • Title Ö-A
  • Publication type A-Ö
  • Publication type Ö-A
  • Issued (Oldest first)
  • Issued (Newest first)
  • Created (Oldest first)
  • Created (Newest first)
  • Last updated (Oldest first)
  • Last updated (Newest first)
  • Disputation date (earliest first)
  • Disputation date (latest first)
Select
The maximal number of hits you can export is 250. When you want to export more records please use the Create feeds function.
  • 1.
    Tsiamis, Anastasios
    et al.
    Automatic Control Laboratory, ETH Zurich, Switzerland.
    Ziemann, Ingvar
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Morari, Manfred
    Department of Electrical and Systems Engineering, University of Pennsylvania, United States.
    Matni, Nikolai
    Department of Electrical and Systems Engineering, University of Pennsylvania, United States.
    Pappas, George J.
    Department of Electrical and Systems Engineering, University of Pennsylvania, United States.
    Learning to Control Linear Systems can be Hard2022In: Proceedings of 35th Conference on Learning Theory, COLT 2022, ML Research Press , 2022, p. 3820-3857Conference paper (Refereed)
    Abstract [en]

    In this paper, we study the statistical difficulty of learning to control linear systems. We focus on two standard benchmarks, the sample complexity of stabilization, and the regret of the online learning of the Linear Quadratic Regulator (LQR). Prior results state that the statistical difficulty for both benchmarks scales polynomially with the system state dimension up to system-theoretic quantities. However, this does not reveal the whole picture. By utilizing minimax lower bounds for both benchmarks, we prove that there exist nontrivial classes of systems for which learning complexity scales dramatically, i.e. exponentially, with the system dimension. This situation arises in the case of underactuated systems, i.e. systems with fewer inputs than states. Such systems are structurally difficult to control and their system theoretic quantities can scale exponentially with the system dimension dominating learning complexity. Under some additional structural assumptions (bounding systems away from uncontrollability), we provide qualitatively matching upper bounds. We prove that learning complexity can be at most exponential with the controllability index of the system, that is the degree of underactuation.

  • 2.
    Ziemann, Ingvar
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Applications of Information Inequalities to Linear Systems: Adaptive Control and Security2021Licentiate thesis, monograph (Other academic)
    Abstract [en]

    This thesis considers the application of information inequalities, Cramér-Rao type bounds, based on Fisher information, to linear systems. These tools are used to study the trade-offs between learning and performance in two application areas: adaptive control and control systems security.

    In the first part of the thesis, we study stochastic adaptive control of linear quadratic regulators (LQR). Here, information inequalities are used to derive instance-dependent  regret lower bounds. First, we consider a simplified version of LQR, a memoryless reference tracking model, and show how regret can be linked to a cumulative estimation error. This is then exploited to derive a regret lower bound in terms of the Fisher information generated by the experiment of the optimal policy. It is shown that if the optimal policy has ill-conditioned Fisher information, then so does any low-regret policy. This is combined with a Cramér-Rao bound to give a regret lower bound on the order of magnitude square-root T in the time-horizon  for a class of instances we call uninformative. The lower bound holds for all policies which depend smoothly on the underlying parametrization.

    Second, we extend these results to the general LQR model, and to arbitrary affine parametrizations of the instance parameters. The notion of uninformativeness is generalized to this situation to give a structure-dependent rank condition for when logarithmic regret is impossible. This is done by reduction of regret to a cumulative Bellman error. Due to the quadratic nature of LQR, this Bellman error turns out to be a quadratic form, which again can be interpreted as an estimation error. Using this, we prove a local minimax regret lower bound, of which the proof relies on relating the minimax regret to a Bayesian estimation problem, and then using Van Trees' inequality. Again, it is shown that an appropriate information quantity of any low regret policy is similar to that of the optimal policy and that any uninformative instance suffers local minimax regret at least on the order of magnitude square-root T. Moreover, it shown that the notion of uninformativeness when specialized to certain well-understood scenarios yields a tight characterization of square-root-regret.

    In the second part of this thesis, we study control systems security problems from a Fisher information point of view. First, we consider a secure state estimation problem and characterize the maximal impact an adversary can cause by means of least informative distributions -- those which maximize the Cramér-Rao bound. For a linear measurement equation, it is shown that the least informative distribution, subjected to variance and sparsity constraints, can be solved for by a semi-definite program, which becomes mixed-integer in the presence of sparsity constraints. Furthermore, by relying on well-known results on minimax and robust estimation, a game-theoretic interpretation for this characterization of the maximum impact is offered.

    Last, we consider a Fisher information regularized minimum variance control objective, to study the trade-offs between parameter privacy and control performance. It is noted that this can be motivated for instance by learning-based attacks, in which case one seeks to leak as little information as possible to a system-identification adversary. Supposing that the feedback law is linear, the noise distribution minimizing the trace of Fisher information subject to a state variance penalty is found to be conditionally Gaussian. 

    Download full text (pdf)
    fulltext
  • 3.
    Ziemann, Ingvar
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Statistical Learning, Dynamics and Control: Fast Rates and Fundamental Limits for Square Loss2022Doctoral thesis, monograph (Other academic)
    Abstract [en]

    Learning algorithms play an ever increasing role in modern engineering solutions. However, despite many recent successes, their performance in the context of dynamical and control systems is not exactly well-understood. This becomes especially pertinent as we move toward deploying these algorithms in safety-critical systems such as our power grids or smart mobility systems. These shortcomings motivate the present thesis. Broadly, by blending tools from control theory, information theory and statistical learning theory, this thesis seeks to further our understanding of when learning in the context of (controlled) dynamical systems is easy or hard. That is, we seek to quantify those properties which are key, and those which are not, for learning to succeed in this context.

    In the first part of the thesis, we study learning with square loss (regression) in a realizable time-series framework with martingale difference noise. Our main result is a fast rate excess risk boundwhich shows that whenever a\emph{trajectory hypercontractivity} condition holds, the risk of the least-squares estimator on dependent data matches the \iid\ rate order-wise after a burn-in time.In comparison, many existing results in learning from dependent data have rates where the effective sample size is deflated by a factor of the mixing-time of the underlying process, even after the burn-in time. However, recent work on linear dynamical systems and generalized linear models has shown that this is deflation is not always necessary. The main contribution of the first part of the thesis is to extend this observation to a much wider class of examples. Furthermore, our results allow the covariate process to exhibit long range correlations which are substantially weaker than geometric ergodicity. We present several examples for when this occurs: bounded function classes forwhich the $L^2$ and $L^{2+\e}$ norms are equivalent, ergodic finite state Markov chains, various parametric models including linear dynamical systems and generalized linear models, and a broad family of infinite dimensional $\ell^2(\mathbb{N})$-ellipsoids. 

    In the second part of the thesis, we study fundamental limits to learning-based solutions of the linear quadratic Gaussian (LQG) problem. The majority of this part is devoted to the online (or adaptive) control problem, in which a learner (or control engineer) seeks to control an unknown linear system subject to a quadratic (square) loss function with minimal regret (cumulative suboptimality). %To solve this problem, the learner must solve a trade-off between exploration and exploitation. Put differently,The learner faces the dual objectives of controlling the system to the best of their present knowledge while simultaneously ensuring that enough information about the system is gathered. We prove fundamental limits - regret lower bounds - to this problem by relating the regret of any policy to an associated Fisher information and applying the Van Trees' Inequality (Bayesian Cramér-Rao). We instantiate our bounds for state-feedback and partially observed systems. We show that ill-conditioned systems - in terms of their controllability and observability structure - lead to larger regret. In other words, classical control-theoretic limitations carry over to the learning-based setting. We further show that systems as simple as integrators lead to \emph{exponential} in the problem dimension regret lower bounds. Finally, we also study the variance of policy gradient methods. We derive results similar in spirit to those for the online LQG problem: ill-conditioned systems, as per poor controllability or observability, yield noisier gradients. 

    Download full text (pdf)
    PhD_Thesis_Ingvar_Ziemann
  • 4.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Sandberg, Henrik
    KTH, School of Electrical Engineering and Computer Science (EECS), Centres, ACCESS Linnaeus Centre. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    On a Phase Transition of Regret in Linear Quadratic Control: The Memoryless Case2021In: IEEE Control Systems Letters, E-ISSN 2475-1456, Vol. 5, no 2, p. 695-700, article id 9126856Article in journal (Refereed)
    Abstract [en]

    We consider an idealized version of adaptive control of a multiple input multiple output (MIMO) system without state. We demonstrate how rank deficient Fisher information in this simple memoryless problem leads to the impossibility of logarithmic rates of regret. Our analysis rests on a version of the Cramèr-Rao inequality that takes into account possible ill-conditioning of Fisher information and a pertubation result on the corresponding singular subspaces. This is used to define a sufficient condition, which we term uniformativeness, for regret to be at least order square root in the samples. 

  • 5.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control). KTH Royal Inst Technol, Sch Elect Engn & Comp Sci, SE-10044 Stockholm, Sweden..
    Sandberg, Henrik
    KTH, School of Electrical Engineering and Computer Science (EECS), Centres, ACCESS Linnaeus Centre. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control). KTH Royal Inst Technol, Sch Elect Engn & Comp Sci, SE-10044 Stockholm, Sweden..
    Parameter Privacy versus Control Performance: Fisher Information Regularized Control2020In: 2020 AMERICAN CONTROL CONFERENCE (ACC), IEEE , 2020, p. 1259-1265Conference paper (Refereed)
    Abstract [en]

    This article introduces and solves a new privacy-related optimization problem for cyber-physical systems where an adversary tries to learn the system dynamics. In the context of linear quadratic systems, we consider the problem of achieving a small cost while balancing the need for keeping knowledge about the model's parameters private. To this end, we formulate a Fisher information regularized version of the linear quadratic regulator with cheap cost. Here the control operator is allowed to not only control the plant but also mask its state by injecting further noise. Within the class of linear policies with additive noise, we solve this problem and show that the optimal noise distribution is Gaussian with state dependent covariance. Next, we prove that the optimal linear feedback law is the same as without regularization. Finally, to motivate our proposed scheme, we formulate for scalar systems an equivalent maximin problem for the worst-case scenario in which the adversary has full knowledge of all other inputs and outputs. Here, our policies are maximin optimal with respect to maximizing the variance over all asymptotically unbiased estimators.

  • 6.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Sandberg, Henrik
    KTH, School of Electrical Engineering and Computer Science (EECS), Centres, ACCESS Linnaeus Centre. KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Regret Lower Bounds for Unbiased Adaptive Control of Linear Quadratic Regulators2020In: IEEE Control Systems Letters, E-ISSN 2475-1456, Vol. 4, no 3, p. 785-790Article in journal (Refereed)
    Abstract [en]

    We present lower bounds for the regret of adaptive control of the linear quadratic regulator. These are given in terms of problem specific expected regret lower bounds valid for unbiased policies linear in the state. Our approach is based on the insight that the adaptive control problem can, given our assumptions, be reduced to a sequential estimation problem. This enables the use of the Cramer-Rao information inequality which yields a scaling limit lower bound of logarithmic order. The bound features both information-theoretic and control-theoretic quantities. By leveraging existing results, we are able to show that the bound is tight in a special case.

  • 7.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Sandberg, Henrik
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Resource Constrained Sensor Attacks by Minimizing Fisher Information2021In: 2021 American control conference (ACC), Institute of Electrical and Electronics Engineers Inc. , 2021, p. 4580-4585, article id 9482865Conference paper (Refereed)
    Abstract [en]

    We analyze the impact of sensor attacks on a linear state estimation problem subject to variance and sparsity constraints. We show that the maximum impact in a leader-follower game where the attacker first chooses the distribution of an adversarial perturbation and the defender follows by choosing an estimator is characterized by a minimum Fisher information principle. In general, this is a nonlinear variational problem, but we show that it can be reduced to a finite-dimensional mixed integer SDP. Alternatively, the proposed solution can be seen as a lower bound on the maximum impact for a game in which the defender plays first.

  • 8.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Sandberg, Henrik
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Matni, Nikolai
    University of Pennsylvania, University of Pennsylvania.
    Single Trajectory Nonparametric Learning of Nonlinear Dynamics2022In: Proceedings of 35th Conference on Learning Theory, COLT 2022, ML Research Press , 2022, p. 3333-3364Conference paper (Refereed)
    Abstract [en]

    Given a single trajectory of a dynamical system, we analyze the performance of the nonparametric least squares estimator (LSE). More precisely, we give nonasymptotic expected l2-distance bounds between the LSE and the true regression function, where expectation is evaluated on a fresh, counterfactual, trajectory. We leverage recently developed information-theoretic methods to establish the optimality of the LSE for nonparametric hypotheses classes in terms of supremum norm metric entropy and a subgaussian parameter. Next, we relate this subgaussian parameter to the stability of the underlying process using notions from dynamical systems theory. When combined, these developments lead to rate-optimal error bounds that scale as T−1/(2+q) for suitably stable processes and hypothesis classes with metric entropy growth of order δ−q. Here, T is the length of the observed trajectory, δ ∈ R+ is the packing granularity and q ∈ (0, 2) is a complexity term. Finally, we specialize our results to a number of scenarios of practical interest, such as Lipschitz dynamics, generalized linear models, and dynamics described by functions in certain classes of Reproducing Kernel Hilbert Spaces (RKHS).

  • 9.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Tsiamis, Anastasios
    Swiss Fed Inst Technol, Automat Control Lab, Zurich, Switzerland..
    Sandberg, Henrik
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Matni, Nikolai
    Univ Penn, Dept Elect & Syst Engn, Philadelphia, PA 19104 USA..
    How are policy gradient methods affected by the limits of control?2022In: 2022 IEEE 61ST CONFERENCE ON DECISION AND CONTROL (CDC), Institute of Electrical and Electronics Engineers (IEEE) , 2022, p. 5992-5999Conference paper (Refereed)
    Abstract [en]

    We study stochastic policy gradient methods from the perspective of control-theoretic limitations. Our main result is that ill-conditioned linear systems in the sense of Doyle inevitably lead to noisy gradient estimates. We also give an example of a class of stable systems in which policy gradient methods suffer from the curse of dimensionality. Finally, we show how our results extend to partially observed systems.

  • 10.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Tu, Stephen
    Robotics at Google, Robotics at Google.
    Learning with little mixing2022In: Advances in Neural Information Processing Systems 35 - 36th Conference on Neural Information Processing Systems, NeurIPS 2022, Neural information processing systems foundation , 2022Conference paper (Refereed)
    Abstract [en]

    We study square loss in a realizable time-series framework with martingale difference noise. Our main result is a fast rate excess risk bound which shows that whenever a trajectory hypercontractivity condition holds, the risk of the least-squares estimator on dependent data matches the iid rate order-wise after a burn-in time. In comparison, many existing results in learning from dependent data have rates where the effective sample size is deflated by a factor of the mixing-time of the underlying process, even after the burn-in time. Furthermore, our results allow the covariate process to exhibit long range correlations which are substantially weaker than geometric ergodicity. We call this phenomenon learning with little mixing, and present several examples for when it occurs: bounded function classes for which the L2 and L2+ε norms are equivalent, ergodic finite state Markov chains, various parametric models, and a broad family of infinite dimensional ℓ2(N) ellipsoids. By instantiating our main result to system identification of nonlinear dynamics with generalized linear model transitions, we obtain a nearly minimax optimal excess risk bound after only a polynomial burn-in time.

  • 11.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Zhou, Yishao
    Stockholm Univ, Dept Math, SE-10691 Stockholm, Sweden..
    Model Reduction of Semistable Distributed Parameter Systems2019In: Proceedings 2019 18th European Control Conference (ECC), IEEE , 2019, p. 1944-1950Conference paper (Refereed)
    Abstract [en]

    The model reduction problem for semistable infinite-dimensional control systems is studied in this paper. In relation to these systems, we study an object we call the semistability Gramian, which serves as a generalization of the ordinary controllability Gramian valid for semistable systems. This Gramian is then given geometric as well as algebraic characterization via a Lyapunov equation. We then proceed to show that under a commutativity assumption relating the original and reduced systems, and as long as the semistability is preserved, we may derive a priori error formulas in H-2-norm in terms of the trace of this Gramian.

  • 12.
    Ziemann, Ingvar
    et al.
    KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control).
    Ziemann, Volker
    Uppsala Univ, S-75120 Uppsala, Sweden..
    Noninvasively improving the orbit-response matrix while continuously correcting the orbit2021In: Physical Review Accelerators and Beams, E-ISSN 2469-9888, Vol. 24, no 7, article id 072804Article in journal (Refereed)
    Abstract [en]

    Based on continuously recorded beam positions and corrector excitations from, for example, a closedorbit feedback system we describe an algorithm that continuously updates an estimate of the orbit response matrix. The speed of convergence can be increased by adding very small perturbations, so-called dither, to the corrector excitations. Estimates for the rate of convergence and the asymptotically achievable accuracies are provided.

1 - 12 of 12
CiteExportLink to result list
Permanent link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf