The Fundamental Limitations of Learning Linear-Quadratic RegulatorsShow others and affiliations
2023 (English)In: 2023 62nd IEEE Conference on Decision and Control, CDC 2023, Institute of Electrical and Electronics Engineers (IEEE) , 2023, p. 4053-4060Conference paper, Published paper (Refereed)
Abstract [en]
We present a local minimax lower bound on the excess cost of designing a linear-quadratic controller from offline data. The bound is valid for any offline exploration policy that consists of a stabilizing controller and an energy bounded exploratory input. The derivation leverages a relaxation of the minimax estimation problem to Bayesian estimation, and an application of van Trees inequality. We show that the bound aligns with system-theoretic intuition. In particular, we demonstrate that the lower bound increases when the optimal control objective value increases. We also show that the lower bound increases when the system is poorly excitable, as characterized by the spectrum of the controllability gramian of the system mapping the noise to the state and the H∞ norm of the system mapping the input to the state. We further show that for some classes of systems, the lower bound may be exponential in the state dimension, demonstrating exponential sample complexity for learning the linear-quadratic regulator.
Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2023. p. 4053-4060
Series
Proceedings of the IEEE Conference on Decision and Control, ISSN 0743-1546
National Category
Control Engineering
Identifiers
URN: urn:nbn:se:kth:diva-343731DOI: 10.1109/CDC49753.2023.10383608ISI: 001166433803057Scopus ID: 2-s2.0-85184820524OAI: oai:DiVA.org:kth-343731DiVA, id: diva2:1839926
Conference
62nd IEEE Conference on Decision and Control, CDC 2023, Singapore, Singapore, Dec 13 2023 - Dec 15 2023
Note
QC 20240326
Part of ISBN 979-8-3503-0124-3
2024-02-222024-02-222025-03-19Bibliographically approved