kth.sePublications
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Robust learning and control of linear dynamical systems
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Decision and Control Systems (Automatic Control). KTH, School of Electrical Engineering and Computer Science (EECS), Centres, ACCESS Linnaeus Centre.ORCID iD: 0000-0003-1014-502x
2020 (English)Licentiate thesis, monograph (Other academic)
Abstract [en]

We consider the linear quadratic regulation problem when the plant is an unknown linear dynamical system. We present robust model-based methods based on convex optimization, which minimize the worst-case cost with respect to uncertainty around model estimates. To quantify uncertainty, we derive a methodbased on Bayesian inference, which is directly applicable to robust control synthesis.We focus on control policies that can be iteratively updated after sequentially collecting data. More specifically, we seek to design control policies that balance exploration (reducing model uncertainty) and exploitation (control of the system) when exploration must be safe (robust).First, we derive a robust controller to minimize the worst-case cost, with high probability, given the empirical observation of the system. This robust controller synthesis is then used to derive a robust dual controller, which updates its control policy after collecting data. An episode in which data is collected is called exploration, and the episode using an updated control policy is exploitation. The objective is to minimize the worst-case cost of the updated control policy, requiring that a given exploration budget constrains the worst-case cost during exploration.We look into robust dual control in both finite and infinite horizon settings. The main difference between the finite and infinite horizon settings is that the latter does not consider the length of the exploration and exploitation phase, but it rather approximates the cost using the infinite horizon cost. In the finite horizon setting, we discuss how different exploration lengths affect the trade-off between exploration and exploitation.Additionally, we derive methods that balance exploration and exploitation to minimize the cumulative worst-case cost for a fixed number of episodes. In this thesis, we refer to such a problem as robust reinforcement learning. Essentially, it is a robust dual controller aiming to minimize the cumulative worst-case cost, and that updates its control policy in each episode.Numerical experiments show that the proposed methods have better performance compared to existing state-of-the-art algorithms. Moreover, experiments also indicate that the exploration prioritizes the uncertainty reduction in the parameters that matter most for control.

Place, publisher, year, edition, pages
Stockholm, Sweden: Kungliga Tekniska högskolan, 2020.
Series
TRITA-EECS-AVL ; 2020:42
National Category
Control Engineering
Research subject
Electrical Engineering
Identifiers
URN: urn:nbn:se:kth:diva-280121ISBN: 978-91-7873-628-7 (print)OAI: oai:DiVA.org:kth-280121DiVA, id: diva2:1464024
Presentation
2020-10-01, https://kth-se.zoom.us/j/62734499438, Stockholm, 10:00 (English)
Opponent
Supervisors
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Note

QC 20200904

Available from: 2020-09-04 Created: 2020-09-03 Last updated: 2022-06-25Bibliographically approved

Open Access in DiVA

fulltext(1871 kB)813 downloads
File information
File name FULLTEXT02.pdfFile size 1871 kBChecksum SHA-512
6a06386f3a66124826e9d29726e57c758f965101c4c36464456d494aff45e5ee4deed5aaa10a4fe99feb82de09a1a64673b8c0ecc618ea153915703716d9963c
Type fulltextMimetype application/pdf

Other links

zoom link for online defense

Authority records

Ferizbegovic, Mina

Search in DiVA

By author/editor
Ferizbegovic, Mina
By organisation
Decision and Control Systems (Automatic Control)ACCESS Linnaeus Centre
Control Engineering

Search outside of DiVA

GoogleGoogle Scholar
Total: 861 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

isbn
urn-nbn

Altmetric score

isbn
urn-nbn
Total: 1346 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf