Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Dynamic programming subject to total variation distance ambiguity
KTH, School of Electrical Engineering (EES), Automatic Control.
2015 (English)In: SIAM Journal of Control and Optimization, ISSN 0363-0129, E-ISSN 1095-7138, Vol. 53, no 4, p. 2040-2075Article in journal (Refereed) Published
Abstract [en]

The aim of this paper is to address optimality of stochastic control strategies via dynamic programming subject to total variation distance ambiguity on the conditional distribution of the controlled process. We formulate the stochastic control problem using minimax theory, in which the control minimizes the payoff while the conditional distribution, from the total variation distance set, maximizes it. First, we investigate the maximization of a linear functional on the space of probability measures on abstract spaces, among those probability measures which are within a total variation distance from a nominal probability measure, and then we give the maximizing probability measure in closed form. Second, we utilize the solution of the maximization to solve minimax stochastic control with deterministic control strategies, under a Markovian and a non-Markovian assumption, on the conditional distributions of the controlled process. The results of this part include (1) minimax optimization subject to total variation distance ambiguity constraint; (2) new dynamic programming recursions, which involve the oscillator seminorm of the value function, in addition to the standard terms; and (3) a new infinite horizon discounted dynamic programming equation, the associated contractive property, and a new policy iteration algorithm. Finally, we provide illustrative examples for both the finite and infinite horizon cases. For the infinite horizon case, we invoke the new policy iteration algorithm to compute the optimal strategies.

Place, publisher, year, edition, pages
2015. Vol. 53, no 4, p. 2040-2075
Keywords [en]
stochastic control, minimax, dynamic programming, total variational distance
National Category
Control Engineering
Identifiers
URN: urn:nbn:se:kth:diva-174009DOI: 10.1137/140955707ISI: 000360666700014Scopus ID: 2-s2.0-84940707638OAI: oai:DiVA.org:kth-174009DiVA, id: diva2:857771
Note

QC 20150930

Available from: 2015-09-30 Created: 2015-09-24 Last updated: 2017-12-01Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Search in DiVA

By author/editor
Charalambous, Themistoklis
By organisation
Automatic Control
In the same journal
SIAM Journal of Control and Optimization
Control Engineering

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 44 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • harvard1
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf