Data-driven control, the task of designing a controller based on process data, finds application in a wide range of disciplines and the topic has been intensively studied over more than half a century. The main purpose of this contribution is to elucidate on the commonalities between data-driven control and parameter estimation. In particular, we discuss the bias-variance trade-off, i.e. rather than aiming for the optimal controller one should aim for a constrained version, that may be characterized by tunable parameters, corresponding to hyperparameters in parameter estimation. As a result we shift attention from indirect vs direct data driven control by highlighting the important role played by (complete) minimal sufficient statistics. To keep technicalities at a minimum, still capturing the essential features of the problem, we consider the problem of minimizing the expected control cost for a quadratic open loop control problem applied to a finite impulse response system. In a Gaussian white noise setting, the maximum-likelihood parameter estimate constitutes a complete minimal sufficient statistic which allows us to focus on controllers that are functions of this model estimate without loss of statistical accuracy. We make a systematic study of three different controller structures and two different tuning techniques and illustrate their behaviours numerically.

We study the problem of determining an effective exploration strategy in static and non-linear optimization problems, which depend on an unknown scalar parameter to be learned from online collected noisy data. An optimal trade-off between exploration and exploitation is crucial for effective optimization under uncertainties, and to achieve this we consider a cumulative regret minimization approach over a finite horizon, with each time instant in the horizon characterized by a stochastic exploration signal, whose variance is to be designed. We aim to extend the well-established concepts of regret minimization from linear to non-linear systems, with a focus on the subsequent conceptual differences and challenges. Thus, under an idealized assumption on an appropriately defined information function associated with the excitation, we are able to show that an optimal exploration strategy is either to use no exploration at all (called lazy exploration) or adding an exploration excitation only at the first time instant of the horizon (called immediate exploration). A quadratic numerical example is presented to demonstrate the effectiveness of the proposed strategy.

MEMS gyroscopes are generally made up of two resonant systems: the so-called drive and sense modes. It is well known that the tracking of the drive-mode resonance frequency is crucial to make the device operate accurately. In this paper, we propose an approach based on recursive identification that allows to estimate this resonance frequency over the time. The proposed approach pertains to a recently developed control configuration which is based on the H∞ control framework and allows this configuration to give satisfactory control performance even when the drive-mode resonance frequency changes due to environment effects.

Data informativity is a crucial property to ensure the consistency of the prediction error estimate. This property has thus been extensively studied in the open-loop and in the closed-loop cases, but has only been briefly touched upon in the dynamic network case. In this paper, we consider the prediction error identification of the modules in a row of a dynamic network using the full input approach. Our main contribution is to propose a number of easily verifiable data informativity conditions for this identification problem. Among these conditions, we distinguish a sufficient data informativity condition that can be verified based on the topology of the network and a necessary and sufficient data informativity condition that can be verified via a rank condition on a matrix of coefficients that are related to a full-order model structure of the network. These data informativity conditions allow to determine different situations (i.e., different excitation patterns) leading to data informativity. In order to be able to distinguish between these different situations, we also propose an optimal experiment design problem that allows to determine the excitation pattern yielding a certain pre-specified accuracy with the least excitation power.

In this work, we consider the problem of regret minimization in adaptive minimum variance and linear quadratic control problems. Regret minimization has been extensively studied in the literature for both types of adaptive control problems. Most of these works give results of the optimal rate of the regret in the asymptotic regime. In the minimum variance case, the optimal asymptotic rate for the regret is log(T) which can be reached without any additional external excitation. On the contrary, for most adaptive linear quadratic problems, it is necessary to add an external excitation in order to get the optimal asymptotic rate of √T. In this paper, we will actually show from a theoretical study, as well as, in simulations that when the control horizon is pre-specified a lower regret can be obtained with either no external excitation or a new exploration type termed immediate.

In this work a heuristic to speed up the convergence of a feedback-based optimization scheme, when experiments can be run in batches, is discussed. The proposed approach allows to select the most promising experiment in the batch, as the one maximising the decrease of an associated Lyapunov function, and to define the inputs for the next batch, based on this. We suggest the application of the scheme to a biological setting, with the goal of maximizing the concentration of a product of interest in a bioreactor under a continuous perfusion framework, while at the same time minimizing the yield of a toxic byproduct. The potential of the approach is exposed by means of a simple synthetic example. 

In bioprocesses, it is important to model the kinetics of the macroscopic rates of reactions since these are required to catch the dynamical aspects of a process. In [Wang et al. 2020], a modeling method involving Gaussian processes has been developed, using a kernel especially designed for the modeling of Monod-type kinetics (activation, inhibition, double component, neutral effect). However, as will be illustrated in this paper, when the number of training data is limited or the metabolite concentration data do not have large variations (which is generally the case for real-life data), this kernel can yield inaccurate models for the kinetics. In this paper, we develop a new kernel better tailored for the modeling of Monod-type kinetics and we show that it has good modeling performances in the case of a limited number of data. The idea is to use the particular structure of Monod-type functions in the design of the kernel, i.e., we incorporate prior knowledge in the modeling.

In this letter, we study the trade-off between exploration and exploitation for linear quadratic adaptive control. This trade-off can be expressed as a function of the exploration and exploitation costs, called cumulative regret. It has been shown over the years that the optimal asymptotic rate of the cumulative regret is in many instances O(root T). In particular, this rate can be obtained by adding a white noise external excitation, with a variance decaying as O(1/root T). As the amount of excitation is pre-determined, such approaches can be viewed as open loop control of the external excitation. In this contribution, we approach the problem of designing the external excitation from a feedback perspective leveraging the well known benefits of feedback control for decreasing sensitivity to external disturbances and system-model mismatch, as compared to open loop strategies. We base the feedback on the Fisher information matrix which is a measure of the accuracy of the model. Specifically, the amplitude of the exploration signal is seen as the control input while the minimum eigenvalue of the Fisher matrix is the variable to be controlled. We call such exploration strategies Fisher Feedback Exploration (F2E). We propose one explicit F2E design, called Inverse Fisher Feedback Exploration (IF2E), and argue that this design guarantees the optimal asymptotic rate for the cumulative regret. We provide theoretical support for IF2E and in a numerical example we illustrate benefits of IF2E and compare it with the open loop approach as well as a method based on Thompson sampling.

In the Prediction Error identification framework, it is crucial that the collected data are informative with respect to the chosen model structure to get a consistent estimate. In this work, we focus on the data informativity property for the identification of multi-inputs single-output ARX systems in closed-loop and we derive a necessary and sufficient condition to verify if a given multisine external excitation combined with the feedback introduced by the controller yields informative data with respect to the chosen model structure.