Regularized system identification has become a significant complement to more classical system identification. It has been numerically shown that kernel-based regularized estimators often perform better than the maximum likelihood estimator in terms of minimizing mean squared error (MSE). However, regularized estimators often require hyper-parameter estimation. This letter focuses on ridge regression and the regularized estimator by employing the empirical Bayes hyper-parameter estimator. We utilize the excess MSE to quantify the MSE difference between the empirical-Bayes-based regularized estimator and the maximum likelihood estimator for large sample sizes. We then exploit the excess MSE expressions to develop both a family of generalized Bayes estimators and a family of closed-form biased estimators. They have the same excess MSE as the empirical-Bayes-based regularized estimator but eliminate the need for hyper-parameter estimation. Moreover, we conduct numerical simulations to show that the performance of these new estimators is comparable to the empirical-Bayes-based regularized estimator, while computationally, they are more efficient.

The dual control problem, first introduced by Feldbaum in the 1960s, is recognized as encapsulating the "exploration versus exploitation"dilemma, central to online learning and control. Numerous heuristic-based exploration methods have been developed to facilitate active learning. However, the theoretically optimal solution provided by dynamic programming (DP) remains computationally intractable for most problems due to the curse of dimensionality. In this paper, we revisit the DP framework within the context of regret minimization for a simple real-time optimization problem, aiming to identify valuable insights and uncover new avenues for simplified DP-based exploration strategies. By deriving the two-horizon DP solution in our simple setting, we observe that the optimal input is obtained by solving a closed-form optimization problem composed of two distinct components representing exploration and exploitation separately, clearly highlighting their inherent trade-off. For longer horizons, receding and cyclic horizon control based on the iterative application of the two-horizon DP provide possible approximations, reducing computational complexity while yielding useful suboptimal control policies. A key advantage of the DP-based exploration method is its ability to automatically adjust the exploration based on the current exploitation and system uncertainty. The proposed method is studied numerically through comparative evaluations against classical heuristic exploration methods from the literature.

Data-driven control, the task of designing a controller based on process data, finds application in a wide range of disciplines and the topic has been intensively studied over more than half a century. The main purpose of this contribution is to elucidate on the commonalities between data-driven control and parameter estimation. In particular, we discuss the bias-variance trade-off, i.e. rather than aiming for the optimal controller one should aim for a constrained version, that may be characterized by tunable parameters, corresponding to hyperparameters in parameter estimation. As a result we shift attention from indirect vs direct data driven control by highlighting the important role played by (complete) minimal sufficient statistics. To keep technicalities at a minimum, still capturing the essential features of the problem, we consider the problem of minimizing the expected control cost for a quadratic open loop control problem applied to a finite impulse response system. In a Gaussian white noise setting, the maximum-likelihood parameter estimate constitutes a complete minimal sufficient statistic which allows us to focus on controllers that are functions of this model estimate without loss of statistical accuracy. We make a systematic study of three different controller structures and two different tuning techniques and illustrate their behaviours numerically.

Regularized system identification has become the research frontier of system identification in the past decade. One related core subject is to study the convergence properties of various hyper-parameter estimators as the sample size goes to infinity. In this paper, we consider one commonly used hyper-parameter estimator, the empirical Bayes (EB). Its convergence in distribution has been studied, and the explicit expression of the covariance matrix of its limiting distribution has been given. However, what we are truly interested in are factors contained in the covariance matrix of the EB hyper-parameter estimator, and then, the convergence of its covariance matrix to that of its limiting distribution is required. In general, the convergence in distribution of a sequence of random variables does not necessarily guarantee the convergence of its covariance matrix. Thus, the derivation of such convergence is a necessary complement to our theoretical analysis about factors that influence the convergence properties of the EB hyper-parameter estimator. In this paper, we consider the regularized finite impulse response (FIR) model estimation with deterministic inputs, and show that the covariance matrix of the EB hyper-parameter estimator converges to that of its limiting distribution. Moreover, we run numerical simulations to demonstrate the efficacy of our theoretical results.

When applying the Bayesian manifold regularization method to function estimation problem with manifold constraints, the direct implementation has computational complexity O(N3), where N is the number of input-output data measurements. This becomes particularly costly when N is large. In this paper, we propose a more efficient implementation based on the Kalman filter and smoother using a state-space model realization of the underlying Gaussian process. Moreover, we explore the sequentially semi-separable structure of the Laplacian matrix and the posterior covariance matrix. Our proposed implementation has computational complexity O(N) and thus can be applied to large data problems. We exemplify the effectiveness of our proposed implementation through numerical simulations.