Identification of systems operating in closed loop is an important problem in industrial applications, where model-based control is used to an increasing extent. For model-based controllers, plant changes over time eventually result in a mismatch between the dynamics of any initial model in the controller and the actual plant dynamics. When the mismatch becomes too large, control performance suffers and it becomes necessary to re-identify the plant to restore performance. Often the available data are not informative enough when the identification is performed in closed loop and extra excitation needs to be injected. This paper considers the problem of generating such excitation with the least possible disruption to the normal operations of the plant. The methods explicitly take time domain constraints into account. The formulation leads to optimal control problems which are in general very difficult optimization problems. Computationally tractable solutions based on Markov decision processes and model predictive control are presented. The performance of the suggested algorithms is illustrated in two simulation examples comparing the novel methods and algorithms available in the literature.