This paper focuses on two inverse problems of the Kalman filter in which the process and measurement noises are correlated. The unknown covariance matrix in a stochastic system is reconstructed from observations of its posterior beliefs. For the standard inverse Kalman filtering problem, a novel duality-based formulation is proposed, where a well-defined inverse optimal control (IOC) problem is solved instead. Identifiability of the underlying model is proved, and a least squares estimator is designed that is statistically consistent. The time-invariant case using the steady-state Kalman gain is further studied. Since this inverse problem is ill-posed, a canonical class of covariance matrices is constructed, which can be uniquely identified from the dataset with asymptotic convergence. Finally, the performances of the proposed methods are illustrated by numerical examples.