In this paper, we investigate fundamental limits for physical implementations of the Kalman-Bucy filter for state estimation of a class of linear port-Hamiltonian systems. In particular, for the studied class of systems we show the Kalman-Bucy filter itself is a port-Hamiltonian systems and by invoking the second law of thermodynamics, we can characterize the external power supply needed to generate an optimal state estimate. We also show how the required external power supply can be decreased by allowing the filter to perturb the measured system to a larger extent. Hence, it is possible to decrease the so-called back action of the filter by spending more energy. We illustrate our results using passive electric circuits.