We consider a joint sensor and controller design problem for linear Gaussian stochastic systems in which a weighted sum of quadratic control cost and the amount of information acquired by the sensor is minimized. This problem formulation is motivated by situations where a control law must be designed in the presence of sensing, communication, and privacy constraints. We show that an optimal linear joint sensor-controller policy is comprised of a linear sensor, Kalman filter, and a certainty equivalence controller, and can be synthesized by a numerically efficient algorithm based on semidefinite programming (SDP).