Suppose a linear model Y = Hx+n, where inputs x, n are independent Gaussian mixtures. The problem is to design the transfer matrix so as to minimize the mean square error (MSE) when estimating x from . This problem has important applications, but faces at least three hurdles. Firstly, even for a fixed H, the minimum MSE (MMSE) has no analytical form. Secondly, theMMSE is generally not convex in . Thirdly, derivatives of the MMSEw.r.t. are hard to obtain. This paper casts theproblemas a stochastic program and invokes gradient methods. The study is motivated by two applications in signal processing. One concerns the choice of error-reducing precoders; the other deals with selection of pilot matrices for channel estimation. In either setting, our numerical results indicate improved estimation accuracy-markedly better than those obtained by optimal design based on standard linear estimators. Some implications of the non-convexities of the MMSE are noteworthy, yet, to our knowledge, not well known. For example, there are cases in which more pilot power is detrimental for channel estimation. This paper explains why.