For prediction of a non-negative target signal using a non-negative input, we design a feed-forward neural network to achieve a better performance than a non-negative matrix factorization (NMF) algorithm. We provide a mathematical relation between the neural network and NMF. The architecture of the neural network is built on a property of rectified-linearunit (ReLU) activation function and a convex optimization layerwise training approach. For an illustrative example, we choose a speech enhancement application where a clean speech spectrum is estimated from a noisy spectrum.