In this paper, we consider device-to-device (D2D) communication that is underlaid in a multi-cell massive multiple-input multiple-output (MIMO) system and proposes a new framework for power control and pilot allocation. In this scheme, the cellular users (CUs) in each cell get orthogonal pilots which are reused with reuse factor one across cells, while all the D2D pairs share another set of orthogonal pilots. We derive a closed-form capacity lower bound for the CUs with different receive processing schemes. In addition, we derive a capacity lower bound for the D2D receivers and a closed-form approximation of it. We provide power control algorithms to maximize the minimum spectral efficiency (SE) and to maximize the product of the signal-to-interference-plus-noise ratios in the network. Different from prior works, in our proposed power control schemes, we consider joint pilot and data transmission optimization. Finally, we provide a numerical evaluation, where we compare our proposed power control schemes with the maximum transmit power case and the case of conventional multi-cell massive MIMO without D2D communication. Based on the provided results, we conclude that our proposed scheme increases the sum SE of multi-cell massive MIMO networks.