This paper considers the precoder designs for successive zero-forcing dirty paper coding (SZF-DPC), a suboptimal transmission technique for MIMO broadcast channels (MIMO BCs). Existing precoder designs for SZF-DPC often consider a sum power constraint. In this paper, we address the precoder design for SZF-DPC with per-antenna power constraints (PAPCs), which has not been well studied. First, we formulate the precoder design as a rank-constrained optimization problem, which is generally difficult to handle. To solve this problem, we follow a relaxation approach, and prove that the optimal solution of the relaxed problem is also optimal for the original problem. Considering the relaxed problem, we propose a numerically efficient algorithm to find the optimal solution, which exhibits a fast convergence rate. Suboptimal precoder designs, with lower computational complexity, are also presented, and compared with the optimal ones in terms of achievable sum rate and computational complexity.