This paper considers a privacy-concerned distributed optimization problem over multi-agent networks, in which malicious agents exist and try to infer the privacy information of the normal ones. We propose a novel dual averaging algorithm which involves the use of a correlated perturbation mechanism to preserve the privacy of the normal agents. It is shown that our algorithm achieves deterministic convergence under arbitrary initial conditions and the privacy preservation is guaranteed. Moreover, a probability density function of the perturbation is given to maximize the degree of privacy measured by the trace of the Fisher information matrix. Finally, a numerical example is provided to illustrate the effectiveness of our algorithm.