In distributed optimization of multi-agent systems, agents cooperate to minimize a global function which is a sum of local objective functions. Motivated by applications including power systems, sensor networks, smart buildings, and smart manufacturing, various distributed optimization algorithms have been developed. In these algorithms, each agent performs local computation based on its own information and information received from its neighboring agents through the underlying communication network, so that the optimization problem can be solved in a distributed manner. This survey paper aims to offer a detailed overview of existing distributed optimization algorithms and their applications in power systems. More specifically, we first review discrete-time and continuous-time distributed optimization algorithms for undirected graphs. We then discuss how to extend these algorithms in various directions to handle more realistic scenarios. Finally, we focus on the application of distributed optimization in the optimal coordination of distributed energy resources.