Optimal control problem with partial derivative equation (PDE) constraint is a numericalwise difficult problem because the optimality conditions lead to PDEs with mixed types of boundary values. The authors provide a new approach to solve this type of problem by space discretization and transform it into a standard optimal control for a multi-agent system. This resulting problem is formulated from a microscopic perspective while the solution only needs limited the macroscopic measurement due to the approach of Hamilton-Jacobi-Bellman (HJB) equation approximation. For solving the problem, only an HJB equation (a PDE with only terminal boundary condition) needs to be solved, although the dimension of that PDE is increased as a drawback. A pollutant elimination problem is considered as an example and solved by this approach. A numerical method for solving the HJB equation is proposed and a simulation is carried out.