A new multi-agent based congestion evacuation model incorporating panic behavior is proposed in this paper for simulating pedestrian evacuation in public places such as a stadium. Different from the existing results, pedestrians in this model are divided into four classes and each pedestrian's status can be either normal, being overtaken, or casualty. The direction of action for each individual is affected by competitive ability, distance to the exits as well as number and density of occupants within the view field of the agent. Our simulations exhibit that during the evacuation process: (1) The agents gather in front of the exits spontaneously and present arched shapes close to the exits. (2) Under the panic state the agents cohere closely and almost do not change the target exit. So other alternative exits are ignored. (3) For the case without obstacle, the casualties under panic increase greatly. But if there are obstacles (chairs), the congestion can be alleviated. Thus the casualties are reduced. (4) If certain exit is partly clogged, the evacuation becomes more efficient when adding a virtual leader. The overall simulation results show that the proposed model can reproduce the real evacuation process in a stadium quite well.