This paper investigates initial undershoots during the power response of grid-forming (GFM) converters. At first, the small signal model of the active power control loops (APC) is established. Based on the analytical model, the locations of the closed-loop RHP zero is utilized to indicate the existence and the peak of the initial undershoots. Subsequently, the impacts of the virtual impedance (VI) control and operating points on the initial power undershoots are comprehensively investigated. It is found that the VI control helps to dampen the oscillation but only has minimal influence on the RHP zero. In addition, the larger active power reference and smaller grid voltage leads to stronger undershoots. Finally, simulation results are presented to verify the accuracy of the established model and analysis.