This article investigates the impact of antenna dispersion on wideband Terahertz (THz) imaging systems. It is demonstrated that the effective bandwidth is reduced from the nominal system bandwidth, and thus, the expected image resolution cannot be reached when the antenna system has inappropriate frequency dispersion characteristics. Besides a theoretical analysis and simulations, experiments were conducted using antennas with different dispersion characteristics in an ultra-wideband 500–750 GHz imaging setup to assess the impact on achievable resolution. When utilizing low dispersion dielectric-lens antennas, the system successfully detected multiple targets with a radius of 0.6 mm even when positioned at close distance (2 mm in cross-range and 3 mm in range). In contrast to that, when using higher dispersion horn antennas, the resolution of a dual-target scenario was reduced to 3 mm in cross-range and 4 mm in range. Furthermore, it is shown that the target position reconstruction accuracy, as well as the signal-to-clutter ratio (SCR), are also improved by 68% and 2.7 dB, respectively, when using low dispersion antennas in the same setup. This investigation, for the first time, highlights the importance of considering antenna dispersion for accurate image reconstruction, particularly for high-resolution wideband THz imaging systems.