The anomalous Hall effect (AHE) has emerged as a key indicator of time-reversal symmetry breaking (TRSB) and topological features in electronic band structures. Absent of a magnetic field, the AHE requires spontaneous TRSB but has proven hard to probe due to averaging over domains. The anomalous component of the Hall effect is thus frequently derived from extrapolating the magnetic field dependence of the Hall response. We show that discerning whether the AHE is an intrinsic property of the field-free system becomes intricate in the presence of strong magnetic fluctuations. As a study case, we use the Weyl semimetal PrAlGe, where TRSB can be toggled via a ferromagnetic transition, providing a transparent view of the AHE's topological origin. Through a combination of thermodynamic, transport, and muon spin relaxation measurements, we contrast the behavior below the ferromagnetic transition temperature to that of strong magnetic fluctuations above. Our results on PrAlGe provide general insights into the interpretation of anomalous Hall signals in systems where TRSB is debated, such as families of kagome metals or certain transition metal dichalcogenides.