LIGO and Virgo’s third observing run revealed the first neutron star–black hole (NSBH) merger candidates in gravitational waves. These events are predicted to synthesize r-process elements1,2 creating optical/near-infrared ‘kilonova’ emission. The joint gravitational wave and electromagnetic detection of an NSBH merger could be used to constrain the equation of state of dense nuclear matter3, and independently measure the local expansion rate of the Universe4. Here, we present the optical follow-up and analysis of two of the only three high-significance NSBH merger candidates detected to date, S200105ae and S200115j, with the Zwicky Transient Facility5. The Zwicky Transient Facility observed ~48% of S200105ae and ~22% of S200115j’s localization probabilities, with observations sensitive to kilonovae brighter than −17.5 mag fading at 0.5 mag d−1 in the g- and r-bands; extensive searches and systematic follow-up of candidates did not yield a viable counterpart. We present state-of-the-art kilonova models tailored to NSBH systems that place constraints on the ejecta properties of these NSBH mergers. We show that with observed depths of apparent magnitude ~22 mag, attainable in metre-class, wide-field-of-view survey instruments, strong constraints on ejecta mass are possible, with the potential to rule out low mass ratios, high black hole spins and large neutron star radii.