Clock synchronization is ubiquitous in wireless systems for communication, sensing, and control. In this paper, we design a scalable system in which an indefinite number of passively receiving wireless units can synchronize to a single master clock at the level of discrete clock ticks. Accurate synchronization requires an estimate of the node positions to compensate the time-of-flight transmission delay in line-of-sight environments. If such information is available, the framework developed here takes position uncertainties into account. In the absence of such information, as in indoor scenarios, we propose an auxiliary localization mechanism. Furthermore, we derive the Cramer-Rao bounds for the system, which show that it enables synchronization accuracy at sub-nanosecond levels. Finally, we develop and evaluate an online estimation method, which is statistically efficient.