We propose a spatiotemporal characterization of the entanglement dynamics in many-body localized (MBL) systems, which exhibits a striking resemblance to dynamical heterogeneity in classical glasses. Specifically, we find that the relaxation times of local entanglement, as measured by the concurrence, are spatially correlated yielding a dynamical length scale for quantum entanglement. As a consequence of this spatiotemporal analysis, we observe that the considered MBL system is made up of dynamically correlated clusters with a size set by this entanglement length scale. The system decomposes into compartments of different activity such as active regions with fast quantum entanglement dynamics and inactive regions where the dynamics is slow. We further find that the relaxation times of the on-site concurrence become broadly distributed and more spatially correlated, as disorder increases or the energy of the initial state decreases. Through this spatiotemporal characterization of entanglement, our work unravels a previously unrecognized connection between the behavior of classical glasses and the genuine quantum dynamics of MBL systems.