This paper addresses mobility management in the downlink of a mmWave cell-free massive MIMO (multiple-input multiple-output) network with imperfect channel knowledge obtained from pilot training. The network consists of a large number of geographically distributed access points (APs) simultaneously serving multiple user equipments (UEs) via coherent joint transmission. The objective is to extend traditional handover concepts to the challenging AP-UE association strategies of cell-free networks. To this end, we propose a distributed algorithm for joint pilot assignment and cluster formation in a dynamic environment considering UE mobility. The primary goal is to limit the necessary number of AP and pilot changes, with reasonable computational complexity. We evaluate the performance in terms of the spectral efficiency with maximum ratio and regularized zero-forcing precoding. Results show that our proposed distributed algorithm effectively identifies the essential AP-UE association refinements. Moreover, it provides a significantly lower average number of pilot changes compared to an ultra-dense network.