Turbulent flow through pipe bends has been extensively studied, but several phenomena still miss an exhaustive explanation. Due to centrifugal forces, the fluid flowing through a curved pipe forms two symmetric, counter-rotating Dean vortices. It has been observed, experimentally and numerically, that these vortices change their size, intensity and location in a quasi-periodic, oscillatory fashion, a phenomenon known as swirl-switching. These oscillations are responsible for failure due to fatigue in pipes, and their origin has been attributed to a recirculation bubble, disturbances coming from the upstream straight section and others. The present study tackles the problem by direct numerical simulations (DNS) of turbulent pipe flow at moderate Reynolds number, analysed, for the first time, with three-dimensional proper orthogonal decomposition (POD) in an effort to distinguish between the spatial and temporal contributions to the oscillations. The simulations are performed at a friction Reynolds number of about 360 with a divergence-free synthetic turbulence inflow, which is crucial to avoid the interference of low-frequency oscillations generated by a standard recycling method. Two different bends are considered, with curvature 0.1 and 0.3, preceded and followed by straight pipe segments. Our results indicate that a single low-frequency, three-dimensional POD mode is responsible for the swirl-switching. This mode represents a travelling wave, and was previously mistaken by 2D POD for two different modes. Low-order reconstruction clearly shows that the upstream turbulent flow does not play a role for the swirl-switching.