In Chain Inflation the universe tunnels along a series of false vacua of ever-decreasing energy. The main goal of this paper is to embed Chain Inflation in high energy fundamental physics. We begin by illustrating a simple effective formalism for calculating Cosmic Microwave Background (CMB) observables in Chain Inflation. Density perturbations seeding the anisotropies emerge from the probabilistic nature of tunneling (rather than from quantum fluctuations of the inflation). To obtain the correct normalization of the scalar power spectrum and the scalar spectral index, we find an upper limit on the scale of inflation at horizon crossing of CMB scales, V-*(1/4) < 10(12) GeV. We then provide an explicit realization of chain inflation, in which the inflaton is identified with an axion in supergravity. The axion enjoys a perturbative shift symmetry which is broken to a discrete remnant by instantons. The model, which we dub 'natural chain inflation' satisfies all cosmological constraints and can be embedded into a standard Lambda CDM cosmology. Our work provides a major step towards the ultraviolet completion of chain inflation in string theory.