We prove several rigidity results about the centralizer of a smooth diffeomorphism, concentrating on two families of examples: diffeomorphisms with transitive centralizer, and perturbations of isometric extensions of Anosov diffeomorphisms of nilmanifolds. We classify all smooth diffeomorphisms with transitive centralizer: they are exactly the maps that preserve a principal fiber bundle structure, acting minimally on the fibers and trivially on the base. We also show that for any smooth, accessible isometric extension f0: M → M of an Anosov diffeomorphism of a nilmanifold, subject to a spectral bunching condition, any f ∈ Diff∞(M) sufficiently C1-close to f0 has centralizer a Lie group. If the dimension of this Lie group equals the dimension of the fiber, then f is a principal fiber bundle morphism covering an Anosov diffeomorphism. Using the results of this paper, we classify the centralizer of any partially hyperbolic diffeomorphism of a 3-dimensional, nontoral nilmanifold: either the centralizer is virtually trivial, or the diffeomorphism is an isometric extension of an Anosov diffeomorphism, and the centralizer is virtually Z × T.