A central question in dynamics is whether the topology of a system determines its geometry. This is known as rigidity. Under mild topological conditions rigidity holds for many classical cases, including: Kleinian groups, circle diffeomorphisms, unimodal interval maps, critical circle maps, and circle maps with a break point. More recent developments show that under similar topological conditions, rigidity does not hold for slightly more general systems. In this paper we state a conjecture which describes how topological classes are organized into rigidity classes.