We study rigidity properties of ABC group actions on the three torus $\mathbb T^3$, by affine transformations. The linear part of such an action is an ABC subgroup of $SL(3,\mathbb Z)$. We investigate when such a linear ABC action on $\mathbb T^3$ can be extended to an affine action that has no identity factors. For such actions, we show KAM rigidity; the main reason for the existence of the conjugacy is KAM rigidity of the parabolic $\mathbb Z^2$ action inside the ABC group action. The main novelty in the opposite case is that we introduce and prove a new type of local rigidity phenomenon, which we label fiberwise KAM rigidity. Even though such affine actions are never KAM rigid, we show that all perturbations of specific form are conjugate to the initial action. We classify fiberwise perturbations of such actions. One important new ingredient is that we use the whole non-commutative action. The method of proof is the KAM iterative method. A detailed analysis of non-commutative group relations is required. Moreover, the systems we consider can be parabolic or partially hyperbolic.