This paper gives an overview of a theory for modelling the interaction between geometric image transformations and receptive field responses for a visual observer that views objects and spatio-temporal events in the environment. Specifically, the paper gives an in-depth treatment of the influence on the receptive field responses due to the following types of locally linearized geometric image transformations: (i) spatial scaling transformations caused by varying the distance between object and the observer, (ii) non-isotropic spatial affine transformations caused by varying the viewing direction relative to the object, (iii) Galilean transformations caused by relative motions between the object and the viewing direction, and (iv) temporal scaling transformations caused by spatio-temporal events occurring either faster or slower relative to a previously observed reference view. By postulating that the family of receptive fields should be covariant under these classes of geometric image transformations, it follows that the receptive field shapes should be expanded over the degrees of freedom of the corresponding image transformations, to enable a formal matching between the receptive field responses computed under different viewing conditions for the same scene or for a structurally similar spatio-temporal event.

We develop this theory for the idealized generalized Gaussian derivative model of visual receptive fields in terms of combinations of (i) smoothing with affine Gaussian kernels over the spatial domain, (ii) smoothing with either the non-causal Gaussian kernel or the time-causal limit kernel over the temporal domain and (iii) the computation of scale-normalized spatial and temporal derivatives from the spatio-temporally smoothed image data. Formal transformation properties are stated for these computational primitives for the 4 main types of primitive geometric image transformations, and it is shown that a visual system based on such computational primitives will have the ability to match the spatio-temporal receptive responses computed from dynamic scenes under the variabilities caused by composed variations in the viewing conditions.

We conclude the treatment by discussing and providing potential support for a working hypothesis that the receptive fields of simple cells in the primary visual cortex ought to be covariant under these classes of geometric image transformations, and thus have the shapes of their receptive fields expanded over the degrees of freedom of the corresponding geometric image transformations.