Because of the variabilities of real-world image structures under the natural image transformations that arise when observing similar objects or spatio-temporal events under different viewing conditions, the receptive field responses computed in the earliest layers of the visual hierarchy may be strongly influenced by such geometric image transformations. One way of handling this variability is by basing the vision system on covariant receptive field families, which expand the receptive field shapes over the degrees of freedom in the image transformations.

This paper addresses the problem of deriving relationships between spatial and spatio-temporal receptive field responses obtained for different values of the shape parameters in the resulting multi-parameter families of receptive fields. For this purpose, we derive both (i) infinitesimal relationships, roughly corresponding to a combination of notions from semi-groups and Lie groups, as well as (ii) macroscopic cascade smoothing properties, which describe how receptive field responses at coarser spatial and temporal scales can be computed by applying smaller support incremental filters to the output from corresponding receptive fields at finer spatial and temporal scales, structurally related to the notion  of Lie algebras, although with directional preferences. 

For the receptive field models based on spatio-temporal smoothing using a sole combination of affine Gaussian smoothing kernels over image space with non-causal temporal Gaussian kernels over the temporal domain, we derive reasonably complete results in this respect, by exploiting the specific structure of the joint 2+1-D Gaussian spatio-temporal model. This permits characterizations in terms of higher-dimensional generalizations of the notion of Hermite polynomials, as well as characterizations in terms of joint spatio-temporal covariance matrices.

For the time-causal spatio-temporal receptive field model, where the temporal smoothing is instead performed using the time-causal limit kernel, we derive less complete results, however, still highly useful for the special case when the velocity parameters assume equal values in both the incremental convolution kernel as well as in the corresponding layers of the spatio-temporal scale-space representation.

The presented results provide (i) a deeper understanding of the relationships between spatial and spatio-temporal receptive field responses for different values of the filter parameters, which can be used for both (ii) designing more efficient schemes for computing receptive field responses over populations of multi-parameter families of receptive fields, as well as (iii) formulating idealized theoretical models of the computations of simple cells in biological vision.