This paper presents a theoretical analysis of the orientation selectivity of simple and complex cells that can be well modelled by the generalized Gaussian derivative model for visual receptive fields, with the purely spatial component of the receptive fields determined by oriented affine Gaussian derivatives for different orders of spatial differentiation.

A detailed mathematical analysis is presented for the three different cases of either: (i) purely spatial receptive fields, (ii) space-time separable spatio-temporal receptive fields and (iii)~velocity-adapted spatio-temporal receptive fields. Closed-form theoretical expressions for the orientation selectivity curves for idealized models of simple and complex cells are derived for all these main cases, and it is shown that the degree of orientation selectivity of the receptive fields increases with a scale parameter ratio $\kappa$, defined as the ratio between the scale parameters in the directions perpendicular to vs. parallel with the preferred orientation of the receptive field. It is also shown that the degree of orientation selectivity increases with the order of spatial differentiation in the underlying affine Gaussian derivative operators over the spatial domain.

We describe biological implications of the derived theoretical results, demonstrating that the predictions from the presented theory are consistent with previously established biological results concerning broad vs. sharp orientation tuning of visual neurons in the primary visual cortex. We also demonstrate that the above theoretical predictions, in combination with these biological results, are consistent with a previously formulated biological hypothesis, stating that the biological receptive field shapes should span the degrees of freedom in affine image transformations, to support affine covariance over the population of receptive fields in the primary visual cortex.

Based on the results from the theoretical analysis in the paper, combined with existing results for biological experiments, we formulate a set of testable predictions that could be used to, with neurophysiological experiments, judge if the receptive fields in the primary visual cortex of higher mammals could be regarded as spanning a variability over the eccentricity or the elongation of the receptive fields, and, if so, then also characterize if such a variability would, in a structured way, be related to the pinwheel structure in the visual cortex.

For comparison, we also present a corresponding theoretical orientation selectivity analysis for purely spatial receptive fields according to an affine Gabor model. The results from that analysis are consistent with the results obtained from the affine Gaussian derivative model, in the respect that the orientation selectivity becomes more narrow when making the receptive fields wider in the direction perpendicular to the preferred orientation of the receptive field. The affine Gabor model does, however, comprise one more degree of freedom in its parameter space, compared to the affine Gaussian derivative model, where a variability within that additional dimension of the parameter space does also strongly influence the orientation selectivity of the receptive fields. In this respect, the affine Gaussian derivative model leads to more specific predictions concerning relationships between the orientation selectivity and the elongation of the receptive fields, compared to the affine Gabor model.