kth.sePublications
Change search
Link to record
Permanent link

Direct link
Lindeberg, Tony, ProfessorORCID iD iconorcid.org/0000-0002-9081-2170
Biography [eng]

Tony Lindeberg is a Professor of Computer Science at KTH Royal Institute of Technology in Stockholm, Sweden. He received his MSc degree in 1987, his PhD degree in 1991, became docent in 1996, and was appointed professor in 2000. He was a Research Fellow at the Royal Swedish Academy of Sciences between 2000 and 2010.

His research interests in computer vision relate to scale-space representation, image features, object recognition, video analysis and computational modelling of biological vision. He has developed theories and methodologies for continuous and discrete scale-space representation, visual and auditory receptive fields, hierarchical and deep networks, detection of salient image structures, automatic scale selection, scale-covariant and scale-invariant image features, affine-covariant and affine-invariant features, affine and Galilean normalization, temporal, spatio-temporal and spectro-temporal scale-space concepts as well as spatial and spatio-temporal image descriptors for image-based recognition.

He does also work on computational modelling of hearing and has previously worked on topics in medical image analysis, brain activation and gesture recognition. He is the author of the book Scale-Space Theory in Computer Vision.

Biography [swe]

Tony Lindeberg är professor i datavetenskap vid Kungliga Tekniska Högskolan, KTH, i Stockholm. Han fick sin civilingenjörsexamen i teknisk fysik 1987, blev teknisk doktor i datalogi 1991, blev docent 1996 och utnämndes till professor 2000. Han var akademiforskare vid Kungliga Vetenskapsakademien mellan 2000 och 2010.

Hans forskningsintressen i datorseende omfattar skalrumsrepresentation, särdragsdetektion, objektigenkänning, videoanalys och beräkningsinriktad modellering av biologiskt seende. Han har utvecklat teorier och metodiker för kontinuerliga och diskreta skalrumsrepresentationer, visuella och auditiva receptiva fält, hierarkiska och djupa nätverk, detektion av framträdande särdrag, automatiskt skalval, skalkovarianta och skalinvarianta särdrag, affint kovarianta och invarianta särdrag, affin och galileisk normalisering, temporala, spatio-temporala och spektro-temporala skalrumsbegrepp samt spatiala och spatio-temporala bilddeskriptorer för bildbaserad igenkänning.

Han arbetar också med beräkningsorienterad modellering av hörsel och har tidigare arbetat inom medicinsk bildanalys, hjärnaktivitetsanalys och med gestigenkänning. Han är författare till boken ”Scale-Space Theory in Computer Vision”.

Publications (10 of 171) Show all publications
Lindeberg, T. (2025). A time-causal and time-recursive analogue of the Gabor transform. IEEE Transactions on Information Theory, 71(2), 1450-1480
Open this publication in new window or tab >>A time-causal and time-recursive analogue of the Gabor transform
2025 (English)In: IEEE Transactions on Information Theory, ISSN 0018-9448, E-ISSN 1557-9654, Vol. 71, no 2, p. 1450-1480Article in journal (Refereed) Published
Abstract [en]

This paper presents a time-causal analogue of the Gabor filter, as well as a both time-causal and time-recursive analogue of the Gabor transform, where the proposed time-causal representations obey both temporal scale covariance and a cascade property over temporal scales. The motivation behind these constructions is to enable theoretically well-founded time-frequency analysis over multiple temporal scales for real-time situations, or for physical or biological modelling situations, when the future cannot be accessed, and the non-causal access to the future in Gabor filtering is therefore not viable for a time-frequency analysis of the system.

We develop a principled axiomatically determined theory for formulating these time-causal time-frequency representations, obtained by replacing the Gaussian kernel in the Gabor filtering with a time-causal kernel, referred to as the time-causal limit kernel, and which guarantees simplification properties from finer to coarser levels of scales in a time-causal situation, similar as the Gaussian kernel can be shown to guarantee over a non-causal temporal domain. We do also develop an axiomatically determined theory for implementing a discrete analogue of the proposed time-causal frequency analysis method on discrete data, based on first-order recursive filters coupled in cascade, with provable variation-diminishing properties that strongly suppress the influence from local perturbations and noise, and with specially chosen time constants to achieve self-similarity over scales and temporal scale covariance.

In these ways, the proposed time-frequency representations guarantee well-founded treatment over multiple temporal scales, in situations when the characteristic scales in the signals, or physical or biological phenomena, to be analyzed may vary substantially, and additionally all steps in the time-frequency analysis have to be fully time-causal.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE), 2025
Keywords
Time-frequency analysis, Gabor filter, Gabor transform, Time-causal, Time-recursive, Temporal scale, Scale covariance, Harmonic analysis, Signal processing
National Category
Signal Processing Mathematics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-356929 (URN)10.1109/tit.2024.3507879 (DOI)001407266100007 ()2-s2.0-85210750479 (Scopus ID)
Projects
Covariant and invariant deep networks
Funder
Swedish Research Council, 2022-02969
Note

QC 20250226

Available from: 2024-11-28 Created: 2024-11-28 Last updated: 2025-02-26Bibliographically approved
Lindeberg, T. (2025). Approximation properties relative to continuous scale space for hybrid discretisations of Gaussian derivative operators. Frontiers in Signal Processing, 4, 1-12, Article ID 1447841.
Open this publication in new window or tab >>Approximation properties relative to continuous scale space for hybrid discretisations of Gaussian derivative operators
2025 (English)In: Frontiers in Signal Processing, E-ISSN 2673-8198, Vol. 4, p. 1-12, article id 1447841Article in journal (Refereed) Published
Abstract [en]

 This paper presents an analysis of properties of two hybrid discretisation methods for Gaussian derivatives, based on convolutions with either the normalised sampled Gaussian kernel or the integrated Gaussian kernel followed by central differences. The motivation for studying these discretisation methods is that in situations when multiple spatial derivatives of different orders are needed at the same scale level, they can be computed significantly more efficiently, compared to more direct derivative approximations based on explicit convolutions with either sampled Gaussian derivative kernels or integrated Gaussian derivative kernels. We characterise the properties of these hybrid discretisation methods in terms of quantitative performance measures, concerning the amount of spatial smoothing that they imply, as well as the relative consistency of the scale estimates obtained from scale-invariant feature detectors with automatic scale selection, with an emphasis on the behaviour for very small values of the scale parameter, which may differ significantly from corresponding results obtained from the fully continuous scale-space theory, as well as between different types of discretisation methods. The presented results are intended as a guide, when designing as well as interpreting the experimental results of scale-space algorithms that operate at very fine scale levels.

Place, publisher, year, edition, pages
Frontiers Media SA, 2025
Keywords
scale, discrete, continuous, Gaussian kernel, Gaussian derivative, scale space
National Category
Computer graphics and computer vision Mathematics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-359213 (URN)10.3389/frsip.2024.1447841 (DOI)001418090300001 ()2-s2.0-85218832742 (Scopus ID)
Projects
Covariant and invariant deep networks
Funder
Swedish Research Council, 2022-02969
Note

QC 20250129

Available from: 2025-01-29 Created: 2025-01-29 Last updated: 2025-03-12Bibliographically approved
Lindeberg, T. (2025). Orientation selectivity properties for the affine Gaussian derivative and the affine Gabor models for visual receptive fields. Journal of Computational Neuroscience, 53(1), 61-98
Open this publication in new window or tab >>Orientation selectivity properties for the affine Gaussian derivative and the affine Gabor models for visual receptive fields
2025 (English)In: Journal of Computational Neuroscience, ISSN 0929-5313, E-ISSN 1573-6873, Vol. 53, no 1, p. 61-98Article in journal (Refereed) Published
Abstract [en]

 This paper presents an in-depth theoretical analysis of the orientation selectivity properties of simple cells and complex cells, that can be well modelled by the generalized Gaussian derivative model for visual receptive fields, with the purely spatial component of the receptive fields determined by oriented affine Gaussian derivatives for different orders of spatial differentiation.

A detailed mathematical analysis is presented for the three different cases of either: (i) purely spatial receptive fields, (ii) space-time separable spatio-temporal receptive fields and (iii) velocity-adapted spatio-temporal receptive fields. Closed-form theoretical expressions for the orientation selectivity curves for idealized models of simple and complex cells are derived for all these main cases, and it is shown that the orientation selectivity of the receptive fields becomes more narrow, as a scale parameter ratio κ, defined as the ratio between the scale parameters in the directions perpendicular to vs. parallel with the preferred orientation of the receptive field, increases. It is also shown that the orientation selectivity becomes more narrow with increasing order of spatial differentiation in the underlying affine Gaussian derivative operators over the spatial domain.

A corresponding theoretical orientation selectivity analysis is also presented for purely spatial receptive fields according to an affine Gabor model, showing that: (i) the orientation selectivity becomes more narrow when making the receptive fields wider in the direction perpendicular to the preferred orientation of the receptive field; while (ii) an additional degree of freedom in the affine Gabor model does, however, also strongly affect the orientation selectivity properties.

Place, publisher, year, edition, pages
Springer Nature, 2025
Keywords
Receptive field, Orientation selectivity, Affine covariance, Simple cell, Complex cell, Vision
National Category
Bioinformatics (Computational Biology)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-359222 (URN)10.1007/s10827-024-00888-w (DOI)001408524100001 ()39878929 (PubMedID)2-s2.0-85217424045 (Scopus ID)
Projects
Covariant and invariant deep networks
Funder
Swedish Research Council, 2022-02969
Note

QC 20250318

Available from: 2025-01-29 Created: 2025-01-29 Last updated: 2025-03-18
Perzanowski, A. & Lindeberg, T. (2025). Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations.
Open this publication in new window or tab >>Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations
2025 (English)Report (Other academic)
Abstract [en]

Due to the variabilities in image structures caused by perspective scaling transformations, it is essential for deep networks to have an ability to generalise to scales not seen during training. This paper presents an in-depth analysis of the scale generalisation properties of the scale-covariant and scale-invariant Gaussian derivative networks, complemented with both conceptual and algorithmic extensions. For this purpose, Gaussian derivative networks (GaussDerNets) are evaluated on new rescaled versions of the Fashion-MNIST and the CIFAR-10 datasets, with spatial scaling variations over a factor of 4 in the testing data, that are not present in the training data. Additionally, evaluations on the previously existing STIR datasets show that the GaussDerNets achieve better scale generalisation than previously reported for these datasets for other types of deep networks.

We first experimentally demonstrate that the GaussDerNets have quite good scale generalisation properties on the new datasets, and that average pooling of feature responses over scales may sometimes also lead to better results than the previously used approach of max pooling over scales. Then, we demonstrate that using a spatial max pooling mechanism after the final layer enables localisation of non-centred objects in image domain, with maintained scale generalisation properties. We also show that regularisation during training, by applying dropout across the scale channels, referred to as scale-channel dropout, improves both the performance and the scale generalisation.

In additional ablation studies, we show that, for the rescaled CIFAR-10 dataset, basing the layers in the GaussDerNets on derivatives up to order three leads to better performance and scale generalisation for coarser scales, whereas networks based on derivatives up to order two achieve better scale generalisation for finer scales. Moreover, we demonstrate that discretisations of GaussDerNets based on the discrete analogue of the Gaussian kernel in combination with central difference operators perform best or among the best, compared to a set of other discrete approximations of the Gaussian derivative kernels. Furthermore, we show that the improvement in performance obtained by learning the scale values of the Gaussian derivatives, as opposed to using the previously proposed choice of a fixed logarithmic distribution of the scale levels, is usually only minor, thus supporting the previously postulated choice of using a logarithmic distribution as a very reasonable prior.

Finally, by visualising the activation maps and the learned receptive fields, we demonstrate that the GaussDerNets have very good explainability properties.

Publisher
p. 52
Keywords
Scale covariance, Scale invariance, Scale generalisation, Scale selection, Gaussian derivative, Scale space, Deep learning, Receptive fields
National Category
Computer graphics and computer vision
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-354182 (URN)10.48550/arXiv.2409.11140 (DOI)
Projects
Covariant and invariant deep networks
Funder
Swedish Research Council, 2018-03586, 2022-02969
Note

QC 20241001

Available from: 2024-10-01 Created: 2024-10-01 Last updated: 2025-03-27Bibliographically approved
Perzanowski, A. & Lindeberg, T. (2025). Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations. Journal of Mathematical Imaging and Vision, 67(3), 1-39, Article ID 29.
Open this publication in new window or tab >>Scale generalisation properties of extended scale-covariant and scale-invariant Gaussian derivative networks on image datasets with spatial scaling variations
2025 (English)In: Journal of Mathematical Imaging and Vision, ISSN 0924-9907, E-ISSN 1573-7683, Vol. 67, no 3, p. 1-39, article id 29Article in journal (Refereed) Published
Abstract [en]

Due to the variabilities in image structures caused by perspective scaling transformations, it is essential for deep networks to have an ability to generalise to scales not seen during training. This paper presents an in-depth analysis of the scale generalisation properties of the scale-covariant and scale-invariant Gaussian derivative networks, complemented with both conceptual and algorithmic extensions. For this purpose, Gaussian derivative networks (GaussDerNets) are evaluated on new rescaled versions of the Fashion-MNIST and the CIFAR-10 datasets, with spatial scaling variations over a factor of 4 in the testing data, that are not present in the training data. Additionally, evaluations on the previously existing STIR datasets show that the GaussDerNets achieve better scale generalisation than previously reported for these datasets for other types of deep networks. We first experimentally demonstrate that the GaussDerNets have quite good scale generalisation properties on the new datasets and that average pooling of feature responses over scales may sometimes also lead to better results than the previously used approach of max pooling over scales. Then, we demonstrate that using a spatial max pooling mechanism after the final layer enables localisation of non-centred objects in the image domain, with maintained scale generalisation properties. We also show that regularisation during training, by applying dropout across the scale channels, referred to as scale-channel dropout, improves both the performance and the scale generalisation. In additional ablation studies, we show that, for the rescaled CIFAR-10 dataset, basing the layers in the GaussDerNets on derivatives up to order three leads to better performance and scale generalisation for coarser scales, whereas networks based on derivatives up to order two achieve better scale generalisation for finer scales. Moreover, we demonstrate that discretisations of GaussDerNets based on the discrete analogue of the Gaussian kernel in combination with central difference operators perform best or among the best, compared to a set of other discrete approximations of the Gaussian derivative kernels. Furthermore, we show that the improvement in performance obtained by learning the scale values of the Gaussian derivatives, as opposed to using the previously proposed choice of a fixed logarithmic distribution of the scale levels, is usually only minor, thus supporting the previously postulated choice of using a logarithmic distribution as a very reasonable prior. Finally, by visualising the activation maps and the learned receptive fields, we demonstrate that the GaussDerNets have very good explainability properties.

Place, publisher, year, edition, pages
Springer Nature, 2025
Keywords
Deep learning, Gaussian derivative, Receptive fields, Scale covariance, Scale generalisation, Scale invariance, Scale selection, Scale space
National Category
Computer graphics and computer vision
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-363792 (URN)10.1007/s10851-025-01245-x (DOI)001488261900001 ()2-s2.0-105004740771 (Scopus ID)
Projects
Covariant and invariant deep networks
Funder
Swedish Research Council, 2018-03586, 2022-02969
Note

QC 20250523

Ej dubblett

Available from: 2025-05-21 Created: 2025-05-21 Last updated: 2025-06-03Bibliographically approved
Lindeberg, T. (2024). Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators.
Open this publication in new window or tab >>Approximation properties relative to continuous scale space for hybrid discretizations of Gaussian derivative operators
2024 (English)Report (Other academic)
Abstract [en]

This paper presents an analysis of properties of two hybrid discretization methods for Gaussian derivatives, based on convolutions with either the normalized sampled Gaussian kernel or the integrated Gaussian kernel followed by central differences. The motivation for studying these discretization methods is that in situations when multiple spatial derivatives of different orders are needed at the same scale level, they can be computed significantly more efficiently compared to more direct derivative approximations based on explicit convolutions with either sampled Gaussian derivative kernels or integrated Gaussian derivative kernels.  

While these computational benefits do also hold for the genuinely discrete approach for computing discrete analogues of Gaussian derivatives, based on convolution with the discrete analogue of the Gaussian kernel followed by central differences, the underlying mathematical primitives for the discrete analogue of the Gaussian kernel, in terms of modified Bessel functions of integer order, may not be available in certain frameworks for image processing, such as when performing deep learning based on scale-parameterized filters in terms of Gaussian derivatives, with learning of the scale levels. The hybrid discretizations studied in this paper do, from this perspective, offer a computationally more efficient way of implementing deep networks based on Gaussian derivatives for such use cases.  

In this paper, we present a characterization of the properties of these hybrid discretization methods, in terms of quantitative performance measures concerning the amount of spatial smoothing that they imply, as well as the relative consistency of scale estimates obtained from scale-invariant feature detectors with automatic scale selection, with an emphasis on the behaviour for very small values of the scale parameter, which may differ significantly from corresponding results obtained from the fully continuous scale-space theory, as well as between different types of discretization methods.

The presented results are intended as a guide, when designing as well as interpreting the experimental results of scale-space algorithms that operate at very fine scale levels.

Publisher
p. 23
Keywords
scale, discrete, continuous, smoothing, Gaussian kernel, Gaussian derivative, scale space
National Category
Computer graphics and computer vision Computational Mathematics
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-346267 (URN)
Projects
Covariant and invariant deep networks
Funder
Swedish Research Council, 2022-02969
Note

QC 20240514

Available from: 2024-05-10 Created: 2024-05-10 Last updated: 2025-03-24Bibliographically approved
Pedersen, J., Conradt, J. & Lindeberg, T. (2024). Covariant spatio-temporal receptive fields for neuromorphic computing.
Open this publication in new window or tab >>Covariant spatio-temporal receptive fields for neuromorphic computing
2024 (English)Report (Other academic)
Abstract [en]

Biological nervous systems constitute important sources of inspiration towards computers that are faster, cheaper, and more energy efficient. Neuromorphic disciplines view the brain as a coevolved system, simultaneously optimizing the hardware and the algorithms running on it. There are clear efficiency gains when bringing the computations into a physical substrate, but we presently lack theories to guide efficient implementations. Here, we present a principled computational model for neuromorphic systems in terms of spatio-temporal receptive fields, based on affine Gaussian kernels over space and leaky-integrator and leaky integrate-and-fire models over time. Our theory is provably covariant to spatial affine and temporal scaling transformations, and with close similarities to the visual processing in mammalian brains. We use these spatio-temporal receptive fields as a prior in an event-based vision task, and show that this improves the training of spiking networks, which otherwise is known as problematic for event-based vision. This work combines efforts within scale-space theory and computational neuroscience to identify theoretically well-founded ways to process spatio-temporal signals in neuromorphic systems. Our contributions are immediately relevant for signal processing and event-based vision, and can be extended to other processing tasks over space and time, such as memory and control.

Keywords
Scale-space theory, Neuromorphic computing, Computer vision
National Category
Computer graphics and computer vision
Research subject
Applied and Computational Mathematics
Identifiers
urn:nbn:se:kth:diva-346247 (URN)
Funder
EU, Horizon 2020, 785907Swedish Research Council, 2022-02969EU, Horizon 2020, 945539Swedish Research Council, 2022-0672
Note

See DOI 10.48550/arXiv.2405.00318 

QC 20240508

Available from: 2024-05-08 Created: 2024-05-08 Last updated: 2025-06-12Bibliographically approved
Lindeberg, T. (2024). Do the receptive fields in the primary visual cortex span a variability over the degree of elongation of the receptive fields?.
Open this publication in new window or tab >>Do the receptive fields in the primary visual cortex span a variability over the degree of elongation of the receptive fields?
2024 (English)Report (Other academic)
Abstract [en]

This paper presents the results of combining (i) theoretical analysis regarding connections between the orientation selectivity and the elongation of receptive fields for the affine Gaussian derivative model with (ii) biological measurements of orientation selectivity in the primary visual cortex to investigate if (iii) the receptive fields can be regarded as spanning a variability in the degree of elongation.

From an in-depth theoretical analysis of idealized models for the receptive fields of simple and complex cells in the primary visual cortex, we established that the orientation selectivity becomes more narrow with increasing elongation of the receptivefields. Combined with previously established biological results, concerning broad vs. sharp orientation tuning of visual neurons in the primary visual cortex, as well as previous experimental results concerning distributions of the resultant of the orientation selectivity curves for simple and complex cells, we show that these results are consistent with the receptive fields spanning a variability over the degree of elongation of the receptive fields. We also show that our principled theoretical model for visual receptive fields leads to qualitatively similar types of deviations from a uniform histogram of the resultant descriptor of the orientation selectivity curves for simple cells, as can be observed in the results from biological experiments.

To firmly determine if the underlying working hypothesis, regarding the receptive fields spanning a variability in the degree of elongation, would truly hold for the receptive fields in the primary visual cortex of higher mammals, we formulate a set of testable predictions, that can be used for investigate this property experimentally, and, if applicable, then also characterize if such a variability would, in a structured way, be related to the pinwheel structure in the visual cortex.

Publisher
p. 22
Keywords
Receptive field, Elongation, Affine covariance, Orientation selectivity, Gaussian derivative, Quasi quadrature, Simple cell, Complex cell, Pinwheel, Vision, Theoretical neuroscience
National Category
Bioinformatics (Computational Biology)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-345179 (URN)
Funder
Swedish Research Council, 2022-02969
Note

QC 20240410

Available from: 2024-04-09 Created: 2024-04-09 Last updated: 2025-03-25Bibliographically approved
Lindeberg, T. (2024). Joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields.
Open this publication in new window or tab >>Joint covariance properties under geometric image transformations for spatio-temporal receptive fields according to the generalized Gaussian derivative model for visual receptive fields
2024 (English)Report (Other academic)
Abstract [en]

The influence of natural image transformations on receptive field responses is crucial for modelling visual operations in computer vision and biological vision. In this regard, covariance properties with respect to geometric image transformations in the earliest layers of the visual hierarchy are essential for expressing robust image operations, and for formulating invariant visual operations at higher levels.

This paper defines and proves a set of joint covariance properties under compositions of spatial scaling transformations, spatial affine transformations, Galilean transformations and temporal scaling transformations, which make it possible to characterize how different types of image transformations interact with each other and the associated spatio-temporal receptive field responses. In this regard, we also extend the notion of scale-normalized derivatives to affine-normalized derivatives, to be able to obtain true affine-covariant properties of spatial derivatives, that are computed based on spatial smoothing with affine Gaussian kernels.

The derived relations show how the parameters of the receptive fields need to be transformed, in order to match the output from spatio-temporal receptive fields under composed spatio-temporal image transformations. As a side effect, the presented proof for the joint covariance property over the integrated combination of the different geometric image transformations also provides specific proofs for the individual transformation properties, which have not previously been fully reported in the literature.

We conclude with a geometric analysis, showing how the derived joint covariance properties make it possible to relate or match spatio-temporal receptive field responses, when observing, possibly moving, local surface patches from different views, under locally linearized perspective or projective transformations, as well as when observing different instances of spatio-temporal events that may occur either faster or slower between different views of similar spatio-temporal events. We do furthermore describe how the parameters in the studied composed spatio-temporal image transformation models directly relate to geometric entities in the image formation process and the 3-D scene structure.

In relation to these geometric interpretations, the derived explicit transformation properties for receptive field responses, defined in terms of spatio-temporal derivatives of the underlying covariant spatio-temporal smoothing kernels, do specifically show how to both interpret and relate spatio-temporal receptive field responses, when viewing dynamic scenes under different composed geometric viewing conditions.

Specifically, we propose that this theoretical analysis should have direct relevance, when interpreting the functional properties of biological receptive fields, both computationally and with regard to how the simple cells in the primary visual cortex, whose functional properties we here model with an idealized axiomatically derived spatio-temporal receptive field model. From the viewpoint of the here presented theory, in combination with previous biological modelling results that demonstrate a very good qualitative agreement between idealized receptive field models according to this theory and neurophysiological recordings of actual biological receptive fields in the primary visual cortex of higher mammals, the shapes of these joint spatio-temporal receptive fields can, from this viewpoint, be regarded as very well adapted to the structural properties of the environment.

Publisher
p. 38
Keywords
covariance, receptive field, scaling, affine, Galilean, spatial, temporal, spatio-temporal, image transformations, geometry, vision
National Category
Computer graphics and computer vision Bioinformatics (Computational Biology)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-346012 (URN)
Projects
Covariant and invariant deep networks
Funder
Swedish Research Council, 2018-03586, 2022-02969
Note

QC 20240429

Available from: 2024-04-29 Created: 2024-04-29 Last updated: 2025-02-01Bibliographically approved
Lindeberg, T. (2024). Orientation selectivity properties for the affine Gaussian derivative and the affine Gabor models for visual receptive fields.
Open this publication in new window or tab >>Orientation selectivity properties for the affine Gaussian derivative and the affine Gabor models for visual receptive fields
2024 (English)Report (Other academic)
Abstract [en]

This paper presents an in-depth theoretical analysis of the orientation selectivity properties of simple cells and complex cells, that can be well modelled by the generalized Gaussian derivative model for visual receptive fields, with the purely spatial component of the receptive fields determined by oriented affine Gaussian derivatives for different orders of spatial differentiation.

A detailed mathematical analysis is presented for the three different cases of either: (i) purely spatial receptive fields, (ii) space-time separable spatio-temporal receptive fields and (iii) velocity-adapted spatio-temporal receptive fields. Closed-form theoretical expressions for the orientation selectivity curves for idealized models of simple and complex cells are derived for all these main cases, and it is shown that the  orientation selectivity of the receptive fields becomes more narrow, as a scale parameter ratio $\kappa$, defined as the ratio between the scale parameters in the directions perpendicular to vs. parallel with the preferred orientation of the receptive field, increases. It is also shown that the orientation selectivity becomes more narrow with increasing order of spatial differentiation in the underlying affine Gaussian derivative operators over the spatial domain.

Additionally, we also derive closed-form expressions for the resultant and the bandwidth descriptors of the orientation selectivity curves, which have previously been used as compact descriptors of the orientation selectivity properties for biological neurons. These results together show that the properties of the affine Gaussian derivative model for visual receptive fields can be analyzed in closed form, which can be highly useful when to relate the results from biological experiments to computational models of the functional properties of simple cells and complex cells in the primary visual cortex.

For comparison, we also present a corresponding theoretical orientation selectivity analysis for purely spatial receptive fields according to an affine Gabor model. The results from that analysis are consistent with the results obtained from the affine Gaussian derivative model, in the respect that the orientation selectivity becomes more narrow when making the receptive fields wider in the direction perpendicular to the preferred orientation of the receptive field.

The affine Gabor model does, however, comprise one more degree of freedom in its parameter space, compared to the affine Gaussian derivative model, where a variability within that additional dimension of the parameter space does also strongly influence the orientation selectivity of the receptive fields. In this respect, the relationship between the orientation selectivity properties and the degree of elongation of the receptive fields is more direct for the affine Gaussian derivative model than for the affine Gabor model.

Publisher
p. 35
Keywords
Receptive field, Orientation selectivity, Affine covariance, Gaussian derivative, Quasi quadrature, Simple cell, Complex cell, Vision, Theoretical neuroscience
National Category
Bioinformatics (Computational Biology)
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-345184 (URN)
Funder
Swedish Research Council, 2022-02969
Note

QC 20240410

Available from: 2024-04-09 Created: 2024-04-09 Last updated: 2024-12-18Bibliographically approved
Organisations
Identifiers
ORCID iD: ORCID iD iconorcid.org/0000-0002-9081-2170

Search in DiVA

Show all publications