On the geometry of rectifier convolutional neural networks
2019 (English)In: Proceedings - 2019 International Conference on Computer Vision Workshop, ICCVW 2019, Institute of Electrical and Electronics Engineers Inc. , 2019, p. 793-797Conference paper, Published paper (Refereed)
Abstract [en]
While recent studies have shed light on the expressivity, complexity and compositionality of convolutional networks, the real inductive bias of the family of functions reachable by gradient descent on natural data is still unknown. By exploiting symmetries in the preactivation space of convolutional layers, we present preliminary empirical evidence of regularities in the preimage of trained rectifier networks, in terms of arrangements of polytopes, and relate it to the nonlinear transformations applied by the network to its input.
Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers Inc. , 2019. p. 793-797
Keywords [en]
Convolutional networks, Deep learning, Heometry, Preimage, Understanding, Computer vision, Convolution, Gradient methods, Rectifying circuits, Compositionality, Gradient descent, Inductive bias, Non-linear transformations, Pre images, Convolutional neural networks
National Category
Robotics and automation
Identifiers
URN: urn:nbn:se:kth:diva-274163DOI: 10.1109/ICCVW.2019.00106ISI: 000554591600099Scopus ID: 2-s2.0-85082492932OAI: oai:DiVA.org:kth-274163DiVA, id: diva2:1444972
Conference
17th IEEE/CVF International Conference on Computer Vision Workshop, ICCVW 2019, 27 October 2019 through 28 October 2019
Note
QC 20200622
Part of ISBN 9781728150239
2020-06-222020-06-222025-02-09Bibliographically approved