kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
Generalized Jensen-Shannon Divergence Loss for Learning with Noisy Labels
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.
KTH, School of Electrical Engineering and Computer Science (EECS), Intelligent systems, Robotics, Perception and Learning, RPL.ORCID iD: 0000-0001-5211-6388
2021 (English)In: Proceedings 35th Conference on Neural Information Processing Systems (NeurIPS 2021)., NIPS , 2021Conference paper, Published paper (Refereed)
Abstract [en]

Prior works have found it beneficial to combine provably noise-robust loss functions e.g. mean absolute error (MAE) with standard categorical loss function e.g. crossentropy (CE) to improve their learnability. Here, we propose to use Jensen-Shannondivergence as a noise-robust loss function and show that it interestingly interpolatebetween CE and MAE with a controllable mixing parameter. Furthermore, wemake a crucial observation that CE exhibits lower consistency around noisy datapoints. Based on this observation, we adopt a generalized version of the Jensen-Shannon divergence for multiple distributions to encourage consistency arounddata points. Using this loss function, we show state-of-the-art results on bothsynthetic (CIFAR), and real-world (e.g. WebVision) noise with varying noise rates.

Place, publisher, year, edition, pages
NIPS , 2021.
National Category
Computer graphics and computer vision
Identifiers
URN: urn:nbn:se:kth:diva-305931ISI: 000922928202022Scopus ID: 2-s2.0-85124784124OAI: oai:DiVA.org:kth-305931DiVA, id: diva2:1618481
Conference
35th Conference on Neural Information Processing Systems, NeurIPS 2021, Virtual, Online, Dec 14 2021 - Dec 6 2021
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Note

Part of proceedings ISBN 978-171384539-3 

QC 20211220

Available from: 2021-12-09 Created: 2021-12-09 Last updated: 2025-02-07Bibliographically approved
In thesis
1. On Label Noise in Image Classification: An Aleatoric Uncertainty Perspective
Open this publication in new window or tab >>On Label Noise in Image Classification: An Aleatoric Uncertainty Perspective
2024 (English)Doctoral thesis, comprehensive summary (Other academic)
Abstract [en]

Deep neural networks and large-scale datasets have revolutionized the field of machine learning. However, these large networks are susceptible to overfitting to label noise, resulting in generalization degradation. In response, the thesis closely examines the problem both from an empirical and theoretical perspective. We empirically analyse the input smoothness of networks as they overfit to label noise, and we theoretically explore the connection to aleatoric uncertainty. These analyses improve our understanding of the problem and have led to our novel methods aimed at enhancing robustness against label noise in classification.

Abstract [sv]

Djupa neurala nätverk och storskaliga dataset har revolutionerat maskininlärningsområdet. Dock är dessa stora nätverk känsliga för överanpassning till felmarkerade etiketter, vilket leder till försämrad generalisering. Som svar på detta undersöker avhandlingen noggrant problemet både från en empirisk och teoretisk synvinkel. Vi analyserar empiriskt nätverkens känslighet försmå ändringar i indatan när de över anpassar till felmarkerade etiketter, och vi utforskar teoretiskt kopplingen till aleatorisk osäkerhet. Dessa analyser förbättrar vår förståelse av problemet och har lett till våra nya metoder med syfte att vara robusta mot felmarkerade etiketter i klassificering.

Place, publisher, year, edition, pages
Stockholm: KTH Royal Institute of Technology, 2024. p. xi, 68
Series
TRITA-EECS-AVL ; 2024:45
Keywords
Label noise, aleatoric uncertainty, noisy labels, robustness, etikettbrus, osäkerhet, felmarkerade etiketter, robusthet
National Category
Computer Sciences
Research subject
Computer Science
Identifiers
urn:nbn:se:kth:diva-346453 (URN)978-91-8040-925-4 (ISBN)
Public defence
2024-06-03, https://kth-se.zoom.us/w/61097277235, F3 (Flodis), Lindstedsvägen 26 & 28, Stockholm, 09:00 (English)
Opponent
Supervisors
Funder
Wallenberg AI, Autonomous Systems and Software Program (WASP)
Note

QC 20240516

Available from: 2024-05-16 Created: 2024-05-16 Last updated: 2025-12-03Bibliographically approved

Open Access in DiVA

fulltext(1476 kB)253 downloads
File information
File name FULLTEXT01.pdfFile size 1476 kBChecksum SHA-512
a674c586fef6f66608cc131342008c3e40554e5084592652a47c3c0c20e69f4a1915cf06ba9e30fa73797792937ee423ff913b91e77319c44c3c10aacb1ba647
Type fulltextMimetype application/pdf
attachment(585 kB)115 downloads
File information
File name ATTACHMENT01.pdfFile size 585 kBChecksum SHA-512
ace62317a9ac498cd6103f2c413d1c9f7744ffc792bb2cc60566869770c8dc4506de86eccb39484a0409359bcc4644bf0a33ee3e54babc44fa552cecc9795758
Type attachmentMimetype application/pdf

Other links

ScopusConferencearXiv

Authority records

Englesson, ErikAzizpour, Hossein

Search in DiVA

By author/editor
Englesson, ErikAzizpour, Hossein
By organisation
Robotics, Perception and Learning, RPL
Computer graphics and computer vision

Search outside of DiVA

GoogleGoogle Scholar
Total: 253 downloads
The number of downloads is the sum of all downloads of full texts. It may include eg previous versions that are now no longer available

urn-nbn

Altmetric score

urn-nbn
Total: 431 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf