kth.sePublications KTH
Change search
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf
MSMT-LCL: Multiscale Spatial-Spectral Masked Transformer With Local Contrastive Learning for Hyperspectral Image Classification
East China Jiaotong Univ, Sch Informat & Software Engn, Nanchang 330013, Peoples R China..
East China Jiaotong Univ, Sch Informat & Software Engn, Nanchang 330013, Peoples R China..
Guangzhou Univ, Sch Elect & Commun Engn, Guangzhou 511370, Peoples R China..ORCID iD: 0000-0003-2458-6774
Hubei Univ, Hubei Key Lab Appl Math, Key Lab Intelligent Sensing Syst & Secur, Minist Educ, Wuhan 430062, Peoples R China.;Hubei Univ, Fac Math & Stat, Wuhan 430062, Peoples R China..ORCID iD: 0000-0002-4759-0584
Show others and affiliations
2024 (English)In: IEEE Transactions on Geoscience and Remote Sensing, ISSN 0196-2892, E-ISSN 1558-0644, Vol. 62, article id 5534316Article in journal (Refereed) Published
Abstract [en]

Deep learning plays a crucial role in hyperspectral image (HSI) classification, with the Transformer being highly favored by researchers due to its exceptional ability to model long-range dependencies. However, the Transformer necessitates a substantial amount of labeled training samples to train its numerous parameters, exacerbating the challenge of training an effective HSI classification Transformer model, particularly given the inherent scarcity of HSI data. Therefore, we propose a novel method for HSI classification, termed multiscale spatial-spectral masked Transformer with local contrastive learning (MSMT-LCL). This method consists of two stages: self-supervised pretraining and supervised fine-tuning. Initially, we utilize the multiscale augmented feature mapping module (MAFM) to project original HSI data into two mixed-scale feature maps, which are then separately fed into two masked Transformer branches for reconstruction. To facilitate the model in learning the dependency relationships between central pixel land-cover information and neighboring land cover, we introduce a novel mask strategy based on center-patch. Furthermore, in the pretraining stage, we integrate local contrastive learning (LCL) to enable the model to focus on local center information at varying scales. Upon completion of pretraining, the network undergoes fine-tuning to obtain feature maps at two different scales. Subsequently, we devise a novel adaptive multiscale feature fusion module (AMFM) to adaptively aggregate these two features and produce the final classification results. Extensive experiments on three real datasets demonstrate the superiority of our proposed MSMT-LCL method over several state-of-the-art HSI classification methods.

Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2024. Vol. 62, article id 5534316
Keywords [en]
Contrastive learning (CL), deep learning, hyperspectral image (HSI) classification, masked Transformer
National Category
Computer graphics and computer vision
Identifiers
URN: urn:nbn:se:kth:diva-357240DOI: 10.1109/TGRS.2024.3472066ISI: 001338406700037Scopus ID: 2-s2.0-85206296225OAI: oai:DiVA.org:kth-357240DiVA, id: diva2:1921366
Note

QC 20241216

Available from: 2024-12-16 Created: 2024-12-16 Last updated: 2025-02-07Bibliographically approved

Open Access in DiVA

No full text in DiVA

Other links

Publisher's full textScopus

Authority records

Ban, Yifang

Search in DiVA

By author/editor
Yang, XiaofeiPeng, JiangtaoBan, YifangJiang, Nan
By organisation
Geoinformatics
In the same journal
IEEE Transactions on Geoscience and Remote Sensing
Computer graphics and computer vision

Search outside of DiVA

GoogleGoogle Scholar

doi
urn-nbn

Altmetric score

doi
urn-nbn
Total: 83 hits
CiteExportLink to record
Permanent link

Direct link
Cite
Citation style
  • apa
  • ieee
  • modern-language-association-8th-edition
  • vancouver
  • Other style
More styles
Language
  • de-DE
  • en-GB
  • en-US
  • fi-FI
  • nn-NO
  • nn-NB
  • sv-SE
  • Other locale
More languages
Output format
  • html
  • text
  • asciidoc
  • rtf