DCTN: Dual-Branch Convolutional Transformer Network With Efficient Interactive Self-Attention for Hyperspectral Image ClassificationShow others and affiliations
2024 (English)In: IEEE Transactions on Geoscience and Remote Sensing, ISSN 0196-2892, E-ISSN 1558-0644, Vol. 62, p. 1-16, article id 5508616Article in journal (Refereed) Published
Abstract [en]
Hyperspectral image (HSI) classification is an essential task in remote sensing with substantial practical significance. However, most existing convolutional neural network (CNN)-based classification methods focus only on local spatial features while neglecting global spectral dependencies. Meanwhile, Transformer-based methods exhibit robust capabilities for global spectral feature modeling but struggle to extract local spatial features effectively. To fully exploit the local spatial feature extraction capabilities of CNN-based networks and the global spectral feature extraction capabilities of Transformer-based networks, this article proposes a dual-branch convolutional Transformer method with efficient interactive self-attention (EISA) for HSI classification, namely the dual-branch convolutional transformer network (DCTN), which can aggregate local and global spatial-spectral features fully. Specifically, DCTN includes two core modules: the spatial-spectral fusion projection module (SFPM) and the EISA module. The former utilizes 3-D convolution with adaptive pooling and 2-D group convolution with residual connection to parallel extract fused and grouped spatial-spectral features, respectively. The latter performs EISA across height, width, and spectral dimensions, enabling deep fusion of spatial-spectral features. Extensive experiments on three real HSI datasets demonstrate that the proposed DCTN method outperforms existing classification methods, yielding state-of-the-art classification performance. The code is available at https://github.com/AllFever/DeepHyperX-DCTN for reproducibility.
Place, publisher, year, edition, pages
Institute of Electrical and Electronics Engineers (IEEE) , 2024. Vol. 62, p. 1-16, article id 5508616
Keywords [en]
Convolution neural networks (CNNs), hyperspectral image (HIS) classification, self-attention mechanism, transformer
National Category
Computer graphics and computer vision
Identifiers
URN: urn:nbn:se:kth:diva-343990DOI: 10.1109/TGRS.2024.3364143ISI: 001164472500043Scopus ID: 2-s2.0-85185545267OAI: oai:DiVA.org:kth-343990DiVA, id: diva2:1841360
Note
QC 20240304
2024-02-282024-02-282025-02-07Bibliographically approved