SpectralFormer: Rethinking Hyperspectral Image Classification With Transformers

IEEE Transactions on Geoscience and Remote Sensing, Journal Year: 2021, Volume and Issue: 60, P. 1 - 15

Published: Nov. 25, 2021

Hyperspectral (HS) images are characterized by approximately contiguous spectral information, enabling the fine identification of materials capturing subtle discrepancies. Owing to their excellent locally contextual modeling ability, convolutional neural networks (CNNs) have been proven be a powerful feature extractor in HS image classification. However, CNNs fail mine and represent sequence attributes signatures well due limitations inherent network backbone. To solve this issue, we rethink classification from sequential perspective with transformers, propose novel backbone called \ul{SpectralFormer}. Beyond band-wise representations classic SpectralFormer is capable learning spectrally local information neighboring bands images, yielding group-wise embeddings. More significantly, reduce possibility losing valuable layer-wise propagation process, devise cross-layer skip connection convey memory-like components shallow deep layers adaptively fuse "soft" residuals across layers. It worth noting that proposed highly flexible network, which can applicable both pixel- patch-wise inputs. We evaluate performance on three datasets conducting extensive experiments, showing superiority over transformers achieving significant improvement comparison state-of-the-art networks. The codes work will available at https://github.com/danfenghong/IEEE_TGRS_SpectralFormer for sake reproducibility.

Language: Английский

---

Artificial Intelligence for Remote Sensing Data Analysis: A review of challenges and opportunities

IEEE Geoscience and Remote Sensing Magazine, Journal Year: 2022, Volume and Issue: 10(2), P. 270 - 294

Published: April 13, 2022

Artificial intelligence (AI) plays a growing role in remote sensing (RS). Applications of AI, particularly machine learning algorithms, range from initial image processing to high-level data understanding and knowledge discovery. AI techniques have emerged as powerful strategy for analyzing RS led remarkable breakthroughs all fields. Given this period breathtaking evolution, work aims provide comprehensive review the recent achievements algorithms applications analysis. The includes more than 270 research papers, covering following major aspects innovation RS: learning, computational intelligence, explicability, mining, natural language (NLP), security. We conclude by identifying promising directions future research.

Language: Английский

---

Deep learning in multimodal remote sensing data fusion: A comprehensive review

International Journal of Applied Earth Observation and Geoinformation, Journal Year: 2022, Volume and Issue: 112, P. 102926 - 102926

Published: July 26, 2022

With the extremely rapid advances in remote sensing (RS) technology, a great quantity of Earth observation (EO) data featuring considerable and complicated heterogeneity are readily available nowadays, which renders researchers an opportunity to tackle current geoscience applications fresh way. joint utilization EO data, much research on multimodal RS fusion has made tremendous progress recent years, yet these developed traditional algorithms inevitably meet performance bottleneck due lack ability comprehensively analyze interpret strongly heterogeneous data. Hence, this non-negligible limitation further arouses intense demand for alternative tool with powerful processing competence. Deep learning (DL), as cutting-edge witnessed remarkable breakthroughs numerous computer vision tasks owing its impressive representation reconstruction. Naturally, it been successfully applied field fusion, yielding improvement compared methods. This survey aims present systematic overview DL-based fusion. More specifically, some essential knowledge about topic is first given. Subsequently, literature conducted trends field. Some prevalent sub-fields then reviewed terms to-be-fused modalities, i.e., spatiospectral, spatiotemporal, light detection ranging-optical, synthetic aperture radar-optical, RS-Geospatial Big Data Furthermore, We collect summarize valuable resources sake development Finally, remaining challenges potential future directions highlighted.

Language: Английский

---

Hyperspectral Image Classification—Traditional to Deep Models: A Survey for Future Prospects

IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, Journal Year: 2021, Volume and Issue: 15, P. 968 - 999

Published: Dec. 9, 2021

Hyperspectral imaging (HSI) has been extensively utilized in many real-life applications because it benefits from the detailed spectral information contained each pixel. Notably, complex characteristics, i.e., nonlinear relation among captured and corresponding object of HSI data, make accurate classification challenging for traditional methods. In last few years, deep learning (DL) substantiated as a powerful feature extractor that effectively addresses problems appeared number computer vision tasks. This prompts deployment DL (HSIC) which revealed good performance. survey enlists systematic overview HSIC compared state-of-the-art strategies said topic. Primarily, we will encapsulate main challenges TML then acquaint superiority to address these problems. article breaks down frameworks into spectral-features, spatial-features, together spatial–spectral features systematically analyze achievements (future research directions well) HSIC. Moreover, consider fact requires large labeled training examples whereas acquiring such is terms time cost. Therefore, this discusses some improve generalization performance can provide future guidelines.

Language: Английский

---

Multimodal Fusion Transformer for Remote Sensing Image Classification

IEEE Transactions on Geoscience and Remote Sensing, Journal Year: 2023, Volume and Issue: 61, P. 1 - 20

Published: Jan. 1, 2023

Vision transformers (ViTs) have been trending in image classification tasks due to their promising performance when compared convolutional neural networks (CNNs). As a result, many researchers tried incorporate ViTs hyperspectral (HSI) tasks. To achieve satisfactory performance, close that of CNNs, need fewer parameters. and other similar use an external (CLS) token which is randomly initialized often fails generalize well, whereas sources multimodal datasets, such as light detection ranging (LiDAR) offer the potential improve these models by means CLS. In this paper, we introduce new fusion transformer (MFT) network comprises multihead cross patch attention (mCrossPA) for HSI land-cover classification. Our mCrossPA utilizes complementary information addition encoder better generalization. The concept tokenization used generate CLS tokens, helping learn {distinctive representation} reduced hierarchical feature space. Extensive experiments are carried out on {widely benchmark} datasets {i.e.,} University Houston, Trento, Southern Mississippi Gulfpark (MUUFL), Augsburg. We compare results proposed MFT model with state-of-the-art transformers, classical conventional classifiers models. superior achieved attention. source code will be made available publicly at \url{https://github.com/AnkurDeria/MFT}.}

Language: Английский

---

Convolutional Neural Networks for Multimodal Remote Sensing Data Classification

IEEE Transactions on Geoscience and Remote Sensing, Journal Year: 2021, Volume and Issue: 60, P. 1 - 10

Published: Nov. 2, 2021

In recent years, enormous research has been made to improve the classification performance of single-modal remote sensing (RS) data. However, with ever-growing availability RS data acquired from satellite or airborne platforms, simultaneous processing and analysis multimodal pose a new challenge researchers in community. To this end, we propose deep-learning-based framework for classification, where convolutional neural networks (CNNs) are taken as backbone an advanced cross-channel reconstruction module, called CCR-Net. As name suggests, CCR-Net learns more compact fusion representations different sources by means strategy across modalities that can mutually exchange information effective way. Extensive experiments conducted on two datasets, including hyperspectral (HS) light detection ranging (LiDAR) data, i.e., Houston2013 dataset, HS synthetic aperture radar (SAR) Berlin demonstrate effectiveness superiority proposed comparison several state-of-the-art methods. The codes will be openly freely available at https://github.com/danfenghong/IEEE_TGRS_CCR-Net sake reproducibility.

Language: Английский

---

Multi-feature fusion: Graph neural network and CNN combining for hyperspectral image classification

Neurocomputing, Journal Year: 2022, Volume and Issue: 501, P. 246 - 257

Published: June 9, 2022

Language: Английский

---

Endmember-Guided Unmixing Network (EGU-Net): A General Deep Learning Framework for Self-Supervised Hyperspectral Unmixing

IEEE Transactions on Neural Networks and Learning Systems, Journal Year: 2021, Volume and Issue: 33(11), P. 6518 - 6531

Published: May 28, 2021

Over the past decades, enormous efforts have been made to improve performance of linear or nonlinear mixing models for hyperspectral unmixing (HU), yet their ability simultaneously generalize various spectral variabilities (SVs) and extract physically meaningful endmembers still remains limited due poor in data fitting reconstruction sensitivity SVs. Inspired by powerful learning deep (DL), we attempt develop a general DL approach HU, fully considering properties extracted from imagery, called endmember-guided network (EGU-Net). Beyond alone autoencoder-like architecture, EGU-Net is two-stream Siamese network, which learns an additional pure nearly correct weights another sharing parameters adding spectrally constraints (e.g., nonnegativity sum-to-one) toward more accurate interpretable solution. Furthermore, resulting framework not only pixelwise but also applicable spatial information modeling with convolutional operators spatial-spectral unmixing. Experimental results conducted on three different datasets ground truth abundance maps corresponding each material demonstrate effectiveness superiority over state-of-the-art algorithms. The codes will be available website: https://github.com/danfenghong/IEEE_TNNLS_EGU-Net.

Language: Английский

---

Semi-Supervised Locality Preserving Dense Graph Neural Network With ARMA Filters and Context-Aware Learning for Hyperspectral Image Classification

IEEE Transactions on Geoscience and Remote Sensing, Journal Year: 2021, Volume and Issue: 60, P. 1 - 12

Published: Aug. 4, 2021

The application of graph convolutional networks (GCNs) to hyperspectral image (HSI) classification is a heavily researched topic. However, GCNs are based on spectral filters, which computationally costly and fail suppress noise effectively. In addition, the current GCN-based methods prone oversmoothing (the representation each node tends be congruent) problems. To circumvent these problems, novel semi-supervised locality-preserving dense neural network (GNN) with autoregressive moving average (ARMA) filters context-aware learning (DARMA-CAL) proposed for HSI classification. this work, we introduce ARMA filter instead apply GNNs. can better capture global structure more robust noise. More importantly, simplify calculations compared filter. show that approximated by recursive method. Furthermore, propose structure, not only implements in but also locality-preserving. Finally, design layerwise mechanism extract useful local information generated layer network. experimental results three real datasets DARMA-CAL outperforms state-of-the-art methods.

Language: Английский

---

Rotation-Invariant Attention Network for Hyperspectral Image Classification

IEEE Transactions on Image Processing, Journal Year: 2022, Volume and Issue: 31, P. 4251 - 4265

Published: Jan. 1, 2022

Hyperspectral image (HSI) classification refers to identifying land-cover categories of pixels based on spectral signatures and spatial information HSIs. In recent deep learning-based methods, explore the HSIs, HSI patch is usually cropped from original as input. And 3×3 convolution utilized a key component capture features for classification. However, sensitive rotation inputs, which results in that methods perform worse rotated To alleviate this problem, rotation-invariant attention network (RIAN) proposed First, center (CSpeA) module designed avoid influence other suppress redundant bands. Then, rectified (RSpaA) replace extracting spectral-spatial patches. The CSpeA module, 1×1 RSpaA are build RIAN Experimental demonstrate invariant HSIs has superior performance, e.g., achieving an overall accuracy 86.53% (1.04% improvement) Houston database.

Language: Английский

---