Multi-interactive Feature Learning and a Full-time Multi-modality Benchmark for Image Fusion and Segmentation

2021 IEEE/CVF International Conference on Computer Vision (ICCV), Год журнала: 2023, Номер unknown, С. 8081 - 8090

Опубликована: Окт. 1, 2023

Multi-modality image fusion and segmentation play a vital role in autonomous driving robotic operation. Early efforts focus on boosting the performance for only one task, e.g., or segmentation, making it hard to reach 'Best of Both Worlds'. To overcome this issue, paper, we propose Multi-interactive Feature learning architecture Segmentation, namely SegMiF, exploit dual-task correlation promote both tasks. The SegMiF is cascade structure, containing sub-network commonly used sub-network. By slickly bridging intermediate features between two components, knowledge learned from task can effectively assist task. Also, benefited network supports perform more pretentiously. Besides, hierarchical interactive attention block established ensure fine-grained mapping all information tasks, so that modality/semantic be fully mutual-interactive. In addition, dynamic weight factor introduced automatically adjust corresponding weights each which balance feature correspondence break through limitation laborious tuning. Furthermore, construct smart multi-wave binocular imaging system collect full-time multi-modality benchmark with 15 annotated pixel-level categories segmentation. Extensive experiments several public datasets our demonstrate proposed method outputs visually appealing fused images averagely 7.66% higher mIoU real-world scene than state-of-the-art approaches. source code are available at https://github.com/JinyuanLiu-CV/SegMiF.

Язык: Английский

---

CrossFuse: A novel cross attention mechanism based infrared and visible image fusion approach

Information Fusion, Год журнала: 2023, Номер 103, С. 102147 - 102147

Опубликована: Ноя. 15, 2023

Язык: Английский

---

CoCoNet: Coupled Contrastive Learning Network with Multi-level Feature Ensemble for Multi-modality Image Fusion

International Journal of Computer Vision, Год журнала: 2023, Номер 132(5), С. 1748 - 1775

Опубликована: Дек. 8, 2023

Язык: Английский

---

Diff-IF: Multi-modality image fusion via diffusion model with fusion knowledge prior

Information Fusion, Год журнала: 2024, Номер 110, С. 102450 - 102450

Опубликована: Май 3, 2024

Язык: Английский

---

Equivariant Multi-Modality Image Fusion

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Год журнала: 2024, Номер abs/2004.10934, С. 25912 - 25921

Опубликована: Июнь 16, 2024

---

A Task-guided, Implicitly-searched and Metainitialized Deep Model for Image Fusion

IEEE Transactions on Pattern Analysis and Machine Intelligence, Год журнала: 2024, Номер 46(10), С. 6594 - 6609

Опубликована: Март 27, 2024

Image fusion plays a key role in variety of multi-sensor-based vision systems, especially for enhancing visual quality and/or extracting aggregated features perception. However, most existing methods just consider image as an individual task, thus ignoring its underlying relationship with these downstream problems. Furthermore, designing proper architectures often requires huge engineering labor. It also lacks mechanisms to improve the flexibility and generalization ability current approaches. To mitigate issues, we establish Task-guided, Implicit-searched Meta-initialized (TIM) deep model address problem challenging real-world scenario. Specifically, first propose constrained strategy incorporate information from tasks guide unsupervised learning process fusion. Within this framework, then design implicit search scheme automatically discover compact our high efficiency. In addition, pretext meta initialization technique is introduced leverage divergence data support fast adaptation different kinds tasks. Qualitative quantitative experimental results on categories problems related (e.g., enhancement semantic understanding) substantiate effectiveness TIM.

Язык: Английский

---

KDFuse: A high-level vision task-driven infrared and visible image fusion method based on cross-domain knowledge distillation

Information Fusion, Год журнала: 2025, Номер unknown, С. 102944 - 102944

Опубликована: Янв. 1, 2025

Язык: Английский

---

Combining transformers with CNN for multi-focus image fusion

Expert Systems with Applications, Год журнала: 2023, Номер 235, С. 121156 - 121156

Опубликована: Авг. 11, 2023

Язык: Английский

---

Bi-level Dynamic Learning for Jointly Multi-modality Image Fusion and Beyond

Опубликована: Авг. 1, 2023

Recently, multi-modality scene perception tasks, e.g., image fusion and understanding, have attracted widespread attention for intelligent vision systems. However, early efforts always consider boosting a single task unilaterally neglecting others, seldom investigating their underlying connections joint promotion. To overcome these limitations, we establish the hierarchical dual tasks-driven deep model to bridge tasks. Concretely, firstly construct an module fuse complementary characteristics cascade task-related modules, including discriminator visual effects semantic network feature measurement. We provide bi-level perspective formulate follow-up downstream incorporate distinct responses fusion, as primary goal modules learnable constraints. Furthermore, develop efficient first-order approximation compute corresponding gradients present dynamic weighted aggregation balance learning. Extensive experiments demonstrate superiority of our method, which not only produces visually pleasant fused results but also realizes significant promotion detection segmentation than state-of-the-art approaches.

Язык: Английский

---

Text-IF: Leveraging Semantic Text Guidance for Degradation-Aware and Interactive Image Fusion

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Год журнала: 2024, Номер unknown, С. 27016 - 27025

Опубликована: Июнь 16, 2024

Язык: Английский

---