Vision-language models for medical report generation and visual question answering: a review

Frontiers in Artificial Intelligence, Journal Year: 2024, Volume and Issue: 7

Published: Nov. 19, 2024

Medical vision-language models (VLMs) combine computer vision (CV) and natural language processing (NLP) to analyze visual textual medical data. Our paper reviews recent advancements in developing VLMs specialized for healthcare, focusing on publicly available designed report generation question answering (VQA). We provide background NLP CV, explaining how techniques from both fields are integrated into VLMs, with data often fused using Transformer-based architectures enable effective learning multimodal Key areas we address include the exploration of 18 public datasets, in-depth analyses pre-training strategies 16 noteworthy comprehensive discussion evaluation metrics assessing VLMs' performance VQA. also highlight current challenges facing VLM development, including limited availability, concerns privacy, lack proper metrics, among others, while proposing future directions these obstacles. Overall, our review summarizes progress harness improved healthcare applications.

Language: Английский

---

Building Flexible, Scalable, and Machine Learning-Ready Multimodal Oncology Datasets

Sensors, Journal Year: 2024, Volume and Issue: 24(5), P. 1634 - 1634

Published: March 2, 2024

The advancements in data acquisition, storage, and processing techniques have resulted the rapid growth of heterogeneous medical data. Integrating radiological scans, histopathology images, molecular information with clinical is essential for developing a holistic understanding disease optimizing treatment. need integrating from multiple sources further pronounced complex diseases such as cancer enabling precision medicine personalized treatments. This work proposes Multimodal Integration Oncology Data System (MINDS)—a flexible, scalable, cost-effective metadata framework efficiently fusing disparate public Cancer Research Commons (CRDC) into an interconnected, patient-centric framework. MINDS consolidates over 41,000 cases across repositories while achieving high compression ratio relative to 3.78 PB source size. It offers sub-5-s query response times interactive exploration. interface exploring relationships types building cohorts large-scale multimodal machine learning models. By harmonizing data, aims potentially empower researchers greater analytical ability uncover diagnostic prognostic insights enable evidence-based care. tracks granular end-to-end provenance, ensuring reproducibility transparency. cloud-native architecture can handle exponential secure, cost-optimized manner substantial storage optimization, replication avoidance, dynamic access capabilities. Auto-scaling, controls, other mechanisms guarantee pipelines’ scalability security. overcomes limitations existing biomedical silos via interoperable metadata-driven approach that represents pivotal step toward future oncology integration.

Language: Английский

---

Utilizing Feature Selection Techniques for AI-Driven Tumor Subtype Classification: Enhancing Precision in Cancer Diagnostics

Biomolecules, Journal Year: 2025, Volume and Issue: 15(1), P. 81 - 81

Published: Jan. 8, 2025

Cancer's heterogeneity presents significant challenges in accurate diagnosis and effective treatment, including the complexity of identifying tumor subtypes their diverse biological behaviors. This review examines how feature selection techniques address these by improving interpretability performance machine learning (ML) models high-dimensional datasets. Feature methods-such as filter, wrapper, embedded techniques-play a critical role enhancing precision cancer diagnostics relevant biomarkers. The integration multi-omics data ML algorithms facilitates more comprehensive understanding heterogeneity, advancing both personalized therapies. However, such ensuring quality, mitigating overfitting, addressing scalability remain limitations methods. Artificial intelligence (AI)-powered offers promising solutions to issues automating refining extraction process. highlights transformative potential approaches while emphasizing future directions, incorporation deep (DL) integrative strategies for robust reproducible findings.

Language: Английский

---

Artificial Intelligence in Head and Neck Cancer: Innovations, Applications, and Future Directions

Current Oncology, Journal Year: 2024, Volume and Issue: 31(9), P. 5255 - 5290

Published: Sept. 6, 2024

Artificial intelligence (AI) is revolutionizing head and neck cancer (HNC) care by providing innovative tools that enhance diagnostic accuracy personalize treatment strategies. This review highlights the advancements in AI technologies, including deep learning natural language processing, their applications HNC. The integration of with imaging techniques, genomics, electronic health records explored, emphasizing its role early detection, biomarker discovery, planning. Despite noticeable progress, challenges such as data quality, algorithmic bias, need for interdisciplinary collaboration remain. Emerging innovations like explainable AI, AI-powered robotics, real-time monitoring systems are poised to further advance field. Addressing these fostering among experts, clinicians, researchers crucial developing equitable effective applications. future HNC holds significant promise, offering potential breakthroughs diagnostics, personalized therapies, improved patient outcomes.

Language: Английский

---

Raman Spectroscopy and AI Applications in Cancer Grading: An Overview

IEEE Access, Journal Year: 2024, Volume and Issue: 12, P. 54816 - 54852

Published: Jan. 1, 2024

Raman spectroscopy (RS) is a label-free molecular vibrational technique that able to identify the fingerprint of various samples making use inelastic scattering monochromatic light. Because its advantages non-destructive and accurate detection, RS finding more for benign malignant tissues, tumor differentiation, subtype classification, section pathology diagnosis, operating either in vivo or vitro . However, high specificity comes at cost. The acquisition rate low, depth information cannot be directly accessed, sampling area limited. Such limitations can contained if data pre- post-processing methods are combined with current Artificial Intelligence (AI), essentially, Machine Learning (ML) Deep (DL). latter modifying approach cancer diagnosis currently used automate many analyses, it has emerged as promising option improving healthcare accuracy patient outcomes by abiliting prediction diseases tools. In very broad context, applications in oncology include risk assessment, early prognosis estimation, treatment selection based on deep knowledge. application autonomous datasets generated analysis tissues could make rapid stand-alone help pathologists diagnose accuracy. This review describes milestones achieved applying AI-based algorithms analysis, grouped according seven major types cancers (Pancreatic, Breast, Skin, Brain, Prostate, Ovarian Oral cavity). Additionally, provides theoretical foundation tackle both present forthcoming challenges this domain. By exploring achievements discussing relative methodologies, offers recapitulative insights recent ongoing efforts position effective screening tool pathologists. Accordingly, we aim encourage future research endeavors facilitate realization full potential AI grading.

Language: Английский

---

Recent Advancement in Bioinformatics

Published: Jan. 13, 2025

Language: Английский

---

Machine learning in ocular oncology and oculoplasty: Transforming diagnosis and treatment

IP International Journal of Ocular Oncology and Oculoplasty, Journal Year: 2025, Volume and Issue: 10(4), P. 196 - 207

Published: Jan. 14, 2025

In the domains of ocular oncology and oculoplasty, machine learning (ML) has become a game-changing technology, providing previously unheard-of levels precision in diagnosis, treatment planning, outcome prediction. Using imaging modalities, genomic data, clinical characteristics, this chapter investigates integration algorithms detection tumours, including retinoblastoma uveal melanoma. Through predictive modelling real-time decision-making, it also emphasises how ML might improve surgical outcomes orbital reconstruction eyelid correction. Automated examination fundus photographs, histological slides, 3D been made possible by methods like deep natural language processing, which have improved individualised therapeutic approaches decreased diagnostic errors. Additionally, use augmented reality robotics surgery is significant development oculoplasty. Notwithstanding its potential, issues data heterogeneity, algorithm interpretability, ethical considerations are roadblocks that need to be addressed. This explores cutting-edge developments, real-world uses, potential future paths, offering researchers doctors thorough resource. Dipali Vikas Mane, Associate Professor, Shriram Shikshan Sanstha’s College Pharmacy, Paniv-413113

Language: Английский

---

Contrastive Learning for Omics-guided Whole-slide Visual Embedding Representation

bioRxiv (Cold Spring Harbor Laboratory), Journal Year: 2025, Volume and Issue: unknown

Published: Jan. 15, 2025

Abstract While computational pathology has transformed cancer diagnosis and prognosis prediction, existing methods remain limited in their ability to decipher the complex molecular characteristics within tumors. We present CLOVER (Contrastive Learning for Omics-guided whole-slide Visual Embedding Representation), a novel deep learning framework that leverages self-supervised contrastive integrate multi-omics data (genomics, epigenomics, transcriptomics) with slide representations, connecting morphological features of Using breast cohorts comprising diagnostic slides paired from 610 patients, we validated CLOVER’s excellence by demonstrating its generate effective slide-level representations consider states cancer. outperforms few-shot scenarios, particularly subtype classification clinical biomarker prediction tasks (ER, PR, HER2 status). Through comprehensive interpretability analysis, identified tumor microenvironment components revealed associated Our results demonstrate enables detailed characterization single suggesting potential utilization future studies.

Language: Английский

---

Current status and future prospects of molecular imaging in targeting the tumor immune microenvironment

Frontiers in Immunology, Journal Year: 2025, Volume and Issue: 16

Published: Jan. 22, 2025

Molecular imaging technologies have significantly transformed cancer research and clinical practice, offering valuable tools for visualizing understanding the complex tumor immune microenvironment. These allow non-invasive examination of key components within microenvironment, including cells, cytokines, stromal providing crucial insights into biology treatment responses. This paper reviews latest advancements in molecular imaging, with a focus on its applications assessing interactions Additionally, challenges faced by are discussed, such as need highly sensitive specific agents, issues data integration, difficulties translation. The future outlook emphasizes potential to enhance personalized through integration artificial intelligence development novel probes. Addressing these is essential fully realizing improving diagnosis, treatment, patient outcomes.

Language: Английский

---

A Survey on Multimodal Large Language Models in Radiology for Report Generation and Visual Question Answering

Information, Journal Year: 2025, Volume and Issue: 16(2), P. 136 - 136

Published: Feb. 12, 2025

Large language models (LLMs) and large vision (LVMs) have driven significant advancements in natural processing (NLP) computer (CV), establishing a foundation for multimodal (MLLMs) to integrate diverse data types real-world applications. This survey explores the evolution of MLLMs radiology, focusing on radiology report generation (RRG) visual question answering (RVQA), where leverage combined capabilities LLMs LVMs improve clinical efficiency. We begin by tracing history development MLLMs, followed an overview MLLM applications RRG RVQA, detailing core datasets, evaluation metrics, leading that demonstrate their potential generating reports image-based questions. then discuss challenges face including dataset scarcity, privacy security, issues within such as bias, toxicity, hallucinations, catastrophic forgetting, limitations traditional metrics. Finally, this paper proposes future research directions address these challenges, aiming help AI researchers radiologists overcome obstacles advance study radiology.

Language: Английский

---