Multimodal biomedical AI

Nature Medicine, Journal Year: 2022, Volume and Issue: 28(9), P. 1773 - 1784

Published: Sept. 1, 2022

Language: Английский

---

Pan-cancer integrative histology-genomic analysis via multimodal deep learning

Cancer Cell, Journal Year: 2022, Volume and Issue: 40(8), P. 865 - 878.e6

Published: Aug. 1, 2022

The rapidly emerging field of computational pathology has demonstrated promise in developing objective prognostic models from histology images. However, most are either based on or genomics alone and do not address how these data sources can be integrated to develop joint image-omic models. Additionally, identifying explainable morphological molecular descriptors that govern such prognosis is interest. We use multimodal deep learning jointly examine whole-slide images profile 14 cancer types. Our weakly supervised, deep-learning algorithm able fuse heterogeneous modalities predict outcomes discover features correlate with poor favorable outcomes. present all analyses for correlates patient across the types at both a disease level an interactive open-access database allow further exploration, biomarker discovery, feature assessment.

Language: Английский

---

Scaling Vision Transformers to Gigapixel Images via Hierarchical Self-Supervised Learning

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2022, Volume and Issue: unknown, P. 16123 - 16134

Published: June 1, 2022

Vision Transformers (ViTs) and their multi-scale hierarchical variations have been successful at capturing image representations but use has generally studied for low-resolution images (e.g. 256 × 256, 384 384). For gigapixel whole-slide imaging (WSI) in computational pathology, WSIs can be as large 150000 pixels 20 magnification exhibit a structure of visual tokens across varying resolutions: from 16 individual cells, to 4096 characterizing interactions within the tissue microenvironment. We introduce new ViT architecture called Hierarchical Image Pyramid Transformer (HIPT), which leverages natural inherent using two levels self-supervised learning learn high-resolution representations. HIPT is pretrained 33 cancer types 10,678 WSIs, 408,218 images, 104M images. benchmark on 9 slide-level tasks, demonstrate that: 1) with pretraining outperforms current state-of-the-art methods subtyping survival prediction, 2) ViTs are able model important inductive biases about phenotypes tumor

Language: Английский

---

Artificial intelligence in histopathology: enhancing cancer research and clinical oncology

Nature Cancer, Journal Year: 2022, Volume and Issue: 3(9), P. 1026 - 1038

Published: Sept. 22, 2022

Language: Английский

---

Algorithmic fairness in artificial intelligence for medicine and healthcare

Nature Biomedical Engineering, Journal Year: 2023, Volume and Issue: 7(6), P. 719 - 742

Published: June 28, 2023

Language: Английский

---

The impact of site-specific digital histology signatures on deep learning model accuracy and bias

Nature Communications, Journal Year: 2021, Volume and Issue: 12(1)

Published: July 20, 2021

The Cancer Genome Atlas (TCGA) is one of the largest biorepositories digital histology. Deep learning (DL) models have been trained on TCGA to predict numerous features directly from histology, including survival, gene expression patterns, and driver mutations. However, we demonstrate that these vary substantially across tissue submitting sites in for over 3,000 patients with six cancer subtypes. Additionally, show histologic image differences between can easily be identified DL. Site detection remains possible despite commonly used color normalization augmentation methods, quantify characteristics constituting this site-specific histology signature. We signatures lead biased accuracy prediction genomic mutations, tumor stage. Furthermore, ethnicity also inferred signatures, which must accounted ensure equitable application These overoptimistic estimates model performance, propose a quadratic programming method abrogates bias by ensuring are not validated samples same site.

Language: Английский

---

Artificial intelligence for assisting cancer diagnosis and treatment in the era of precision medicine

Cancer Communications, Journal Year: 2021, Volume and Issue: 41(11), P. 1100 - 1115

Published: Oct. 6, 2021

Abstract Over the past decade, artificial intelligence (AI) has contributed substantially to resolution of various medical problems, including cancer. Deep learning (DL), a subfield AI, is characterized by its ability perform automated feature extraction and great power in assimilation evaluation large amounts complicated data. On basis quantity data novel computational technologies, especially DL, been applied aspects oncology research potential enhance cancer diagnosis treatment. These applications range from early detection, diagnosis, classification grading, molecular characterization tumors, prediction patient outcomes treatment responses, personalized treatment, automatic radiotherapy workflows, anti‐cancer drug discovery, clinical trials. In this review, we introduced general principle summarized major areas application for discussed future directions remaining challenges. As adoption AI use increasing, anticipate arrival AI‐powered care.

Language: Английский

---

Artificial intelligence-based multi-omics analysis fuels cancer precision medicine

Seminars in Cancer Biology, Journal Year: 2022, Volume and Issue: 88, P. 187 - 200

Published: Dec. 31, 2022

Language: Английский

---

Swarm learning for decentralized artificial intelligence in cancer histopathology

Nature Medicine, Journal Year: 2022, Volume and Issue: 28(6), P. 1232 - 1239

Published: April 25, 2022

Abstract Artificial intelligence (AI) can predict the presence of molecular alterations directly from routine histopathology slides. However, training robust AI systems requires large datasets for which data collection faces practical, ethical and legal obstacles. These obstacles could be overcome with swarm learning (SL), in partners jointly train models while avoiding transfer monopolistic governance. Here, we demonstrate successful use SL large, multicentric gigapixel images over 5,000 patients. We show that trained using BRAF mutational status microsatellite instability hematoxylin eosin (H&E)-stained pathology slides colorectal cancer. on three patient cohorts Northern Ireland, Germany United States, validated prediction performance two independent Kingdom. Our SL-trained outperform most locally models, perform par are merged datasets. In addition, SL-based efficient. future, used to distributed any image analysis task, eliminating need transfer.

Language: Английский

---

Benchmarking weakly-supervised deep learning pipelines for whole slide classification in computational pathology

Medical Image Analysis, Journal Year: 2022, Volume and Issue: 79, P. 102474 - 102474

Published: May 5, 2022

Language: Английский

---