SCD-Tron: Leveraging Large Clinical Language Model for Early Detection of Cognitive Decline from Electronic Health Records DOI Creative Commons
Hao Guan,

John Novoa-Laurentiev,

Zhou Li

et al.

medRxiv (Cold Spring Harbor Laboratory), Journal Year: 2024, Volume and Issue: unknown

Published: Nov. 2, 2024

Background: Early detection of cognitive decline during the preclinical stage Alzheimer's disease and related dementias (AD/ADRD) is crucial for timely intervention treatment. Clinical notes in electronic health record contain valuable information that can aid early identification decline. In this study, we utilize advanced large clinical language models, fine-tuned on notes, to improve Methods: We collected from 2,166 patients spanning 4 years preceding their initial mild impairment (MCI) diagnosis Enterprise Data Warehouse Mass General Brigham. To train model, developed CD-Tron, built upon a model was finetuned using 4,949 expert-labeled note sections. For evaluation, trained applied 1,996 independent sections assess its performance real-world unstructured data. Additionally, used explainable AI techniques, specifically SHAP values (SHapley Additive exPlanations), interpret model's predictions provide insight into most influential features. Error analysis also facilitated further analyze prediction. Results: CD-Tron significantly outperforms baseline achieving notable improvements precision, recall, AUC metrics detecting (CD). Tested many demonstrated high sensitivity with only one false negative, applications prioritizing accurate CD detection. SHAP-based interpretability highlighted key textual features contributing predictions, supporting transparency clinician understanding. Conclusion: offers novel approach by applying models free-text EHR Pretrained it accurately identifies integrates interpretability, enhancing predictions.

Language: Английский

AI-Driven Innovations in Alzheimer's Disease: Integrating Early Diagnosis, Personalized Treatment, and Prognostic Modelling DOI
Mayur B. Kale, Nitu L. Wankhede,

Rupali S. Pawar

et al.

Ageing Research Reviews, Journal Year: 2024, Volume and Issue: unknown, P. 102497 - 102497

Published: Sept. 1, 2024

Language: Английский

Citations

26
Improving large language model applications in biomedicine with retrieval-augmented generation: a systematic review, meta-analysis, and clinical development guidelines DOI Creative Commons
Siru Liu, Allison B. McCoy, Adam Wright

et al.

Journal of the American Medical Informatics Association, Journal Year: 2025, Volume and Issue: unknown

Published: Jan. 15, 2025

The objectives of this study are to synthesize findings from recent research retrieval-augmented generation (RAG) and large language models (LLMs) in biomedicine provide clinical development guidelines improve effectiveness. We conducted a systematic literature review meta-analysis. report was created adherence the Preferred Reporting Items for Systematic Reviews Meta-Analyses 2020 analysis. Searches were performed 3 databases (PubMed, Embase, PsycINFO) using terms related "retrieval augmented generation" "large model," articles published 2023 2024. selected studies that compared baseline LLM performance with RAG performance. developed random-effect meta-analysis model, odds ratio as effect size. Among 335 studies, 20 included review. pooled size 1.35, 95% confidence interval 1.19-1.53, indicating statistically significant (P = .001). reported tasks, LLMs, retrieval sources strategies, well evaluation methods. Building on our review, we Guidelines Unified Implementation Development Enhanced Applications Clinical Settings inform applications RAG. Overall, implementation showed 1.35 increase LLMs. Future should focus (1) system-level enhancement: combination agent, (2) knowledge-level deep integration knowledge into LLM, (3) integration-level integrating systems within electronic health records.

Language: Английский

Citations

6
Generative Large Language Models in Electronic Health Records for Patient Care Since 2023: A Systematic Review DOI Creative Commons
Xinsong Du, Yifei Wang, Zhengyang Zhou

et al.

medRxiv (Cold Spring Harbor Laboratory), Journal Year: 2024, Volume and Issue: unknown

Published: Aug. 12, 2024

Background: Generative Large language models (LLMs) represent a significant advancement in natural processing, achieving state-of-the-art performance across various tasks. However, their application clinical settings using real electronic health records (EHRs) is still rare and presents numerous challenges. Objective: This study aims to systematically review the use of generative LLMs, effectiveness relevant techniques patient care-related topics involving EHRs, summarize challenges faced, suggest future directions. Methods: A Boolean search for peer-reviewed articles was conducted on May 19th, 2024 PubMed Web Science include research published since 2023, which one month after release ChatGPT. The results were deduplicated. Multiple reviewers, including biomedical informaticians, computer scientists, physician, screened publications eligibility data extraction. Only studies utilizing LLMs analyze EHR included. We summarized prompt engineering, fine-tuning, multimodal data, evaluation matrices. Additionally, we identified current applying as reported by included proposed Results: initial 6,328 unique studies, with 76 screening. Of these, 67 (88.2%) employed zero-shot prompting, five them 100% accuracy specific Nine used advanced prompting strategies; four tested these strategies experimentally, finding that engineering improved performance, noting non-linear relationship between number examples improvement. Eight explored fine-tuning all improvements tasks, but three noted potential degradation certain two utilized LLM-based decision-making enabled accurate disease diagnosis prognosis. 55 different metrics 22 purposes, such correctness, completeness, conciseness. Two investigated LLM bias, detecting no bias other male patients received more appropriate suggestions. Six hallucinations, fabricating names structured thyroid ultrasound reports. Additional not limited impersonal tone consultations, made uncomfortable, difficulty had understanding responses. Conclusion: Our indicates few have computational enhance performance. diverse highlight need standardization. currently cannot replace physicians due

Language: Английский

Citations

7
Assessing the Potential of USMLE-Like Exam Questions Generated by GPT-4 DOI Creative Commons
Suhana Bedi, Scott L. Fleming, Chia‐Chun Chiang

et al.

medRxiv (Cold Spring Harbor Laboratory), Journal Year: 2023, Volume and Issue: unknown

Published: April 28, 2023

The United States Medical Licensing Examination (USMLE) is a critical step in assessing the competence of future physicians, yet process creating exam questions and study materials both time-consuming costly. While Large Language Models (LLMs), such as OpenAI’s GPT-4, have demonstrated proficiency answering medical questions, their potential generating remains underexplored. This presents QUEST-AI, novel system that utilizes LLMs to (1) generate USMLE-style (2) identify flag incorrect (3) correct errors flagged questions. We evaluated this system’s output by constructing test set 50 LLM-generated mixed with human-generated conducting two-part assessment three physicians two students. assessors attempted distinguish between LLM validity content. A majority generated QUEST-AI were deemed valid panel clinicians, strong correlations performance on pioneering application education could significantly increase ease efficiency developing content, offering cost-effective accessible alternative for preparation.

Language: Английский

Citations

13
Identification of an ANCA-associated vasculitis cohort using deep learning and electronic health records DOI
Liqin Wang, John Laurentiev,

Claire Cook

et al.

International Journal of Medical Informatics, Journal Year: 2025, Volume and Issue: 196, P. 105797 - 105797

Published: Jan. 18, 2025

Language: Английский

Citations

0
AI-powered model for accurate prediction of MCI-to-AD progression DOI Creative Commons
Ahmed Abdelhameed, Jingna Feng, Xinyue Hu

et al.

Acta Pharmaceutica Sinica B, Journal Year: 2025, Volume and Issue: unknown

Published: Feb. 1, 2025

Language: Английский

Citations

0
Evaluating local open-source large language models for data extraction from unstructured reports on mechanical thrombectomy in patients with ischemic stroke DOI Creative Commons
Aymen Meddeb,

Philipe Ebert,

Keno K. Bressem

et al.

Journal of NeuroInterventional Surgery, Journal Year: 2024, Volume and Issue: unknown, P. jnis - 022078

Published: Aug. 2, 2024

Background A study was undertaken to assess the effectiveness of open-source large language models (LLMs) in extracting clinical data from unstructured mechanical thrombectomy reports patients with ischemic stroke caused by a vessel occlusion. Methods We deployed local LLMs extract points free-text procedural who underwent between September 2020 and June 2023 our institution. The external dataset obtained second university hospital comprised consecutive cases treated March 2024. Ground truth labeling facilitated human-in-the-loop (HITL) approach, time metrics recorded for both automated manual extractions. tested three models—Mixtral, Qwen, BioMistral—assessing their performance on precision, recall, F1 score across 15 categories such as National Institute Health Stroke Scale (NIHSS) scores, occluded vessels, medication details. Results included 1000 primary institution 50 secondary Mixtral showed highest achieving 0.99 first series extraction 0.69 identification within internal dataset. In dataset, precision ranged 1.00 NIHSS scores 0.70 vessels. Qwen moderate high 0.85 low 0.28 BioMistral had broadest range 0.81 times 0.14 HITL approach yielded an average savings 65.6% per case, variations 45.95% 79.56%. Conclusion This highlights potential using medical reports. Incorporating annotations enhances also ensures reliability extracted data. methodology presents scalable privacy-preserving option that can significantly support documentation research endeavors.

Language: Английский

Citations

2
SCD-Tron: Leveraging Large Clinical Language Model for Early Detection of Cognitive Decline from Electronic Health Records DOI Creative Commons
Hao Guan,

John Novoa-Laurentiev,

Zhou Li

et al.

medRxiv (Cold Spring Harbor Laboratory), Journal Year: 2024, Volume and Issue: unknown

Published: Nov. 2, 2024

Background: Early detection of cognitive decline during the preclinical stage Alzheimer's disease and related dementias (AD/ADRD) is crucial for timely intervention treatment. Clinical notes in electronic health record contain valuable information that can aid early identification decline. In this study, we utilize advanced large clinical language models, fine-tuned on notes, to improve Methods: We collected from 2,166 patients spanning 4 years preceding their initial mild impairment (MCI) diagnosis Enterprise Data Warehouse Mass General Brigham. To train model, developed CD-Tron, built upon a model was finetuned using 4,949 expert-labeled note sections. For evaluation, trained applied 1,996 independent sections assess its performance real-world unstructured data. Additionally, used explainable AI techniques, specifically SHAP values (SHapley Additive exPlanations), interpret model's predictions provide insight into most influential features. Error analysis also facilitated further analyze prediction. Results: CD-Tron significantly outperforms baseline achieving notable improvements precision, recall, AUC metrics detecting (CD). Tested many demonstrated high sensitivity with only one false negative, applications prioritizing accurate CD detection. SHAP-based interpretability highlighted key textual features contributing predictions, supporting transparency clinician understanding. Conclusion: offers novel approach by applying models free-text EHR Pretrained it accurately identifies integrates interpretability, enhancing predictions.

Language: Английский

Citations

0