Influence of a Large Language Model on Diagnostic Reasoning: A Randomized Clinical Vignette Study

medRxiv (Cold Spring Harbor Laboratory), Journal Year: 2024, Volume and Issue: unknown

Published: March 14, 2024

ABSTRACT Importance Diagnostic errors are common and cause significant morbidity. Large language models (LLMs) have shown promise in their performance on both multiple-choice open-ended medical reasoning examinations, but it remains unknown whether the use of such tools improves diagnostic reasoning. Objective To assess impact GPT-4 LLM physicians’ compared to conventional resources. Design Multi-center, randomized clinical vignette study. Setting The study was conducted using remote video conferencing with physicians across country in-person participation multiple academic institutions. Participants Resident attending training family medicine, internal or emergency medicine. Intervention(s) were access addition resources just They allocated 60 minutes review up six vignettes adapted from established exams. Main Outcome(s) Measure(s) primary outcome based differential diagnosis accuracy, appropriateness supporting opposing factors, next evaluation steps. Secondary outcomes included time spent per case final diagnosis. Results 50 (26 attendings, 24 residents) participated, an average 5.2 cases completed participant. median score 76.3 percent (IQR 65.8 86.8) for group 73.7 63.2 84.2) group, adjusted difference 1.6 percentage points (95% CI -4.4 7.6; p=0.60). 519 seconds 371 668 seconds), 565 456 788 seconds) a -82 -195 31; p=0.20). alone scored 15.5 1.5 29, p=0.03) higher than group. Conclusions Relevance In vignette-based study, availability as aid did not significantly improve resources, although may components efficiency. demonstrated physician groups, suggesting opportunities further improvement physician-AI collaboration practice.

Language: Английский

---

The TRIPOD-LLM Statement: A Targeted Guideline For Reporting Large Language Models Use

medRxiv (Cold Spring Harbor Laboratory), Journal Year: 2024, Volume and Issue: unknown

Published: July 25, 2024

Large Language Models (LLMs) are rapidly being adopted in healthcare, necessitating standardized reporting guidelines. We present TRIPOD-LLM, an extension of the TRIPOD+AI statement, addressing unique challenges LLMs biomedical applications. TRIPOD-LLM provides a comprehensive checklist 19 main items and 50 subitems, covering key aspects from title to discussion. The guidelines introduce modular format accommodating various LLM research designs tasks, with 14 32 subitems applicable across all categories. Developed through expedited Delphi process expert consensus, emphasizes transparency, human oversight, task-specific performance reporting. also interactive website ( https://tripod-llm.vercel.app/ ) facilitating easy guideline completion PDF generation for submission. As living document, will evolve field, aiming enhance quality, reproducibility, clinical applicability healthcare

Language: Английский

---

Genome-scale models in human metabologenomics

Nature Reviews Genetics, Journal Year: 2024, Volume and Issue: unknown

Published: Sept. 19, 2024

Language: Английский

---

Physician Versus Large Language Model Chatbot Responses to Web-Based Questions From Autistic Patients in Chinese: Cross-Sectional Comparative Analysis

Journal of Medical Internet Research, Journal Year: 2024, Volume and Issue: 26, P. e54706 - e54706

Published: April 2, 2024

Background There is a dearth of feasibility assessments regarding using large language models (LLMs) for responding to inquiries from autistic patients within Chinese-language context. Despite Chinese being one the most widely spoken languages globally, predominant research focus on applying these in medical field has been English-speaking populations. Objective This study aims assess effectiveness LLM chatbots, specifically ChatGPT-4 (OpenAI) and ERNIE Bot (version 2.2.3; Baidu, Inc), advanced LLMs China, addressing individuals setting. Methods For this study, we gathered data DXY—a acknowledged, web-based, consultation platform China with user base over 100 million individuals. A total patient samples were rigorously selected January 2018 August 2023, amounting 239 questions extracted publicly available autism-related documents platform. To maintain objectivity, both original responses anonymized randomized. An evaluation team 3 chief physicians assessed across 4 dimensions: relevance, accuracy, usefulness, empathy. The completed 717 evaluations. initially identified best response then used Likert scale 5 categories gauge responses, each representing distinct level quality. Finally, compared collected different sources. Results Among evaluations conducted, 46.86% (95% CI 43.21%-50.51%) assessors displayed varying preferences physicians, 34.87% 31.38%-38.36%) favoring ChatGPT 18.27% 15.44%-21.10%) Bot. average relevance scores ChatGPT, 3.75 3.69-3.82), 3.69 3.63-3.74), 3.41 3.35-3.46), respectively. Physicians (3.66, 95% 3.60-3.73) (3.73, 3.69-3.77) demonstrated higher accuracy ratings (3.52, 3.47-3.57). In terms usefulness scores, (3.54, 3.47-3.62) received than (3.40, 3.34-3.47) (3.05, 2.99-3.12). concerning empathy dimension, (3.64, 3.57-3.71) outperformed (3.13, 3.04-3.21) (3.11, 3.04-3.18). Conclusions cross-sectional physicians’ exhibited superiority present Nonetheless, can provide valuable guidance may even surpass demonstrating However, it crucial acknowledge that further optimization are imperative prerequisites before effective integration clinical settings diverse linguistic environments be realized. Trial Registration Clinical Registry ChiCTR2300074655; https://www.chictr.org.cn/bin/project/edit?pid=199432

Language: Английский

---

Evaluating Large Language Model (LLM) Performance on Established Breast Classification Systems

Diagnostics, Journal Year: 2024, Volume and Issue: 14(14), P. 1491 - 1491

Published: July 11, 2024

Medical researchers are increasingly utilizing advanced LLMs like ChatGPT-4 and Gemini to enhance diagnostic processes in the medical field. This research focuses on their ability comprehend apply complex classification systems for breast conditions, which can significantly aid plastic surgeons making informed decisions diagnosis treatment, ultimately leading improved patient outcomes. Fifty clinical scenarios were created evaluate accuracy of each LLM across five established breast-related systems. Scores from 0 2 assigned responses denote incorrect, partially correct, or completely correct classifications. Descriptive statistics employed compare performances Gemini. exhibited superior overall performance, achieving 98% compared ChatGPT-4's 71%. While both models performed well Baker capsular contracture UTSW gynecomastia, consistently outperformed other systems, such as Fischer Grade Classification gender-affirming mastectomy, Kajava ectopic tissue, Regnault ptosis. With further development, integrating into surgery practice will likely support decision making.

Language: Английский

---

Preventing unrestricted and unmonitored AI experimentation in healthcare through transparency and accountability

npj Digital Medicine, Journal Year: 2025, Volume and Issue: 8(1)

Published: Jan. 18, 2025

The integration of large language models (LLMs) into electronic health records offers potential benefits but raises significant ethical, legal, and operational concerns, including unconsented data use, lack governance, AI-related malpractice accountability. Sycophancy, feedback loop bias, reuse risk amplifying errors without proper oversight. To safeguard patients, especially the vulnerable, clinicians must advocate for patient-centered education, ethical practices, robust oversight to prevent harm.

Language: Английский

---

Harnessing AI for Understanding Scientific Literature: Innovations and Applications of Chat-Agent System in Battery Recycling Research

Materials Today Energy, Journal Year: 2025, Volume and Issue: unknown, P. 101818 - 101818

Published: Jan. 1, 2025

Language: Английский

---

Precision Management in Chronic Disease: An AI Empowered Perspective on Medicine-Engineering Crossover

iScience, Journal Year: 2025, Volume and Issue: 28(3), P. 112044 - 112044

Published: Feb. 17, 2025

Language: Английский

---

Advantages and limitations of large language models for antibiotic prescribing and antimicrobial stewardship

npj Antimicrobials and Resistance, Journal Year: 2025, Volume and Issue: 3(1)

Published: Feb. 27, 2025

Antibiotic prescribing requires balancing optimal treatment for patients with reducing antimicrobial resistance. There is a lack of standardization in research on using large language models (LLMs) supporting antibiotic prescribing, necessitating more efforts to identify biases and misinformation their outputs. Educating future medical professionals these aspects crucial ensuring the proper use LLMs providing deeper understanding strengths limitations.

Language: Английский

---

The new paradigm in machine learning – foundation models, large language models and beyond: a primer for physicians

Internal Medicine Journal, Journal Year: 2024, Volume and Issue: 54(5), P. 705 - 715

Published: May 1, 2024

Abstract Foundation machine learning models are deep capable of performing many different tasks using data modalities such as text, audio, images and video. They represent a major shift from traditional task‐specific prediction models. Large language (LLM), brought to wide public prominence in the form ChatGPT, text‐based foundational that have potential transform medicine by enabling automation range tasks, including writing discharge summaries, answering patients questions assisting clinical decision‐making. However, not without risk can potentially cause harm if their development, evaluation use devoid proper scrutiny. This narrative review describes types LLM, emerging applications limitations bias likely future translation into practice.

Language: Английский

---