Assessing the Responses of Large Language Models (ChatGPT-4, Gemini, and Microsoft Copilot) to Frequently Asked Questions in Breast Imaging: A Study on Readability and Accuracy

Cureus, Journal Year: 2024, Volume and Issue: unknown

Published: May 9, 2024

Background Large language models (LLMs), such as ChatGPT-4, Gemini, and Microsoft Copilot, have been instrumental in various domains, including healthcare, where they enhance health literacy aid patient decision-making. Given the complexities involved breast imaging procedures, accurate comprehensible information is vital for engagement compliance. This study aims to evaluate readability accuracy of provided by three prominent LLMs, response frequently asked questions imaging, assessing their potential improve understanding facilitate healthcare communication. Methodology We collected most common on from clinical practice posed them LLMs. then evaluated responses terms accuracy. Responses LLMs were analyzed using Flesch Reading Ease Flesch-Kincaid Grade Level tests through a radiologist-developed Likert-type scale. Results The found significant variations among Gemini Copilot scored higher scales (p < 0.001), indicating easier understand. In contrast, ChatGPT-4 demonstrated greater its 0.001). Conclusions While show promise providing responses, issues may limit utility education. Conversely, despite being less accurate, are more accessible broader audience. Ongoing adjustments evaluations these essential ensure meet diverse needs patients, emphasizing need continuous improvement oversight deployment artificial intelligence technologies healthcare.

Language: Английский

---

Digital twins as global learning health and disease models for preventive and personalized medicine

Genome Medicine, Journal Year: 2025, Volume and Issue: 17(1)

Published: Feb. 7, 2025

Abstract Ineffective medication is a major healthcare problem causing significant patient suffering and economic costs. This issue stems from the complex nature of diseases, which involve altered interactions among thousands genes across multiple cell types organs. Disease progression can vary between patients over time, influenced by genetic environmental factors. To address this challenge, digital twins have emerged as promising approach, led to international initiatives aiming at clinical implementations. Digital are virtual representations health disease processes that integrate real-time data simulations predict, prevent, personalize treatments. Early applications DTs shown potential in areas like artificial organs, cancer, cardiology, hospital workflow optimization. However, widespread implementation faces several challenges: (1) characterizing dynamic molecular changes biological scales; (2) developing computational methods into DTs; (3) prioritizing mechanisms therapeutic targets; (4) creating interoperable DT systems learn each other; (5) designing user-friendly interfaces for clinicians; (6) scaling technology globally equitable access; (7) addressing ethical, regulatory, financial considerations. Overcoming these hurdles could pave way more predictive, preventive, personalized medicine, potentially transforming delivery improving outcomes.

Language: Английский

---

Evaluation of LLMs accuracy and consistency in the registered dietitian exam through prompt engineering and knowledge retrieval

Scientific Reports, Journal Year: 2025, Volume and Issue: 15(1)

Published: Jan. 9, 2025

Large language models (LLMs) are fundamentally transforming human-facing applications in the health and well-being domains: boosting patient engagement, accelerating clinical decision-making, facilitating medical education. Although state-of-the-art LLMs have shown superior performance several conversational applications, evaluations within nutrition diet still insufficient. In this paper, we propose to employ Registered Dietitian (RD) exam conduct a standard comprehensive evaluation of LLMs, GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, assessing both accuracy consistency queries. Our includes 1050 RD questions encompassing topics proficiency levels. addition, for first time, examine impact Zero-Shot (ZS), Chain Thought (CoT), with Self Consistency (CoT-SC), Retrieval Augmented Prompting (RAP) on responses. findings revealed that while these obtained acceptable overall performance, their results varied considerably different prompts question domains. GPT-4o CoT-SC prompting outperformed other approaches, whereas Pro ZS recorded highest consistency. For 3.5, CoT improved accuracy, RAP was particularly effective answer Expert level questions. Consequently, choosing appropriate LLM technique, tailored specific domain, can mitigate errors potential risks chatbots.

Language: Английский

---

Beyond transparency and explainability: on the need for adequate and contextualized user guidelines for LLM use

Ethics and Information Technology, Journal Year: 2024, Volume and Issue: 26(3)

Published: July 17, 2024

Language: Английский

---

Prompt Engineering in Healthcare

Electronics, Journal Year: 2024, Volume and Issue: 13(15), P. 2961 - 2961

Published: July 26, 2024

The rapid advancements in artificial intelligence, particularly generative AI and large language models, have unlocked new possibilities for revolutionizing healthcare delivery. However, harnessing the full potential of these technologies requires effective prompt engineering—designing optimizing input prompts to guide systems toward generating clinically relevant accurate outputs. Despite importance engineering, medical education has yet fully incorporate comprehensive training on this critical skill, leading a knowledge gap among clinicians. This article addresses educational by providing an overview its applications primary care medicine, best practices implementation. role well-crafted eliciting accurate, relevant, valuable responses from models is discussed, emphasizing need grounded aligned with evidence-based guidelines. explores various engineering care, including enhancing patient–provider communication, streamlining clinical documentation, supporting education, facilitating personalized shared decision-making. Incorporating domain-specific knowledge, engaging iterative refinement validation prompts, addressing ethical considerations biases are highlighted. Embracing as core competency will be crucial successfully adopting implementing ultimately improved patient outcomes enhanced

Language: Английский

---

Large Language Models in Oncology: Revolution or Cause for Concern?

Current Oncology, Journal Year: 2024, Volume and Issue: 31(4), P. 1817 - 1830

Published: March 29, 2024

The technological capability of artificial intelligence (AI) continues to advance with great strength. Recently, the release large language models has taken world by storm concurrent excitement and concern. As a consequence their impressive ability versatility, provide potential opportunity for implementation in oncology. Areas possible application include supporting clinical decision making, education, contributing cancer research. Despite promises that these novel systems can offer, several limitations barriers challenge implementation. It is imperative concerns, such as accountability, data inaccuracy, protection, are addressed prior integration progression continues, new ethical practical dilemmas will also be approached; thus, evaluation concerns dynamic nature. This review offers comprehensive overview oncology, well surrounding care.

Language: Английский

---

Large‐Language‐Model‐Based AI Agent for Organic Semiconductor Device Research

Advanced Materials, Journal Year: 2024, Volume and Issue: 36(32)

Published: May 31, 2024

Abstract Large language models (LLMs) have attracted widespread attention recently, however, their application in specialized scientific fields still requires deep adaptation. Here, an artificial intelligence (AI) agent for organic field‐effect transistors (OFETs) is designed by integrating the generative pre‐trained transformer 4 (GPT‐4) model with well‐trained machine learning (ML) algorithms. It can efficiently extract experimental parameters of OFETs from literature and reshape them into a structured database, achieving precision recall rates both exceeding 92%. Combined ML models, this AI further provide targeted guidance suggestions device design. With prompt engineering human‐in‐loop strategies, extracts sufficient information 709 277 research articles across different publishers gathers standardized database containing more than 10 000 parameters. Using based on Extreme Gradient Boosting trained performance judgment. interpretation high‐precision model, has provided feasible optimization scheme that tripled charge transport properties 2,6‐diphenyldithieno[3,2‐ b :2′,3′‐ d ]thiophene OFETs. This work effective practice LLMs field optoelectronic devices expands paradigm materials devices.

Language: Английский

---

Large language model use in clinical oncology

npj Precision Oncology, Journal Year: 2024, Volume and Issue: 8(1)

Published: Oct. 23, 2024

Large language models (LLMs) are undergoing intensive research for various healthcare domains. This systematic review and meta-analysis assesses current applications, methodologies, the performance of LLMs in clinical oncology. A mixed-methods approach was used to extract, summarize, compare methodological approaches outcomes. includes 34 studies. primarily evaluated on their ability answer oncologic questions across The highlights a significant variance, influenced by diverse methodologies evaluation criteria. Furthermore, differences inherent model capabilities, prompting strategies, oncological subdomains contribute heterogeneity. lack use standardized LLM-specific reporting protocols leads disparities, which must be addressed ensure comparability LLM ultimately leverage reliable integration technologies into practice.

Language: Английский

---

Evaluating and Enhancing Large Language Models’ Performance in Domain-specific Medicine: Explainable LLM with DocOA (Preprint)

Journal of Medical Internet Research, Journal Year: 2024, Volume and Issue: 26, P. e58158 - e58158

Published: June 4, 2024

The efficacy of large language models (LLMs) in domain-specific medicine, particularly for managing complex diseases such as osteoarthritis (OA), remains largely unexplored.

Language: Английский

---

Optimizing Large Language Models in Radiology and Mitigating Pitfalls: Prompt Engineering and Fine-tuning

Radiographics, Journal Year: 2025, Volume and Issue: 45(4)

Published: March 6, 2025

Large language models (LLMs) such as generative pretrained transformers (GPTs) have had a major impact on society, and there is increasing interest in using these for applications medicine radiology. This article presents techniques to optimize describes their known challenges limitations. Specifically, the authors explore how best craft natural prompts, process prompt engineering, elicit more accurate desirable responses. The also explain fine-tuning conducted, which general model, GPT-4, further trained specific use case, summarizing clinical notes, improve reliability relevance. Despite enormous potential of models, substantial limit widespread implementation. These tools differ substantially from traditional health technology complexity probabilistic nondeterministic nature, differences lead issues "hallucinations," biases, lack reliability, security risks. Therefore, provide radiologists with baseline knowledge underpinning an understanding them, addition exploring practices engineering fine-tuning. Also discussed are current proof-of-concept cases LLMs radiology literature, decision support report generation, limitations preventing adoption ©RSNA, 2025 See invited commentary by Chung Mongan this issue.

Language: Английский

---