Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies

Computers & Education, Journal Year: 2024, Volume and Issue: unknown, P. 105224 - 105224

Published: Dec. 1, 2024

Language: Английский

---

Consistent Performance of GPT-4o in Rare Disease Diagnosis Across Nine Languages and 4967 Cases

medRxiv (Cold Spring Harbor Laboratory), Journal Year: 2025, Volume and Issue: unknown

Published: Feb. 28, 2025

Large language models (LLMs) are increasingly used in the medical field for diverse applications including differential diagnostic support. The estimated training data to create LLMs such as Generative Pretrained Transformer (GPT) predominantly consist of English-language texts, but could be across globe support diagnostics if barriers overcome. Initial pilot studies on utility diagnosis languages other than English have shown promise, a large-scale assessment relative performance these variety European and non-European comprehensive corpus challenging rare-disease cases is lacking. We created 4967 clinical vignettes using structured captured with Human Phenotype Ontology (HPO) terms Global Alliance Genomics Health (GA4GH) Phenopacket Schema. These span total 378 distinct genetic diseases 2618 associated phenotypic features. translations together language-specific templates generate prompts English, Chinese, Czech, Dutch, German, Italian, Japanese, Spanish, Turkish. applied GPT-4o, version gpt-4o-2024-08-06, task delivering ranked zero-shot prompt. An ontology-based approach Mondo disease ontology was map synonyms subtypes diagnoses order automate evaluation LLM responses. For GPT-4o placed correct at first rank 19·8% within top-3 ranks 27·0% time. In comparison, eight non-English tested here 1 between 16·9% 20·5%, 25·3% 27·7% cases. consistent nine tested. This suggests that may settings. NHGRI 5U24HG011449 5RM1HG010860. P.N.R. supported by Professorship Alexander von Humboldt Foundation; P.L. National Grant (PMP21/00063 ONTOPREC-ISCIII, Fondos FEDER).

Language: Английский

---

Accuracy of Large Language Models When Answering Clinical Research Questions: Systematic Review and Network Meta-Analysis

Journal of Medical Internet Research, Journal Year: 2025, Volume and Issue: 27, P. e64486 - e64486

Published: April 30, 2025

Large language models (LLMs) have flourished and gradually become an important research application direction in the medical field. However, due to high degree of specialization, complexity, specificity medicine, which results extremely accuracy requirements, controversy remains about whether LLMs can be used More studies evaluated performance various types but conclusions are inconsistent. This study uses a network meta-analysis (NMA) assess when answering clinical questions provide high-level evidence-based evidence for its future development In this systematic review NMA, we searched PubMed, Embase, Web Science, Scopus from inception until October 14, 2024. Studies on were included screened by reading published reports. The NMA conducted compare different questions, including objective open-ended top 1 diagnosis, 3 5 triage classification. was performed using Bayesian frequency theory methods. Indirect intercomparisons between programs grading scale. A larger surface under cumulative ranking curve (SUCRA) value indicates higher corresponding LLM accuracy. examined 168 articles encompassing 35,896 3063 cases. Of studies, 40 (23.8%) considered low risk bias, 128 (76.2%) had moderate risk, none rated as having risk. ChatGPT-4o (SUCRA=0.9207) demonstrated strong terms followed Aeyeconsult (SUCRA=0.9187) ChatGPT-4 (SUCRA=0.8087). (SUCRA=0.8708) excelled at questions. diagnosis cases, human experts (SUCRA=0.9001 SUCRA=0.7126, respectively) ranked highest, while Claude Opus (SUCRA=0.9672) well diagnosis. Gemini (SUCRA=0.9649) highest SUCRA area Our that has advantage For may more credible. Humans accurate performs better classification, is advantageous. analysis offers valuable insights clinicians practitioners, empowering them effectively leverage improved decision-making learning, management scenarios. PROSPERO CRD42024558245; https://www.crd.york.ac.uk/PROSPERO/view/CRD42024558245.

Language: Английский

---

Evaluating ChatGPT's Multilingual Performance in Clinical Nutrition Advice Using Synthetic Medical Text: Insights from Central Asia

Journal of Nutrition, Journal Year: 2024, Volume and Issue: 155(3), P. 729 - 735

Published: Dec. 26, 2024

Language: Английский

---

Research on the Application of Intelligent Writing Teaching System Based on Computer Vision and Natural Language Processing in Higher Vocational English

Published: Oct. 18, 2024

Language: Английский

---