How to Design, Create, and Evaluate an Instruction-Tuning Dataset for Large Language Model Training in Health Care: Tutorial From a Clinical Perspective (Preprint) DOI
Wojciech Nazar,

Grzegorz Nazar,

Aleksandra Kamińska

и другие.

Опубликована: Дек. 23, 2024

UNSTRUCTURED High-quality data are critical in health care, forming the cornerstone for accurate diagnoses, effective treatment plans, and reliable conclusions. Similarly, high-quality datasets underpin development performance of large language models (LLMs). Among these, instruction-tuning (ITDs) used instruction fine-tuning have been pivotal enhancing LLM generalization capabilities across diverse tasks. This tutorial provides a comprehensive guide to designing, creating, evaluating ITDs care applications. Written from clinical perspective, it aims make concepts accessible broad audience, especially medical practitioners. Key topics include identifying useful sources, defining characteristics well-designed datasets, crafting instruction-input-output examples. We explore practical approaches dataset construction, examining advantages limitations 3 primary methods: fully manual preparation by expert annotators, synthetic generation using artificial intelligence (AI), an innovative hybrid approach which experts draft initial AI generates additional data. Moreover, we discuss strategies metadata selection human evaluation ensure quality effectiveness ITDs. By integrating these elements, this structured framework establishing It bridges technical domains, supporting continued interdisciplinary advancement medicine. Additionally, address current practices propose future directions, emphasizing need global, unified also argue that general (AGI), if realized, will not replace empirical research AGI depend on human-curated process apply knowledge. At same time, likely remain most method supplying knowledge AGI, positioning them as tool AI-driven care.

Язык: Английский

Agentic Large Language Models for Healthcare: Current Progress and Future Opportunities DOI Creative Commons
Han Yuan

Medicine Advances, Год журнала: 2025, Номер unknown

Опубликована: Март 3, 2025

Язык: Английский

Процитировано

3
Large Language Models in Worldwide Medical Exams: Platform Development and Comprehensive Analysis (Preprint) DOI Creative Commons
Hui Zong, Rongrong Wu, Jiaxue Cha

и другие.

Journal of Medical Internet Research, Год журнала: 2024, Номер 26, С. e66114 - e66114

Опубликована: Дек. 10, 2024

Background Large language models (LLMs) are increasingly integrated into medical education, with transformative potential for learning and assessment. However, their performance across diverse exams globally has remained underexplored. Objective This study aims to introduce MedExamLLM, a comprehensive platform designed systematically evaluate the of LLMs on worldwide. Specifically, seeks (1) compile curate data worldwide exams; (2) analyze trends disparities in LLM capabilities geographic regions, languages, contexts; (3) provide resource researchers, educators, developers explore advance integration artificial intelligence education. Methods A systematic search was conducted April 25, 2024, PubMed database identify relevant publications. Inclusion criteria encompassed peer-reviewed, English-language, original research articles that evaluated at least one exams. Exclusion included review articles, non-English publications, preprints, studies without performance. The screening process candidate publications independently by 2 researchers ensure accuracy reliability. Data, including exam information, model performance, availability, references, were manually curated, standardized, organized. These curated MedExamLLM platform, enabling its functionality visualize geographic, linguistic, characteristics. web developed focus accessibility, interactivity, scalability support continuous updates user engagement. Results total 193 final analysis. comprised information 16 198 28 countries 15 languages from year 2009 2023. United States accounted highest number related English being dominant used these Generative Pretrained Transformer (GPT) series models, especially GPT-4, demonstrated superior achieving pass rates significantly higher than other LLMs. analysis revealed significant variability different linguistic contexts. Conclusions is an open-source, freely accessible, publicly available online providing evaluation evidence knowledge about around world. serves as valuable fields clinical medicine intelligence. By synthesizing capabilities, provides insights Limitations include biases source exclusion literature. Future should address gaps methods enhance

Язык: Английский

Процитировано

9
Tackling algorithmic bias and promoting transparency in health datasets: the STANDING Together consensus recommendations DOI Creative Commons
Joseph Alderman, Joanne Palmer, Elinor Laws

и другие.

The Lancet Digital Health, Год журнала: 2024, Номер 7(1), С. e64 - e88

Опубликована: Дек. 18, 2024

Язык: Английский

Процитировано

9
Natural language processing to evaluate texting conversations between patients and healthcare providers during COVID-19 Home-Based Care in Rwanda at scale DOI Creative Commons
Richard Lester,

Matthew Manson,

Muhammed Semakula

и другие.

PLOS Digital Health, Год журнала: 2025, Номер 4(1), С. e0000625 - e0000625

Опубликована: Янв. 15, 2025

Community isolation of patients with communicable infectious diseases limits spread pathogens but our understanding isolated patients’ needs and challenges is incomplete. Rwanda deployed a digital health service nationally to assist public clinicians remotely monitor support SARS-CoV-2 cases via their mobile phones using daily interactive short message (SMS) check-ins. We aimed assess the texting patterns communicated topics better understand patient experiences. extracted data on all COVID-19 exposed contacts who were enrolled in WelTel text messaging program between March 18, 2020, 31, 2022, linked demographic clinical from national registry. A sample conversation corpus was English-translated labeled interest defined by medical experts. Multiple natural language processing (NLP) topic classification models trained compared F1 scores. Best performing applied classify unlabeled conversations. Total 33,081 (mean age 33·9, range 0–100), 44% female, including 30,398 2,683 contacts) registered WelTel. Registered generated 12,119 conversations Kinyarwanda (n = 8,183, 67%), English 3,069, 25%) other languages. Sufficiently large (LLMs) unavailable for Kinyarwanda. Traditional machine learning (ML) outperformed fine-tuned transformer architecture native untranslated corpus, however, reverse observed English-only data. The most frequently identified discussed included symptoms (69%), diagnostics (38%), social issues (19%), prevention (18%), healthcare logistics (16%), treatment (8·5%). Education, advice, triage these provided patients. Interactive can be used pandemics at scale. NLP help evaluate factors that affect which could ultimately inform precision responses future pandemics.

Язык: Английский

Процитировано

0
Natural language processing in plastic surgery patient consultations DOI Open Access
Ankoor A. Talwar, Chen Shen,

Joseph H. Shin

и другие.

Artificial Intelligence Surgery, Год журнала: 2025, Номер 5(1), С. 46 - 52

Опубликована: Янв. 10, 2025

Natural language processing (NLP) is the study of systems that allow machines to understand, interpret, and generate human language. With advent large models (LLMs), non-technical industries can also harness power NLP. This includes healthcare, specifically surgical care plastic surgery. manuscript an introductory review for surgeons understand current state future potential NLP in patient consultations. The integration into surgery consultations transform both documentation communication. These applications include information extraction, chart summarization, ambient transcription, coding, enhancing understanding, translation, a patient-facing chatbot. We discuss progress toward building these highlight their challenges. has personalize care, enhance satisfaction, improve workflows surgeons. Altogether, radically our model consultation one more patient-centered.

Язык: Английский

Процитировано

0
Analysis of longitudinal social media for monitoring symptoms during a pandemic DOI

Shixu Lin,

Lucas Garay, Yining Hua

и другие.

Journal of Biomedical Informatics, Год журнала: 2025, Номер unknown, С. 104778 - 104778

Опубликована: Янв. 1, 2025

Язык: Английский

Процитировано

0
Bridging the Divide: A Systems Thinking Approach to Inclusivity in AI Development and Education DOI

Arkapravo Sarkar

Опубликована: Янв. 1, 2025

Язык: Английский

Процитировано

0
FAIR-EC: A Global Research Network for Fair, Accountable, Interpretable, and Responsible AI in Emergency Care (Preprint) DOI Creative Commons
Chuan Hong,

Jonathan Chong Kai Liew,

Jae Yong Yu

и другие.

Опубликована: Март 22, 2025

BACKGROUND The current landscape of Emergency Care (EC) is marked by high demand leading to issues such as Department boarding, overcrowding and subsequent delays that impact the quality safety patient care. Integrating data science into EC can enhance decision-making with predictive, preventative, personalized, participatory approaches. However, gaps in adherence fairness, accountability, interpretability, responsibility are evident, particularly due barriers data-sharing, which often result a lack transparency robust oversight these applications. OBJECTIVE Fair, Accountable, Interpretable Responsible (FAIR)-EC collaboration adapts existing FAIR principles address emerging challenges integrates EC. This initiative aims transform establishing ethical artificial intelligence (AI) standards specifically tailored for this integration. By bridging gap between professionals, scientists other stakeholders, promotes international cooperation leverages advanced techniques outcomes across different care settings. METHODS We propose federated research design enables analyses extensive datasets from various global institutions without compromising privacy. approach transforms epidemiological techniques, emphasizing harmonization comprehensive healthcare systems. RESULTS FAIR-EC has facilitated collection analysis diverse geographical regions, enabling examination regional variations practices. Initial projects have demonstrated promising outcomes, including successful development scoring system adaptation association studies predictive models regions. These efforts highlight feasibility leveraging complexities while preserving CONCLUSIONS ethically effectively EC, addressing like fragmented data, real-time handoffs, public health crises. Its harmonizes streams privacy, its emphasis on AI aligns dynamic nature Despite variability complexity, establishes strong foundation innovation

Язык: Английский

Процитировано

0
Diversity and Inclusion Within Datasets in Heart Failure DOI Creative Commons
Elinor Laws, Maria Charalambides,

Sonam Vadera

и другие.

JACC Advances, Год журнала: 2025, Номер 4(3), С. 101610 - 101610

Опубликована: Март 1, 2025

Heart failure (HF) is a life-threatening disease affecting 64 million people worldwide. Artificial intelligence (AI) technologies are being developed for use in HF to support early diagnosis and stratification of treatment. The performance characteristics AI influenced by whether the data used during lifecycle reflects populations which used. aim study was identify characterize datasets across HF, focusing on diversity inclusivity. MEDLINE Embase were systematically searched from January 1, 2012, until August 30, 2022, articles relating development HF. Articles independently screened 2 reviewers datasets. Dataset documentation analyzed with focus accessibility, geographical origin, relevant metadata reporting, dataset composition. 72 identified represented 23 countries over individuals. In total, 62 (86%) reported "age," 61 (85%) sex or gender, 21 (29%) race and/or ethnicity, 8 (11%) socioeconomic status. that 89% individuals within "White" "Caucasian" category. Only 20 (28%) fully accessible. Reporting sex, status inconsistent. There need generate transparently Although collecting reporting demographic attributes complex needs be undertaken appropriate safeguards, it also an essential step toward building equitable AI-based health technologies.

Язык: Английский

Процитировано

0
The imperative of diversity and equity for the adoption of responsible AI in healthcare DOI Creative Commons
Denise E. Hilling,

Imane Ihaddouchen,

Stefan Buijsman

и другие.

Frontiers in Artificial Intelligence, Год журнала: 2025, Номер 8

Опубликована: Апрель 16, 2025

Artificial Intelligence (AI) in healthcare holds transformative potential but faces critical challenges ethical accountability and systemic inequities. Biases AI models, such as lower diagnosis rates for Black women or gender stereotyping Large Language Models, highlight the urgent need to address historical structural inequalities data development processes. Disparities clinical trials datasets, often skewed toward high-income, English-speaking regions, amplify these issues. Moreover, underrepresentation of marginalized groups among developers researchers exacerbates challenges. To ensure equitable AI, diverse collection, federated data-sharing frameworks, bias-correction techniques are essential. Structural initiatives, fairness audits, transparent model processes, early registration alongside inclusive global collaborations like TRAIN-Europe CHAI, can drive responsible adoption. Prioritizing diversity datasets researchers, well implementing governance will foster systems that uphold principles deliver outcomes globally.

Язык: Английский

Процитировано

0