Performance of large language artificial intelligence models on solving restorative dentistry and endodontics student assessments DOI Creative Commons

Paul Künzle,

Sebastian Paris

Clinical Oral Investigations, Год журнала: 2024, Номер 28(11)

Опубликована: Окт. 7, 2024

Abstract Objectives The advent of artificial intelligence (AI) and large language model (LLM)-based AI applications (LLMAs) has tremendous implications for our society. This study analyzed the performance LLMAs on solving restorative dentistry endodontics (RDE) student assessment questions. Materials methods 151 questions from a RDE question pool were prepared prompting using OpenAI (ChatGPT-3.5,-4.0 -4.0o) Google (Gemini 1.0). Multiple-choice sorted into four subcategories, entered answers recorded analysis. P-value chi-square statistical analyses performed Python 3.9.16. Results total answer accuracy ChatGPT-4.0o was highest, followed by ChatGPT-4.0, Gemini 1.0 ChatGPT-3.5 (72%, 62%, 44% 25%, respectively) with significant differences between all except GPT-4.0 models. subcategories direct restorations caries indirect endodontics. Conclusions Overall, there are among LLMAs. Only ChatGPT-4 models achieved success ratio that could be used caution to support dental academic curriculum. Clinical relevance While clinicians field-related questions, this capacity depends strongly employed model. most performant acceptable rates in some subject sub-categories analyzed.

Язык: Английский

Shaping the Future of Education: Exploring the Potential and Consequences of AI and ChatGPT in Educational Settings DOI Creative Commons
Simone Grassini

Education Sciences, Год журнала: 2023, Номер 13(7), С. 692 - 692

Опубликована: Июль 7, 2023

Over the last decade, technological advancements, especially artificial intelligence (AI), have significantly transformed educational practices. Recently, development and adoption of Generative Pre-trained Transformers (GPT), particularly OpenAI’s ChatGPT, has sparked considerable interest. The unprecedented capabilities these models, such as generating humanlike text facilitating automated conversations, broad implications in various sectors, including education health. Despite their immense potential, concerns regarding widespread use opacity been raised within scientific community. latest version GPT series, displayed remarkable proficiency, passed US bar law exam, amassed over a million subscribers shortly after its launch. However, impact on sector elicited mixed reactions, with some educators heralding it progressive step others raising alarms potential to reduce analytical skills promote misconduct. This paper aims delve into discussions, exploring problems associated applying advanced AI models education. It builds extant literature contributes understanding how technologies reshape norms “new gold rush” era.

Язык: Английский

Процитировано

553

A systematic review of large language models and their implications in medical education DOI Creative Commons

Harrison C. Lucas,

Jeffrey S. Upperman, Jamie R. Robinson

и другие.

Medical Education, Год журнала: 2024, Номер unknown

Опубликована: Апрель 19, 2024

Abstract Introduction In the past year, use of large language models (LLMs) has generated significant interest and excitement because their potential to revolutionise various fields, including medical education for aspiring physicians. Although students undergo a demanding educational process become competent health care professionals, emergence LLMs presents promising solution challenges like information overload, time constraints pressure on clinical educators. However, integrating into raises critical concerns educators, professionals students. This systematic review aims explore LLM applications in education, specifically impact students' learning experiences. Methods A search was performed PubMed, Web Science Embase articles discussing using selected keywords related from ChatGPT's debut until February 2024. Only available full text or English were reviewed. The credibility each study critically appraised by two independent reviewers. Results identified 166 studies, which 40 found be relevant study. Among key themes included capabilities, benefits such as personalised regarding content accuracy. Importantly, 42.5% these studies evaluated novel way, ChatGPT, contexts exams clinical/biomedical information, highlighting replicating human‐level performance knowledge. remaining broadly discussed prospective role reflecting keen future despite current constraints. Conclusions responsible implementation offers opportunity enhance ensuring accuracy, emphasising skill‐building maintaining ethical safeguards are crucial. Continuous evaluation interdisciplinary collaboration essential appropriate integration education.

Язык: Английский

Процитировано

54

Hallucination Rates and Reference Accuracy of ChatGPT and Bard for Systematic Reviews: Comparative Analysis DOI Creative Commons
Mikaël Chelli, Jules Descamps,

Vincent Lavoué

и другие.

Journal of Medical Internet Research, Год журнала: 2024, Номер 26, С. e53164 - e53164

Опубликована: Май 22, 2024

Background Large language models (LLMs) have raised both interest and concern in the academic community. They offer potential for automating literature search synthesis systematic reviews but raise concerns regarding their reliability, as tendency to generate unsupported (hallucinated) content persist. Objective The aim of study is assess performance LLMs such ChatGPT Bard (subsequently rebranded Gemini) produce references context scientific writing. Methods replicating results human-conducted was assessed. Using pertaining shoulder rotator cuff pathology, these were tested by providing same inclusion criteria comparing with original review references, serving gold standards. used 3 key metrics: recall, precision, F1-score, alongside hallucination rate. Papers considered “hallucinated” if any 2 following information wrong: title, first author, or year publication. Results In total, 11 across 4 fields yielded 33 prompts (3 LLMs×11 reviews), 471 analyzed. Precision rates GPT-3.5, GPT-4, 9.4% (13/139), 13.4% (16/119), 0% (0/104) respectively (P<.001). Recall 11.9% (13/109) GPT-3.5 13.7% (15/109) failing retrieve relevant papers Hallucination stood at 39.6% (55/139) 28.6% (34/119) 91.4% (95/104) Further analysis nonhallucinated retrieved GPT revealed significant differences identifying various criteria, randomized studies, participant intervention criteria. also noted geographical open-access biases LLMs. Conclusions Given current performance, it not recommended be deployed primary exclusive tool conducting reviews. Any generated warrant thorough validation researchers. high occurrence hallucinations highlights necessity refining training functionality before confidently using them rigorous purposes.

Язык: Английский

Процитировано

51

Roles, Users, Benefits, and Limitations of Chatbots in Health Care: Rapid Review DOI Creative Commons
Moustafa Laymouna, Yuanchao Ma, David Lessard

и другие.

Journal of Medical Internet Research, Год журнала: 2024, Номер 26, С. e56930 - e56930

Опубликована: Апрель 12, 2024

Background Chatbots, or conversational agents, have emerged as significant tools in health care, driven by advancements artificial intelligence and digital technology. These programs are designed to simulate human conversations, addressing various care needs. However, no comprehensive synthesis of chatbots’ roles, users, benefits, limitations is available inform future research application the field. Objective This review aims describe characteristics, focusing on their diverse roles pathway, user groups, limitations. Methods A rapid published literature from 2017 2023 was performed with a search strategy developed collaboration sciences librarian implemented MEDLINE Embase databases. Primary studies reporting chatbot benefits were included. Two reviewers dual-screened results. Extracted data subjected content analysis. Results The categorized into 2 themes: delivery remote services, including patient support, management, education, skills building, behavior promotion, provision administrative assistance providers. User groups spanned across patients chronic conditions well cancer; individuals focused lifestyle improvements; demographic such women, families, older adults. Professionals students also alongside seeking mental behavioral change, educational enhancement. chatbots classified improvement quality efficiency cost-effectiveness delivery. identified encompassed ethical challenges, medicolegal safety concerns, technical difficulties, experience issues, societal economic impacts. Conclusions Health offer wide spectrum applications, potentially impacting aspects care. While they promising for improving quality, integration system must be approached consideration ensure optimal, safe, equitable use.

Язык: Английский

Процитировано

44

Large Language Models and User Trust: Consequence of Self-Referential Learning Loop and the Deskilling of Health Care Professionals DOI Creative Commons
Avishek Choudhury, Zaira S. Chaudhry

Journal of Medical Internet Research, Год журнала: 2024, Номер 26, С. e56764 - e56764

Опубликована: Март 20, 2024

As the health care industry increasingly embraces large language models (LLMs), understanding consequence of this integration becomes crucial for maximizing benefits while mitigating potential pitfalls. This paper explores evolving relationship among clinician trust in LLMs, transition data sources from predominantly human-generated to artificial intelligence (AI)–generated content, and subsequent impact on performance LLMs competence. One primary concerns identified is LLMs’ self-referential learning loops, where AI-generated content feeds into algorithms, threatening diversity pool, potentially entrenching biases, reducing efficacy LLMs. While theoretical at stage, feedback loop poses a significant challenge as deepens, emphasizing need proactive dialogue strategic measures ensure safe effective use LLM technology. Another key takeaway our investigation role user expertise necessity discerning approach trusting validating outputs. The highlights how expert users, particularly clinicians, can leverage enhance productivity by off-loading routine tasks maintaining critical oversight identify correct inaccuracies content. balance skepticism vital ensuring that augment rather than undermine quality patient care. We also discuss risks associated with deskilling professionals. Frequent reliance could result decline providers’ diagnostic thinking skills, affecting training development future legal ethical considerations surrounding deployment are examined. medicolegal challenges, including liability cases erroneous diagnoses or treatment advice generated references recent legislative efforts, such Algorithmic Accountability Act 2023, steps toward establishing framework responsible AI-based technologies In conclusion, advocates integrating By importance expertise, fostering engagement outputs, navigating landscape, we serve valuable tools enhancing supporting addresses immediate challenges posed sets foundation their maintainable future.

Язык: Английский

Процитировано

36

AI Hallucinations: A Misnomer Worth Clarifying DOI
Negar Maleki, Balaji Padmanabhan, Kaushik Dutta

и другие.

Опубликована: Июнь 25, 2024

Язык: Английский

Процитировано

33

Ethical dilemmas posed by the rise of artificial intelligence: a view from transhumanism DOI Creative Commons
Fernando Antonio Zapata Muriel, Santiago Montoya Zapata, Diego Montoya-Zapata

и другие.

Región Científica, Год журнала: 2024, Номер unknown

Опубликована: Янв. 15, 2024

Artificial intelligence has generated several concerns and discussions, especially about the possible risks consequences if ethical principles are not critically observed. Information was collected through documentary hermeneutic research methods, in which interpretation critical analysis prevail, followed by study of relevant bibliographic references on these topics. The results were triangulated with answers from artificial chat (ChatGPT 3.5) Spanish. It found that there significant differences between human beings, transhuman, intelligence, generating different spiritual-transcendent dilemmas today, can make intelligent machine a danger to humanity. Concepts such as singularity, autonomy, conscience, decision-making, freedom, among others, allow us glimpse difference programmed, automated certain functionality autonomy. is concluded everything techno-scientifically ethically acceptable, nor it equate programmed algorithms beings capable self-awareness, self-determination, thinking their existence, being aware uniqueness, other vital differences.

Язык: Английский

Процитировано

28

Reference Hallucination Score for Medical Artificial Intelligence Chatbots: Development and Usability Study DOI Creative Commons
Fadi Aljamaan, Mohamad‐Hani Temsah, Ibraheem Altamimi

и другие.

JMIR Medical Informatics, Год журнала: 2024, Номер 12, С. e54345 - e54345

Опубликована: Июль 3, 2024

Artificial intelligence (AI) chatbots have recently gained use in medical practice by health care practitioners. Interestingly, the output of these AI was found to varying degrees hallucination content and references. Such hallucinations generate doubts about their implementation.

Язык: Английский

Процитировано

27

Integrating the adapted UTAUT model with moral obligation, trust and perceived risk to predict ChatGPT adoption for assessment support: A survey with students DOI Creative Commons
Chung Yee Lai,

Kwok Yip Cheung,

Chee Seng Chan

и другие.

Computers and Education Artificial Intelligence, Год журнала: 2024, Номер 6, С. 100246 - 100246

Опубликована: Май 27, 2024

• Trust is the strongest predictor of ChatGPT adoption in assessments Moral obligation barrier acceptance for assessment support Perceived risk a significant demotivator but not mediator between trust and use intention Recommending clear guidelines on responsible university workshops about ethical AI

Язык: Английский

Процитировано

24

Exploring ChatGPT and its impact on society DOI
Md. Asraful Haque, Shuai Li

AI and Ethics, Год журнала: 2024, Номер unknown

Опубликована: Фев. 21, 2024

Язык: Английский

Процитировано

21