Evaluating the Efficacy of Large Language Models in CPT Coding for Craniofacial Surgery: A Comparative Analysis DOI

Emily L Isch,

Advith Sarikonda,

Abhijeet Sambangi

et al.

Journal of Craniofacial Surgery, Journal Year: 2024, Volume and Issue: unknown

Published: Sept. 2, 2024

Background: The advent of Large Language Models (LLMs) like ChatGPT has introduced significant advancements in various surgical disciplines. These developments have led to an increased interest the utilization LLMs for Current Procedural Terminology (CPT) coding surgery. With CPT being a complex and time-consuming process, often exacerbated by scarcity professional coders, there is pressing need innovative solutions enhance efficiency accuracy. Methods: This observational study evaluated effectiveness 5 publicly available large language models—Perplexity.AI, Bard, BingAI, 3.5, 4.0—in accurately identifying codes craniofacial procedures. A consistent query format was employed test each model, ensuring inclusion detailed procedure components where necessary. responses were classified as correct, partially or incorrect based on their alignment with established specified Results: results indicate that while no overall association between type AI model correctness code identification, are notable differences performance simple among models. Specifically, 4.0 showed higher accuracy codes, whereas Perplexity.AI Bard more codes. Discussion: use chatbots surgery presents promising avenue reducing administrative burden associated costs manual coding. Despite lower rates compared specialized, trained algorithms, accessibility minimal training requirements make them attractive alternatives. also suggests priming models operative notes may accuracy, offering resource-efficient strategy improving clinical practice. Conclusions: highlights feasibility potential benefits integrating into process findings advocate further refinement improve practicality, suggesting future AI-assisted could become standard component workflows, aligning ongoing digital transformation health care.

Language: Английский

Transforming Education: A Comprehensive Review of Generative Artificial Intelligence in Educational Settings through Bibliometric and Content Analysis DOI Open Access
Zied Bahroun, Chiraz Anane, Vian Ahmed

et al.

Sustainability, Journal Year: 2023, Volume and Issue: 15(17), P. 12983 - 12983

Published: Aug. 29, 2023

In the ever-evolving era of technological advancements, generative artificial intelligence (GAI) emerges as a transformative force, revolutionizing education. This review paper, guided by PRISMA framework, presents comprehensive analysis GAI in education, synthesizing key insights from selection 207 research papers to identify gaps and future directions field. study begins with content that explores GAI’s impact specific educational domains, including medical education engineering The versatile applications encompass assessment, personalized learning support, intelligent tutoring systems. Ethical considerations, interdisciplinary collaboration, responsible technology use are highlighted, emphasizing need for transparent models addressing biases. Subsequently, bibliometric is conducted, examining prominent AI tools, focus, geographic distribution, collaboration. ChatGPT dominant tool, reveals significant exponential growth 2023. Moreover, this paper identifies promising directions, such GAI-enhanced curriculum design longitudinal studies tracking its long-term on outcomes. These findings provide understanding potential reshaping offer valuable researchers, educators, policymakers interested intersection

Language: Английский

Citations

323
Role of AI chatbots in education: systematic literature review DOI Creative Commons
Lasha Labadze, Maya Grigolia,

Lela Machaidze

et al.

International Journal of Educational Technology in Higher Education, Journal Year: 2023, Volume and Issue: 20(1)

Published: Oct. 31, 2023

Abstract AI chatbots shook the world not long ago with their potential to revolutionize education systems in a myriad of ways. can provide immediate support by answering questions, offering explanations, and providing additional resources. Chatbots also act as virtual teaching assistants, supporting educators through various means. In this paper, we try understand full benefits education, opportunities, challenges, limitations, concerns, prospects using educational settings. We conducted an extensive search across academic databases, after applying specific predefined criteria, selected final set 67 relevant studies for review. The research findings emphasize numerous integrating seen from both students' educators' perspectives. found that students primarily gain AI-powered three key areas: homework study assistance, personalized learning experience, development skills. For educators, main advantages are time-saving assistance improved pedagogy. However, our emphasizes significant challenges critical factors need handle diligently. These include concerns related applications such reliability, accuracy, ethical considerations.

Language: Английский

Citations

229
Mapping the global evidence around the use of ChatGPT in higher education: A systematic scoping review DOI
Aisha Naz Ansari, Sohail Ahmad, Sadia Muzaffar Bhutta

et al.

Education and Information Technologies, Journal Year: 2023, Volume and Issue: 29(9), P. 11281 - 11321

Published: Oct. 20, 2023

Language: Английский

Citations

64
Large language models for generating medical examinations: systematic review DOI Creative Commons
Yaara Artsi, Vera Sorin, Eli Konen

et al.

BMC Medical Education, Journal Year: 2024, Volume and Issue: 24(1)

Published: March 29, 2024

Abstract Background Writing multiple choice questions (MCQs) for the purpose of medical exams is challenging. It requires extensive knowledge, time and effort from educators. This systematic review focuses on application large language models (LLMs) in generating MCQs. Methods The authors searched studies published up to November 2023. Search terms focused LLMs generated MCQs examinations. Non-English, out year range not focusing AI multiple-choice were excluded. MEDLINE was used as a search database. Risk bias evaluated using tailored QUADAS-2 tool. Results Overall, eight between April 2023 October included. Six Chat-GPT 3.5, while two employed GPT 4. Five showed that can produce competent valid exams. Three write but did evaluate validity questions. One study conducted comparative analysis different models. other compared LLM-generated with those written by humans. All presented faulty deemed inappropriate Some required additional modifications order qualify. Conclusions be However, their limitations cannot ignored. Further this field essential more conclusive evidence needed. Until then, may serve supplementary tool writing 2 at high risk bias. followed Preferred Reporting Items Systematic Reviews Meta-Analyses (PRISMA) guidelines.

Language: Английский

Citations

32
ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: a literature review DOI Creative Commons
Yavuz Selim Kıyak, Emre Emekli

Postgraduate Medical Journal, Journal Year: 2024, Volume and Issue: unknown

Published: May 16, 2024

Abstract ChatGPT’s role in creating multiple-choice questions (MCQs) is growing but the validity of these artificial-intelligence-generated unclear. This literature review was conducted to address urgent need for understanding application ChatGPT generating MCQs medical education. Following database search and screening 1920 studies, we found 23 relevant studies. We extracted prompts MCQ generation assessed evidence MCQs. The findings showed that varied, including referencing specific exam styles adopting personas, which align with recommended prompt engineering tactics. covered various domains, showing mixed accuracy rates, some studies indicating comparable quality human-written questions, others highlighting differences difficulty discrimination levels, alongside a significant reduction question creation time. Despite its efficiency, highlight necessity careful suggest further research optimize use generation. Main messages Ensure high-quality outputs by utilizing well-designed prompts; educators should prioritize detailed, clear when Avoid using ChatGPT-generated directly examinations without thorough prevent inaccuracies ensure relevance. Leverage potential streamline test development process, enhancing efficiency compromising quality.

Language: Английский

Citations

32
Battle of the authors: Comparing neurosurgery articles written by humans and AI DOI
Mehmet Yiğit Akgün,

Melihcan Savasci,

Caner Günerbüyük

et al.

Journal of Clinical Neuroscience, Journal Year: 2025, Volume and Issue: 135, P. 111152 - 111152

Published: Feb. 25, 2025

Language: Английский

Citations

2
A descriptive study based on the comparison of ChatGPT and evidence-based neurosurgeons DOI Creative Commons
Jiayu Liu, Jiqi Zheng, Xintian Cai

et al.

iScience, Journal Year: 2023, Volume and Issue: 26(9), P. 107590 - 107590

Published: Aug. 9, 2023

ChatGPT is an artificial intelligence product developed by OpenAI. This study aims to investigate whether can respond in accordance with evidence-based medicine neurosurgery. We generated 50 neurosurgical questions covering diseases. Each question was posed three times GPT-3.5 and GPT-4.0. also recruited neurosurgeons high, middle, low seniority questions. The results were analyzed regarding ChatGPT's overall performance score, mean scores the items' specialty classification, type. In conclusion, GPT-3.5's ability comparable that of seniority, GPT-4.0's high seniority. Although yet be a neurosurgeon future upgrades could enhance its abilities.

Language: Английский

Citations

34
Evaluating AI‐powered text‐to‐image generators for anatomical illustration: A comparative study DOI Creative Commons
Geoffroy Noël

Anatomical Sciences Education, Journal Year: 2023, Volume and Issue: 17(5), P. 979 - 983

Published: Sept. 11, 2023

Medical illustration, which involves the creation of visual representations anatomy, has long been an essential tool for medical professionals and educators. The integration AI illustration potential to revolutionize field anatomy education, providing highly accurate, customizable images. authors evaluated three AI-powered text-to-image generators in producing anatomical illustrations human skulls, heart, brain. were assessed their accurate depiction foramina, suture lines, coronary arteries, aortic pulmonary trunk branching, gyri, sulci, relationship between cerebellum temporal lobes. None produced with comprehensive details. Foramina, such as mental supraorbital frequently omitted, lines inaccurately represented. heart failed indicate proper artery origins, branching aorta was often incorrect. Brain lacked gyri sulci depiction, lobes remained unclear. Although tended toward esoteric imagery, they exhibited significant speed cost advantages over illustrators. However, improving accuracy necessitates augmenting training databases anatomically correct study emphasizes ongoing role illustrators, especially ensuring provision accessible illustrations.

Language: Английский

Citations

32
Large Language Models and Artificial Intelligence: A Primer for Plastic Surgeons on the Demonstrated and Potential Applications, Promises, and Limitations of ChatGPT DOI
Jad Abi‐Rafeh, Hong Hao Xu, Roy Kazan

et al.

Aesthetic Surgery Journal, Journal Year: 2023, Volume and Issue: 44(3), P. 329 - 343

Published: Aug. 9, 2023

The rapidly evolving field of artificial intelligence (AI) holds great potential for plastic surgeons. ChatGPT, a recently released AI large language model (LLM), promises applications across many disciplines, including healthcare.

Language: Английский

Citations

28
The Significance of Artificial Intelligence Platforms in Anatomy Education: An Experience With ChatGPT and Google Bard DOI Open Access
Hasan Barış Ilgaz,

Zehra Çelik

Cureus, Journal Year: 2023, Volume and Issue: unknown

Published: Sept. 15, 2023

This study evaluated the use of two large language models (LLMs), ChatGPT and Google Bard, in anatomy education. The were asked to answer questions, generate multiple-choice write articles on topics. results showed that able perform these tasks with varying degrees accuracy. Bard did not differ significantly terms answering questions. Both questions a high degree However, performance article writing was yet at sufficient level. also found LLMs medical education requires caution. is because are still under development they can sometimes inaccurate or misleading information. It important carefully evaluate output before using them educational settings. Overall, have potential be valuable tools for more research needed improve accuracy better understand how used effectively

Language: Английский

Citations

25