Evaluation of LLMs accuracy and consistency in the registered dietitian exam through prompt engineering and knowledge retrieval

Scientific Reports, Journal Year: 2025, Volume and Issue: 15(1)

Published: Jan. 9, 2025

Large language models (LLMs) are fundamentally transforming human-facing applications in the health and well-being domains: boosting patient engagement, accelerating clinical decision-making, facilitating medical education. Although state-of-the-art LLMs have shown superior performance several conversational applications, evaluations within nutrition diet still insufficient. In this paper, we propose to employ Registered Dietitian (RD) exam conduct a standard comprehensive evaluation of LLMs, GPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro, assessing both accuracy consistency queries. Our includes 1050 RD questions encompassing topics proficiency levels. addition, for first time, examine impact Zero-Shot (ZS), Chain Thought (CoT), with Self Consistency (CoT-SC), Retrieval Augmented Prompting (RAP) on responses. findings revealed that while these obtained acceptable overall performance, their results varied considerably different prompts question domains. GPT-4o CoT-SC prompting outperformed other approaches, whereas Pro ZS recorded highest consistency. For 3.5, CoT improved accuracy, RAP was particularly effective answer Expert level questions. Consequently, choosing appropriate LLM technique, tailored specific domain, can mitigate errors potential risks chatbots.

Language: Английский

---

Large language models in patient education: a scoping review of applications in medicine

Frontiers in Medicine, Journal Year: 2024, Volume and Issue: 11

Published: Oct. 29, 2024

Large Language Models (LLMs) are sophisticated algorithms that analyze and generate vast amounts of textual data, mimicking human communication. Notable LLMs include GPT-4o by Open AI, Claude 3.5 Sonnet Anthropic, Gemini Google. This scoping review aims to synthesize the current applications potential uses in patient education engagement.

Language: Английский

---

From AI to the Table: A Systematic Review of ChatGPT’s Potential and Performance in Meal Planning and Dietary Recommendations

Dietetics, Journal Year: 2025, Volume and Issue: 4(1), P. 7 - 7

Published: Feb. 14, 2025

A balanced diet is crucial for preventing diseases and managing existing health conditions. ChatGPT as garnered attention from researchers, including nutrition scientists dietitians, an innovative tool personalized meal planning dietary recommendations. Objectives: The purpose of this study was to review scientific evidence on ChatGPT’s performance in providing plans generating Methods: This systematic conducted following the PRISMA guidelines. Keyword-based database searches were performed PubMed, Web Science, EBSCO, Embase. Inclusion criteria included (1) empirical studies (2) primary research Results: Twenty-three met inclusion criteria, comprising fourteen validation studies, five comparative four qualitative studies. Most reported that achieved satisfactory accuracy often indistinguishable human dietitians. One even outperformed However, limitations risks, such safety concerns a lack real-world implementation, also identified. Conclusions: shows promise relatively reliable recommendations, offering more accessible cost-effective solutions. Nevertheless, further are needed address its challenges.

Language: Английский

---

Assessing the Quality of ChatGPT’s Dietary Advice for College Students from Dietitians’ Perspectives

Nutrients, Journal Year: 2024, Volume and Issue: 16(12), P. 1939 - 1939

Published: June 19, 2024

Background: As ChatGPT becomes a primary information source for college students, its performance in providing dietary advice is under scrutiny. This study assessed ChatGPT’s nutritional guidance to students. Methods: on was evaluated by 30 experienced dietitians and using an objective nutrition literacy (NL) test. The were recruited assess the quality of advice, including NL achievement response quality. Results: results indicate that varies across scenarios suboptimal achieving with full rates from 7.50% 37.56%. While responses excelled readability, they lacked understandability, practicality, completeness. In test, showed 84.38% accuracy rate, surpassing level Taiwanese top concern among dietitians, cited 52 times 242 feedback entries, “response lacks thoroughness or rigor, leading misunderstandings misuse”. Despite potential as supplementary educational tool, significant gaps must be addressed, especially detailed inquiries. Conclusion: highlights need improved AI approaches suggests developing teaching guides usage instructions train students support dietitians.

Language: Английский

---

Diet Quality and Caloric Accuracy in AI-Generated Diet Plans: A Comparative Study Across Chatbots

Nutrients, Journal Year: 2025, Volume and Issue: 17(2), P. 206 - 206

Published: Jan. 7, 2025

Background/Objectives: With the rise of artificial intelligence (AI) in nutrition and healthcare, AI-driven chatbots are increasingly recognised as potential tools for generating personalised diet plans. This study aimed to evaluate capabilities three popular chatbots-Gemini, Microsoft Copilot, ChatGPT 4.0-in designing weight-loss plans across varying caloric levels genders. Methods: comparative assessed quality meal generated by a calorie range 1400-1800 kcal, using identical prompts tailored male female profiles. The Diet Quality Index-International (DQI-I) was used dimensions variety, adequacy, moderation, balance. Caloric accuracy analysed calculating percentage deviations from requested targets categorising discrepancies into defined ranges. Results: All achieved high total DQI-I scores (DQI-I > 70), demonstrating satisfactory overall quality. However, balance sub-scores related macronutrient fatty acid distributions were consistently lowest, showing critical limitation AI algorithms. 4.0 exhibited highest precision adherence, while Gemini showed greater variability, with over 50% its deviating target more than 20%. Conclusions: show significant promise nutritionally adequate diverse Nevertheless, gaps achieving optimal emphasise need algorithmic refinement. While these have revolutionise offering precise inclusive dietary solutions, they should enhance rather replace expertise dietetic professionals.

Language: Английский

---

Prospects and perils of ChatGPT in diabetes

World Journal of Diabetes, Journal Year: 2025, Volume and Issue: 16(3)

Published: Jan. 20, 2025

ChatGPT, a popular large language model developed by OpenAI, has the potential to transform management of diabetes mellitus. It is conversational artificial intelligence trained on extensive datasets, although not specifically health-related. The development and core components ChatGPT include neural networks machine learning. Since current yet diabetes-related it limitations such as risk inaccuracies need for human supervision. Nevertheless, aid in patient engagement, medical education, clinical decision support. In management, can contribute personalized dietary guidelines, providing emotional Specifically, being tested scenarios assessment obesity, screening diabetic retinopathy, provision guidelines ketoacidosis. Ethical legal considerations are essential before be integrated into healthcare. Potential concerns relate data privacy, accuracy responses, maintenance patient-doctor relationship. Ultimately, while models hold immense revolutionize care, one needs weigh their limitations, ethical implications, integration promises future proactive, personalized, patient-centric care management.

Language: Английский

---

Developing a Public-Friendly Sarcopenia Guideline Framework: A model integrating ChatGPT, human experts, and Google (Preprint)

Published: Feb. 26, 2025

The highly academic and complex nature of the current expert consensus on sarcopenia may limit public awareness understanding this disease. This study aims to develop a more public-friendly framework for future guidelines by utilizing model that incorporates Google, Chat Generative Pre-Trained Transformer 4.0 (ChatGPT 4.0), experts. first step human-centered involved identifying most popular questions using “People Also Ask” feature Google. In second step, these were input into ChatGPT generate answers, while experts reviewed existing provide answers. third assessed relevance questions. They compared analyzed responses from reviews, offering suggestions guidelines. Finally, results analysis used as prompts We scored identified 9 was inadequate in answering consensus, diagnostic criteria received highest emphasis, scoring 25.5 points (out 40 points), followed exercise nutrition, each with 13 points. contrast, topics such prognosis symptoms addressed less, score only 5.5 4.5, respectively. integrates 4.0, human experts, Google created guidelines, addressing inadequately covered topics. combines has great potential creating accessible clinical none

Language: Английский

---

Generative artificial intelligence ChatGPT in clinical nutrition — Advances and challenges

Nutrición Hospitalaria, Journal Year: 2025, Volume and Issue: unknown

Published: Jan. 1, 2025

ChatGPT and other artificial intelligence (AI) tools can modify nutritional management in clinical settings. These technologies, based on machine learning deep learning, enable the identification of risks, proposal personalized interventions, monitoring patient progress using data extracted from records. excels areas such as assessment by calculating caloric needs suggesting nutrient-rich foods, diagnosis, identifying issues with technical terminology. In it offers dietary educational strategies but lacks critical abilities interpreting non-verbal cues or performing physical examinations. Recent studies indicate that achieves high accuracy questions related to guidelines shows deficiencies integrating multiple medical conditions ensuring meal plans. Additionally, generated plans may exhibit significant deviations imbalances micronutrients vitamin D B12. Despite its limitations, this AI has potential complement practice improving accessibility personalization care. However, effective implementation requires professional supervision, integration existing healthcare systems, constant updates databases. conclusion, while does not replace nutrition experts, serve a valuable tool optimize education our patiens, always under guidance trained professionals.

Language: Английский

---

Artificial Intelligence‐Based Diets: A Role in the Nutritional Treatment of Metabolic Dysfunction‐Associated Steatotic Liver Disease?

Journal of Human Nutrition and Dietetics, Journal Year: 2025, Volume and Issue: 38(2)

Published: Feb. 27, 2025

ABSTRACT Background Metabolic dysfunction‐associated steatotic liver disease (MASLD) is a growing global health concern. Effective management of this condition relies heavily on lifestyle modifications and dietary interventions. In study, we sought to evaluate the plans for MASLD generated by ChatGPT (GPT‐4o) according current guideline recommendations. Methods was used create single‐day meal 48 simulated patients with MASLD, tailored individual characteristics such as age, gender, height, weight transient elastography parameters. The were assessed appropriateness disease‐specific guidelines. Results mean energy content menus planned 1596.9 ± 141.5 kcal accuracy 91.3 11.0%, fibre 22.0 0.6 g 88.1 2.5%. However, they exhibited elevated levels protein, fat saturated acids. Conversely, carbohydrate lower. recommended loss obese but did not extend advice normal‐weight overweight individuals. Notably, recommendations Mediterranean diet physical activity absent. Conclusions shows potential in developing management. discrepancies macronutrient distributions omission key evidence‐based highlight need further refinement. To enhance effectiveness AI tools recommendations, alignment established guidelines must be improved.

Language: Английский

---

ChatGPT-4o and 4o1 Preview as Dietary Support Tools in a Real-World Medicated Obesity Program: A Prospective Comparative Analysis

Healthcare, Journal Year: 2025, Volume and Issue: 13(6), P. 647 - 647

Published: March 16, 2025

Background/Objectives: Clinicians are becoming increasingly interested in the use of large language models (LLMs) obesity services. While most experts agree that LLM integration would increase access to care and its efficiency, many remain skeptical their scientific accuracy capacity convey human empathy. Recent studies have shown ChatGPT-3 capable emulating dietitian responses a range basic dietary questions. Methods: This study compared two ChatGPT-4o those from dietitians across 10 complex questions (5 broad; 5 narrow) derived patient–clinician interactions within real-world medicated digital weight loss service. Results: Investigators found neither nor Chat GPT-4o1 preview were statistically outperformed (p < 0.05) by on any study’s The same finding was made when scores aggregated ten following four individual criteria: correctness, comprehensibility, empathy/relatability, actionability. Conclusions: These results provide preliminary evidence advanced LLMs may be able play significant supporting role Research other contexts is needed before stronger conclusions about lifestyle coaching whether such initiatives access.

Language: Английский

---