Competency of Large Language Models in Evaluating Appropriate Responses to Suicidal Ideation: Comparative Study (Preprint) DOI
Ryan McBain, Jonathan Cantor, Li Ang Zhang

и другие.

Опубликована: Окт. 23, 2024

BACKGROUND With suicide rates in the United States at an all-time high, individuals experiencing suicidal ideation are increasingly turning to large language models (LLMs) for guidance and support. OBJECTIVE The objective of this study was assess competency 3 widely used LLMs distinguish appropriate versus inappropriate responses when engaging who exhibit ideation. METHODS This observational, cross-sectional evaluated revised Suicidal Ideation Response Inventory (SIRI-2) generated by ChatGPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro. Data collection analyses were conducted July 2024. A common training module mental health professionals, SIRI-2 provides 24 hypothetical scenarios which a patient exhibits depressive symptoms ideation, followed two clinician responses. Clinician scored from –3 (highly inappropriate) +3 appropriate). All provided with standardized set instructions rate We compared LLM those expert suicidologists, conducting linear regression converting z scores identify outliers (z score&gt;1.96 or &lt;–1.96; <i>P</i>&lt;0.05). Furthermore, we final produced professionals prior studies. RESULTS rated as more than ratings suicidologists. item-level mean difference 0.86 ChatGPT (95% CI 0.61-1.12; <i>P</i>&lt;.001), 0.61 0.41-0.81; 0.73 0.35-1.11; <i>P</i>&lt;.001). In terms scores, 19% (9 48) Similarly, 11% (5 Additionally, 36% (17 score 45.7, roughly equivalent master’s level counselors 36.7, exceeding performance after intervention skills training. 54.5, untrained K-12 school staff. CONCLUSIONS Current versions major demonstrated upward bias their evaluations ideation; however, 2 performed exceeded professionals.

Язык: Английский

Is Artificial Intelligence the Next Co-Pilot for Primary Care in Diagnosing and Recommending Treatments for Depression? DOI Creative Commons
Inbar Levkovich

Medical Sciences, Год журнала: 2025, Номер 13(1), С. 8 - 8

Опубликована: Янв. 11, 2025

Depression poses significant challenges to global healthcare systems and impacts the quality of life individuals their family members. Recent advancements in artificial intelligence (AI) have had a transformative impact on diagnosis treatment depression. These innovations potential significantly enhance clinical decision-making processes improve patient outcomes settings. AI-powered tools can analyze extensive data—including medical records, genetic information, behavioral patterns—to identify early warning signs depression, thereby enhancing diagnostic accuracy. By recognizing subtle indicators that traditional assessments may overlook, these enable providers make timely precise decisions are crucial preventing onset or escalation depressive episodes. In terms treatment, AI algorithms assist personalizing therapeutic interventions by predicting effectiveness various approaches for individual patients based unique characteristics history. This includes recommending tailored plans consider patient’s specific symptoms. Such personalized strategies aim optimize overall efficiency healthcare. theoretical review uniquely synthesizes current evidence applications primary care depression management, offering comprehensive analysis both personalization capabilities. Alongside advancements, we also address conflicting findings field presence biases necessitate important limitations.

Язык: Английский

Процитировано

1
Evaluating Diagnostic Accuracy and Treatment Efficacy in Mental Health: A Comparative Analysis of Large Language Model Tools and Mental Health Professionals DOI Creative Commons
Inbar Levkovich

European Journal of Investigation in Health Psychology and Education, Год журнала: 2025, Номер 15(1), С. 9 - 9

Опубликована: Янв. 18, 2025

Large language models (LLMs) offer promising possibilities in mental health, yet their ability to assess disorders and recommend treatments remains underexplored. This quantitative cross-sectional study evaluated four LLMs (Gemini 2.0 Flash Experimental), Claude (Claude 3.5 Sonnet), ChatGPT-3.5, ChatGPT-4) using text vignettes representing conditions such as depression, suicidal ideation, early chronic schizophrenia, social phobia, PTSD. Each model’s diagnostic accuracy, treatment recommendations, predicted outcomes were compared with norms established by health professionals. Findings indicated that for certain conditions, including depression PTSD, like ChatGPT-4 achieved higher accuracy human However, more complex cases, LLM performance varied, achieving only 55% while other professionals performed better. tended suggest a broader range of proactive treatments, whereas recommended targeted psychiatric consultations specific medications. In terms outcome predictions, generally optimistic regarding full recovery, especially treatment, lower recovery rates partial rates, particularly untreated cases. While range, conservative highlight the need professional oversight. provide valuable support diagnostics planning but cannot replace discretion.

Язык: Английский

Процитировано

1
Fine-Tuning LLMs for Specialized Use Cases DOI Creative Commons
D. M. Anisuzzaman, Jeffrey G. Malins, Paul A. Friedman

и другие.

Mayo Clinic Proceedings Digital Health, Год журнала: 2024, Номер 3(1), С. 100184 - 100184

Опубликована: Ноя. 29, 2024

Large language models (LLMs) are a type of artificial intelligence, which operate by predicting and assembling sequences words that statistically likely to follow from given text input. With this basic ability, LLMs able answer complex questions extremely instructions. Products created using such as ChatGPT OpenAI Claude Anthropic have huge amount traction user engagements revolutionized the way we interact with technology, bringing new dimension human-computer interaction. Fine-tuning is process in pretrained model, an LLM, further trained on custom data set adapt it for specialized tasks or domains. In review, outline some major methodologic approaches techniques can be used fine-tune use cases enumerate general steps required carrying out LLM fine-tuning. We then illustrate few these describing several specific fine-tuning across medical subspecialties. Finally, close consideration benefits limitations associated cases, emphasis concerns field medicine.

Язык: Английский

Процитировано

7
From statistics to deep learning: Using large language models in psychiatric research DOI Creative Commons
Yining Hua,

Andrew Beam,

Lori B. Chibnik

и другие.

International Journal of Methods in Psychiatric Research, Год журнала: 2025, Номер 34(1)

Опубликована: Янв. 8, 2025

Abstract Background Large Language Models (LLMs) hold promise in enhancing psychiatric research efficiency. However, concerns related to bias, computational demands, data privacy, and the reliability of LLM‐generated content pose challenges. Gap Existing studies primarily focus on clinical applications LLMs, with limited exploration their potentials broader research. Objective This study adopts a narrative review format assess utility LLMs research, beyond settings, focusing effectiveness literature review, design, subject selection, statistical modeling, academic writing. Implication provides clearer understanding how can be effectively integrated process, offering guidance mitigating associated risks maximizing potential benefits. While for advancing careful oversight, rigorous validation, adherence ethical standards are crucial such as privacy concerns, issues, thereby ensuring effective responsible use improving

Язык: Английский

Процитировано

0
Large language models and psychiatry DOI
Graziella Orrù, Giulia Melis, Giuseppe Sartori

и другие.

International Journal of Law and Psychiatry, Год журнала: 2025, Номер 101, С. 102086 - 102086

Опубликована: Фев. 27, 2025

Язык: Английский

Процитировано

0
Feasibility of using ChatGPT to generate exposure hierarchies for treating obsessive compulsive disorder DOI Creative Commons
Emily E. Bernstein, Adam C. Jaroszewski, Ryan J. Jacoby

и другие.

Behavior Therapy, Год журнала: 2025, Номер unknown

Опубликована: Март 1, 2025

Язык: Английский

Процитировано

0
GPT-4 generated psychological reports in psychodynamic perspective: a pilot study on quality, risk of hallucination and client satisfaction DOI Creative Commons
Namwoo Kim,

Ji-Seon Lee,

Sung Park

и другие.

Frontiers in Psychiatry, Год журнала: 2025, Номер 16

Опубликована: Март 19, 2025

Background Recently, there have been active proposals on how to utilize large language models (LLMs) in the fields of psychiatry and counseling. It would be interesting develop programs with LLMs that generate psychodynamic assessments help individuals gain insights about themselves, evaluate features such services. However, studies this subject are rare. This pilot study aims quality, risk hallucination (incorrect AI-generated information), client satisfaction psychological reports generated by GPT-4. Methods The report comprised five components: formulation, psychopathology, parental influence, defense mechanisms, strengths. Participants were recruited from distressed repetitive interpersonal issues. was conducted three steps: 1) Questions provided participants, designed create formulations: 14 questions GPT for inferring formulations, while 6 fixed focused participants’ relationship their parents. A total 20 provided. Using responses these questions, GPT-4 reports. 2) Seven professors different university hospitals evaluated quality hallucinations reading only, without meeting participants. assessment compared those inferred experts. 3) All using self-report questionnaires based a Likert scale developed study. Results 10 participants recruited, average age 32 years. median response indicated all components similar level assessed as ranging unlikely minor. According evaluation, agreed is clearly understandable, insightful, credible, useful, satisfying, recommendable. Conclusion suggests possibility artificial intelligence could assist users providing interpretations.

Язык: Английский

Процитировано

0
Competency of Large Language Models in Evaluating Appropriate Responses to Suicidal Ideation: Comparative Study DOI Creative Commons
Ryan McBain, Jonathan Cantor, Li Ang Zhang

и другие.

Journal of Medical Internet Research, Год журнала: 2025, Номер 27, С. e67891 - e67891

Опубликована: Март 5, 2025

With suicide rates in the United States at an all-time high, individuals experiencing suicidal ideation are increasingly turning to large language models (LLMs) for guidance and support. The objective of this study was assess competency 3 widely used LLMs distinguish appropriate versus inappropriate responses when engaging who exhibit ideation. This observational, cross-sectional evaluated revised Suicidal Ideation Response Inventory (SIRI-2) generated by ChatGPT-4o, Claude 3.5 Sonnet, Gemini 1.5 Pro. Data collection analyses were conducted July 2024. A common training module mental health professionals, SIRI-2 provides 24 hypothetical scenarios which a patient exhibits depressive symptoms ideation, followed two clinician responses. Clinician scored from -3 (highly inappropriate) +3 appropriate). All provided with standardized set instructions rate We compared LLM those expert suicidologists, conducting linear regression converting z scores identify outliers (z score>1.96 or <-1.96; P<0.05). Furthermore, we final produced professionals prior studies. rated as more than ratings suicidologists. item-level mean difference 0.86 ChatGPT (95% CI 0.61-1.12; P<.001), 0.61 0.41-0.81; 0.73 0.35-1.11; P<.001). In terms scores, 19% (9 48) Similarly, 11% (5 Additionally, 36% (17 score 45.7, roughly equivalent master's level counselors 36.7, exceeding performance after intervention skills training. 54.5, untrained K-12 school staff. Current versions major demonstrated upward bias their evaluations ideation; however, 2 performed exceeded professionals.

Язык: Английский

Процитировано

0
Clinical decision support systems in mental health: A scoping review of health professionals’ experiences DOI Creative Commons

Fangziyun Tong,

Reeva Lederman, Simon D’Alfonso

и другие.

International Journal of Medical Informatics, Год журнала: 2025, Номер unknown, С. 105881 - 105881

Опубликована: Март 1, 2025

Clinical decision support systems (CDSSs) have the potential to assist health professionals in making informed and cost-effective clinical decisions while reducing medical errors. However, compared physical health, CDSSs been less investigated within mental context. In particular, despite being primary users of CDSSs, few studies explored their experiences and/or views on these systems. Furthermore, we are not aware any reviews specifically focusing this topic. To address gap, conducted a scoping review map state art examining from perspectives professionals. review, following Preferred Reporting Items for Systematic Meta-Analyses (PRISMA) guideline, systematically searched relevant literature two databases, PubMed PsycINFO. We identified 23 articles describing 20 Through synthesis qualitative findings, four key barriers three facilitators adoption were identified. Although did synthesize quantitative findings due heterogeneity results methodologies, emphasize issue lack valid methods evaluating best our knowledge, is first professionals' CDSSs. adopting highlighted need standardizing research evaluate space.

Язык: Английский

Процитировано

0
Teachers vs. Chat GPT: identifying and naming acts of online sexual abuse DOI Creative Commons
Michal Dolev-Cohen, Nicola Henry

Information Communication & Society, Год журнала: 2025, Номер unknown, С. 1 - 19

Опубликована: Апрель 17, 2025

Язык: Английский

Процитировано

0