Evaluating Diagnostic Accuracy and Treatment Efficacy in Mental Health: A Comparative Analysis of Large Language Model Tools and Mental Health Professionals

European Journal of Investigation in Health Psychology and Education, Год журнала: 2025, Номер 15(1), С. 9 - 9

Опубликована: Янв. 18, 2025

Large language models (LLMs) offer promising possibilities in mental health, yet their ability to assess disorders and recommend treatments remains underexplored. This quantitative cross-sectional study evaluated four LLMs (Gemini 2.0 Flash Experimental), Claude (Claude 3.5 Sonnet), ChatGPT-3.5, ChatGPT-4) using text vignettes representing conditions such as depression, suicidal ideation, early chronic schizophrenia, social phobia, PTSD. Each model’s diagnostic accuracy, treatment recommendations, predicted outcomes were compared with norms established by health professionals. Findings indicated that for certain conditions, including depression PTSD, like ChatGPT-4 achieved higher accuracy human However, more complex cases, LLM performance varied, achieving only 55% while other professionals performed better. tended suggest a broader range of proactive treatments, whereas recommended targeted psychiatric consultations specific medications. In terms outcome predictions, generally optimistic regarding full recovery, especially treatment, lower recovery rates partial rates, particularly untreated cases. While range, conservative highlight the need professional oversight. provide valuable support diagnostics planning but cannot replace discretion.

Язык: Английский

---

Réalité virtuelle, intelligence artificielle et métaverses en santé mentale

Soins Cadres, Год журнала: 2025, Номер unknown

Опубликована: Фев. 1, 2025

---

Supervised machine learning compared to large language models for identifying functional seizures from medical records

Epilepsia, Год журнала: 2025, Номер unknown

Опубликована: Фев. 17, 2025

Abstract Objective The Functional Seizures Likelihood Score (FSLS) is a supervised machine learning–based diagnostic score that was developed to differentiate functional seizures (FS) from epileptic (ES). In contrast this targeted approach, large language models (LLMs) can identify patterns in data for which they were not specifically trained. To evaluate the relative benefits of each we compared performance FSLS two LLMs: ChatGPT and GPT‐4. Methods total, 114 anonymized cases constructed based on patients with documented FS, ES, mixed ES or physiologic seizure‐like events (PSLEs). Text‐based presented three sequential prompts LLMs, showing history present illness (HPI), electroencephalography (EEG) results, neuroimaging results. We accuracy (number correct predictions/number cases) area under receiver‐operating characteristic (ROC) curves (AUCs) LLMs using mixed‐effects logistic regression. Results 74% (95% confidence interval [CI] 65%–82%) AUC 85% CI 77%–92%). GPT‐4 superior both ( p <.001), an 77%–91%) 87% 79%–95%). Cohen's kappa between 40% (fair). provided different predictions days when same note 33% patients, LLM's self‐rated certainty moderately correlated observed variability (Spearman's rho 2 : 30% [fair, ChatGPT] 63% [substantial, GPT‐4]). Significance Both identified substantial subset FS clinical history. fair agreement highlights differently structured score. inconsistency LLMs' across incomplete insight into their own consistency concerning. This comparison cautions about how learning artificial intelligence could practice.

Язык: Английский

---

Empowering pediatric, adolescent, and young adult patients with cancer utilizing generative AI chatbots to reduce psychological burden and enhance treatment engagement: a pilot study

Frontiers in Digital Health, Год журнала: 2025, Номер 7

Опубликована: Фев. 25, 2025

Pediatric and adolescent/young adult (AYA) cancer patients face profound psychological challenges, exacerbated by limited access to continuous mental health support. While conventional therapeutic interventions often follow structured protocols, the potential of generative artificial intelligence (AI) chatbots provide conversational support remains unexplored. This study evaluates feasibility impact AI in alleviating distress enhancing treatment engagement this vulnerable population. Two age-appropriate chatbots, leveraging GPT-4, were developed natural, empathetic conversations without protocols. Five pediatric AYA participated a two-week intervention, engaging with via messaging platform. Pre- post-intervention anxiety stress levels self-reported, usage patterns analyzed assess chatbots' effectiveness. Four out five participants reported significant reductions post-intervention. Participants engaged chatbot every 2-3 days, sessions lasting approximately 10 min. All noted improved motivation, 80% disclosing personal concerns they had not shared healthcare providers. The 24/7 availability particularly benefited experiencing nighttime anxiety. pilot demonstrates complement traditional services addressing unmet needs patients. findings suggest these tools can serve as accessible, systems. Further large-scale studies are warranted validate promising results.

Язык: Английский

---

Automated Digital Safety Planning Interventions for Young Adults: Qualitative Study Using Online Co-design Methods

JMIR Formative Research, Год журнала: 2025, Номер 9, С. e69602 - e69602

Опубликована: Фев. 26, 2025

Background Young adults in the United States are experiencing accelerating rates of suicidal thoughts and behaviors but have lowest formal mental health care. Digital suicide prevention interventions potential to increase access care by circumventing attitudinal structural barriers that prevent These tools should be designed collaboration with young who lived experience suicide-related optimize acceptability use. Objective This study aims identify needs, preferences, features for an automated SMS text messaging–based safety planning service support self-management among adults. Methods We enrolled 30 (age 18-24 years) recent participate asynchronous remote focus groups via online private forum. Participants responded researcher-posted prompts were encouraged reply fellow participants—creating a threaded digital conversation. Researcher-posted centered on participants’ experiences thought behavior-related coping, planning, technologies behavior self-management. Focus group transcripts analyzed using thematic analysis extract key feature considerations tool. Results adult participants indicated message–based intervention must meet their needs 2 ways. First, empowering them manage symptoms own acquiring effective coping skills. Second, leveraging adults’ existing social connections. also shared 3 technological intervention: (1) transparency about how functions, kinds actions it does not take, limits confidentiality, role human oversight within program; (2) strong privacy practices—data security around content data created would maintained used was extremely important given sensitive nature data; (3) usability, convenience, accessibility particularly participants—this includes having approachable engaging message tone, customizable delivery options (eg, length, number, focus), straightforward menu navigation. highlighted specific could core skill acquisition self-tracking, idea generation, reminders). Conclusions Engaging design process tool revealed critical addressed if is effectively expand evidence-based reach people at risk behaviors. Specifically, building skillfulness cope crises, deepening interpersonal connections, system transparency, privacy.

Язык: Английский

---

Leveraging large language models to assist philosophical counseling: prospective techniques, value, and challenges

Humanities and Social Sciences Communications, Год журнала: 2025, Номер 12(1)

Опубликована: Март 6, 2025

Язык: Английский

---

A Deep Learning-Based Technique for Detection of Generalized Anxiety Disorder using CNN and ResNet-like Approach

Arabian Journal for Science and Engineering, Год журнала: 2025, Номер unknown

Опубликована: Март 17, 2025

Язык: Английский

---

Comparative analysis of a standard (GPT-4o) and reasoning-enhanced (o1 pro) large language model on complex clinical questions from the Japanese orthopaedic board examination

Journal of Orthopaedic Science, Год журнала: 2025, Номер unknown

Опубликована: Апрель 1, 2025

Язык: Английский

---

Extracting Pulmonary Embolism Diagnoses From Radiology Impressions Using GPT-4o: Large Language Model Evaluation Study

JMIR Medical Informatics, Год журнала: 2025, Номер 13, С. e67706 - e67706

Опубликована: Апрель 9, 2025

Background Pulmonary embolism (PE) is a critical condition requiring rapid diagnosis to reduce mortality. Extracting PE diagnoses from radiology reports manually time-consuming, highlighting the need for automated solutions. Advances in natural language processing, especially transformer models like GPT-4o, offer promising tools improve diagnostic accuracy and workflow efficiency clinical settings. Objective This study aimed develop an automatic extraction system using GPT-4o extract report impressions, enhancing decision-making efficiency. Methods In total, 2 approaches were developed evaluated: fine-tuned Clinical Longformer as baseline model GPT-4o-based extractor. Longformer, encoder-only model, was chosen its robustness text classification tasks, particularly on smaller scales. decoder-only instruction-following LLM, selected advanced understanding capabilities. The evaluate GPT-4o’s ability perform compared Longformer. trained dataset of 1000 impressions validated separate set 200 samples, while extractor same 200-sample set. Postdeployment performance further assessed additional operational records efficacy real-world setting. Results outperformed metrics, achieving sensitivity 1.0 (95% CI 1.0-1.0; Wilcoxon test, P<.001) F1-score 0.975 0.9495-0.9947; across validation dataset. evaluations also showed strong deployed with 1.0-1.0), specificity 0.94 0.8913-0.9804), 0.97 0.9479-0.9908). high level supports reduction manual review, streamlining workflows improving precision. Conclusions provides effective solution reports, offering reliable tool that aids timely accurate decision-making. approach has potential significantly patient outcomes by expediting treatment pathways conditions PE.

Язык: Английский

---

Describing the Framework for AI Tool Assessment in Mental Health and Applying It to a Generative AI Obsessive-Compulsive Disorder Platform: Tutorial

JMIR Formative Research, Год журнала: 2024, Номер 8, С. e62963 - e62963

Опубликована: Окт. 18, 2024

As artificial intelligence (AI) technologies occupy a bigger role in psychiatric and psychological care become the object of increased research attention, industry investment, public scrutiny, tools for evaluating their clinical, ethical, user-centricity standards have essential. In this paper, we first review history rating systems used to evaluate AI mental health interventions. We then describe recently introduced Framework Tool Assessment Mental Health (FAITA-Mental Health), whose scoring system allows users grade platforms on key domains, including credibility, user experience, crisis management, agency, equity, transparency. Finally, demonstrate use FAITA-Mental scale by systematically applying it OCD Coach, generative tool readily available ChatGPT store designed help manage symptoms obsessive-compulsive disorder. The results offer insights into utility limitations when applied “real-world” space, suggesting that framework effectively identifies strengths gaps AI-driven tools, particularly areas such as acute management. also highlight need stringent guide integration manner is not only effective but safe protective users’ rights welfare.

Язык: Английский

---