Development of a Comprehensive Evaluation Scale for LLM-Powered Counseling Chatbots (CES-LCC) Using the eDelphi Method

Informatics, Год журнала: 2025, Номер 12(1), С. 33 - 33

Опубликована: Март 20, 2025

Background/Objectives: With advancements in Large Language Models (LLMs), counseling chatbots are becoming essential tools for delivering scalable and accessible mental health support. Traditional evaluation scales, however, fail to adequately capture the sophisticated capabilities of these systems, such as personalized interactions, empathetic responses, memory retention. This study aims design a robust comprehensive scale, Comprehensive Evaluation Scale LLM-Powered Counseling Chatbots (CES-LCC), using eDelphi method address this gap. Methods: A panel 16 experts psychology, artificial intelligence, human-computer interaction, digital therapeutics participated two iterative rounds. The process focused on refining dimensions items based qualitative quantitative feedback. Initial validation, conducted after assembling final version involved 49 participants CES-LCC evaluate an LLM-powered chatbot Self-Help Plus (SH+), Acceptance Commitment Therapy-based intervention stress management. Results: features 27 grouped into nine dimensions: Understanding Requests, Providing Helpful Information, Clarity Relevance Responses, Quality, Trust, Emotional Support, Guidance Direction, Memory, Overall Satisfaction. real-world validation revealed high internal consistency (Cronbach’s alpha = 0.94), although minor adjustments required specific dimensions, Responses. Conclusions: fills critical gap chatbots, offering standardized tool assessing their multifaceted capabilities. While preliminary results promising, further research is needed validate scale across diverse populations settings.

Язык: Английский

---

“Like Someone Is Paying Attention to You, Listening to You, and Guiding You”: Acceptability of a Mental Health Chatbot Among Caregivers of Adolescents Living With HIV

Journal of the International Association of Providers of AIDS Care (JIAPAC), Год журнала: 2025, Номер 24

Опубликована: Март 1, 2025

Background This study assessed the acceptability, among caregivers, of a mental health chatbot designed for adolescents living with HIV aged 10 to 19 years. Methods Fifteen caregivers interacted chatbot. Pre–post assessments and semistructured interviews evaluated acceptability. Data were analyzed using Framework Analysis approach. Results Caregivers 31 70 years found acceptable on individual, interpersonal, environmental levels. They appreciated educational content self-help tools, feeling would benefit them personally. also saw potential in improve communication their children, particularly during critical periods like diagnosis. Despite concerns about data costs or internet access, most viewed as an accessible supplement traditional services. Conclusion suggests that Peruvian was potentially benefiting caregivers’ health, enhancing caregiver–adolescent interactions, fostering better communication.

Язык: Английский

---

Evaluating the ability of artificial intelligence to predict suicide: A systematic review of reviews

Journal of Affective Disorders, Год журнала: 2025, Номер unknown

Опубликована: Апрель 1, 2025

Язык: Английский

---

Santé mentale au travail et intelligence artificielle : entre soutien psychologique et risque de dépendance

Enjeux numériques., Год журнала: 2025, Номер n° 29(1), С. 85 - 90

Опубликована: Апрель 10, 2025

---

Applications, Challenges, and Future Perspectives of Artificial Intelligence in Psychopharmacology, Psychological Disorders and Physiological Psychology: A Comprehensive Review

Journal of Pharmacy And Bioallied Sciences, Год журнала: 2025, Номер unknown

Опубликована: Апрель 29, 2025

A BSTRACT Artificial Intelligence (AI) is revolutionizing psychopharmacology and psychological research, enhancing diagnostics, treatments, accessibility. This review examines AI’s transformative role, applications, challenges, future directions in these fields. AI tools improve diagnostic accuracy by analyzing brain imaging, health records, behavioral data, enabling precise identification of disorders like depression schizophrenia. Personalized medicine, powered AI, predicts individual medication responses, minimizing side effects optimizing outcomes. Innovative therapies, such as virtual psychotherapists AI-assisted social robots, expand access to mental care underserved areas. psycho-radiology leverages imaging for tailored interventions treatment prediction, while wearable technologies digital phenotyping enable real-time monitoring early intervention. However, challenges persist, including data privacy, algorithmic bias, ethical dilemmas, regulatory hurdles, emphasizing the need robust governance. Future advancements include refining diagnostics through machine learning natural language processing integrating collaborative models holistic, personalized care. Ensuring ethical, transparent, culturally sensitive applications essential trust sustainability. aims explore potential highlighting its ability revolutionize addressing inherent adoption implementation.

Язык: Английский

---

Ethical Considerations in AI-Powered Health Communities to Promote Good Health and Wellbeing

Advances in computational intelligence and robotics book series, Год журнала: 2025, Номер unknown, С. 67 - 94

Опубликована: Апрель 25, 2025

The proliferation of Artificial Intelligence (AI) use not only has transformed the health communities but also raised concerns about its ethical implications. Employing a narrative review method, this chapter explores different types AI-powered to examine functionalities and purposes each community serves. This delves into identifying key considerations challenges associated by developers, experts participants in communities. Besides, it will existing regulations frameworks governing AI healthcare industry relation their effectiveness, potential gaps areas for improvement addressing identified concerns. Finally, propose recommendations development implementation synthesizing literature. Findings from offer insights best practices data governance.

Язык: Английский

---

LLM-based robot personality simulation and cognitive system

Scientific Reports, Год журнала: 2025, Номер 15(1)

Опубликована: Май 16, 2025

The inherence of personality in human-robot interaction enhances conversational dynamics and user experience. deployment Chat GPT-4 within a cognitive robot framework is designed by using state-space realization to emulate specific traits, incorporating elements emotion, motivation, visual attention, both short-term long-term memory. encoding retrieval memory are facilitated through document embedding techniques, while emotions generated based on predictions future events. This processes textual information, responding or initiating actions accordance with the configured settings processes. constancy effectiveness simulation have been compared human baseline validated via two assessments: International Personality Item Pool - Neuroticism, Extraversion Openness (IPIP-NEO) Big Five test. Our proposed model Kelly's role construct repertory, Cattell's 16 factors preferences, which analyzed validity subjects. Theory mind observed simulation, perform better second-order belief other agent improved theory dataset (ToMi dataset). Based methods, our robot, Mobi, enable chat its own personality, handle social conflicts understand user's intent. Such simulations can achieve high degree likeness, characterized conversations that flexible imbued intention.

Язык: Английский

---

Balancing risks and benefits: clinicians’ perspectives on the use of generative AI chatbots in mental healthcare

Frontiers in Digital Health, Год журнала: 2025, Номер 7

Опубликована: Май 29, 2025

Introduction The use of generative-AI chatbots has proliferated in mental health, to support both clients and clinicians across a range uses. This paper aimed explore the perspectives health regarding risks benefits integrating into landscape. Methods Twenty-three participated 45-minute virtual interview, which series open-ended scale-based questions were asked, demonstration chatbot's potential capabilities was presented. Results Participants highlighted several chatbots, such as their ability administer homework tasks, provide multilingual support, enhance accessibility affordability healthcare, offer access up-to-date research, increase engagement some client groups. However, they also identified risks, including lack regulation, data privacy concerns, chatbots' limited understanding backgrounds, for over-reliance on incorrect treatment recommendations, inability detect subtle communication cues, tone eye contact. There no significant finding suggest that participants viewed either or outweighing other. Moreover, chatbot not found influence whether favoured chatbots. Discussion Qualitative responses revealed balance is highly contextual, varying based case population group being served. study contributes important insights from critical stakeholders developers consider future iterations AI tools health.

Язык: Английский

---

Who do I trust more? Public Perception on AI-driven Mental health Interventions: A Survey research (Preprint)

JMIR Formative Research, Год журнала: 2024, Номер 8, С. e64380 - e64380

Опубликована: Сен. 30, 2024

Artificial intelligence (AI) has become increasingly important in health care, generating both curiosity and concern. With a doctor-patient ratio of 1:834 India, AI the potential to alleviate significant care burden. Public perception plays crucial role shaping attitudes that can facilitate adoption new technologies. Similarly, acceptance AI-driven mental interventions is determining their effectiveness widespread adoption. Therefore, it essential study public perceptions usage existing by exploring user experiences opinions on future applicability, particularly comparison traditional, human-based interventions.

Язык: Английский

---

The Efficacy of Conversational Artificial Intelligence in Rectifying the Theory of Mind and Autonomy Biases: Comparative Analysis (Preprint)

JMIR Mental Health, Год журнала: 2024, Номер 12, С. e64396 - e64396

Опубликована: Окт. 29, 2024

The increasing deployment of conversational artificial intelligence (AI) in mental health interventions necessitates an evaluation their efficacy rectifying cognitive biases and recognizing affect human-AI interactions. These are particularly relevant contexts as they can exacerbate conditions such depression anxiety by reinforcing maladaptive thought patterns or unrealistic expectations This study aimed to assess the effectiveness therapeutic chatbots (Wysa Youper) versus general-purpose language models (GPT-3.5, GPT-4, Gemini Pro) identifying user used constructed case scenarios simulating typical user-bot interactions examine how effectively address selected biases. assessed included theory-of-mind (anthropomorphism, overtrust, attribution) autonomy (illusion control, fundamental attribution error, just-world hypothesis). Each chatbot response was evaluated based on accuracy, quality, adherence behavioral therapy principles using ordinal scale ensure consistency scoring. To enhance reliability, responses underwent a double review process 2 scientists, followed secondary clinical psychologist specializing therapy, ensuring robust assessment across interdisciplinary perspectives. revealed that outperformed biases, overtrust bias, hypothesis. GPT-4 achieved highest scores all whereas bot Wysa scored lowest. Notably, bots showed more consistent accuracy adaptability addressing bias-related cues different contexts, suggesting broader flexibility handling complex patterns. In addition, recognition tasks, not only excelled but also demonstrated quicker adaptation subtle emotional nuances, outperforming 67% (4/6) tested shows that, while hold promise for support bias intervention, current capabilities limited. Addressing AI-human requires systems both rectify analyze integral human cognition, promoting precision empathy. findings reveal need improved simulated design provide adaptive, personalized reduce overreliance encourage independent coping skills. Future research should focus enhancing affective mechanisms ethical concerns mitigation data privacy safe, effective AI-based support.

Язык: Английский

---