Evaluating the quality of medical content on YouTube using large language models DOI Creative Commons
Mahmoud I. Khalil, Fatma Mohamed, Abdulhadi Shoufan

et al.

Scientific Reports, Journal Year: 2025, Volume and Issue: 15(1)

Published: March 22, 2025

YouTube has become a dominant source of medical information and health-related decision-making. Yet, many videos on this platform contain inaccurate or biased information. Although expert reviews could help mitigate situation, the vast number daily uploads makes solution impractical. In study, we explored potential Large Language Models (LLMs) to assess quality content YouTube. We collected set previously evaluated by experts prompted twenty models rate their using DISCERN instrument. then analyzed inter-rater agreement between language models' experts' ratings Brennan–Prediger's (BP) Kappa. found that LLMs exhibited wide range agreements with (ranging from −1.10 0.82). All tended give higher scores than human experts. The individual questions be lower, some showing significant disagreement Including scoring guidelines in prompt improved model performance. conclude are capable evaluating videos. If used as stand-alone systems embedded into traditional recommender systems, these can issue online

Language: Английский

Promises and challenges of generative artificial intelligence for human learning DOI
Lixiang Yan, Samuel Greiff, Ziwen Teuber

et al.

Nature Human Behaviour, Journal Year: 2024, Volume and Issue: 8(10), P. 1839 - 1850

Published: Oct. 22, 2024

Language: Английский

Citations

43

The promise and challenges of generative AI in education DOI Creative Commons
Michail N. Giannakos, Roger Azevedo, Peter Brusilovsky

et al.

Behaviour and Information Technology, Journal Year: 2024, Volume and Issue: unknown, P. 1 - 27

Published: Sept. 2, 2024

Generative artificial intelligence (GenAI) tools, such as large language models (LLMs), generate natural and other types of content to perform a wide range tasks. This represents significant technological advancement that poses opportunities challenges educational research practice. commentary brings together contributions from nine experts working in the intersection learning technology presents critical reflections on opportunities, challenges, implications related GenAI technologies context education. In commentary, it is acknowledged GenAI's capabilities can enhance some teaching practices, design, regulation learning, automated content, feedback, assessment. Nevertheless, we also highlight its limitations, potential disruptions, ethical consequences, misuses. The identified avenues for further include development new insights into roles human play, strong continuous evidence, human-centric design technology, necessary policy, support competence mechanisms. Overall, concur with general skeptical optimism about use tools LLMs Moreover, danger hastily adopting education without deep consideration efficacy, ecosystem-level implications, ethics, pedagogical soundness practices.

Language: Английский

Citations

32

Do teachers spot AI? Evaluating the detectability of AI-generated texts among student essays DOI Creative Commons
Johanna Fleckenstein, Jennifer Meyer, Thorben Jansen

et al.

Computers and Education Artificial Intelligence, Journal Year: 2024, Volume and Issue: 6, P. 100209 - 100209

Published: Jan. 25, 2024

The potential application of generative artificial intelligence (AI) in schools and universities poses great challenges, especially for the assessment students' texts. Previous research has shown that people generally have difficulty distinguishing AI-generated from human-written texts; however, ability teachers to identify an text among student essays not yet been investigated. Here we show two experimental studies novice (N = 89) experienced 200) could texts generated by ChatGPT student-written However, there are some indications more made differentiated accurate judgments. Furthermore, both groups were overconfident their Effects real assumed source on quality heterogeneous. Our findings demonstrate with relatively little prompting, current AI can generate detectable teachers, which a challenge grading essays. study provides empirical evidence debate regarding exam strategies light latest technological developments.

Language: Английский

Citations

24

Does ChatGPT enhance student learning? A systematic review and meta-analysis of experimental studies DOI Creative Commons
Ruiqi Deng,

Mingyu Jiang,

Xiao Yu

et al.

Computers & Education, Journal Year: 2024, Volume and Issue: unknown, P. 105224 - 105224

Published: Dec. 1, 2024

Language: Английский

Citations

12

Synergizing collaborative writing and AI feedback: An investigation into enhancing L2 writing proficiency in wiki-based environments DOI Creative Commons
Watcharapol Wiboolyasarin, Kanokpan Wiboolyasarin,

Kanpabhat Suwanwihok

et al.

Computers and Education Artificial Intelligence, Journal Year: 2024, Volume and Issue: 6, P. 100228 - 100228

Published: April 24, 2024

Language: Английский

Citations

11

A language model-powered simulated patient with automated feedback for history taking: Prospective study (Preprint) DOI Creative Commons
Friederike Holderried, Christian Stegemann–Philipps, Anne Herrmann–Werner

et al.

JMIR Medical Education, Journal Year: 2024, Volume and Issue: 10, P. e59213 - e59213

Published: June 27, 2024

Although history taking is fundamental for diagnosing medical conditions, teaching and providing feedback on the skill can be challenging due to resource constraints. Virtual simulated patients web-based chatbots have thus emerged as educational tools, with recent advancements in artificial intelligence (AI) such large language models (LLMs) enhancing their realism potential provide feedback.

Language: Английский

Citations

11

The life cycle of large language models in education: A framework for understanding sources of bias DOI
Jin-Sook Lee, Yann Hicke, Renzhe Yu

et al.

British Journal of Educational Technology, Journal Year: 2024, Volume and Issue: 55(5), P. 1982 - 2002

Published: July 12, 2024

Abstract Large language models (LLMs) are increasingly adopted in educational contexts to provide personalized support students and teachers. The unprecedented capacity of LLM‐based applications understand generate natural can potentially improve instructional effectiveness learning outcomes, but the integration LLMs education technology has renewed concerns over algorithmic bias, which may exacerbate inequalities. Building on prior work that mapped traditional machine life cycle, we a framework LLM cycle from initial development customizing pre‐trained for various settings. We explain each step identify potential sources bias arise context education. discuss why current measures fail transfer LLM‐generated text (eg, tutoring conversations) because encodings high‐dimensional, there be multiple correct responses, tailoring responses pedagogically desirable rather than unfair. proposed clarifies complex nature provides practical guidance their evaluation promote equity. Practitioner notes What is already known about this topic (ML) focus predicting labels well understood. Biases enter ML at points methods measure mitigate these biases have been developed tested. other forms generative artificial intelligence (GenAI) technologies (EdTech), approaches not specific domain paper adds A holistic perspective with domain‐specific examples highlight opportunities challenges incorporating understanding (NLU) generation (NLG) into EdTech. Potential identified discussed where expect harms students, teachers, users GenAI education, guide measurement mitigation. Implications practice and/or policy Education practitioners policymakers should aware originate multitude steps offers them heuristic asking developers assess risk bias. Measuring systems use more ML, large part highly context‐dependent what counts as good feedback an assignment varies). EdTech play important role collecting curating datasets benchmarking moving forward.

Language: Английский

Citations

11

The emotional impact of generative AI: negative emotions and perception of threat DOI
Alessandro Gabbiadini, Dimitri Ognibene, Cristina Baldissarri

et al.

Behaviour and Information Technology, Journal Year: 2024, Volume and Issue: unknown, P. 1 - 18

Published: March 26, 2024

Generative Artificial Intelligence (AI) is a rapidly expanding field that aims to develop machines capable of performing tasks were previously considered unique humans, such as learning, reasoning, problem-solving, and decision-making. The recent release several tools based on AI (e.g. ChatGPT) has sparked debates the potential this technology garnered widespread attention in mainstream media.

Language: Английский

Citations

10

Goal setting in higher education: how, why, and when are students prompted to set goals? A systematic review DOI Creative Commons

Gabrielle Martins van Jaarsveld,

Jacqueline Wong, Martine Baars

et al.

Frontiers in Education, Journal Year: 2025, Volume and Issue: 9

Published: Jan. 8, 2025

The increasingly digital landscape of higher education has highlighted the importance self-regulated learning in environments. To support this, academic goal setting is frequently used to enhance order improve performance. Although many studies have explored implementation activities as behavioral modifiers, across these varied, and there little consensus on components which should be included reported when studying activities. provide an overview current state field, a systematic review was carried out examining implemented within over last 14 years (2010–2024) determine for whom, what contexts, how been implemented. results from 60 reveal wide array implementations covering countries disciplines. Overall, are highly heterogeneous, with large differences between out. However, also show strong trend toward partial digitalization, most using technology deliver their activities, but very few adopting technologies any further enhancements or support. reveals focus non-experimental exploring content student goals, only small selection testing effect experimental studies. Based we suggest future work focuses setting, especially focusing interplay design individual needs, well investigation emerging educational can scale

Language: Английский

Citations

1

The Critical Role of Trust in Adopting AI-Powered Educational Technology for Learning: An Instrument for Measuring Student Perceptions DOI Creative Commons
Tanya Nazaretsky, Paola Mejia-Domenzain, Vinitra Swamy

et al.

Computers and Education Artificial Intelligence, Journal Year: 2025, Volume and Issue: unknown, P. 100368 - 100368

Published: Jan. 1, 2025

Language: Английский

Citations

1