A Bayesian Exploration on the Motivational and Behavioral Impacts of Chatbots in Language Learning

Published: April 23, 2025

Language: Английский

---

Generalization bias in large language model summarization of scientific research

Royal Society Open Science, Journal Year: 2025, Volume and Issue: 12(4)

Published: April 1, 2025

Artificial intelligence chatbots driven by large language models (LLMs) have the potential to increase public science literacy and support scientific research, as they can quickly summarize complex information in accessible terms. However, when summarizing texts, LLMs may omit details that limit scope of research conclusions, leading generalizations results broader than warranted original study. We tested 10 prominent LLMs, including ChatGPT-4o, ChatGPT-4.5, DeepSeek, LLaMA 3.3 70B, Claude 3.7 Sonnet, comparing 4900 LLM-generated summaries their texts. Even explicitly prompted for accuracy, most produced those with 70B overgeneralizing 26-73% cases. In a direct comparison human-authored summaries, LLM were nearly five times more likely contain broad (odds ratio = 4.85, 95% CI [3.06, 7.70], p < 0.001). Notably, newer tended perform worse generalization accuracy earlier ones. Our indicate strong bias many widely used towards posing significant risk large-scale misinterpretations findings. highlight mitigation strategies, lowering temperature settings benchmarking accuracy.

Language: Английский

---

GPT models for text annotation: An empirical exploration in public policy research

Policy Studies Journal, Journal Year: 2025, Volume and Issue: unknown

Published: May 2, 2025

Abstract Text annotation, the practice of labeling text following a predetermined scheme, is essential to qualitative public policy research. Despite its importance, annotating large data faces challenges high labor and time costs. Recent developments in language models (LLMs), specifically with generative pretrained transformers (GPTs), show potential approach that may alleviate burden manual annotation. In this report, we first introduce small sample pretest strategy for researchers decide whether use Open AI's GPT addition, test if can substitute human coders by comparing results two different prompting strategies against Using email messages collected from national corresponding experiment US nursing home market as an example, on average, demonstrate 86.25% percentage agreement between annotations. We also possess context‐based limitations. Our report ends reflections suggestions readers who are interested using

Language: Английский

---

Colour/shape-taste correspondences across three languages in ChatGPT

Cognition, Journal Year: 2024, Volume and Issue: 253, P. 105936 - 105936

Published: Aug. 31, 2024

Crossmodal correspondences, the tendency for a sensory feature / attribute in one modality (either physically present or merely imagined), to be associated with another modality, have been studied extensively, revealing consistent patterns, such as sweet tastes being pink colours and round shapes across languages. The research explores whether correspondences are captured by ChatGPT, large language model developed OpenAI. Across twelve studies, this investigates colour/shapes-taste crossmodal ChatGPT-3.5 -4o, focusing on associations between shapes/colours five basic three languages (English, Japanese, Spanish). Studies 1A-F examined taste-shape associations, using prompts assess ChatGPT's association of angular tastes. results indicated significant, consistent, shape taste, with, example, strongly sweet/umami bitter/salty/sour magnitude shape-taste matching appears greater ChatGPT-4o than ChatGPT-3.5, ChatGPT prompted English Spanish Japanese. 2A-F focused colour-taste eleven that ChatGPT-4o, but not generally replicates patterns previously observed human participants. Specifically, associates pink, sour yellow, salty white/blue, bitter black, umami red However, magnitude/similarity shape/colour-taste more pronounced (i.e., having little variance, mean difference), which does adequately reflect subtle nuances typically seen correspondences. These findings suggest captures language- GPT version-specific variations, albeit some differences when compared previous studies involving contribute valuable knowledge field explore possibility generative AI resembles perceptual systems cognition languages, provide insight into development evolution capture

Language: Английский

---

Should Artificial Intelligence Play a Durable Role in Biomedical Research and Practice?

International Journal of Molecular Sciences, Journal Year: 2024, Volume and Issue: 25(24), P. 13371 - 13371

Published: Dec. 13, 2024

During the last decade, artificial intelligence (AI) was applied to nearly all domains of human activity, including scientific research. It is thus warranted ask whether AI thinking should be durably involved in biomedical This problem addressed by examining three complementary questions (i) What are major barriers currently met investigators? suggested that during 2 decades there a shift towards growing need elucidate complex systems, and this not sufficiently fulfilled previously successful methods such as theoretical modeling or computer simulation (ii) potential meet aforementioned need? it recent well-suited perform classification prediction tasks on multivariate possibly help data interpretation, provided their efficiency properly validated. (iii) Recent representative results obtained with machine learning suggest may comparable displayed operators. concluded play an important role practice. Also, already other physics, combining conventional might generate further progress new applications, involving heuristic interpretation.

Language: Английский

---