Machine Bias. How Do Generative Language Models Answer Opinion Polls? DOI
Julien Boelaert, Samuel Coavoux, Étienne Ollion

и другие.

Sociological Methods & Research, Год журнала: 2025, Номер unknown

Опубликована: Апрель 21, 2025

Generative artificial intelligence (AI) is increasingly presented as a potential substitute for humans, including research subjects. However, there no scientific consensus on how closely these in silico clones can emulate survey respondents. While some defend the use of “synthetic users,” others point toward social biases responses provided by large language models (LLMs). In this article, we demonstrate that critics are right to be wary using generative AI respondents, but probably not reasons. Our results show (i) date, cannot replace subjects opinion or attitudinal research; (ii) they display strong bias and low variance each topic; (iii) randomly varies from one topic next. We label pattern “machine bias,” concept define, whose consequences LLM-based further explore.

Язык: Английский

A Cross-Cultural Confusion Model for Detecting and Evaluating Students’ Confusion In a Large Classroom DOI
Yu Fang, Shihong Huang, Amy Ogan

и другие.

Опубликована: Фев. 21, 2025

Язык: Английский

Процитировано

0
A Primer for Evaluating Large Language Models in Social-Science Research DOI Creative Commons
Suhaib Abdurahman,

Alireza S. Ziabari,

Alexander Moore

и другие.

Advances in Methods and Practices in Psychological Science, Год журнала: 2025, Номер 8(2)

Опубликована: Апрель 1, 2025

Autoregressive large language models (LLMs) exhibit remarkable conversational and reasoning abilities exceptional flexibility across a wide range of tasks. Subsequently, LLMs are being increasingly used in scientific research to analyze data, generate synthetic or even write articles. This trend necessitates that authors follow best practices for conducting reporting LLM journal reviewers can evaluate the quality works use LLMs. We provide social-scientific with essential recommendations ensure replicable robust results using Our also highlight considerations reviewers, focusing on methodological rigor, replicability, validity when evaluating studies automate data processing simulate human data. offer practical advice assessing appropriateness applications submitted studies, emphasizing need transparency challenges posed by nondeterministic continuously evolving nature these models. By providing framework critical review, this primer, we aim high-quality, innovative landscape social-science

Язык: Английский

Процитировано

0
Machine Bias. How Do Generative Language Models Answer Opinion Polls? DOI
Julien Boelaert, Samuel Coavoux, Étienne Ollion

и другие.

Sociological Methods & Research, Год журнала: 2025, Номер unknown

Опубликована: Апрель 21, 2025

Generative artificial intelligence (AI) is increasingly presented as a potential substitute for humans, including research subjects. However, there no scientific consensus on how closely these in silico clones can emulate survey respondents. While some defend the use of “synthetic users,” others point toward social biases responses provided by large language models (LLMs). In this article, we demonstrate that critics are right to be wary using generative AI respondents, but probably not reasons. Our results show (i) date, cannot replace subjects opinion or attitudinal research; (ii) they display strong bias and low variance each topic; (iii) randomly varies from one topic next. We label pattern “machine bias,” concept define, whose consequences LLM-based further explore.

Язык: Английский

Процитировано

0