Wait, It’s All Token Noise? Always Has Been: Interpreting LLM Behavior Using Shapley Value DOI
Behnam Mohammadi

SSRN Electronic Journal, Год журнала: 2024, Номер unknown

Опубликована: Янв. 1, 2024

The emergence of large language models (LLMs) has opened up exciting possibilities for simulating human behavior and cognitive processes, with potential applications in various domains, including marketing research consumer analysis. However, the validity utilizing LLMs as stand-ins subjects remains uncertain due to glaring divergences that suggest fundamentally different underlying processes at play sensitivity LLM responses prompt variations. This paper presents a novel approach based on Shapley values from cooperative game theory interpret quantify relative contribution each component model's output. Through two applications—a discrete choice experiment an investigation biases—we demonstrate how value method can uncover what we term "token noise" effects, phenomenon where decisions are disproportionately influenced by tokens providing minimal informative content. raises concerns about robustness generalizability insights obtained context simulation. Our model-agnostic extends its utility proprietary LLMs, valuable tool marketers researchers strategically optimize prompts mitigate apparent biases. findings underscore need more nuanced understanding factors driving before relying them substitutes settings. We emphasize importance reporting results conditioned specific templates exercising caution when drawing parallels between LLMs.

Язык: Английский

On the creativity of large language models DOI Creative Commons
Giorgio Franceschelli, Mirco Musolesi

AI & Society, Год журнала: 2024, Номер unknown

Опубликована: Ноя. 28, 2024

Abstract Large language models (LLMs) are revolutionizing several areas of Artificial Intelligence. One the most remarkable applications is creative writing, e.g., poetry or storytelling: generated outputs often astonishing quality. However, a natural question arises: can LLMs be really considered creative? In this article, we first analyze development under lens creativity theories, investigating key open questions and challenges. particular, focus our discussion on dimensions value, novelty, surprise as proposed by Margaret Boden in her work. Then, consider different classic perspectives, namely product, process, press, person. We discuss set “easy” “hard” problems machine creativity, presenting them relation to LLMs. Finally, examine societal impact these technologies with particular industries, analyzing opportunities offered, challenges arising from them, potential associated risks, both legal ethical points view.

Язык: Английский

Процитировано

17
A Manager and an AI Walk into a Bar: Does ChatGPT Make Biased Decisions Like We Do? DOI
Yang Chen, Meena Andiappan, Tracy A. Jenkin

и другие.

SSRN Electronic Journal, Год журнала: 2023, Номер unknown

Опубликована: Янв. 1, 2023

Large language models (LLMs) such as ChatGPT have garnered global attention recently, with a promise to disrupt and revolutionize business operations. As managers rely more on artificial intelligence (AI) technology, there is an urgent need understand whether are systematic biases in AI decision-making since they trained human data feedback, both may be highly biased. This paper tests broad range of behavioral commonly found humans that especially relevant operations management. We although can much less biased accurate than problems explicit mathematical/probabilistic natures, it also exhibits many possess, when the complicated, ambiguous, implicit. It suffer from conjunction bias probability weighting. Its preference influenced by framing, salience anticipated regret, choice reference. struggles process ambiguous information evaluates risks differently humans. produce responses similar heuristics employed humans, prone confirmation bias. To make these issues worse, overconfident. Our research characterizes ChatGPT's behaviors showcases for researchers businesses consider potentialAI developing employing

Язык: Английский

Процитировано

36
Theory Is All You Need: AI, Human Cognition, and Decision Making DOI
Teppo Felin, Matthias Holweg

SSRN Electronic Journal, Год журнала: 2024, Номер unknown

Опубликована: Янв. 1, 2024

Artificial intelligence (AI) now matches or outperforms human in an astonishing array of games, tests, and other cognitive tasks that involve high-level reasoning thinking. Many scholars argue that—due to bias bounded rationality—humans should (or will soon) be replaced by AI situations involving cognition strategic decision making. We disagree. In this paper we first trace the historical origins idea artificial as a form computation information processing. highlight problems with analogy between computers minds input-output devices, using large language models example. Human cognition—in important instances—is better conceptualized theorizing rather than data processing, prediction, even Bayesian updating. Our argument, when it comes cognition, is AI's data-based prediction different from theory-based causal logic. introduce belief-data (a)symmetries difference use "heavier-than-air flight" example our arguments. Theories provide mechanism for identifying new evidence, way "intervening" world, experimenting, problem solving. conclude discussion implications arguments making, including role human-AI hybrids might play process.

Язык: Английский

Процитировано

9
(Ir)rationality and cognitive biases in large language models DOI Creative Commons
Olivia Macmillan-Scott, Mirco Musolesi

Royal Society Open Science, Год журнала: 2024, Номер 11(6)

Опубликована: Июнь 1, 2024

Do large language models (LLMs) display rational reasoning? LLMs have been shown to contain human biases due the data they trained on; whether this is reflected in reasoning remains less clear. In paper, we answer question by evaluating seven using tasks from cognitive psychology literature. We find that, like humans, irrationality these tasks. However, way displayed does not reflect that humans. When incorrect answers are given tasks, often ways differ human-like biases. On top of this, reveal an additional layer significant inconsistency responses. Aside experimental results, paper seeks make a methodological contribution showing how can assess and compare different capabilities types models, case with respect reasoning.

Язык: Английский

Процитировано

9
Anchoring Bias in Large Language Models: An Experimental Study DOI Creative Commons

Jiaxu Lou,

Yifan Sun

Опубликована: Янв. 28, 2025

Large Language Models (LLMs) like GPT-4 and Gemini have significantly advanced artificial intelligence by enabling machines to generate comprehend human-like text. Despite their impressive capabilities, LLMs are not free of limitations. They shown various biases. While much research has explored demographic biases, the cognitive biases in been equally studied. This study delves into anchoring bias, a bias where initial information disproportionately influences judgment. Utilizing an experimental dataset, we examine how manifests verify effectiveness mitigation strategies. Our findings highlight sensitivity LLM responses biased hints. At same time, our experiments show that, mitigate one needs collect hints from comprehensive angles prevent being anchored individual pieces information, while simple algorithms such as Chain-of-Thought, Thoughts Principles, Ignoring Anchor Hints, Reflection sufficient.

Язык: Английский

Процитировано

0
Biases, evolutionary mismatch and the comparative analysis of human versus artificial cognition: a comment on Macmillan-Scott and Musolesi (2024) DOI Creative Commons
Pier Luigi Sacco

Royal Society Open Science, Год журнала: 2025, Номер 12(2)

Опубликована: Фев. 1, 2025

Язык: Английский

Процитировано

0
Natural Language Processing and Large Language Models DOI Creative Commons
Peter Wulff, Marcus Kubsch, Christina Krist

и другие.

Springer texts in education, Год журнала: 2025, Номер unknown, С. 117 - 142

Опубликована: Янв. 1, 2025

Язык: Английский

Процитировано

0
theoraizer: AI-assisted Theory Construction DOI Open Access

Meike Waaijers,

Hannes Rosenbusch, Caspar J. Van Lissa

и другие.

Опубликована: Авг. 2, 2024

The Causal Loop Diagram (CLD) method is a technique for theory construction in which domain experts collaborate to identify causal relationships between variables. However, CLD labor-intensive, and the input required from grows quadratically with number of variables involved. This limits small graphs. Large Language Models (LLMs), their advanced text processing capabilities extensive knowledge base, can efficiently generate large amounts content, offering potential overcome these limitations. paper presents theoraizer, an R package Shiny app that enhances by integrating LLMs as digital extension expert group. Researchers use theoraizer define list putative variables, after it queries LLM links drastically reduces amount work arrive at candidate provides scientists standardized, multi-stage framework constructing theories.

Язык: Английский

Процитировано

0
Wait, It’s All Token Noise? Always Has Been: Interpreting LLM Behavior Using Shapley Value DOI
Behnam Mohammadi

SSRN Electronic Journal, Год журнала: 2024, Номер unknown

Опубликована: Янв. 1, 2024

The emergence of large language models (LLMs) has opened up exciting possibilities for simulating human behavior and cognitive processes, with potential applications in various domains, including marketing research consumer analysis. However, the validity utilizing LLMs as stand-ins subjects remains uncertain due to glaring divergences that suggest fundamentally different underlying processes at play sensitivity LLM responses prompt variations. This paper presents a novel approach based on Shapley values from cooperative game theory interpret quantify relative contribution each component model's output. Through two applications—a discrete choice experiment an investigation biases—we demonstrate how value method can uncover what we term "token noise" effects, phenomenon where decisions are disproportionately influenced by tokens providing minimal informative content. raises concerns about robustness generalizability insights obtained context simulation. Our model-agnostic extends its utility proprietary LLMs, valuable tool marketers researchers strategically optimize prompts mitigate apparent biases. findings underscore need more nuanced understanding factors driving before relying them substitutes settings. We emphasize importance reporting results conditioned specific templates exercising caution when drawing parallels between LLMs.

Язык: Английский

Процитировано

0