A hierarchy of linguistic predictions during natural language comprehension

Proceedings of the National Academy of Sciences, Journal Year: 2022, Volume and Issue: 119(32)

Published: Aug. 3, 2022

Understanding spoken language requires transforming ambiguous acoustic streams into a hierarchy of representations, from phonemes to meaning. It has been suggested that the brain uses prediction guide interpretation incoming input. However, role in processing remains disputed, with disagreement about both ubiquity and representational nature predictions. Here, we address issues by analyzing recordings participants listening audiobooks, using deep neural network (GPT-2) precisely quantify contextual First, establish responses words are modulated ubiquitous Next, disentangle model-based predictions distinct dimensions, revealing dissociable signatures syntactic category (parts speech), phonemes, semantics. Finally, show high-level (word) inform low-level (phoneme) predictions, supporting hierarchical predictive processing. Together, these results underscore processing, showing spontaneously predicts upcoming at multiple levels abstraction.

Language: Английский

---

Dissociating language and thought in large language models

Trends in Cognitive Sciences, Journal Year: 2024, Volume and Issue: 28(6), P. 517 - 540

Published: March 19, 2024

Language: Английский

---

Large Language Models Demonstrate the Potential of Statistical Learning in Language

Cognitive Science, Journal Year: 2023, Volume and Issue: 47(3)

Published: Feb. 25, 2023

Abstract To what degree can language be acquired from linguistic input alone? This question has vexed scholars for millennia and is still a major focus of debate in the cognitive science language. The complexity human hampered progress because studies language–especially those involving computational modeling–have only been able to deal with small fragments our skills. We suggest that most recent generation Large Language Models (LLMs) might finally provide tools determine empirically how much ability experience. LLMs are sophisticated deep learning architectures trained on vast amounts natural data, enabling them perform an impressive range tasks. argue that, despite their clear semantic pragmatic limitations, have already demonstrated human‐like grammatical without need built‐in grammar. Thus, while there learn about humans acquire use language, full‐fledged models scientists evaluate just far statistical take us explaining full

Language: Английский

---

Language Model Behavior: A Comprehensive Survey

Computational Linguistics, Journal Year: 2023, Volume and Issue: 50(1), P. 293 - 350

Published: Nov. 15, 2023

Abstract Transformer language models have received widespread public attention, yet their generated text is often surprising even to NLP researchers. In this survey, we discuss over 250 recent studies of English model behavior before task-specific fine-tuning. Language possess basic capabilities in syntax, semantics, pragmatics, world knowledge, and reasoning, but these are sensitive specific inputs surface features. Despite dramatic increases quality as scale hundreds billions parameters, the still prone unfactual responses, commonsense errors, memorized text, social biases. Many weaknesses can be framed over-generalizations or under-generalizations learned patterns text. We synthesize results highlight what currently known about large capabilities, thus providing a resource for applied work research adjacent fields that use models.

Language: Английский

---

A-maze of Natural Stories: Comprehension and surprisal in the Maze task

Glossa Psycholinguistics, Journal Year: 2023, Volume and Issue: 2(1)

Published: April 11, 2023

Behavioral measures of word-by-word reading time provide experimental evidence to test theories language processing. A-maze is a recent method for measuring incremental sentence processing that can localize slowdowns related syntactic ambiguities in individual sentences. We adapted use on longer passages and tested it the Natural Stories corpus. Participants were able comprehend these text they read via Maze task. Moreover, task yielded useable reaction data with word predictability effects linearly surprisal, same pattern found other methods. Crucially, times show tight relationship properties current word, little spillover effects from previous words. This superior localization an advantage compared Overall, we expanded scope materials, thus theoretical questions, be studied

Language: Английский

---

Why large language models are poor theories of human linguistic cognition: A reply to Piantadosi

Biolinguistics, Journal Year: 2023, Volume and Issue: 17

Published: Dec. 15, 2023

In a recent manuscript entitled “Modern language models refute Chomsky’s approach to language”, Steven Piantadosi proposes that large such as GPT-3 can serve serious theories of human linguistic cognition. In fact, he maintains these are significantly better than proposals emerging from within generative linguistics. The present note explains why this claim is wrong.

Language: Английский

---

Acquiring constraints on filler-gap dependencies from structural collocations: Assessing a computational learning model of island-insensitivity in Norwegian

Language Acquisition, Journal Year: 2025, Volume and Issue: unknown, P. 1 - 44

Published: March 13, 2025

Language: Английский

---

How poor is the stimulus? Evaluating hierarchical generalization in neural networks trained on child-directed speech

Published: Jan. 1, 2023

When acquiring syntax, children consistently choose hierarchical rules over competing non-hierarchical possibilities. Is this preference due to a learning bias for structure, or more general biases that interact with cues in children's linguistic input? We explore these possibilities by training LSTMs and Transformers - two types of neural networks without on data similar quantity content input: text from the CHILDES corpus. then evaluate what models have learned about English yes/no questions, phenomenon which structure is crucial. find that, though they perform well at capturing surface statistics child-directed speech (as measured perplexity), both model generalize way consistent an incorrect linear rule than correct rule. These results suggest human-like generalization alone requires stronger sequence-processing standard network architectures.

Language: Английский

---

Surprisal From Language Models Can Predict ERPs in Processing Predicate-Argument Structures Only if Enriched by an Agent Preference Principle

Neurobiology of Language, Journal Year: 2023, Volume and Issue: 5(1), P. 167 - 200

Published: Sept. 7, 2023

Language models based on artificial neural networks increasingly capture key aspects of how humans process sentences. Most notably, model-based surprisals predict event-related potentials such as N400 amplitudes during parsing. Assuming that these represent realistic estimates human linguistic experience, their success in modeling language processing raises the possibility system relies no other principles than general architecture and sufficient input. Here, we test this hypothesis effects observed verb-final sentences German, Basque, Hindi. By stacking Bayesian generalised additive models, show that, each language, topographies region verb are best predicted when complemented by an Agent Preference principle transiently interprets initial role-ambiguous noun phrases agents, leading to reanalysis interpretation fails. Our findings demonstrate need for independently usage frequencies structural differences between languages. The has unequal force, however. Compared surprisal, its effect is weakest stronger Hindi, still Basque. This gradient correlated with extent which grammars allow unmarked NPs be patients, a feature boosts effects. We conclude gain more neurobiological plausibility incorporating Preference. Conversely, theories profit from surprisal addition like Preference, arguably have distinct evolutionary roots.

Language: Английский

---

Large Language Models: The Need for Nuance in Current Debates and a Pragmatic Perspective on Understanding

Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Journal Year: 2023, Volume and Issue: unknown

Published: Jan. 1, 2023

Current Large Language Models (LLMs) are unparalleled in their ability to generate grammatically correct, fluent text. LLMs appearing rapidly, and debates on LLM capacities have taken off, but reflection is lagging behind. Thus, this position paper, we first zoom the debate critically assess three points recurring critiques of capacities: i) that only parrot statistical patterns training data; ii) master formal not functional language competence; iii) learning cannot inform human learning. Drawing empirical theoretical arguments, show these need more nuance. Second, outline a pragmatic perspective issue 'real' understanding intentionality LLMs. Understanding pertain unobservable mental states attribute other humans because they value: allow us abstract away from complex underlying mechanics predict behaviour effectively. We reflect circumstances under which it would make sense for similarly LLMs, thereby outlining philosophical context as an increasingly prominent technology society.

Language: Английский

---