Lexical semantic content, not syntactic structure, is the main contributor to ANN-brain similarity of fMRI responses in the language network

bioRxiv (Cold Spring Harbor Laboratory), Journal Year: 2023, Volume and Issue: unknown

Published: May 6, 2023

Abstract Representations from artificial neural network (ANN) language models have been shown to predict human brain activity in the network. To understand what aspects of linguistic stimuli contribute ANN-to-brain similarity, we used an fMRI dataset responses n=627 naturalistic English sentences (Pereira et al., 2018) and systematically manipulated for which ANN representations were extracted. In particular, i) perturbed sentences’ word order, ii) removed different subsets words, or iii) replaced with other varying semantic similarity. We found that lexical content sentence (largely carried by words) rather than sentence’s syntactic form (conveyed via order function is primarily responsible follow-up analyses, perturbation manipulations adversely affect predictivity also lead more divergent ANN’s embedding space decrease ability upcoming tokens those stimuli. Further, results are robust whether mapping model trained on intact stimuli, conditioned same context humans saw. The critical result—that lexical- main contributor similarity between ones—aligns idea goal system extract meaning strings. Finally, this work highlights strength systematic experimental evaluating how close accurate generalizable

Language: Английский

---

An information-theoretic analysis of targeted regressions during reading

Cognition, Journal Year: 2024, Volume and Issue: 249, P. 105765 - 105765

Published: May 20, 2024

Regressions, or backward saccades, are common during reading, accounting for between 5% and 20% of all saccades. And yet, relatively little is known about what causes them. We provide an information-theoretic operationalization two previous qualitative hypotheses regressions, which we dub reactivation reanalysis. argue that these make different predictions the pointwise mutual information pmi a regression's source target. Intuitively, words measures how much more (or less) likely one word to be present given other. On hand, hypothesis predicts regressions occur associated, implying high positive values pmi. other reanalysis should not associated with each other, negative, low As second theoretical contribution, expand on theories by considering only but also expected pmi, E[pmi], where expectation taken over possible realizations The rationale this language processing involves making inferences under uncertainty, readers may uncertain they have read, especially if was skipped. To test both theories, use contemporary models estimate pmi-based statistics pairs in three corpora eye tracking data English, as well six languages across families (Indo-European, Uralic, Turkic). Our results consistent tested: Positive E[pmi] consistently help predict patterns whereas negative do not. interpretation increases predictive scope our studies first systematic crosslinguistic analysis literature. support and, broadly, number behaviors can linked principles.

Language: Английский

---

Compositionality Under Time Pressure

Published: May 11, 2023

Compositionality is a central component of the human faculty for generalization and flexibility. However, computations involved are poorly understood, especially in terms their cognitive costs. On one hand, compositionality requires searching combinatorially large hypothesis spaces, raising issues tractability.On other compositional representations afford efficient compact compression. To shed light on resource required compositionality, we used within-subject time pressure manipulation to study how participants navigated series mazes, generated using recursive operations over spatial primitives. We find evidence that behavior guided by use primitives abstract them, where degree structure increases performance speeds up decisions.And while led more random errors, it did not impair capacity compositionality. Rather, increased reliance reusing recombining previous computations, suggesting remarkable robustness reasoning.

Language: Английский

---

Cloze probability, predictability ratings, and computational estimates for 205 English sentences, aligned with existing EEG and reading time data

Behavior Research Methods, Journal Year: 2023, Volume and Issue: 56(5), P. 5190 - 5213

Published: Oct. 25, 2023

We release a database of cloze probability values, predictability ratings, and computational estimates for sample 205 English sentences (1726 words), aligned with previously released word-by-word reading time data (both self-paced eye-movement records; Frank et al., Behavior Research Methods, 45(4), 1182-1190. 2013) EEG responses (Frank Brain Language, 140, 1-11. 2015). Our analyses show that ratings are the best predictors signal (N400, P600, LAN) times, eye movement patterns, when spillover effects taken into account. The particularly effective at explaining variance in eye-tracking without spillover. Cloze have decent overall psychometric accuracy early fixation patterns (first duration). results indicate choice measurement word context critically depends on processing index being considered.

Language: Английский

---

Interplay of Semantic Plausibility and Word Order Canonicity in Sentence Processing of People With Aphasia Using a Verb-Final Language

American Journal of Speech-Language Pathology, Journal Year: 2024, Volume and Issue: unknown, P. 1 - 11

Published: July 11, 2024

Purpose: The Western Aphasia Battery is widely used to assess people with aphasia (PWA). Sequential Commands (SC) one of the most challenging subtests for PWA. However, test items confound linguistic factors that make sentences difficult current study systematically manipulated semantic plausibility and word order in like those SC examine how these affect comprehension deficits aphasia. Method: Fifty Korean speakers (25 PWA 25 controls) completed a sentence–picture matching task (canonical vs. noncanonical) (plausible less plausible). Analyses focused on accuracy aimed identify sentence types best discriminate groups. Additionally, we explored which type serves as predictor severity. Results: demonstrated greater difficulties processing plausible than ones compared controls. Across groups, noncanonical elicited lower canonical sentences. Notably, control groups differed severity significantly correlated Conclusion: Even languages flexible order, find it process syntactic structures roles.

Language: Английский

---