Deep language algorithms predict semantic comprehension from brain activity

Scientific Reports, Год журнала: 2022, Номер 12(1)

Опубликована: Сен. 29, 2022

Abstract Deep language algorithms, like GPT-2, have demonstrated remarkable abilities to process text, and now constitute the backbone of automatic translation, summarization dialogue. However, whether these models encode information that relates human comprehension still remains controversial. Here, we show representations GPT-2 not only map onto brain responses spoken stories, but they also predict extent which subjects understand corresponding narratives. To this end, analyze 101 recorded with functional Magnetic Resonance Imaging while listening 70 min short stories. We then fit a linear mapping model activity from GPT-2’s activations. Finally, reliably correlates ( $$\mathcal {R}=0.50, p<10^{-15}$$ R=0.50,p<10-15 ) subjects’ scores as assessed for each story. This effect peaks in angular, medial temporal supra-marginal gyri, is best accounted by long-distance dependencies generated deep layers GPT-2. Overall, study shows how help clarify computations underlying comprehension.

Язык: Английский

---

Effects of Age on Cortical Tracking of Word-Level Features of Continuous Competing Speech

Frontiers in Neuroscience, Год журнала: 2021, Номер 15

Опубликована: Апрель 1, 2021

Speech-in-noise comprehension difficulties are common among the elderly population, yet traditional objective measures of speech perception largely insensitive to this deficit, particularly in absence clinical hearing loss. In recent years, a growing body research young normal-hearing adults has demonstrated that high-level features related semantics and lexical predictability elicit strong centro-parietal negativity EEG signal around 400 ms following word onset. Here we investigate effects age on cortical tracking these word-level within two-talker mixture, their relationship with self-reported speech-in-noise understanding. While undergoing recordings, younger older adult participants listened continuous narrative story presence distractor story. We then utilized forward encoding models estimate four features: (1) onsets, (2) “semantic” dissimilarity each relative preceding context, (3) surprisal for word, (4) overall audibility. Our results revealed robust all attended speech, audibility showing significantly stronger contributions neural activity than dissimilarity. Additionally, exhibited adults, especially over frontal electrode sites, potentially reflecting increased listening effort. Finally, neuro-behavioral analyses trends negative between subjective model goodness-of-fit as well positive task performance goodness-of-fit, indicating behavioral relevance measures. Together, our demonstrate utility modeling responses multi-talker using complex, potential use study changes processing due aging

Язык: Английский

---

Exploring Age Differences in Absorption and Enjoyment during Story Listening

Deleted Journal, Год журнала: 2024, Номер 6(2), С. 667 - 684

Опубликована: Июнь 13, 2024

Using naturalistic spoken narratives to investigate speech processes and comprehension is becoming increasingly popular in experimental hearing research. Yet, little known about how individuals engage with story materials listening experiences change age. We investigated absorption the context of stories, explored predictive factors for engagement, examined utility a scale developed written assess auditory materials. Adults aged 20–78 years (N = 216) participated an online study. Participants listened one ten stories intended be engaging different degrees rated terms enjoyment. ages similarly absorbing enjoyable. Further, higher mood scores predicted enjoyment ratings. Factor analysis showed items approximately grouped according original dimensions, suggesting that may similar although certain discriminated less effectively between more or engaging. The present study provides novel insights into adults supports using stimuli

Язык: Английский

---

Heard or Understood? Neural Tracking of Language Features in a Comprehensible Story, an Incomprehensible Story and a Word List

eNeuro, Год журнала: 2023, Номер 10(7), С. ENEURO.0075 - 23.2023

Опубликована: Июль 1, 2023

Speech comprehension is a complex neural process on which relies activation and integration of multiple brain regions. In the current study, we evaluated whether speech can be investigated by tracking. Neural tracking phenomenon in responses time-lock to rhythm specific features continuous speech. These acoustic, i.e., acoustic tracking, or derived from content using language properties, We differs between comprehensible story, an incomprehensible word list. 19 participants (six men). No significant difference regarding was found. However, only found for story. The most prominent effect visible surprisal, feature at level. response surprisal showed negativity 300 400 ms, similar N400 evoked paradigms. This significantly more negative when story comprehended, words could integrated context previous words. results show that capture comprehension.

Язык: Английский

---

Reliability and generalizability of neural speech tracking in younger and older adults

Neurobiology of Aging, Год журнала: 2023, Номер 134, С. 165 - 180

Опубликована: Ноя. 21, 2023

Язык: Английский

---

A listening advantage for native speech is reflected by attention-related activity in auditory cortex

Communications Biology, Год журнала: 2025, Номер 8(1)

Опубликована: Фев. 5, 2025

Язык: Английский

---

Effects of Noise and Reward on Pupil Size and Electroencephalographic Speech Tracking in a Word‐Detection Task

European Journal of Neuroscience, Год журнала: 2025, Номер 61(3)

Опубликована: Фев. 1, 2025

ABSTRACT Speech is hard to understand when there background noise. intelligibility and listening effort both affect our ability speech, but the relative contribution of these factors disentangle. Previous studies suggest that speech could be assessed with EEG tracking via pupil size. However, measures may confounded, because poor require a larger effort. To address this, we developed novel word‐detection paradigm allows for rapid behavioural assessment processing. In this paradigm, words appear on screen during continuous similar closed captioning. two experiments total 51 participants, manipulated by changing signal‐to‐noise ratios (SNRs) modulated varying monetary reward. Increasing SNR improved detection performance along tracking. Additionally, find size increases increased SNR. Surprisingly, reward SNR, found only size, whereas We interpret as effects arousal The experimental here beneficial assessing hearing devices in terms

Язык: Английский

---

Dynamic modeling of EEG responses to natural speech reveals earlier processing of predictable words

PLoS Computational Biology, Год журнала: 2025, Номер 21(4), С. e1013006 - e1013006

Опубликована: Апрель 28, 2025

In recent years, it has become clear that EEG indexes the comprehension of natural, narrative speech. One particularly compelling demonstration this fact can be seen by regressing responses to speech against measures how individual words in linguistically relate their preceding context. This approach produces a so-called temporal response function displays centro-parietal negativity reminiscent classic N400 component event-related potential. shortcoming previous implementations is they have typically assumed linear, time-invariant relationship between linguistic features and responses. other words, analysis assumes same shape timing for every word – only varies (linearly) terms its amplitude. present work, we relax assumption under hypothesis may processed more rapidly when are predictable. Specifically, introduce framework wherein standard linear modulated amplitude, latency, scale based on predictability current prior words. We use proposed model recorded from set participants who listened an audiobook narrated single talker, separate attended one two concurrently presented audiobooks. show expected faster evoking lower amplitude N400-like with earlier peaks effect driven both word’s own immediately word. Additional suggests finding not simply explained quickly disambiguated phonetic neighbors. As such, our study demonstrates brain natural depend predictability. By accounting these effects, also improves accuracy which neural modeled.

Язык: Английский

---

Never Mind the Repeat: How Speech Expectations Reduce Tracking at the Cocktail Party

Cortex, Год журнала: 2025, Номер 189, С. 1 - 19

Опубликована: Май 20, 2025

Язык: Английский

---

Electrophysiological indices of hierarchical speech processing differentially reflect the comprehension of speech in noise

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2023, Номер unknown

Опубликована: Март 31, 2023

ABSTRACT The past few years have seen an increase in the use of encoding models to explain neural responses natural speech. goal these is characterize how human brain converts acoustic speech energy into different linguistic representations that enable everyday comprehension. For example, researchers shown electroencephalography (EEG) data can be modeled terms features speech, such as its amplitude envelope or spectrogram, phonemes and phoneme probability, higher-level like context-based word predictability. However, it unclear reliably EEG indices reflect comprehension listening conditions. To address this, we recorded from neurotypical adults who listened segments audiobook levels background noise. We their reflected this varied with across noise levels. In line our hypothesis, signatures predictability phonetic were more closely correlated behavioral measures percentage words heard than based on low-level features. markers influence top- down, prediction bottom-up processing also behavior. These findings help relationship between behavior by comprehensively linking hierarchical language metrics. SIGNIFICANCE STATEMENT Acoustic been consistently tracked activity even noisy low- high- level covary one another relate Here, find categorical feature affected probability-based better correlate intelligibility results extend understanding various are comparatively electrical they perception challenging

Язык: Английский

---