The impact of face masks on face-to-face neural tracking of speech: auditory and visual obstacles

bioRxiv (Cold Spring Harbor Laboratory), Journal Year: 2024, Volume and Issue: unknown

Published: Feb. 12, 2024

Abstract Face masks provide fundamental protection against the transmission of respiratory viruses but hamper communication. We estimated auditory and visual obstacles generated by face on communication measuring neural tracking face-to-face speech. To this end, we recorded EEG while participants were exposed to naturalistic audio-visual speech, embedded in multi-talker noise, three contexts: (i) no-mask (audio-visual information was fully available), (ii) virtual mask (occluded lips, intact audio), (iii) real lips degraded audio). The lip movements sound envelope speech measured through backward modeling, that is, reconstructing stimulus properties from activity. Behaviorally, increased listening -phonological-errors content retrieval perceived difficulty. At level, observed occlusion mouth abolished dampened at earliest processing stages. Degraded acoustic due filtering altered later stages instead. Finally, a consistent link emerged between increment difficulty drop reconstruction performance when attending speaker wearing mask. Results clearly dissociated impacts While hampered ability predict integrate filter impacted typically associated with selective attention. provided evidence major impact metacognitive levels subtending processing.

Language: Английский

---

Cortical tracking of naturalistic music and speech across frequency bands and brain regions: functional mapping and temporal dynamics

bioRxiv (Cold Spring Harbor Laboratory), Journal Year: 2024, Volume and Issue: unknown

Published: June 13, 2024

Abstract Music and speech encode hierarchically organized structural complexity at the service of human expressiveness communication. Previous research has shown that populations neurons in auditory regions track envelope acoustic signals within range slow fast oscillatory activity. However, extent to which cortical tracking is influenced by interplay between stimulus type, frequency band, brain anatomy remains an open question. In this study, we reanalyzed intracranial recordings from thirty subjects implanted with electrocorticography (ECoG) grids left cerebral hemisphere, drawn existing open-access ECoG database. Participants passively watched a movie where visual scenes were accompanied either music or stimuli. Cross-correlation activity signals, along density-based clustering analyses linear mixed effect modeling, revealed both anatomically overlapping functionally distinct mapping as function type band. We observed widespread left-hemisphere Slow Frequency Band (SFB, band-passed filtered low-frequency signal 1-8Hz), near zero temporal lags. contrast, High (HFB, 70-120Hz signal) was higher during perception, more densely concentrated classical language processing areas, showed frontal-to-temporal gradient lag values not perception musical Our results highlight complex interaction region band shapes dynamics naturalistic signals.

Language: Английский

---

Dynamic modeling of EEG responses to natural speech reveals earlier processing of predictable words

bioRxiv (Cold Spring Harbor Laboratory), Journal Year: 2024, Volume and Issue: unknown

Published: Aug. 26, 2024

Abstract In recent years, it has become clear that EEG indexes the comprehension of natural, narrative speech. One particularly compelling demonstration this fact can be seen by regressing responses to speech against measures how individual words in linguistically relate their preceding context. This approach produces a so-called temporal response function displays centro-parietal negativity reminiscent classic N400 component event-related potential. shortcoming previous implementations is they have typically assumed linear, time-invariant relationship between linguistic features and responses. other words, analysis assumes same shape timing for every word – only varies (linearly) terms its amplitude. present work, we relax assumption under hypothesis may processed more rapidly when are predictable. Specifically, introduce framework wherein standard linear modulated amplitude, latency, scale based on expectedness current prior words. We use proposed model recorded from set participants who listened an audiobook narrated single talker, separate attended one two concurrently presented audiobooks. show expected faster evoking lower amplitude N400-like with earlier peaks effect driven both word’s own predictability immediately word. Additional suggests finding not simply explained quickly disambiguated phonetic neighbors. As such, our study demonstrates brain natural depend predictability. By accounting these effects, also improves accuracy which neural modeled.

Language: Английский

---

The effect of gaze on EEG measures of multisensory integration in a cocktail party scenario

bioRxiv (Cold Spring Harbor Laboratory), Journal Year: 2023, Volume and Issue: unknown

Published: Aug. 24, 2023

Seeing the speaker's face greatly improves our speech comprehension in noisy environments. This is due to brain's ability combine auditory and visual information around us, a process known as multisensory integration. Selective attention also strongly influences what we comprehend scenarios with multiple speakers - an effect cocktail-party phenomenon. However, interaction between integration not fully understood, especially when it comes natural, continuous speech. In recent electroencephalography (EEG) study, explored this issue showed that enhanced audiovisual speaker attended compared unattended. Here, extend work investigate how varies depending on person's gaze behavior, which affects quality of they have access to. To do so, recorded EEG from 31 healthy adults performed selective tasks several paradigms involving two concurrently presented speakers. We then modeled related audio (envelope) Crucially, classes model one assumed underlying (AV) versus another independent unisensory processes (A+V). comparison revealed evidence strong attentional effects participants were looking directly at speaker. was apparent peripheral vision participants. Overall, findings suggest influence high fidelity (articulatory) available. More generally, suggests interplay during natural dynamic adaptable based specific task environment.

Language: Английский

---