Linear Modeling of Neurophysiological Responses to Speech and Other Continuous Stimuli: Methodological Considerations for Applied Research

Frontiers in Neuroscience, Journal Year: 2021, Volume and Issue: 15

Published: Nov. 22, 2021

Cognitive neuroscience, in particular research on speech and language, has seen an increase the use of linear modeling techniques for studying processing natural, environmental stimuli. The availability such computational tools prompted similar investigations many clinical domains, facilitating study cognitive sensory deficits under more naturalistic conditions. However, (and often highly heterogeneous) cohorts introduces added layer complexity to procedures, potentially leading instability and, as a result, inconsistent findings. Here, we outline some key methodological considerations applied research, referring hypothetical experiment involving worked examples simulated electrophysiological (EEG) data. In particular, focus experimental design, data preprocessing, stimulus feature extraction, model training evaluation, interpretation weights. Throughout paper, demonstrate implementation each step MATLAB using mTRF-Toolbox discuss how address issues that could arise research. doing so, hope provide better intuition these technical points resource researchers investigating ecologically rich

Language: Английский

---

Neural tracking as a diagnostic tool to assess the auditory pathway

Hearing Research, Journal Year: 2022, Volume and Issue: 426, P. 108607 - 108607

Published: Sept. 14, 2022

Language: Английский

---

The effects of data quantity on performance of temporal response function analyses of natural speech processing

Frontiers in Neuroscience, Journal Year: 2023, Volume and Issue: 16

Published: Jan. 12, 2023

In recent years, temporal response function (TRF) analyses of neural activity recordings evoked by continuous naturalistic stimuli have become increasingly popular for characterizing properties within the auditory hierarchy. However, despite this rise in TRF usage, relatively few educational resources these tools exist. Here we use a dual-talker speech paradigm to demonstrate how key parameter experimental design, quantity acquired data, influences fit either individual data (subject-specific analyses), or group (generic analyses). We show that although model prediction accuracy increases monotonically with quantity, amount required achieve significant accuracies can vary substantially based on whether fitted contains densely (e.g., acoustic envelope) sparsely lexical surprisal) spaced features, especially when goal is capture aspect responses uniquely explained specific features. Moreover, generic models exhibit high performance small amounts test (2–8 min), if they are trained sufficiently large set. As such, may be particularly useful clinical and multi-task study designs limited recording time. Finally, regularization procedure used fitting interact models, larger training quantities resulting systematically amplitudes. Together, demonstrations work should aid new users analyses, combination other tools, such as piloting power serve detailed reference choosing acquisition duration future studies.

Language: Английский

---

Lip movements enhance speech representations and effective connectivity in auditory dorsal stream

NeuroImage, Journal Year: 2022, Volume and Issue: 257, P. 119311 - 119311

Published: May 16, 2022

Viewing speaker's lip movements facilitates speech perception, especially under adverse listening conditions, but the neural mechanisms of this perceptual benefit at phonemic and feature levels remain unclear. This fMRI study addressed question by quantifying regional multivariate representation network organization underlying audiovisual speech-in-noise perception. Behaviorally, valid improved recognition place articulation to aid phoneme identification. Meanwhile, enhanced representations phonemes in left auditory dorsal stream regions, including frontal motor areas supramarginal gyrus (SMG). Moreover, voicing features were promoted differentially these with Broca's area while better encoded ventral premotor cortex SMG. Next, dynamic causal modeling (DCM) analysis showed that such local changes accompanied strengthened effective connectivity along stream. neurite orientation dispersion arcuate fasciculus, bearing skeleton stream, predicted visual enhancements connectivity. Our findings provide novel insight science promote both encoding pathway functional enhancement is mediated microstructural architecture circuit.

Language: Английский

---

The integration of continuous audio and visual speech in a cocktail-party environment depends on attention

NeuroImage, Journal Year: 2023, Volume and Issue: 274, P. 120143 - 120143

Published: April 29, 2023

In noisy environments, our ability to understand speech benefits greatly from seeing the speaker's face. This is attributed brain's integrate audio and visual information, a process known as multisensory integration. addition, selective attention plays an enormous role in what we understand, so-called cocktail-party phenomenon. But how integration interact remains incompletely understood, particularly case of natural, continuous speech. Here, addressed this issue by analyzing EEG data recorded participants who undertook task using natural To assess integration, modeled responses two ways. The first assumed that audiovisual processing simply linear combination (i.e., A + V model), while second allows for possibility interactions AV model). Applying these models revealed attended were better explained model, providing evidence contrast, unattended best captured suggesting suppressed Follow up analyses some limited early speech, with no occurring at later levels processing. We take findings occurs multiple brain, each which can be differentially affected attention.

Language: Английский

---

A convolutional neural network provides a generalizable model of natural sound coding by neural populations in auditory cortex

PLoS Computational Biology, Journal Year: 2023, Volume and Issue: 19(5), P. e1011110 - e1011110

Published: May 5, 2023

Convolutional neural networks (CNNs) can provide powerful and flexible models of sensory processing. However, the utility CNNs in studying auditory system has been limited by their requirement for large datasets complex response properties single neurons. To address these limitations, we developed a population encoding model: CNN that simultaneously predicts activity several hundred neurons recorded during presentation set natural sounds. This approach defines shared spectro-temporal space pools statistical power across Population varying architecture performed consistently substantially better than traditional linear-nonlinear on data from primary non-primary cortex. Moreover, were highly generalizable. The output layer model pre-trained one could be fit to novel units, achieving performance equivalent original data. ability generalize suggests capture complete representational an cortical field.

Language: Английский

---

Auditory cortex encodes lipreading information through spatially distributed activity

Current Biology, Journal Year: 2024, Volume and Issue: 34(17), P. 4021 - 4032.e5

Published: Aug. 16, 2024

Language: Английский

---

Objectively Measuring Audiovisual Effects in Noise Using Virtual Human Speakers

Trends in Hearing, Journal Year: 2025, Volume and Issue: 29

Published: April 1, 2025

Speech intelligibility in challenging listening environments relies on the integration of audiovisual cues. Measuring effectiveness these can be difficult due to complexity such environments. The Audiovisual True-to-Life Assessment Auditory Rehabilitation (AVATAR) is a paradigm that was developed provide an ecological environment capture both audio and visual aspects speech measures. Previous research has shown benefit from cues measured using behavioral (e.g., word recognition) electrophysiological neural tracking) current examines, when AVATAR paradigm, if measures yield similar outcomes as We hypothesized would enhance scores signal-to-noise ratio (SNR) signal decreased. Twenty young (18-25 years old) participants (1 male 19 female) with normal hearing participated our study. For experiment, we administered lists sentences adaptive procedure estimate reception threshold (SRT). 35 randomized across five SNR levels (silence, 0, -3, -6, -9 dB) two conditions (audio-only audiovisual). used tracking decoder measure reconstruction accuracies for each participant. observed most had higher condition compared audio-only moderate high noise. found may correlate shows benefit.

Language: Английский

---

Resting-state functional connectivity changes following audio-tactile speech training

Frontiers in Neuroscience, Journal Year: 2025, Volume and Issue: 19

Published: April 29, 2025

Understanding speech in background noise is a challenging task, especially when the signal also distorted. In series of previous studies, we have shown that comprehension can improve if, simultaneously with auditory speech, person receives speech-extracted low-frequency signals on their fingertips. The effect increases after short audio-tactile training. this study, used resting-state functional magnetic resonance imaging (rsfMRI) to measure spontaneous oscillations brain while at rest assess training-induced changes connectivity. We observed enhanced connectivity (FC) within right-hemisphere cluster corresponding middle temporal motion area (MT), extrastriate body (EBA), and lateral occipital cortex (LOC), which, before training, was found be more connected bilateral dorsal anterior insula. Furthermore, early visual areas demonstrated switch from increased training sensory/multisensory association parietal hub, contralateral palm receiving vibrotactile inputs, addition, right sensorimotor cortex, including finger representations, internally results altogether interpreted two main complementary frameworks. first, speech-specific, factor relates pre-existing for audio–visual processing, visual, motion, regions involved lip-reading gesture analysis under difficult acoustic conditions, upon which new network might built. other framework refers spatial/body awareness integration, both are necessary performing revealed insular regions. It possible an extended period directly strengthen connections between utterly novel multisensory task. contribute better understanding largely unknown neuronal mechanisms underlying tactile benefits may relevant rehabilitation hearing-impaired population.

Language: Английский

---

A representation of abstract linguistic categories in the visual system underlies successful lipreading

NeuroImage, Journal Year: 2023, Volume and Issue: 282, P. 120391 - 120391

Published: Sept. 25, 2023

There is considerable debate over how visual speech processed in the absence of sound and whether neural activity supporting lipreading occurs brain areas. Much ambiguity stems from a lack behavioral grounding neurophysiological analyses that cannot disentangle high-level linguistic phonetic/energetic contributions speech. To address this, we recorded EEG human observers as they watched silent videos, half which were novel previously rehearsed with accompanying audio. We modeled responses to reflected processing low-level features (motion, lip movements) higher-level categorical representation units, known visemes. The ability these visemes account for – beyond motion movements was significantly enhanced videos way correlated participants' trial-by-trial lipread Source localization viseme showed clear cortex, no strong evidence involvement auditory interpret this support idea system produces its own specialized 1) well-described by features, 2) dissociable movements, 3) predictive ability. also suggest reinterpretation previous findings cortical activation during consistent hierarchical accounts audiovisual perception.

Language: Английский

---