Relating EEG to continuous speech using deep neural networks: a review

Journal of Neural Engineering, Journal Year: 2023, Volume and Issue: 20(4), P. 041003 - 041003

Published: July 13, 2023

Abstract Objective. When a person listens to continuous speech, corresponding response is elicited in the brain and can be recorded using electroencephalography (EEG). Linear models are presently used relate EEG recording speech signal. The ability of linear find mapping between these two signals as measure neural tracking speech. Such limited they assume linearity EEG-speech relationship, which omits nonlinear dynamics brain. As an alternative, deep learning have recently been Approach. This paper reviews comments on deep-learning-based studies that single- or multiple-speakers paradigms. We point out recurrent methodological pitfalls need for standard benchmark model analysis. Main results. gathered 29 studies. main issues we found biased cross-validations, data leakage leading over-fitted models, disproportionate size compared model’s complexity. In addition, address requirements analysis, such public datasets, common evaluation metrics, good practices match-mismatch task. Significance. present review summarizing while addressing important considerations this newly expanding field. Our study particularly relevant given growing application decoding.

Language: Английский

---

SparrKULee: A Speech-Evoked Auditory Response Repository from KU Leuven, Containing the EEG of 85 Participants

Data, Journal Year: 2024, Volume and Issue: 9(8), P. 94 - 94

Published: July 26, 2024

Researchers investigating the neural mechanisms underlying speech perception often employ electroencephalography (EEG) to record brain activity while participants listen spoken language. The high temporal resolution of EEG enables study responses fast and dynamic signals. Previous studies have successfully extracted characteristics from data and, conversely, predicted features. Machine learning techniques are generally employed construct encoding decoding models, which necessitate a substantial quantity data. We present SparrKULee, Speech-evoked Auditory Repository data, measured at KU Leuven, comprising 64-channel recordings 85 young individuals with normal hearing, each whom listened 90–150 min natural speech. This dataset is more extensive than any currently available in terms both number per participant. It suitable for training larger machine models. evaluate using linear state-of-the-art non-linear models encoding/decoding match/mismatch paradigm, providing benchmark scores future research.

Language: Английский

---

Beyond linear neural envelope tracking: a mutual information approach

Journal of Neural Engineering, Journal Year: 2023, Volume and Issue: 20(2), P. 026007 - 026007

Published: Feb. 22, 2023

Objective.The human brain tracks the temporal envelope of speech, which contains essential cues for speech understanding. Linear models are most common tool to study neural tracking. However, information on how is processed can be lost since nonlinear relations precluded. Analysis based mutual (MI), other hand, detect both linear and gradually becoming more popular in field Yet, several different approaches calculating MI applied with no consensus approach use. Furthermore, added value techniques remains a subject debate field. The present paper aims resolve these open questions.Approach.We analyzed electroencephalography (EEG) data participants listening continuous analyses models.Main results.Comparing approaches, we conclude that results reliable robust using Gaussian copula approach, first transforms standard Gaussians. With this analysis valid technique studying Like models, it allows spatial interpretations processing, peak latency analyses, applications multiple EEG channels combined. In final analysis, tested whether components were response by removing all data. We robustly detected single-subject level analysis.Significance.We demonstrate processes way. Unlike detects such relations, proving its addition, retains characteristics an advantage when complex (nonlinear) deep networks.

Language: Английский

---

CCSUMSP: A cross-subject Chinese speech decoding framework with unified topology and multi-modal semantic pre-training

Information Fusion, Journal Year: 2025, Volume and Issue: unknown, P. 103022 - 103022

Published: Feb. 1, 2025

Language: Английский

---

CORGEE: Real-Time Hearing Diagnostics Based on EEG Responses to Natural Speech

Springer briefs in electrical and computer engineering, Journal Year: 2025, Volume and Issue: unknown, P. 39 - 52

Published: Jan. 1, 2025

Language: Английский

---

Speech Reception Threshold Estimation via EEG‐Based Continuous Speech Envelope Reconstruction

European Journal of Neuroscience, Journal Year: 2025, Volume and Issue: 61(6)

Published: March 1, 2025

ABSTRACT This study investigates the potential of speech reception threshold (SRT) estimation through electroencephalography (EEG) based envelope reconstruction techniques with continuous speech. Additionally, we investigate influence stimuli's signal‐to‐noise ratio (SNR) on temporal response function (TRF). Twenty young normal‐hearing participants listened to audiobook excerpts varying background noise levels while EEG was recorded. A linear decoder trained reconstruct from data. The accuracy calculated as Pearson's correlation between reconstructed and actual envelopes. An SRT estimate (SRT neuro ) obtained midpoint a sigmoid fitted versus SNR data points. TRF estimated at each level, followed by statistical analysis reveal significant effects latencies amplitudes most prominent components. within 3 dB behavioral for all participants. showed latency decrease N1 P2 amplitude magnitude increase increasing SNR. results suggest that both components are influenced changes in SNR, indicating they may be linked same underlying neural process.

Language: Английский

---

Structural and functional brain changes in people with knee osteoarthritis: a scoping review

PeerJ, Journal Year: 2023, Volume and Issue: 11, P. e16003 - e16003

Published: Sept. 7, 2023

Background Knee osteoarthritis is a highly prevalent disease worldwide that leads to functional disability and chronic pain. It has been shown not only changes are generated at the joint level in these individuals, but also neuroplastic produced different brain areas, especially those areas related pain perception, therefore, objective of this research was identify compare structural knee OA versus healthy subjects. Methodology Searches MEDLINE (PubMed), EMBASE, WOS, CINAHL, SCOPUS, Health Source, Epistemonikos databases were conducted explore available evidence on occurring people with OA. Data recorded study characteristics, participant assessment techniques. The methodological quality studies analysed Newcastle Ottawa Scale. Results Sixteen met inclusion criteria. A decrease volume gray matter insular region, parietal lobe, cingulate cortex, hippocampus, visual temporal prefrontal basal ganglia found However, opposite occurred frontal nucleus accumbens, amygdala region somatosensory where an increase evidenced. Moreover, decreased connectivity lobe from insula, parietal, insula subcallosal area, shown. Conclusion All findings suggestive affecting matrix

Language: Английский

---

Auditory EEG decoding challenge for ICASSP 2023

IEEE Open Journal of Signal Processing, Journal Year: 2024, Volume and Issue: 5, P. 652 - 661

Published: Jan. 1, 2024

This paper describes the auditory EEG challenge, organized as one of Signal Processing Grand Challenges at ICASSP 2023. The challenge provides recordings 85 subjects who listened to continuous speech, audiobooks or podcasts, while their brain activity was recorded. 71 were provided a training set such that participants could train models on relatively large dataset. remaining 14 used held-out in evaluating challenge. consists two tasks relate electroencephalogram (EEG) signals presented speech stimulus. first task, match-mismatch, aims determine which segments induced given segment. In second regression goal is reconstruct envelope from EEG. For match-mismatch performance different teams close baseline model, and did generalize well unseen subjects. contrast, top significantly improved over stories test failing

Language: Английский

---

Decoding Envelope and Frequency-Following EEG Responses to Continuous Speech Using Deep Neural Networks

IEEE Open Journal of Signal Processing, Journal Year: 2024, Volume and Issue: 5, P. 700 - 716

Published: Jan. 1, 2024

The electroencephalogram (EEG) offers a non-invasive means by which listener's auditory system may be monitored during continuous speech perception. Reliable auditory-EEG decoders could facilitate the objective diagnosis of hearing disorders, or find applications in cognitively-steered aids. Previously, we developed for ICASSP Auditory EEG Signal Processing Grand Challenge (SPGC). These aimed to solve match-mismatch task: given short temporal segment recordings, and two candidate segments, task is identify segments temporally aligned, matched, with segment. made use cortical responses envelope, as well speech-related frequency-following responses, relate recordings stimuli. Here comprehensively document methods were developed. We extend our previous analysis exploring association between speaker characteristics (pitch sex) classification accuracy, provide full statistical final performance evaluated on heldout portion dataset. Finally, generalisation capabilities are characterised, evaluating them using an entirely different dataset contains recorded under variety speech-listening conditions. results show that achieve accurate robust accuracies, they can even serve attention without additional training.

Language: Английский

---

Multi-Head Attention and GRU for Improved Match-Mismatch Classification of Speech Stimulus and EEG Response

ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Journal Year: 2023, Volume and Issue: unknown

Published: May 5, 2023

This work is based on the participation by HyperAttention team in Auditory EEG Decoding Challenge, 2023 (ICASSP Signal Processing Grand Challenge) task 1, which deals with match-mismatch classification of speech stimuli and responses human listeners. We demonstrate benefits using mel-spectrograms instead envelopes as input features well effectiveness Multi-Head Attention GRU for processing. With a total score 79.05 %, we reach second place challenge.

Language: Английский

---