Evidence of a predictive coding hierarchy in the human brain listening to speech

Nature Human Behaviour, Journal Year: 2023, Volume and Issue: 7(3), P. 430 - 441

Published: March 2, 2023

Abstract Considerable progress has recently been made in natural language processing: deep learning algorithms are increasingly able to generate, summarize, translate and classify texts. Yet, these models still fail match the abilities of humans. Predictive coding theory offers a tentative explanation this discrepancy: while optimized predict nearby words, human brain would continuously hierarchy representations that spans multiple timescales. To test hypothesis, we analysed functional magnetic resonance imaging signals 304 participants listening short stories. First, confirmed activations modern linearly map onto responses speech. Second, showed enhancing with predictions span timescales improves mapping. Finally, organized hierarchically: frontoparietal cortices higher-level, longer-range more contextual than temporal cortices. Overall, results strengthen role hierarchical predictive processing illustrate how synergy between neuroscience artificial intelligence can unravel computational bases cognition.

Language: Английский

---

High-resolution image reconstruction with latent diffusion models from human brain activity

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2023, Volume and Issue: unknown, P. 14453 - 14463

Published: June 1, 2023

Reconstructing visual experiences from human brain activity offers a unique way to understand how the represents world, and interpret connection between computer vision models our system. While deep generative have recently been employed for this task, reconstructing realistic images with high semantic fidelity is still challenging problem. Here, we propose new method based on diffusion model (DM) reconstruct obtained via functional magnetic resonance imaging (fMRI). More specifically, rely latent (LDM) termed Stable Diffusion. This reduces computational cost of DMs, while preserving their performance. We also characterize inner mechanisms LDM by studying its different components (such as vector image Z, conditioning inputs C, elements denoising U-Net) relate distinct functions. show that proposed can high-resolution in straight-forward fashion, without need any additional training fine-tuning complex deep-learning models. provide quantitative interpretation neuroscientific perspective. Overall, study proposes promising activity, provides framework understanding DMs. Please check out webpage at https://sites.google.com/view/stablediffusion-withbrain/.

Language: Английский

---

A shared model-based linguistic space for transmitting our thoughts from brain to brain in natural conversations

Neuron, Journal Year: 2024, Volume and Issue: 112(18), P. 3211 - 3222.e5

Published: Aug. 2, 2024

Language: Английский

---

MSBKA: A Multi-Strategy Improved Black-Winged Kite Algorithm for Feature Selection of Natural Disaster Tweets Classification

Biomimetics, Journal Year: 2025, Volume and Issue: 10(1), P. 41 - 41

Published: Jan. 10, 2025

With the advancement of Internet, social media platforms have gradually become powerful in spreading crisis-related content. Identifying informative tweets associated with natural disasters is beneficial for rescue operation. When faced massive text data, choosing pivotal features, reducing calculation expense, and increasing model classification performance a significant challenge. Therefore, this study proposes multi-strategy improved black-winged kite algorithm (MSBKA) feature selection disaster based on wrapper method's principle. Firstly, BKA by utilizing enhanced Circle mapping, integrating hierarchical reverse learning, introducing Nelder-Mead method. Then, MSBKA combined excellent classifier SVM (RBF kernel function) to construct hybrid model. Finally, MSBKA-SVM performs tweet tasks. The empirical analysis data from four shows that proposed has achieved an accuracy 0.8822. Compared GA, PSO, SSA, BKA, increased 4.34%, 2.13%, 2.94%, 6.35%, respectively. This research proves can play supporting role risk.

Language: Английский

---

High-resolution image reconstruction with latent diffusion models from human brain activity

bioRxiv (Cold Spring Harbor Laboratory), Journal Year: 2022, Volume and Issue: unknown

Published: Nov. 21, 2022

Reconstructing visual experiences from human brain activity offers a unique way to understand how the represents world, and interpret connection between computer vision models our system. While deep generative have recently been employed for this task, reconstructing realistic images with high semantic fidelity is still challenging problem. Here, we propose new method based on diffusion model (DM) reconstruct obtained via functional magnetic resonance imaging (fMRI). More specifically, rely latent (LDM) termed Stable Diffusion. This reduces computational cost of DMs, while preserving their performance. We also characterize inner mechanisms LDM by studying its different components (such as vector image Z, conditioning inputs C, elements denoising U-Net) relate distinct functions. show that proposed can high-resolution in straightforward fashion, without need any additional training fine-tuning complex deep-learning models. provide quantitative interpretation neuroscientific perspective. Overall, study proposes promising activity, provides framework understanding DMs. Please check out webpage at https://sites.google.com/view/stablediffusion-with-brain/

Language: Английский

---

Semantic Representations during Language Comprehension Are Affected by Context

Journal of Neuroscience, Journal Year: 2023, Volume and Issue: 43(17), P. 3144 - 3158

Published: March 27, 2023

The meaning of words in natural language depends crucially on context. However, most neuroimaging studies word use isolated and sentences with little Because the brain may process differently from how it processes simplified stimuli, there is a pressing need to determine whether prior results generalize language. fMRI was used record human activity while four subjects (two female) read conditions that vary context: narratives, sentences, blocks semantically similar words, words. We then compared signal-to-noise ratio (SNR) evoked responses, we voxelwise encoding modeling approach compare representation semantic information across conditions. find consistent effects varying First, stimuli more context evoke responses higher SNR bilateral visual, temporal, parietal, prefrontal cortices Second, increasing increases at group level. In individual subjects, only consistently widespread information. Third, affects voxel tuning. Finally, models estimated using do not well These show has large quality data brain. Thus, regime. SIGNIFICANCE STATEMENT Context an important part understanding language, but Here, examined out-of-context improves neuro-imaging changes where represented suggest findings daily life.

Language: Английский

---

A natural language fMRI dataset for voxelwise encoding models

Scientific Data, Journal Year: 2023, Volume and Issue: 10(1)

Published: Aug. 23, 2023

Abstract Speech comprehension is a complex process that draws on humans’ abilities to extract lexical information, parse syntax, and form semantic understanding. These sub-processes have traditionally been studied using separate neuroimaging experiments attempt isolate specific effects of interest. More recently it has become possible study all stages language in single experiment narrative natural stimuli. The resulting data are richly varied at every level, enabling analyses can probe everything from spectral representations high-level meaning. We provide dataset containing BOLD fMRI responses recorded while 8 participants each listened 27 complete, natural, stories (~6 hours). This includes pre-processed raw MRIs, as well hand-constructed 3D cortical surfaces for participant. To address the challenges analyzing naturalistic data, this accompanied by python library basic code creating voxelwise encoding models. Altogether, provides large novel resource understanding speech processing human brain.

Language: Английский

---

Shared functional specialization in transformer-based language models and the human brain

Nature Communications, Journal Year: 2024, Volume and Issue: 15(1)

Published: June 29, 2024

Abstract When processing language, the brain is thought to deploy specialized computations construct meaning from complex linguistic structures. Recently, artificial neural networks based on Transformer architecture have revolutionized field of natural language processing. Transformers integrate contextual information across words via structured circuit computations. Prior work has focused internal representations (“embeddings”) generated by these circuits. In this paper, we instead analyze directly: deconstruct into functionally-specialized “transformations” that words. Using functional MRI data acquired while participants listened naturalistic stories, first verify transformations account for considerable variance in activity cortical network. We then demonstrate emergent performed individual, “attention heads” differentially predict specific regions. These heads fall along gradients corresponding different layers and context lengths a low-dimensional space.

Language: Английский

---

Eye movements track prioritized auditory features in selective attention to natural speech

Nature Communications, Journal Year: 2024, Volume and Issue: 15(1)

Published: May 1, 2024

Abstract Over the last decades, cognitive neuroscience has identified a distributed set of brain regions that are critical for attention. Strong anatomical overlap with oculomotor processes suggests joint network attention and eye movements. However, role this shared in complex, naturalistic environments remains understudied. Here, we investigated movements relation to (un)attended sentences natural speech. Combining simultaneously recorded tracking magnetoencephalographic data temporal response functions, show gaze tracks attended speech, phenomenon termed ocular speech tracking. Ocular even differentiates target from distractor multi-speaker context is further related intelligibility. Moreover, provide evidence its contribution neural differences processing, emphasizing necessity consider activity future research interpretation auditory cognition.

Language: Английский

---

XGBoost-B-GHM: An Ensemble Model with Feature Selection and GHM Loss Function Optimization for Credit Scoring

Systems, Journal Year: 2024, Volume and Issue: 12(7), P. 254 - 254

Published: July 14, 2024

Credit evaluation has always been an important part of the financial field. The existing credit methods have difficulty in solving problems redundant data features and imbalanced samples. In response to above issues, ensemble model combining advanced feature selection algorithm optimized loss function is proposed, which can be applied field improve risk management ability institutions. Firstly, Boruta embedded for selection, effectively reduce dimension noise model’s capacity generalization by automatically identifying screening out that are highly correlated with target variables. Then, GHM incorporated into XGBoost tackle issue skewed sample distribution, common classification, further classification prediction performance model. comparative experiments on four large datasets demonstrate proposed method superior mainstream extract handle problem

Language: Английский

---