A large-scale examination of inductive biases shaping high-level visual representation in brains and machines

Nature Communications, Год журнала: 2024, Номер 15(1)

Опубликована: Окт. 30, 2024

Язык: Английский

---

Self-supervision deep learning models are better models of human high-level visual cortex: The roles of multi-modality and dataset training size

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2025, Номер unknown

Опубликована: Янв. 13, 2025

Abstract With the rapid development of Artificial Neural Network based visual models, many studies have shown that these models show unprecedented potence in predicting neural responses to images cortex. Lately, advances computer vision introduced self-supervised where a model is trained using supervision from natural properties training set. This has led examination their prediction performance, which revealed better than supervised for with language or image-only supervision. In this work, we delve deeper into models’ ability explain representations object categories. We compare differed objectives examine they diverge predict fMRI and MEG recordings while participants are presented different Results both self-supervision was advantageous comparison classification training. addition, predictor later stages perception, shows consistent advantage over longer duration, beginning 80ms after exposure. Examination effect data size large dataset did not necessarily improve predictions, particular models. Finally, correspondence hierarchy each cortex showed image only conclude consistently recordings, type reveals property activity, language-supervision explaining onsets, explains long very early latencies response, naturally sharing corresponding hierarchical structure as brain.

Язык: Английский

---

Predictive Coding algorithms induce brain-like responses in Artificial Neural Networks

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2025, Номер unknown

Опубликована: Янв. 20, 2025

Abstract This study explores whether predictive coding (PC) inspired Deep Neural Networks can serve as biologically plausible neural network models of the brain. We compared two PC-inspired training objectives, a and contrastive approach, to supervised baseline in simple Recurrent Network (RNN) architecture. evaluated on key signatures PC, including mismatch responses, formation priors, learning semantic information. Our results show that models, especially locally trained model, exhibited these PC-like behaviors better than Supervised or an Untrained RNN. Further, we found activity regularization evokes response-like effects across all suggesting it may proxy for energy-saving principles PC. Finally, find Gain Control (an important mechanism PC framework) be implemented using weight regularization. Overall, our findings indicate are able capture computational processing brain, promising foundation building artificial networks. work contributes understanding relationship between biological networks, highlights potential algorithms advancing brain modelling well brain-inspired machine learning.

Язык: Английский

---

Cf-Wiad: Consistency Fusion with Weighted Instance and Adaptive Distribution for Enhanced Semi-Supervised Skin Lesion Classification

Опубликована: Янв. 1, 2025

Язык: Английский

---

Individual variation in the functional lateralization of human ventral temporal cortex: Local competition and long-range coupling

Imaging Neuroscience, Год журнала: 2025, Номер 3

Опубликована: Янв. 1, 2025

The ventral temporal cortex (VTC) of the human cerebrum is critically engaged in high-level vision. One intriguing aspect this region its functional lateralization, with neural responses to words being stronger left hemisphere, and faces right hemisphere; such patterns can be summarized a signed laterality index (LI), positive for leftward laterality. Converging evidence has suggested that word emerges couple efficiently left-lateralized frontotemporal language regions, but more mixed regarding sources lateralization face perception. Here, we use individual differences as tool test three theories VTC organization arising from (1) local competition between driven by long-range coupling processes, (2) other categories, (3) areas exhibiting social processing. First, an in-house MRI experiment, did not obtain negative correlation LIs selectivity relative object responses, find when using fixation baseline, challenging ideas driving rightward lateralization. We next examined broader LI interactions large-scale Human Connectome Project (HCP) dataset. Face were significantly anti-correlated, while body positively correlated, consistent idea generic representational cooperation may shape Last, assessed role development Within our substantial was evident text several nodes distributed text-processing circuit. In HCP data, both negatively correlated processing different subregions posterior lobe (PSL STSp, respectively). summary, no face-word VTC; instead, multiple lateralities within VTC, including Moreover, also influenced lobe, where become lateralized due language.

Язык: Английский

---

A vectorial code for semantics in human hippocampus

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2025, Номер unknown

Опубликована: Фев. 23, 2025

ABSTRACT As we listen to speech, our brains actively compute the meaning of individual words. Inspired by success large language models (LLMs), hypothesized that brain employs vectorial coding principles, such is reflected in distributed activity single neurons. We recorded responses hundreds neurons human hippocampus, which has a well-established role semantic coding, while participants listened narrative speech. find encoding contextual word simultaneous whose selectivities span multiple unrelated categories. Like embedding vectors models, distance between neural population correlates with distance; however, this effect was only observed (like BERT) and reversed non-contextual Word2Vec), suggesting depends critically on contextualization. Moreover, for subset highly semantically similar words, even embedders showed an inverse correlation distances; attribute pattern noise-mitigating benefits contrastive coding. Finally, further support critical context, range covaries lexical polysemy. Ultimately, these results hypothesis hippocampus follows principles.

Язык: Английский

---

Language modulates vision: Evidence from neural networks and human brain-lesion models

Research Square (Research Square), Год журнала: 2025, Номер unknown

Опубликована: Март 27, 2025

Comparing information structures in between deep neural networks (DNNs) and the human brain has become a key method for exploring their similarities differences. Recent research shown better alignment of vision-language DNN models, such as CLIP, with activity ventral occipitotemporal cortex (VOTC) than earlier vision supporting idea that language modulates visual perception. However, interpreting results from comparisons is inherently limited due to "black box" nature DNNs. To address this, we combined model–brain fitness analyses lesion data examine how disrupting communication pathway systems causally affects ability vision–language DNNs explain VOTC. Across four diverse datasets, CLIP consistently outperformed both label-supervised (ResNet) unsupervised (MoCo) models predicting VOTC activity. This advantage was left-lateralized, aligning network. Analyses 33 stroke patients revealed reduced white matter integrity region left angular gyrus correlated decreased performance increased MoCo performance, indicating dynamic influence processing on These findings support integration modulation neurocognitive vision, reinforcing concepts models. The sensitivity similarity specific lesions demonstrates leveraging manipulation promising framework evaluating developing brain-like computer

Язык: Английский

---

Object knowledge representation in the human visual cortex requires a connection with the language system

PLoS Biology, Год журнала: 2025, Номер 23(5), С. e3003161 - e3003161

Опубликована: Май 20, 2025

How world knowledge is stored in the human brain a central question cognitive neuroscience. Object effects have been commonly observed higher-order sensory association cortices, with role of language being highly debated. Using object color as test case, we investigated whether communication system plays necessary neural representation visual cortex and corresponding behaviors, combining diffusion imaging (measuring white-matter structural integrity), functional MRI (fMRI; measuring knowledge), neuropsychological assessments behavioral integrity) group patients who suffered from stroke ( N = 33; 18 left-hemisphere lesions, 11 right-hemisphere 4 bilateral lesions). The integrity loss connection between anterior temporal region ventral had significant effect on strength behavior across modalities. These contributions could not be explained by potential early perception pathway or confounding variables. Our experiments reveal contribution vision-language occipitotemporal (VOTC) highlighting significance language-sensory interface.

Язык: Английский

---

Individual variation in the functional lateralization of human ventral temporal cortex: Local competition and long-range coupling

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2024, Номер unknown

Опубликована: Окт. 16, 2024

Abstract The ventral temporal cortex (VTC) of the human cerebrum is critically engaged in computations related to high-level vision. One intriguing aspect this region its asymmetric organization and functional lateralization. Notably, VTC, neural responses words are stronger left hemisphere, whereas faces right hemisphere. Converging evidence has suggested that left-lateralized word emerge couple efficiently with frontotemporal language regions, but more mixed regarding sources right-lateralization for face perception. Here, we use individual differences as a tool adjudicate between three theories VTC arising from: 1) local competition faces, 2) other categories, 3) long-range coupling areas subject their own competition. First, an in-house MRI experiment, demonstrated laterality both substantial reliable within right-handed population young adults. We found no (anti-)correlation selectivity relative object responses, positive correlation when using fixation baseline, challenging ideas faces. next examined broader large-scale Human Connectome Project (HCP) dataset. Face were significantly anti-correlated, while body positively correlated, consistent idea generic representational cooperation may shape Last, assessed role development laterality. Within our was evident text several nodes distributed text-processing circuit. In HCP data, negatively correlated laterality, social perception same areas, effect processing representations, driven by processing. conclude interactions heterogeneous hemispheric specializations visual cortex.

Язык: Английский

---

Modular representations emerge in neural networks trained to perform context-dependent tasks

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2024, Номер unknown

Опубликована: Окт. 1, 2024

Abstract The brain has large-scale modular structure in the form of regions, which are thought to arise from constraints on connectivity and physical geometry cortical sheet. In contrast, experimental theoretical work argued both for against existence specialized sub-populations neurons (modules) within single regions. By studying artificial neural networks, we show that this local modularity emerges support context-dependent behavior, but only when input is low-dimensional. No anatomical required. We also specialization at population level (different modules correspond orthogonal subspaces). Modularity yields abstract representations, allows rapid learning generalization novel tasks, facilitates related contexts. Non-modular representations facilitate unrelated Our findings reconcile conflicting results make predictions future experiments.

Язык: Английский

---