What can 1.8 billion regressions tell us about the pressures shaping high-level visual representation in brains and machines?

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2022, Номер unknown

Опубликована: Март 29, 2022

Abstract The rapid development and open-source release of highly performant computer vision models offers new potential for examining how different inductive biases impact representation learning emergent alignment with the high-level human ventral visual system. Here, we assess a diverse set 224 models, curated to enable controlled comparison model properties, testing their brain predictivity using large-scale functional magnetic resonance imaging data. We find that qualitatively architectures (e.g. CNNs versus Transformers) markedly task objectives purely contrastive vision-language alignment) achieve near equivalent degrees predictivity, when other factors are held constant. Instead, variation across training diets yields largest, most consistent effect on predictivity. Overarching properties commonly suspected increase greater effective dimensionality; learnable parameter count) were not robust indicators this more extensive survey. highlight standard model-to-brain linear re-weighting methods may be too flexible, as have very similar brain-predictivity scores, despite significant in underlying representations. Broadly, our findings point importance diet, challenge common assumptions about used link brains, concretely outline future directions leveraging full diversity existing tools probe computational principles biological artificial systems.

Язык: Английский

---

A language of thought for the mental representation of geometric shapes

Cognitive Psychology, Год журнала: 2022, Номер 139, С. 101527 - 101527

Опубликована: Ноя. 17, 2022

Язык: Английский

---

The Dorsal Visual Pathway Represents Object-Centered Spatial Relations for Object Recognition

Journal of Neuroscience, Год журнала: 2022, Номер 42(23), С. 4693 - 4710

Опубликована: Май 4, 2022

Although there is mounting evidence that input from the dorsal visual pathway crucial for object processes in ventral pathway, specific functional contributions of cortex to these remain poorly understood. Here, we hypothesized computes spatial relations among an object9s parts, a process forming global shape percepts, and transmits this information support categorization. Using fMRI with human participants (females males), discovered regions intraparietal sulcus (IPS) were selectively involved computing object-centered part relations. These exhibited task-dependent effective connectivity cortex, distinct other regions, such as those representing allocentric relations, 3D shape, tools. In subsequent experiment, found multivariate response posterior (p)IPS, defined on basis part-relations, could be used decode category at levels comparable regions. Moreover, mediation analyses further suggested IPS may account representations pathway. Together, our results highlight recognition. We suggest source ability categorize objects shape. SIGNIFICANCE STATEMENT Humans novel rapidly effortlessly. Such categorization achieved by structure, is, parts. Yet, despite their importance, it unclear how are represented neurally. computed which typically implicated visuospatial processing. fMRI, identified selective cortex. can categorization, even mediate region thought findings shed light broader network brain

Язык: Английский

---

Texture-like representation of objects in human visual cortex

Proceedings of the National Academy of Sciences, Год журнала: 2022, Номер 119(17)

Опубликована: Апрель 19, 2022

Significance Humans are exquisitely sensitive to the spatial arrangement of visual features in objects and scenes, but not textures. Category-selective regions cortex widely believed underlie object perception, suggesting such should distinguish natural images from synthesized containing similar scrambled arrangements. Contrarily, we demonstrate that representations category-selective do discriminate feature-matched scrambles can different categories, a texture-like encoding. We find insensitivity feature Imagenet-trained deep convolutional neural networks. This suggests need reconceptualize role as representing basis set complex features, useful for myriad behaviors.

Язык: Английский

---

Tasks and their role in visual neuroscience

Neuron, Год журнала: 2023, Номер 111(11), С. 1697 - 1713

Опубликована: Апрель 10, 2023

Язык: Английский

---

Many but not all deep neural network audio models capture brain responses and exhibit correspondence between model stages and brain regions

PLoS Biology, Год журнала: 2023, Номер 21(12), С. e3002366 - e3002366

Опубликована: Дек. 13, 2023

Models that predict brain responses to stimuli provide one measure of understanding a sensory system and have many potential applications in science engineering. Deep artificial neural networks emerged as the leading such predictive models visual but are less explored audition. Prior work provided examples audio-trained produced good predictions auditory cortical fMRI exhibited correspondence between model stages regions, left it unclear whether these results generalize other network and, thus, how further improve this domain. We evaluated model-brain for publicly available audio along with in-house trained on 4 different tasks. Most tested outpredicted standard spectromporal filter-bank cortex systematic correspondence: Middle best predicted primary cortex, while deep non-primary cortex. However, some state-of-the-art substantially worse predictions. recognize speech background noise better than quiet, potentially because hearing imposes constraints biological representations. The training task influenced prediction quality specific tuning properties, overall resulting from multiple generally support promise audition, though they also indicate current do not explain their entirety.

Язык: Английский

---

High-performing neural network models of visual cortex benefit from high latent dimensionality

PLoS Computational Biology, Год журнала: 2024, Номер 20(1), С. e1011792 - e1011792

Опубликована: Янв. 10, 2024

Geometric descriptions of deep neural networks (DNNs) have the potential to uncover core representational principles computational models in neuroscience. Here we examined geometry DNN visual cortex by quantifying latent dimensionality their natural image representations. A popular view holds that optimal DNNs compress representations onto low-dimensional subspaces achieve invariance and robustness, which suggests better should lower dimensional geometries. Surprisingly, found a strong trend opposite direction-neural with high-dimensional tended generalization performance when predicting cortical responses held-out stimuli both monkey electrophysiology human fMRI data. Moreover, high was associated learning new categories stimuli, suggesting higher are suited generalize beyond training domains. These findings suggest general principle whereby confers benefits cortex.

Язык: Английский

---

A unifying framework for functional organization in early and higher ventral visual cortex

Neuron, Год журнала: 2024, Номер 112(14), С. 2435 - 2451.e7

Опубликована: Май 10, 2024

Язык: Английский

---

Memorability shapes perceived time (and vice versa)

Nature Human Behaviour, Год журнала: 2024, Номер 8(7), С. 1296 - 1308

Опубликована: Апрель 22, 2024

Язык: Английский

---

Is Ockham’s razor losing its edge? New perspectives on the principle of model parsimony

Proceedings of the National Academy of Sciences, Год журнала: 2025, Номер 122(5)

Опубликована: Янв. 27, 2025

The preference for simple explanations, known as the parsimony principle, has long guided development of scientific theories, hypotheses, and models. Yet recent years have seen a number successes in employing highly complex models inquiry (e.g., 3D protein folding or climate forecasting). In this paper, we reexamine principle light these technological advancements. We review developments, including surprising benefits modeling with more parameters than data, increasing appreciation context-sensitivity data misspecification models, new tools. By integrating insights, reassess utility proxy desirable model traits, such predictive accuracy, interpretability, effectiveness guiding research, resource efficiency. conclude that are sometimes essential progress, discuss ways which complexity can play complementary roles practice.

Язык: Английский

---