THINGS-data, a multimodal collection of large-scale datasets for investigating object representations in human brain and behavior

eLife, Journal Year: 2023, Volume and Issue: 12

Published: Feb. 27, 2023

Understanding object representations requires a broad, comprehensive sampling of the objects in our visual world with dense measurements brain activity and behavior. Here, we present THINGS-data, multimodal collection large-scale neuroimaging behavioral datasets humans, comprising densely sampled functional MRI magnetoencephalographic recordings, as well 4.70 million similarity judgments response to thousands photographic images for up 1,854 concepts. THINGS-data is unique its breadth richly annotated objects, allowing testing countless hypotheses at scale while assessing reproducibility previous findings. Beyond insights promised by each individual dataset, multimodality allows combining much broader view into processing than previously possible. Our analyses demonstrate high quality provide five examples hypothesis-driven data-driven applications. constitutes core public release THINGS initiative (https://things-initiative.org) bridging gap between disciplines advancement cognitive neuroscience.

Language: Английский

---

Better models of human high-level visual cortex emerge from natural language supervision with a large and diverse dataset

Nature Machine Intelligence, Journal Year: 2023, Volume and Issue: 5(12), P. 1415 - 1426

Published: Nov. 13, 2023

Language: Английский

---

Is my “red” your “red”?: Evaluating structural correspondences between color similarity judgments using unsupervised alignment

iScience, Journal Year: 2025, Volume and Issue: 28(3), P. 112029 - 112029

Published: Feb. 15, 2025

Whether one person's subjective experience of the "redness" red is equivalent to another's a fundamental question in consciousness studies. Intersubjective comparison relational structures sensory experiences, termed "qualia structures", can constrain question. We propose an unsupervised alignment method, based on optimal transport, find mapping between similarity experiences without presupposing correspondences (such as "red-to-red"). After collecting judgments for 93 colors, we showed that derived from color-neurotypical participants be "correctly" aligned at group level. In contrast, those color-blind could not with participants. Our results provide quantitative evidence interindividual structural equivalence or difference color qualia, implying people's "red" relationally other color-neurotypical's "red", but "red". This method applicable across modalities, enabling general exploration experiences.

Language: Английский

---

A large and rich EEG dataset for modeling human visual object recognition

NeuroImage, Journal Year: 2022, Volume and Issue: 264, P. 119754 - 119754

Published: Nov. 15, 2022

Language: Английский

---

Cortical topographic motifs emerge in a self-organized map of object space

Science Advances, Journal Year: 2023, Volume and Issue: 9(25)

Published: June 21, 2023

The human ventral visual stream has a highly systematic organization of object information, but the causal pressures driving these topographic motifs are debated. Here, we use self-organizing principles to learn representation data manifold deep neural network representational space. We find that smooth mapping this space showed many brain-like motifs, with large-scale by animacy and real-world size, supported mid-level feature tuning, naturally emerging face- scene-selective regions. While some theories object-selective cortex posit differently tuned regions brain reflect collection distinctly specified functional modules, present work provides computational support for an alternate hypothesis tuning topography unified

Language: Английский

---

THINGSplus: New norms and metadata for the THINGS database of 1854 object concepts and 26,107 natural object images

Behavior Research Methods, Journal Year: 2023, Volume and Issue: 56(3), P. 1583 - 1603

Published: April 24, 2023

To study visual and semantic object representations, the need for well-curated concepts images has grown significantly over past years. address this, we have previously developed THINGS, a large-scale database of 1854 systematically sampled with 26,107 high-quality naturalistic these concepts. With THINGSplus, extend THINGS by adding concept- image-specific norms metadata all one copyright-free image example per concept. Concept-specific were collected properties real-world size, manmadeness, preciousness, liveliness, heaviness, naturalness, ability to move or be moved, graspability, holdability, pleasantness, arousal. Further, provide 53 superordinate categories as well typicality ratings their members. Image-specific includes nameability measure, based on human-generated labels objects depicted in images. Finally, identified new public domain Property (M = 0.97, SD 0.03) 0.01) demonstrate excellent consistency, subsequently arousal only exception (r 0.69). Our property 0.85, 0.11) 0.72, 0.74, 0.88) data correlated strongly external norms, again lowest validity 0.41, 0.08). summarize, THINGSplus provides large-scale, externally validated extension existing an important allowing detailed selection stimuli control variables wide range research interested processing, language, memory.

Language: Английский

---

Distributed representations of behaviour-derived object dimensions in the human visual system

Nature Human Behaviour, Journal Year: 2024, Volume and Issue: unknown

Published: Sept. 9, 2024

Language: Английский

---

Spontaneous eye movements reflect the representational geometries of conceptual spaces

Proceedings of the National Academy of Sciences, Journal Year: 2024, Volume and Issue: 121(17)

Published: April 18, 2024

Functional neuroimaging studies indicate that the human brain can represent concepts and their relational structure in memory using coding schemes typical of spatial navigation. However, whether we read out internal representational geometries conceptual spaces solely from behavior remains unclear. Here, report between might be reflected spontaneous eye movements during verbal fluency tasks: When asked participants to randomly generate numbers, correlated with distances along left-to-right one-dimensional geometry number space (mental line), while they scaled distance ring-like two-dimensional color (color wheel) when generated names. Moreover, produced animal names, low-dimensional similarity word frequencies. These results suggest used internally organize gaze behavior.

Language: Английский

---

Gromov–Wasserstein unsupervised alignment reveals structural correspondences between the color similarity structures of humans and large language models

Scientific Reports, Journal Year: 2024, Volume and Issue: 14(1)

Published: July 10, 2024

Large Language Models (LLMs), such as the General Pre-trained Transformer (GPT), have shown remarkable performance in various cognitive tasks. However, it remains unclear whether these models ability to accurately infer human perceptual representations. Previous research has addressed this question by quantifying correlations between similarity response patterns of humans and LLMs. Correlation provides a measure similarity, but relies pre-defined item labels does not distinguish category- item- level falling short characterizing detailed structural correspondence To assess their equivalence more detail, we propose use an unsupervised alignment method based on Gromov-Wasserstein optimal transport (GWOT). GWOT allows for comparison structures without relying label correspondences can reveal fine-grained similarities differences that may be detected simple correlation analysis. Using large dataset judgments 93 colors, compared color (color-neurotypical color-atypical participants) two GPT (GPT-3.5 GPT-4). Our results show structure color-neurotypical participants remarkably well aligned with GPT-4 and, lesser extent, GPT-3.5. These contribute methodological advancements comparing LLMs perception, highlight potential methods correspondences.

Language: Английский

---

Principles of intensive human neuroimaging

Trends in Neurosciences, Journal Year: 2024, Volume and Issue: unknown

Published: Oct. 1, 2024

Language: Английский

---