High-resolution image reconstruction with latent diffusion models from human brain activity

bioRxiv (Cold Spring Harbor Laboratory), Journal Year: 2022, Volume and Issue: unknown

Published: Nov. 21, 2022

Reconstructing visual experiences from human brain activity offers a unique way to understand how the represents world, and interpret connection between computer vision models our system. While deep generative have recently been employed for this task, reconstructing realistic images with high semantic fidelity is still challenging problem. Here, we propose new method based on diffusion model (DM) reconstruct obtained via functional magnetic resonance imaging (fMRI). More specifically, rely latent (LDM) termed Stable Diffusion. This reduces computational cost of DMs, while preserving their performance. We also characterize inner mechanisms LDM by studying its different components (such as vector image Z, conditioning inputs C, elements denoising U-Net) relate distinct functions. show that proposed can high-resolution in straightforward fashion, without need any additional training fine-tuning complex deep-learning models. provide quantitative interpretation neuroscientific perspective. Overall, study proposes promising activity, provides framework understanding DMs. Please check out webpage at https://sites.google.com/view/stablediffusion-with-brain/

Language: Английский

---

Prediction during language comprehension: what is next?

Trends in Cognitive Sciences, Journal Year: 2023, Volume and Issue: 27(11), P. 1032 - 1052

Published: Sept. 11, 2023

Prediction is often regarded as an integral aspect of incremental language comprehension, but little known about the cognitive architectures and mechanisms that support it. We review studies showing listeners readers use all manner contextual information to generate multifaceted predictions upcoming input. The nature these may vary between individuals owing differences in experience, among other factors. then turn unresolved questions which guide search for underlying mechanisms. (i) Is prediction essential processing or optional strategy? (ii) Are generated from within system by domain-general processes? (iii) What relationship memory? (iv) Does comprehension require simulation via production system? discuss promising directions making progress answering developing a mechanistic understanding language.

Language: Английский

---

Machine learning and artificial intelligence in neuroscience: A primer for researchers

Brain Behavior and Immunity, Journal Year: 2023, Volume and Issue: 115, P. 470 - 479

Published: Nov. 14, 2023

Artificial intelligence (AI) is often used to describe the automation of complex tasks that we would attribute to. Machine learning (ML) commonly understood as a set methods develop an AI. Both have seen recent boom in usage, both scientific and commercial fields. For community, ML can solve bottle necks created by complex, multi-dimensional data generated, for example, functional brain imaging or *omics approaches. here identify patterns could not been found using traditional statistic However, comes with serious limitations need be kept mind: their tendency optimise solutions input means it crucial importance externally validate any findings before considering them more than hypothesis. Their black-box nature implies decisions usually cannot understood, which renders use medical decision making problematic lead ethical issues. Here, present introduction curious field ML/AI. We explain principles well methodological advancements discuss risks what see future directions field. Finally, show practical examples neuroscience illustrate ML.

Language: Английский

---

Functional neuroimaging as a catalyst for integrated neuroscience

Nature, Journal Year: 2023, Volume and Issue: 623(7986), P. 263 - 273

Published: Nov. 8, 2023

Language: Английский

---

Generating meaning: active inference and the scope and limits of passive AI

Trends in Cognitive Sciences, Journal Year: 2023, Volume and Issue: 28(2), P. 97 - 112

Published: Nov. 15, 2023

Prominent accounts of sentient behavior depict brains as generative models organismic interaction with the world, evincing intriguing similarities current advances in artificial intelligence (AI). However, because they contend control purposive, life-sustaining sensorimotor interactions, living organisms are inextricably anchored to body and world. Unlike passive learned by AI systems, must capture sensory consequences action. This allows embodied agents intervene upon their worlds ways that constantly put best test, thus providing a solid bedrock is – we argue essential development genuine understanding. We review resulting implications consider future directions for AI.

Language: Английский

---