Learning with sparse reward in a gap junction network inspired by the insect mushroom body DOI Creative Commons
Tianqi Wei, Qinghai Guo, Barbara Webb

и другие.

PLoS Computational Biology, Год журнала: 2024, Номер 20(5), С. e1012086 - e1012086

Опубликована: Май 23, 2024

Animals can learn in real-life scenarios where rewards are often only available when a goal is achieved. This ‘distal’ or ‘sparse’ reward problem remains challenge for conventional reinforcement learning algorithms. Here we investigate an algorithm such scenarios, inspired by the possibility that axo-axonal gap junction connections, observed neural circuits with parallel fibres as insect mushroom body, could form resistive network. In network, active node represents task state, connections between nodes represent state transitions and their connection to actions, current flow target guide decision making. Building on evidence weights adaptive, propose experience of modulate graph encoding structure. We demonstrate approach be used efficient under sparse rewards, discuss whether it plausible account body.

Язык: Английский

Neural learning rules for generating flexible predictions and computing the successor representation DOI Creative Commons
Ching Fang, Dmitriy Aronov,

LF Abbott

и другие.

eLife, Год журнала: 2023, Номер 12

Опубликована: Март 16, 2023

The predictive nature of the hippocampus is thought to be useful for memory-guided cognitive behaviors. Inspired by reinforcement learning literature, this notion has been formalized as a map called successor representation (SR). SR captures number observations about hippocampal activity. However, algorithm does not provide neural mechanism how such representations arise. Here, we show dynamics recurrent network naturally calculate when synaptic weights match transition probability matrix. Interestingly, horizon can flexibly modulated simply changing gain. We derive simple, biologically plausible rules learn in network. test our model with realistic inputs and data recorded during random foraging. Taken together, results suggest that more accessible circuits than previously support broad range functions.

Язык: Английский

Процитировано

41
The what, how, and why of naturalistic behavior DOI Creative Commons
Ann Kennedy

Current Opinion in Neurobiology, Год журнала: 2022, Номер 74, С. 102549 - 102549

Опубликована: Май 7, 2022

Язык: Английский

Процитировано

37
Local prediction-learning in high-dimensional spaces enables neural networks to plan DOI Creative Commons
Christoph Stöckl, Yukun Yang, Wolfgang Maass

и другие.

Nature Communications, Год журнала: 2024, Номер 15(1)

Опубликована: Март 15, 2024

Planning and problem solving are cornerstones of higher brain function. But we do not know how the does that. We show that learning a suitable cognitive map space suffices. Furthermore, this can be reduced to predict next observation through local synaptic plasticity. Importantly, resulting encodes relations between actions observations, its emergent high-dimensional geometry provides sense direction for reaching distant goals. This quasi-Euclidean simple heuristic online planning works almost as well best offline algorithms from AI. If is physical space, method automatically extracts structural regularities sequence observations it receives so generalize unseen parts. speeds up navigation in 2D mazes locomotion with complex actuator systems, such legged bodies. The learner propose require teacher, similar self-attention networks (Transformers). contrast Transformers, backpropagation errors or very large datasets learning. Hence blue-print future energy-efficient neuromorphic hardware acquires advanced capabilities autonomous on-chip

Язык: Английский

Процитировано

6
Complex behavior from intrinsic motivation to occupy future action-state path space DOI Creative Commons
Jorge Ramírez‐Ruiz,

Dmytro Grytskyy,

Chiara Mastrogiuseppe

и другие.

Nature Communications, Год журнала: 2024, Номер 15(1)

Опубликована: Июль 29, 2024

Abstract Most theories of behavior posit that agents tend to maximize some form reward or utility. However, animals very often move with curiosity and seem be motivated in a reward-free manner. Here we abandon the idea maximization propose goal is maximizing occupancy future paths actions states. According this maximum principle, rewards are means occupy path space, not per se; goal-directedness simply emerges as rational ways searching for resources so movement, understood amply, never ends. We find action-state entropy only measure consistent additivity other intuitive properties expected occupancy. provide analytical expressions relate optimal policy state-value function prove convergence our value iteration algorithm. Using discrete continuous state tasks, including high-dimensional controller, show complex behaviors such “dancing”, hide-and-seek, basic altruistic naturally result from intrinsic motivation space. All all, present theory generates both variability absence maximization.

Язык: Английский

Процитировано

4
A non-Hebbian code for episodic memory DOI Creative Commons
Rich Pang, Stefano Recanatesi

Science Advances, Год журнала: 2025, Номер 11(8)

Опубликована: Фев. 21, 2025

Hebbian plasticity has long dominated neurobiological models of memory formation. Yet, rules operating on one-shot episodic timescales rarely depend both pre- and postsynaptic spiking, challenging theory in this crucial regime. Here, we present an model governed by a simpler rule depending only presynaptic activity. We show that rule, capitalizing high-dimensional neural activity with restricted transitions, naturally stores episodes as paths through complex state spaces like those underlying world model. The resulting traces, which term path vectors, are highly expressive decodable odor-tracking algorithm. vectors robust alternatives to support sequential associative recall, along policy learning, shed light specific hippocampal rules. Thus, non-Hebbian is sufficient for flexible learning well-suited encode policies

Язык: Английский

Процитировано

0
Learning with sparse reward in a gap junction network inspired by the insect mushroom body DOI Creative Commons
Tianqi Wei, Qinghai Guo, Barbara Webb

и другие.

PLoS Computational Biology, Год журнала: 2024, Номер 20(5), С. e1012086 - e1012086

Опубликована: Май 23, 2024

Animals can learn in real-life scenarios where rewards are often only available when a goal is achieved. This ‘distal’ or ‘sparse’ reward problem remains challenge for conventional reinforcement learning algorithms. Here we investigate an algorithm such scenarios, inspired by the possibility that axo-axonal gap junction connections, observed neural circuits with parallel fibres as insect mushroom body, could form resistive network. In network, active node represents task state, connections between nodes represent state transitions and their connection to actions, current flow target guide decision making. Building on evidence weights adaptive, propose experience of modulate graph encoding structure. We demonstrate approach be used efficient under sparse rewards, discuss whether it plausible account body.

Язык: Английский

Процитировано

1