Simulated synapse loss induces depression-like behaviors in deep reinforcement learning DOI Creative Commons
Eric Chalmers,

Santina Duarte,

Xena Al-Hejji

и другие.

Frontiers in Computational Neuroscience, Год журнала: 2024, Номер 18

Опубликована: Ноя. 6, 2024

Deep Reinforcement Learning is a branch of artificial intelligence that uses neural networks to model reward-based learning as it occurs in biological agents. Here we modify approach by imposing suppressive effect on the connections between neurons network—simulating dendritic spine loss observed major depressive disorder (MDD). Surprisingly, this simulated sufficient induce variety MDD-like behaviors artificially intelligent agent, including anhedonia, increased temporal discounting, avoidance, and an altered exploration/exploitation balance. Furthermore, simulating alternative longstanding reward-processing-centric conceptions MDD (dysfunction dopamine system, reward context-dependent rates, exploration) does not produce same range behaviors. These results support conceptual reduction brain connectivity (and thus information-processing capacity) rather than imbalance monoamines—though computational suggests possible explanation for dysfunction systems MDD. Reversing spine-loss our can lead rescue rewarding behavior under some conditions. This supports search treatments increase plasticity synaptogenesis, implications their effective administration.

Язык: Английский

A model of how hierarchical representations constructed in the hippocampus are used to navigate through space DOI Creative Commons
Eric Chalmers,

Matthieu Bardal,

Robert J. McDonald

и другие.

Adaptive Behavior, Год журнала: 2024, Номер 33(1), С. 55 - 71

Опубликована: Авг. 28, 2024

Animals can navigate through complex environments with amazing flexibility and efficiency: they forage over large areas, quickly learning rewarding behavior changing their plans when necessary. Some insights into the neural mechanisms supporting this ability be found in hippocampus (HPC)—a brain structure involved navigation, learning, memory. Neuronal activity HPC provides a hierarchical representation of space, representing an environment at multiple scales. In addition, it has been observed that memory-consolidation processes are inactivated, animals still plan familiar but not new environments. Findings like these suggest three useful principles: spatial is hierarchical, world-model intrinsically valuable, action planning occurs as downstream process separate from learning. Here, we demonstrate computationally how agent could learn models using off-line replay trajectories show empirically allows efficient to reach arbitrary goals within reinforcement setting. Using computational model simulate hippocampal damage reproduces navigation behaviors rodents inactivation. The approach presented here might help clarify different interpretations some studies present implications for future both machine biological intelligence.

Язык: Английский

Процитировано

1
The Hippocampus in Pigeons Contributes to the Model-Based Valuation and the Relationship between Temporal Context States DOI Creative Commons
Lifang Yang, Fuli Jin, Long Yang

и другие.

Animals, Год журнала: 2024, Номер 14(3), С. 431 - 431

Опубликована: Янв. 29, 2024

Model-based decision-making guides organism behavior by the representation of relationships between different states. Previous studies have shown that mammalian hippocampus (Hp) plays a key role in learning structure among experiences. However, hippocampal neural mechanisms birds for model-based rarely been reported. Here, we trained six pigeons to perform two-step task and explore whether their Hp contributes learning. Behavioral performance multi-channel local field potentials (LFPs) were recorded during task. We estimated subjective values using reinforcement model dynamically fitted pigeon’s choice behavior. The results show learner can capture behavioral choices well throughout process. Neural analysis indicated high-frequency (12–100 Hz) power represented temporal context Moreover, dynamic correlation decoding provided further support dependence valuations. In addition, observed significant increase similarity at low-frequency band (1–12 common states after Overall, our findings suggest use inferences learn multi-step tasks, multiple LFP frequency bands collaboratively contribute Specifically, oscillations represent valuations, while is influenced relationship These understanding underlying broaden scope contributions avian

Язык: Английский

Процитировано

0
A bio-inspired reinforcement learning model that accounts for fast adaptation after punishment DOI Creative Commons
Eric Chalmers, Artur Luczak

Neurobiology of Learning and Memory, Год журнала: 2024, Номер 215, С. 107974 - 107974

Опубликована: Авг. 28, 2024

Humans and animals can quickly learn a new strategy when previously-rewarding is punished. It difficult to model this with reinforcement learning methods, because they tend perseverate on previously-learned strategies - hallmark of impaired response punishment. Past work has addressed by augmenting conventional equations ad hoc parameters or parallel systems. This produces models that account for reversal learning, but are more abstract, complex, somewhat detached from neural substrates. Here we use different approach: generalize recently-discovered neuron-level rule, the assumption it captures basic principle may occur at whole-brain-level. Surprisingly, gives rule accounts adaptation lose-shift behavior, uses only same as equations. In normal reward prediction errors drive scaled likelihood agent assigns action triggered The demonstrates quick in card sorting variable Iowa gambling tasks, also exhibits human-like paradox-of-choice effect. will be useful experimental researchers modeling behavior.

Язык: Английский

Процитировано

0
A melancholy machine: simulated synapse loss induces depression-like behaviors in deep reinforcement learning DOI Open Access
Eric Chalmers,

Santina Duarte,

Xena Al-Hejji

и другие.

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2024, Номер unknown

Опубликована: Июнь 1, 2024

Abstract Deep Reinforcement Learning is a branch of artificial intelligence that uses neural networks to model reward-based learning as it occurs in biological agents. Here we modify approach by imposing suppressive effect on the connections between neurons network - simulating dendritic spine loss observed major depressive disorder (MDD). Surprisingly, this simulated sufficient induce variety MDD-like behaviors artificially intelligent agent, including anhedonia, increased temporal discounting, avoidance, and an altered exploration/exploitation balance. Furthermore, alternative longstanding reward-processing-centric conceptions MDD (dysfunction dopamine system, reward context-dependent rates, exploration) does not produce same range behaviors. These results support conceptual reduction brain connectivity (and thus information-processing capacity) rather than imbalance monoamines though computational suggests possible explanation for dysfunction systems MDD. Reversing spine-loss our can lead rescue rewarding behavior under some conditions. This supports search treatments increase plasticity synaptogenesis, implications their effective administration. Significance statement Simulating deep reinforcement agent causes exhibit surprising depression-like restoration allows be re-learned. sees Major Depressive Disorder reversible capacity, providing insights pathology treatment.

Язык: Английский

Процитировано

0
Simulated synapse loss induces depression-like behaviors in deep reinforcement learning DOI Creative Commons
Eric Chalmers,

Santina Duarte,

Xena Al-Hejji

и другие.

Frontiers in Computational Neuroscience, Год журнала: 2024, Номер 18

Опубликована: Ноя. 6, 2024

Deep Reinforcement Learning is a branch of artificial intelligence that uses neural networks to model reward-based learning as it occurs in biological agents. Here we modify approach by imposing suppressive effect on the connections between neurons network—simulating dendritic spine loss observed major depressive disorder (MDD). Surprisingly, this simulated sufficient induce variety MDD-like behaviors artificially intelligent agent, including anhedonia, increased temporal discounting, avoidance, and an altered exploration/exploitation balance. Furthermore, simulating alternative longstanding reward-processing-centric conceptions MDD (dysfunction dopamine system, reward context-dependent rates, exploration) does not produce same range behaviors. These results support conceptual reduction brain connectivity (and thus information-processing capacity) rather than imbalance monoamines—though computational suggests possible explanation for dysfunction systems MDD. Reversing spine-loss our can lead rescue rewarding behavior under some conditions. This supports search treatments increase plasticity synaptogenesis, implications their effective administration.

Язык: Английский

Процитировано

0