Exploration–Exploitation Mechanisms in Recurrent Neural Networks and Human Learners in Restless Bandit Problems

Computational Brain & Behavior, Journal Year: 2024, Volume and Issue: 7(3), P. 314 - 356

Published: May 24, 2024

Abstract A key feature of animal and human decision-making is to balance the exploration unknown options for information gain (directed exploration) versus selecting known immediate reward (exploitation), which often examined using restless bandit tasks. Recurrent neural network models (RNNs) have recently gained traction in both systems neuroscience work on reinforcement learning, due their ability show meta-learning task domains. Here we comprehensively compared performance a range RNN architectures as well learners four-armed problems. The best-performing architecture (LSTM with computation noise) exhibited human-level performance. Computational modeling behavior first revealed that behavioral data contain signatures higher-order perseveration, i.e., perseveration beyond last trial, but this effect was more pronounced RNNs. In contrast, learners, not RNNs, positive uncertainty choice probability exploration). hidden unit dynamics exploratory choices were associated disruption predictive signals during states low state value, resembling win-stay-loose-shift strategy, resonating previous single recording findings monkey prefrontal cortex. Our results highlight similarities differences between it emerges computational mechanisms identified cognitive work.

Language: Английский

---

Understanding Context Dependence in Time Perception in Autism Spectrum Disorders: A Computational Study

Published: May 20, 2024

Language: Английский

---

Intrinsic threshold plasticity: cholinergic activation and role in the neuronal recognition of incomplete input patterns

The Journal of Physiology, Journal Year: 2022, Volume and Issue: 601(15), P. 3221 - 3239

Published: July 26, 2022

Abstract Activity‐dependent changes in membrane excitability are observed neurons across brain areas and represent a cell‐autonomous form of plasticity (intrinsic plasticity; IP) that itself does not involve alterations synaptic strength (synaptic SP). Non‐homeostatic IP may play an essential role learning, e.g. by changing the action potential threshold near soma. A computational problem, however, arises from implication such amplification discriminate between inputs therefore reduce resolution input representation. Here, we investigate consequences for performance artificial neural network (a) discrimination unknown patterns (b) recognition known/learned patterns. While negative potentials output layer indeed its ability to patterns, they benefit known but incompletely presented An analysis thresholds IP‐induced published sets physiological data obtained whole‐cell patch‐clamp recordings L2/3 pyramidal primary visual cortex (V1) awake macaques somatosensory (S1) mice vitro , respectively, reveals difference resting ∼15 mV V1 ∼25 S1, total range ∼10 (S1). The most efficient activity pattern lower is paired cholinergic electric activation. Our findings show reduction promotes shift coding strategies accurate faithful representation interpretative assignment learned object categories. image Key points Intrinsic change soma (threshold plasticity), thus altering input–output function all ‘upstream’ location. problem arising this shared it different assess as well subsequent spike threshold. We observe do performance, at same time improve task, particular when presented. Analysis preferentially result stimulation with activation muscarinic acetylcholine receptors.

Language: Английский

---

Machine learning as a new tool in neurological disease prevention, diagnosis, and treatment

Exploration of Neuroprotective Therapy, Journal Year: 2023, Volume and Issue: unknown, P. 1 - 7

Published: Feb. 21, 2023

More than 600 different neurological diseases affect the human population. Some of these are genetic and can emerge even before birth, some caused by defects, infections, trauma, degeneration, inflammation, cancer. However, they all share disabilities damage to nervous system. In last decades, burden almost disorders has increased in terms absolute incidence, prevalence, mortality, largely due population’s growth aging. This represents a dangerous trend should become our priority for future. But what new goals we going set reach now, how will exploit thought-provoking technological skills making feasible? Machine learning be at root problem. Indeed, most recently, there been push towards medical data analysis machine learning, great improvement training capabilities particularly artificial deep neural networks (DNNs) inspired biological characterizing brain. generated competitive results applications such as biomolecular target protein structure prediction, structure-based rational drug design, repurposing, exerting major impact on neuroscience well-being. By approaching early risks diseases, non-invasive diagnosis, personalized treatment assessment, discovery, automated science, arena thus potential becoming frontier empowering research clinical practice years ahead.

Language: Английский

---

Serotonergic and dopaminergic neurons in the dorsal raphe are differentially altered in a mouse model for Parkinson’s disease

Published: Aug. 21, 2023

Parkinson’s disease (PD) is characterized by motor impairments caused degeneration of dopamine neurons in the substantia nigra pars compacta. In addition to these symptoms, PD patients often suffer from non-motor co-morbidities including sleep and psychiatric disturbances, which are thought depend on concomitant alterations serotonergic noradrenergic transmission. A primary locus dorsal raphe nucleus (DRN), providing brain-wide input. Here, we identified electrophysiological morphological parameters classify dopaminergic murine DRN under control conditions a model, following striatal injection catecholamine toxin, 6-hydroxydopamine (6-OHDA). Electrical properties both neuronal populations were altered 6-OHDA. neurons, most changes reversed when 6-OHDA was injected combination with desipramine, noradrenaline reuptake inhibitor, protecting terminals. Our results show that depletion mouse model causes neural circuitry.

Language: Английский

---