Balancing exploration and exploitation with information and randomization

Current Opinion in Behavioral Sciences, Journal Year: 2020, Volume and Issue: 38, P. 49 - 56

Published: Nov. 6, 2020

Language: Английский

---

Integrative Brain Network and Salience Models of Psychopathology and Cognitive Dysfunction in Schizophrenia

Biological Psychiatry, Journal Year: 2022, Volume and Issue: 94(2), P. 108 - 120

Published: Oct. 4, 2022

Language: Английский

---

Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making

eLife, Journal Year: 2020, Volume and Issue: 9

Published: June 2, 2020

Involvement of dopamine in regulating exploration during decision-making has long been hypothesized, but direct causal evidence humans is still lacking. Here, we use a combination computational modeling, pharmacological intervention and functional magnetic resonance imaging to address this issue. Thirty-one healthy male participants performed restless four-armed bandit task within-subjects design under three drug conditions: 150 mg the precursor L-dopa, 2 D2 receptor antagonist haloperidol, placebo. Choices were best explained by an extension established Bayesian learning model accounting for perseveration, directed random exploration. Modeling revealed attenuated while neural signatures exploration, exploitation prediction error unaffected. Instead, L-dopa representations overall uncertainty insula dorsal anterior cingulate cortex. Our results highlight role these regions suggest that modulates how circuit tracks accumulating decision-making.

Language: Английский

---

Better living through understanding the insula: Why subregions can make all the difference

Neuropharmacology, Journal Year: 2021, Volume and Issue: 198, P. 108765 - 108765

Published: Aug. 27, 2021

Language: Английский

---

Lapses in perceptual decisions reflect exploration

eLife, Journal Year: 2021, Volume and Issue: 10

Published: Jan. 11, 2021

Perceptual decision-makers often display a constant rate of errors independent evidence strength. These ‘lapses’ are treated as nuisance arising from noise tangential to the decision, e.g. inattention or motor errors. Here, we use multisensory decision task in rats demonstrate that these explanations cannot account for lapses’ stimulus dependence. We propose novel explanation: lapses reflect strategic trade-off between exploiting known rewarding actions and exploring uncertain ones. tested this model’s predictions by selectively manipulating one action’s reward magnitude probability. As uniquely predicted model, changes were restricted associated with action. Finally, show powerful tool assigning decision-related computations neural structures based on disruption experiments (here, posterior striatum secondary cortex). results suggest an integral component decision-making informative about action values normal disrupted brain states.

Language: Английский

---

Human complex exploration strategies are enriched by noradrenaline-modulated heuristics

eLife, Journal Year: 2021, Volume and Issue: 10

Published: Jan. 4, 2021

An exploration-exploitation trade-off, the arbitration between sampling a lesser-known against known rich option, is thought to be solved using computationally demanding exploration algorithms. Given limitations in human cognitive resources, we hypothesised presence of additional cheaper strategies. We examined for such heuristics choice behaviour where show this involves value-free random exploration, that ignores all prior knowledge, and novelty targets novel options alone. In double-blind, placebo-controlled drug study, assessing contributions dopamine (400 mg amisulpride) noradrenaline (40 propranolol), attenuated under influence propranolol, but not amisulpride. Our findings demonstrate humans deploy distinct cheap strategies noradrenergic control.

Language: Английский

---

Impaired value-based decision-making in Parkinson’s disease apathy

Brain, Journal Year: 2024, Volume and Issue: 147(4), P. 1362 - 1376

Published: Feb. 2, 2024

Abstract Apathy is a common and disabling complication of Parkinson’s disease characterized by reduced goal-directed behaviour. Several studies have reported dysfunction within prefrontal cortical regions projections from brainstem nuclei whose neuromodulators include dopamine, serotonin noradrenaline. Work in animal human neuroscience confirmed contributions these on aspects motivated decision-making. Specifically, overlapping to encoding the value decisions, influence whether explore alternative courses action or persist an existing strategy achieve rewarding goal. Building upon this work, we hypothesized that apathy should be associated with impairment value-based learning. Using four-armed restless bandit reinforcement learning task, studied decision-making 75 volunteers; 53 patients disease, without clinical apathy, 22 age-matched healthy control subjects. Patients exhibited impaired ability choose highest bandit. Task performance predicted individual patient’s severity measured using Lille Rating Scale (R = −0.46, P < 0.001). Computational modelling choices group made decisions were indifferent learnt options, consistent previous reports reward insensitivity. Further analysis demonstrated shift away exploiting option reduction perseveration, which also correlated scores −0.5, We went acquire functional MRI 59 19 20 controls performing Restless Bandit Task. Analysis signal at point feedback diminished ventromedial cortex was more marked but not predictive their severity. model-based categorization choice type, lower bandits activated similar degree controls. In contrast, significantly increased activation across distributed thalamo-cortical network. Enhanced activity thalamus both patient groups connectivity dorsal anterior cingulate insula. Given task no different subjects, interpret recruitment network as possible compensatory mechanism, compensates against symptomatic manifestation disease.

Language: Английский

---

Differential impacts of healthy cognitive aging on directed and random exploration.

Psychology and Aging, Journal Year: 2024, Volume and Issue: 39(1), P. 88 - 101

Published: Feb. 1, 2024

Deciding whether to explore unknown opportunities or exploit well-known options is a ubiquitous part of our everyday lives. Extensive work in college students suggests that young people make explore-exploit decisions using mixture information seeking and random behavioral variability. Whether, what extent, older adults use the same strategies unknown. To address this question, 51 (ages 65-74) 32 younger 18-25) completed Horizon Task, gambling task quantifies variability as well how these are controlled for purposes exploration. Qualitatively, we found performed similar on task, increasing both their when it was adaptive explore. Quantitively, however, there were substantial differences between age groups, with showing less overall reliance means In addition, subset approximately 26% whose close zero, avoiding informative even they clearly better choice. Unsurprisingly, "information avoiders" worse task. contrast, performance remaining seeking" comparable suggesting age-related decision making may be except taken extremes. (PsycInfo Database Record (c) 2024 APA, all rights reserved).

Language: Английский

---

Distinct computational mechanisms of uncertainty processing explain opposing exploratory behaviors in anxiety and apathy

Biological Psychiatry Cognitive Neuroscience and Neuroimaging, Journal Year: 2025, Volume and Issue: unknown

Published: Jan. 1, 2025

Language: Английский

---

Observational learning of exploration-exploitation strategies in bandit tasks

Cognition, Journal Year: 2025, Volume and Issue: 259, P. 106124 - 106124

Published: March 21, 2025

Language: Английский

---