The Importance of Falsification in Computational Cognitive Modeling

Trends in Cognitive Sciences, Journal Year: 2017, Volume and Issue: 21(6), P. 425 - 433

Published: May 2, 2017

Language: Английский

---

Reinforcement learning across development: What insights can we draw from a decade of research?

Developmental Cognitive Neuroscience, Journal Year: 2019, Volume and Issue: 40, P. 100733 - 100733

Published: Nov. 6, 2019

The past decade has seen the emergence of use reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety tasks and model variants, reached convergent conclusions. In this review, we examine tuning parameters that govern different aspects decision-making processes vary consistently as function age, what neurocognitive changes may account for differences parameter estimates across development. We explore patterns are better described by extent individuals adapt their statistics environments, or more static biases emerge varied contexts. focus specifically on rates inverse temperature estimates, find evidence from childhood adulthood, become at optimally weighting recent outcomes during diverse contexts less exploratory decision-making. provide recommendations how two possibilities — potential alternative accounts can be tested directly build cohesive body research yields greater insight into development core processes.

Language: Английский

---

Confirmation bias in human reinforcement learning: Evidence from counterfactual feedback processing

PLoS Computational Biology, Journal Year: 2017, Volume and Issue: 13(8), P. e1005684 - e1005684

Published: Aug. 11, 2017

Previous studies suggest that factual learning, is, learning from obtained outcomes, is biased, such participants preferentially take into account positive, as compared to negative, prediction errors. However, whether or not the error valence also affects counterfactual forgone unknown. To address this question, we analysed performance of two groups on reinforcement tasks using a computational model was adapted test if influences learning. We carried out experiments: in experiment, learned partial feedback (i.e., outcome chosen option only); complete information outcomes both and unchosen were displayed). In replicated previous findings valence-induced bias, whereby relative contrast, for found opposite bias: negative errors taken account, positive ones. When considering bias context it appears people tend confirms their current choice.

Language: Английский

---

The interpretation of computational model parameters depends on the context

eLife, Journal Year: 2022, Volume and Issue: 11

Published: Nov. 4, 2022

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning complex problem solving, shed light on developmental individual differences, anchor processes in specific mechanisms. However, RL literature increasingly reveals contradictory results, which might cast doubt these claims. We hypothesized that many contradictions arise two commonly-held assumptions about computational model parameters are actually often invalid: That generalize between contexts (e.g. tasks, models) they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8–30 years complete three learning tasks one experimental session, fitted each. found some (exploration / decision noise) showed significant generalization: followed similar trajectories, were reciprocally predictive tasks. Still, generalization was significantly below methodological ceiling. Furthermore, other (learning rates, forgetting) did not show evidence of generalization, sometimes even opposite trajectories. Interpretability low for all parameters. conclude systematic study context factors reward stochasticity; task volatility) will be necessary enhance generalizability interpretability models.

Language: Английский

---

Searching for Rewards Like a Child Means Less Generalization and More Directed Exploration

Psychological Science, Journal Year: 2019, Volume and Issue: 30(11), P. 1561 - 1572

Published: Oct. 25, 2019

How do children and adults differ in their search for rewards? We considered three different hypotheses that attribute developmental differences to (a) children's increased random sampling, (b) more directed exploration toward uncertain options, or (c) narrower generalization. Using a task which noisy rewards were spatially correlated on grid, we compared the ability of 55 younger (ages 7 8 years), older 9-11 50 19-55 years) successfully generalize about unobserved outcomes balance exploration-exploitation dilemma. Our results show explore eagerly than but obtain lower rewards. built predictive model disentangle unique contributions found robust recoverable parameter estimates indicating less rely adults. did not, however, find reliable terms sampling.

Language: Английский

---

What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience

Current Opinion in Behavioral Sciences, Journal Year: 2021, Volume and Issue: 41, P. 128 - 137

Published: July 3, 2021

Language: Английский

---

Disentangling the systems contributing to changes in learning during adolescence

Developmental Cognitive Neuroscience, Journal Year: 2019, Volume and Issue: 41, P. 100732 - 100732

Published: Nov. 14, 2019

Multiple neurocognitive systems contribute simultaneously to learning. For example, dopamine and basal ganglia (BG) are thought support reinforcement learning (RL) by incrementally updating the value of choices, while prefrontal cortex (PFC) contributes different computations, such as actively maintaining precise information in working memory (WM). It is commonly that WM PFC show more protracted development than RL BG systems, yet their contributions rarely assessed tandem. Here, we used a simple task test how changes across adolescence. We tested 187 subjects ages 8 17 53 adults (25-30). Participants learned stimulus-action associations from feedback; load was varied be within or exceed capacity. age 8-12 slower participants 13-17, were sensitive load. computational modeling estimate subjects' use processes. Surprisingly, found during development. rate increased with until 18 parameters showed subtle, gender- puberty-dependent early These results can inform education intervention strategies based on developmental science

Language: Английский

---

Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal

Developmental Cognitive Neuroscience, Journal Year: 2022, Volume and Issue: 55, P. 101106 - 101106

Published: April 22, 2022

During adolescence, youth venture out, explore the wider world, and are challenged to learn how navigate novel uncertain environments. We investigated performance changes across adolescent development in a stochastic, volatile reversal-learning task that uniquely taxes balance of persistence flexibility. In sample 291 participants aged 8-30, we found mid-teen years, adolescents outperformed both younger older participants. developed two independent cognitive models, based on Reinforcement learning (RL) Bayesian inference (BI). The RL parameter for from negative outcomes BI parameters specifying participants' mental models were closest optimal adolescents, suggesting central role processing. By contrast, noise improved monotonically with age. distilled insights using principal component analysis three shared components interacted form peak: adult-like behavioral quality, child-like time scales, developmentally-unique processing positive feedback. This research highlights adolescence as neurodevelopmental window can create advantages It also shows detailed be gleaned by new ways.

Language: Английский

---

Developmental changes in exploration resemble stochastic optimization

Nature Human Behaviour, Journal Year: 2023, Volume and Issue: 7(11), P. 1955 - 1967

Published: Aug. 17, 2023

Human development is often described as a 'cooling off' process, analogous to stochastic optimization algorithms that implement gradual reduction in randomness over time. Yet there ambiguity how interpret this analogy, due lack of concrete empirical comparisons. Using data from n = 281 participants ages 5 55, we show cooling off does not only apply the single dimension randomness. Rather, human resembles an process multiple learning parameters, for example, reward generalization, uncertainty-directed exploration and random temperature. Rapid changes parameters occur during childhood, but these plateau converge efficient values adulthood. We while developmental trajectory strikingly similar several algorithms, are important differences convergence. None tested were able discover reliably better regions strategy space than adult on task.

Language: Английский

---

Goal-directed learning in adolescence: neurocognitive development and contextual influences

Nature reviews. Neuroscience, Journal Year: 2024, Volume and Issue: 25(3), P. 176 - 194

Published: Jan. 23, 2024

Language: Английский

---