Reinforcement learning across development: What insights can we draw from a decade of research?

Developmental Cognitive Neuroscience, Journal Year: 2019, Volume and Issue: 40, P. 100733 - 100733

Published: Nov. 6, 2019

The past decade has seen the emergence of use reinforcement learning models to study developmental change in value-based learning. It is unclear, however, whether these computational modeling studies, which have employed a wide variety tasks and model variants, reached convergent conclusions. In this review, we examine tuning parameters that govern different aspects decision-making processes vary consistently as function age, what neurocognitive changes may account for differences parameter estimates across development. We explore patterns are better described by extent individuals adapt their statistics environments, or more static biases emerge varied contexts. focus specifically on rates inverse temperature estimates, find evidence from childhood adulthood, become at optimally weighting recent outcomes during diverse contexts less exploratory decision-making. provide recommendations how two possibilities — potential alternative accounts can be tested directly build cohesive body research yields greater insight into development core processes.

Language: Английский

---

The interpretation of computational model parameters depends on the context

eLife, Journal Year: 2022, Volume and Issue: 11

Published: Nov. 4, 2022

Reinforcement Learning (RL) models have revolutionized the cognitive and brain sciences, promising to explain behavior from simple conditioning complex problem solving, shed light on developmental individual differences, anchor processes in specific mechanisms. However, RL literature increasingly reveals contradictory results, which might cast doubt these claims. We hypothesized that many contradictions arise two commonly-held assumptions about computational model parameters are actually often invalid: That generalize between contexts (e.g. tasks, models) they capture interpretable (i.e. unique, distinctive) neurocognitive processes. To test this, we asked 291 participants aged 8–30 years complete three learning tasks one experimental session, fitted each. found some (exploration / decision noise) showed significant generalization: followed similar trajectories, were reciprocally predictive tasks. Still, generalization was significantly below methodological ceiling. Furthermore, other (learning rates, forgetting) did not show evidence of generalization, sometimes even opposite trajectories. Interpretability low for all parameters. conclude systematic study context factors reward stochasticity; task volatility) will be necessary enhance generalizability interpretability models.

Language: Английский

---

What do reinforcement learning models measure? Interpreting model parameters in cognition and neuroscience

Current Opinion in Behavioral Sciences, Journal Year: 2021, Volume and Issue: 41, P. 128 - 137

Published: July 3, 2021

Language: Английский

---

Reinforcement learning and Bayesian inference provide complementary models for the unique advantage of adolescents in stochastic reversal

Developmental Cognitive Neuroscience, Journal Year: 2022, Volume and Issue: 55, P. 101106 - 101106

Published: April 22, 2022

During adolescence, youth venture out, explore the wider world, and are challenged to learn how navigate novel uncertain environments. We investigated performance changes across adolescent development in a stochastic, volatile reversal-learning task that uniquely taxes balance of persistence flexibility. In sample 291 participants aged 8-30, we found mid-teen years, adolescents outperformed both younger older participants. developed two independent cognitive models, based on Reinforcement learning (RL) Bayesian inference (BI). The RL parameter for from negative outcomes BI parameters specifying participants' mental models were closest optimal adolescents, suggesting central role processing. By contrast, noise improved monotonically with age. distilled insights using principal component analysis three shared components interacted form peak: adult-like behavioral quality, child-like time scales, developmentally-unique processing positive feedback. This research highlights adolescence as neurodevelopmental window can create advantages It also shows detailed be gleaned by new ways.

Language: Английский

---

Valence biases in reinforcement learning shift across adolescence and modulate subsequent memory

eLife, Journal Year: 2022, Volume and Issue: 11

Published: Jan. 24, 2022

As individuals learn through trial and error, some are more influenced by good outcomes, while others weight bad outcomes heavily. Such valence biases may also influence memory for past experiences. Here, we examined whether asymmetries in reinforcement learning change across adolescence, individual bias the content of subsequent memory. Participants ages 8–27 learned values ‘point machines,’ after which their trial-unique images presented with choice was assessed. Relative to children adults, adolescents overweighted worse-than-expected during learning. Individuals’ modulated incidental memory, such that those who prioritized worse- (or better-) than-expected were likely remember paired these an effect reproduced independent dataset. Collectively, results highlight age-related changes computation subjective value demonstrate a valence-asymmetric valuation process influences how information is episodic

Language: Английский

---

Goal-directed learning in adolescence: neurocognitive development and contextual influences

Nature reviews. Neuroscience, Journal Year: 2024, Volume and Issue: 25(3), P. 176 - 194

Published: Jan. 23, 2024

Language: Английский

---

Revisiting adolescence as a sensitive period for sociocultural processing

Neuroscience & Biobehavioral Reviews, Journal Year: 2024, Volume and Issue: 164, P. 105820 - 105820

Published: July 18, 2024

Language: Английский

---

Mechanisms of learning and plasticity in childhood and adolescence

Developmental Cognitive Neuroscience, Journal Year: 2020, Volume and Issue: 42, P. 100764 - 100764

Published: Jan. 30, 2020

Language: Английский

---

Preference uncertainty accounts for developmental effects on susceptibility to peer influence in adolescence

Nature Communications, Journal Year: 2021, Volume and Issue: 12(1)

Published: June 22, 2021

Adolescents are prone to social influence from peers, with implications for development, both adaptive and maladaptive. Here, using a computer-based paradigm, we replicate cross-sectional effect of more susceptibility peer in large dataset adolescents 14 24 years old. Crucially, extend this finding by adopting longitudinal perspective, showing that within-person decreases over 1.5 year follow-up time period. Exploiting design, show influences at baseline predicts an improvement relations the Using Bayesian computational model, demonstrate younger greater tendency adopt others' preferences arises out higher uncertainty about their own paradigmatic case delay discounting (a phenomenon called 'preference uncertainty'). This preference and, turn, leads reduced one's behaviour others. Neuro-developmentally, measure myelination within medial prefrontal cortex, estimated baseline, developmental decrease follow-up. Thus, neural evidence, reveal mechanisms underpinning during adolescence.

Language: Английский

---

Sensitivity and specificity in affective and social learning in adolescence

Trends in Cognitive Sciences, Journal Year: 2023, Volume and Issue: 27(7), P. 642 - 655

Published: May 16, 2023

Adolescence is a period of heightened affective and social sensitivity. In this review we address how increased sensitivity influences associative learning. Based on recent evidence from human rodent studies, as well advances in computational biology, suggest that, compared to other age groups, adolescents show features Pavlovian learning but tend perform worse than adults at instrumental Because does not involve decision-making, whereas does, propose that these developmental differences might be due rewards threats adolescence, coupled with lower specificity responding. We discuss the implications findings for adolescent mental health education.

Language: Английский

---