Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning DOI Open Access
Sarah Jo C Venditto, Kevin J Miller, Carlos D. Brody

и другие.

Опубликована: Дек. 13, 2024

Different brain systems have been hypothesized to subserve multiple “experts” that compete generate behavior. In reinforcement learning, two general processes, one model-free (MF) and model-based (MB), are often modeled as a mixture of agents (MoA) capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from set temporal dynamics underlying “hidden” states agent contributions over time. Applying this multi-step, reward-guided task rats reveals progression within-session strategies: shift initial MB exploration exploitation, finally reduced engagement. The predict changes both response time OFC neural encoding during task, suggesting these capturing real dynamics.

Язык: Английский

Genetic changes linked to two different syndromic forms of autism enhance reinforcement learning in adolescent male but not female mice DOI Creative Commons
Juliana Chase, Jing‐Jing Li, Wan Chen Lin

и другие.

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2025, Номер unknown

Опубликована: Янв. 15, 2025

Autism Spectrum Disorder (ASD) is characterized by restricted and repetitive behaviors social differences, both of which may manifest, in part, from underlying differences corticostriatal circuits reinforcement learning. Here, we investigated learning mice with mutations either Tsc2 or Shank3 , high-confidence ASD risk genes associated major syndromic forms ASD. Using an odor-based two-alternative forced choice (2AFC) task, tested adolescent sexes found male Shank3B heterozygote (Het) showed enhanced performance compared to their wild type (WT) siblings. No gain function was observed females. a novel (RL) based computational model infer rate as well policy-level task engagement disengagement, that the males driven positive Het mice. The absent when were trained probabilistic reward schedule. These findings two mouse models reveal convergent phenotype shows similar sensitivity sex environmental uncertainty. data can inform our understanding strengths challenges autism, while providing further evidence experience uncertainty modulate autism-related phenotypes. Reinforcement foundational form widely used behavioral interventions for autism. measured carrying genetic linked different We siblings, females no differences. This longer introduced into schedule correct choices. support diverse changes interact generate common phenotypes Our idea autism produce function.

Язык: Английский

Процитировано

1
Disentangling sources of variability in decision-making DOI
Jade S. Duffy, Mark A. Bellgrove, Peter R. Murphy

и другие.

Nature reviews. Neuroscience, Год журнала: 2025, Номер unknown

Опубликована: Март 20, 2025

Язык: Английский

Процитировано

0
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning DOI Open Access
Sarah Jo C Venditto, Kevin J Miller, Carlos D. Brody

и другие.

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2024, Номер unknown

Опубликована: Фев. 28, 2024

Abstract Different brain systems have been hypothesized to subserve multiple “experts” that compete generate behavior. In reinforcement learning, two general processes, one model-free (MF) and model-based (MB), are often modeled as a mixture of agents (MoA) capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from set temporal dynamics underlying “hidden” states agent contributions over time. Applying this multi-step, reward-guided task rats reveals progression within-session strategies: shift initial MB exploration exploitation, finally reduced engagement. The predict changes both response time OFC neural encoding during task, suggesting these capturing real dynamics.

Язык: Английский

Процитировано

1
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning DOI Open Access
Sarah Jo C Venditto, Kevin J Miller, Carlos D. Brody

и другие.

Опубликована: Май 10, 2024

Different brain systems have been hypothesized to subserve multiple “experts” that compete generate behavior. In reinforcement learning, two general processes, one model-free (MF) and model-based (MB), are often modeled as a mixture of agents (MoA) capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from set temporal dynamics underlying “hidden” states agent contributions over time. Applying this multi-step,reward-guided task rats reveals progression within-session strategies: shift initial MB exploration exploitation, finally reduced engagement. The predict changes both response time OFC neural encoding during task, suggesting these capturing real dynamics.

Язык: Английский

Процитировано

1
A Bayesian Hierarchical Model of Trial-To-Trial Fluctuations in Decision Criterion DOI Creative Commons
Robin Vloeberghs, Anne E. Urai, Kobe Desender

и другие.

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2024, Номер unknown

Опубликована: Июль 31, 2024

Classical decision models assume that the parameters giving rise to choice behavior are stable, yet emerging research suggests these may fluctuate over time. Such fluctuations, observed in neural activity and behavioral strategies, have significant implications for understanding decision-making processes. However, empirical studies on fluctuating human strategies been limited due extensive data requirements estimating fluctuations. Here, we introduce hMFC (Hierarchical Model Fluctuations Criterion), a Bayesian framework designed estimate slow fluctuations criterion from data. We first showcase importance of considering criterion: incorrectly assuming stable gives apparent history effects underestimates perceptual sensitivity. then present hierarchical estimation procedure capable reliably recovering underlying state with as few 500 trials per participant, offering robust tool researchers typical datasets. Critically, does not only accurately recover criterion, it also effectively deals confounds caused by Lastly, provide code comprehensive demo at www.github.com/robinvloeberghs/hMFC enable widespread application research.

Язык: Английский

Процитировано

1
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning DOI Open Access
Sarah Jo C Venditto, Kevin J Miller, Carlos D. Brody

и другие.

Опубликована: Май 10, 2024

Different brain systems have been hypothesized to subserve multiple “experts” that compete generate behavior. In reinforcement learning, two general processes, one model-free (MF) and model-based (MB), are often modeled as a mixture of agents (MoA) capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from set temporal dynamics underlying “hidden” states agent contributions over time. Applying this multi-step,reward-guided task rats reveals progression within-session strategies: shift initial MB exploration exploitation, finally reduced engagement. The predict changes both response time OFC neural encoding during task, suggesting these capturing real dynamics.

Язык: Английский

Процитировано

0
Dynamic reinforcement learning reveals time-dependent shifts in strategy during reward learning DOI Open Access
Sarah Jo C Venditto, Kevin J Miller, Carlos D. Brody

и другие.

Опубликована: Дек. 13, 2024

Different brain systems have been hypothesized to subserve multiple “experts” that compete generate behavior. In reinforcement learning, two general processes, one model-free (MF) and model-based (MB), are often modeled as a mixture of agents (MoA) capture differences between automaticity vs. deliberation. However, shifts in strategy cannot be captured by static MoA. To investigate such dynamics, we present the mixture-of-agents hidden Markov model (MoA-HMM), which simultaneously learns inferred action values from set temporal dynamics underlying “hidden” states agent contributions over time. Applying this multi-step, reward-guided task rats reveals progression within-session strategies: shift initial MB exploration exploitation, finally reduced engagement. The predict changes both response time OFC neural encoding during task, suggesting these capturing real dynamics.

Язык: Английский

Процитировано

0