Theoretically Informed Generative Models Can Advance the Psychological and Brain Sciences: Lessons from the Reliability Paradox

Published: Aug. 24, 2020

Theories of individual differences are foundational to psychological and brain sciences, yet they traditionally developed tested using superficial summaries data (e.g., mean response times) that both (1) disconnected from our otherwise rich conceptual theories behavior, (2) contaminated with measurement error. Traditional approaches therefore lack the flexibility required test increasingly complex behavior. To resolve this theory- description gap, we present generative modeling approach, which involves formally specifying how behavior is generated within people processes vary across people. Generative shifts focus away estimating descriptive statistical “effects” toward psychologically interpretable parameters, while simultaneously enhancing reliability validity measures. We demonstrate utility models in context “reliability paradox”, a phenomenon wherein highly replicable group effects Stroop effect) fail capture low test-retest reliability). Simulations empirical Implicit Association Test, Stroop, Flanker, Posner, Delay Discounting tasks show yield more theoretically informative higher estimates relative traditional approaches, illustrating their potential for theory development.

Language: Английский

---

Evidence for a competitive relationship between executive functions and statistical learning

npj Science of Learning, Journal Year: 2024, Volume and Issue: 9(1)

Published: April 12, 2024

Abstract The ability of the brain to extract patterns from environment and predict future events, known as statistical learning, has been proposed interact in a competitive manner with prefrontal lobe-related networks their characteristic cognitive or executive functions. However, it remains unclear whether these functions also possess relationship implicit learning across individuals at level latent function components. In order address this currently unknown aspect, we investigated, two independent experiments (N Study1 = 186, N Study2 157), between measured by Alternating Serial Reaction Time task, functions, multiple neuropsychological tests. both studies, modest, but consistent negative correlation most measures was observed. Factor analysis further revealed that factor representing verbal fluency complex working memory seemed drive correlations. Thus, antagonistic might specifically be mediated updating component or/and long-term access.

Language: Английский

---

Complementary benefits of multivariate and hierarchical models for identifying individual differences in cognitive control

Imaging Neuroscience, Journal Year: 2025, Volume and Issue: 3

Published: Jan. 1, 2025

Abstract Understanding individual differences in cognitive control is a central goal psychology and neuroscience. Reliably measuring these differences, however, has proven extremely challenging, at least when using standard measures neuroscience such as response times or task-based fMRI activity. While prior work pinpointed the source of issue—the vast amount cross-trial variability within measures—solutions remain elusive. Here, we propose one potential way forward: an analytic framework that combines hierarchical Bayesian modeling with multivariate decoding trial-level data. Using this longitudinal data from Dual Mechanisms Cognitive Control project, estimated individuals’ neural responses associated color-word Stroop task, then assessed reliability across time interval several months. We show many prefrontal parietal brain regions, test–retest was near maximal, only models were able to reveal state affairs. Further, compared traditional univariate contrasts, enabled individual-level correlations be significantly greater precision. specifically link improvements precision optimized suppression decoding. Together, findings not indicate control-related individuate people highly stable manner time, but also suggest integrating provides powerful approach for investigating control, can effectively address issue high-variability measures.

Language: Английский

---

A multiverse assessment of the reliability of the self-matching task as a measurement of the self-prioritization effect

Behavior Research Methods, Journal Year: 2025, Volume and Issue: 57(1)

Published: Jan. 2, 2025

Language: Английский

---

Altered Neural Responses to Punishment Learning in Conduct Disorder

Biological Psychiatry Cognitive Neuroscience and Neuroimaging, Journal Year: 2025, Volume and Issue: unknown

Published: Jan. 1, 2025

Conduct disorder (CD) is associated with deficits in the use of punishment for reinforcement learning (RL) and subsequent decision-making, contributing to reckless, antisocial, aggressive behaviors. Here, we used functional magnetic resonance imaging (fMRI) examine whether differences behavioral rates derived from computational modeling, particularly punishment, are reflected aberrant neural responses youths CD compared typically-developing controls (TDCs). 75 99 TDCs (9-18 years, 47% girls) performed a probabilistic RL task reward, neutral contingencies. Using fMRI data conjunction modeling indices (learning rate α), investigated group three conditions whole-brain regions-of-interest (ROI) analyses, including ventral striatum insula. Whole-brain analysis revealed typical both groups. However, linear regression models ROI analyses that only response pattern (anterior) insula during was different TDCs. Youths have atypical (but not reward), specifically This suggests selective dysfunction mechanisms thereby 'punishment insensitivity/hyposensitivity' as hallmark disorder. As involved avoidance behaviors related negative affect or arousal, may contribute inappropriate which increases risk affected youth.

Language: Английский

---

Improving the reliability of cognitive task measures: A narrative review

Published: Oct. 31, 2022

Cognitive tasks are capable of providing researchers with crucial insights into the relationship between cognitive processing and psychiatric phenomena. However, many recent studies have found that task measures exhibit poor reliability, which hampers their usefulness for individual-differences research. Here we provide a narrative review approaches to improve reliability measures. First, methods calculating discuss some nuances specific tasks. Then, introduce taxonomy experiment design analysis strategies improving reliability. Where appropriate, highlight exemplary We hope this article can serve as helpful guide experimenters who wish new task, or an existing one, achieve sufficient use in

Language: Английский

---

Improving the Reliability of the Pavlovian Go/No-Go Task

Published: July 20, 2023

Background: The Pavlovian go/no-go task is commonly used to measure individual differences in biases and their interaction with instrumental learning. However, prior research has found suboptimal reliability for computational model-based performance measures this task, limiting its usefulness individual-differences research. These studies did not make use of several strategies previously shown enhance task-measure (e.g., gamification, hierarchical Bayesian modeling model estimation). Here we investigated if such approaches could improve the task’s reliability. Methods: Across two experiments, recruited independent samples adult participants (N=103, N=110) complete a novel, gamified version multiple times over weeks. We derive reinforcement learning indices participants' performance, additionally estimate these measures. Results: In Experiment 1, observed considerable unexpected practice effects, most reaching near-ceiling levels repeat testing. Consequently, test-retest some parameters was unacceptable (range: 0.379–0.973). 2, completed modified designed lessen effects. greatly reduced effects improved estimates 0.696–0.989). Conclusion: results demonstrate that on can reach sufficient individual- additional investigation necessary validate other populations settings.

Language: Английский

---

Clarifying the reliability paradox: poor test-retest reliability attenuates group differences

Published: May 4, 2024

Cognitive sciences are grappling with the reliability paradox: measures that robustly produce within-group effects tend to have low test-retest reliability, rendering them unsuitable for studying individual differences. Despite growing awareness of this paradox, its full extent remains underappreciated. Specifically, most research focuses exclusively on how affects correlational analyses differences, while largely ignoring group Moreover, by conflating within- and between-group effects, some studies erroneously suggest poor does not pose problems This brief report aims clarify misunderstanding through simple data simulations. To make argument more intuitive, we consider two illustrative cases: comparing patients versus controls groups formed a median split. We demonstrate attenuates observed differences just as much it Given dichotomizing/grouping continuous - which is implicit in many leads loss statistical power, proves be even problematic While here focused cognitive psychiatry, our findings quite general could inform other areas research, including education, sex, gender, age, race, ethnicity, etc.

Language: Английский

---

Does the reliability of computational models truly improve with hierarchical modeling? Some recommendations and considerations for the assessment of model parameter reliability

Psychonomic Bulletin & Review, Journal Year: 2024, Volume and Issue: unknown

Published: May 8, 2024

Abstract Computational modeling of behavior is increasingly being adopted as a standard methodology in psychology, cognitive neuroscience, and computational psychiatry. This approach involves estimating parameters (or cognitive) model that represents the processes underlying behavior. In this approach, reliability parameter estimates an important issue. The use hierarchical (Bayesian) approaches, which place prior on each individual participants, thought to improve parameters. However, characteristics estimates, especially when individual-level priors are assumed, models, have not yet been fully discussed. Furthermore, suitability different measures for assessing thoroughly understood. study, we conduct systematic examination these issues through theoretical analysis numerical simulations, focusing specifically reinforcement learning models. We note heterogeneity estimation precision parameters, particularly with priors, can skew toward individuals higher precision. further there two factors reduce reliability, namely error intersession variation true discuss how evaluate separately. Based considerations present several recommendations cautions

Language: Английский

---

Comparing likelihood-based and likelihood-free approaches to fitting and comparing models of intertemporal choice

Published: June 6, 2024

Machine learning methods have recently begun to be used for fitting and comparing cognitive models, yet they mainly focused on dealing with models that lack tractable likelihoods. Evaluating how these approaches compare traditional likelihood-based is critical understanding the utility of machine modeling determining what role it might play in development new theories. In this paper, we systematically benchmark neural network against model comparison, focusing intertemporal choice as an illustrative application. By applying each approach data from participants substance use problems, show there a high degree convergence between Bayesian when comes making inferences about latent processes real outcomes. For however, classification networks significantly outperformed metrics. Next, extended two ways, using recurrent layers allow them fit variable stimuli numbers trials, dropout posterior sampling. We ultimately suggest are better suited fast parameter estimation, sampling, large sets, while hierarchical should preferred flexible applications across different experimental designs.

Language: Английский

---