Experiment-based calibration in psychology: Optimal design considerations

Journal of Mathematical Psychology, Год журнала: 2023, Номер 117, С. 102818 - 102818

Опубликована: Ноя. 8, 2023

Psychological theories are often formulated at the level of latent, not directly observable, variables. Empirical measurement latent variables ought to be valid. Classical psychometric validity indices can difficult apply in experimental contexts. A complementary index, termed retrodictive validity, is correlation theory-derived predicted scores with actually measured scores, specifically designed calibration experiments. In current note, I analyse how experiments maximise information garnered and specifically, minimise sample variance estimators. First, harness asymptotic limits analytically derive different distribution features that impact on estimator variance. Then, numerically simulate various distributions combinations feature values. This allows deriving recommendations for values, resource investment, Finally, highlight cases which a misspecified theory particularly problematic.

Язык: Английский

Experiment-based calibration in psychology: foundational and data-generating model

Опубликована: Окт. 27, 2023

Experiment-based calibration is a novel method for measurement validation, which unlike classical validity metrics does not require stable between-person variance. In this approach, the latent variable to be measured manipulated by an experiment, and its predicted scores - termed standard are compared against scores. Previous work has shown that under plausible boundary conditions, correlation between retrodictive informative about accuracy, i.e. combined trueness precision. Here, I expand these findings in several directions. First, formalise approach probability-theoretic framework with concept of standardised space. show previously derived conditions hold arbitrary distributions standard, true, Second, relate theory fact apply any form criterion validity, including convergent validity. Thus, state precise empirically quantifiable on Third, confounding variables, correlated variables. limit, will converge most closely related standard. Finally, provide modelling data-generating process Markov kernels, identify sufficient data generation model results sum, article provides formal experiment-based facilitates empirical assessment generating processes.

Язык: Английский