The Hawthorne effect in studies of firearm and toolmark examiners

Journal of Forensic Sciences, Journal Year: 2025, Volume and Issue: unknown

Published: April 10, 2025

Abstract The Hawthorne effect refers to the tendency of individuals behave differently when they know are being studied. In forensic science domain, concerns have been raised about “strategic examiner,” where examiner uses different decision thresholds depending on whether in a test situation or working an actual case. blind testing conducted by Houston Forensic Science Center (“HFSC”) firearms examination presents unique opportunity hypothesis that rate inconclusive calls differs for discovered vs. undiscovered tests firearm examination. Over 5 years, 529 item comparisons were filtered into casework at HFSC. items was 56.4%, while 39.3%. Thus, percentage 43.5% higher among than items. This pattern results held bullet (83% 59%) and cartridge case (29% 20%) both same‐source different‐source comparisons. These findings corroborate examiners tested demonstrate necessity if research goal is evaluate performance conducting casework.

Language: Английский

---

Scientific guidelines for evaluating the validity of forensic feature-comparison methods

Proceedings of the National Academy of Sciences, Journal Year: 2023, Volume and Issue: 120(41)

Published: Oct. 2, 2023

When it comes to questions of fact in a legal context-particularly about measurement, association, and causality-courts should employ ordinary standards applied science. Applied sciences generally develop along path that proceeds from basic scientific discovery some natural process the formation theory how works what causes fail, development an invention intended assess, repair, or improve process, specification predictions instrument's actions and, finally, empirical validation determine instrument achieves effect. These elements are salient deeply embedded cultures medicine engineering, both which primarily grew sciences. However, inventions underlie most forensic science disciplines have few roots science, they do not sound theories justify their predicted results tests prove work as advertised. Inspired by "Bradford Hill Guidelines"-the dominant framework for causal inference epidemiology-we set forth four guidelines can be used establish validity comparison methods generally. This is checklist establishing threshold minimum validity, no magic formula determines when particular hypotheses passed necessary threshold. We illustrate these considering discipline firearm tool mark examination.

Language: Английский

---

Three-Way ROCs for Forensic Decision Making

Statistics and Public Policy, Journal Year: 2023, Volume and Issue: 10(1)

Published: July 21, 2023

Firearm examiners use a comparison microscope to judge whether bullets or cartridge cases were fired by the same gun. Examiners can reach one of three possible conclusions: Identification (a match), Elimination (not Inconclusive. Numerous error rate studies report that firearm commit few errors when they conduct these examinations. However, also many inconclusive judgments (>50%), and how score responses is controversial. There have recently been Signal Detection Theory (SDT) primers in this domain. Unfortunately, analyses rely on hypothetical data fail address response issue adequately. This article reports an SDT analysis using from large study practicing examiners. First, we demonstrate problem relying traditional two-way model, which either drops combines responses; addition lacking ecological validity, approach leads implausible results. Second, introduce readers three-way model. We forensic firearms While statistically complicated, it well suited evaluate performance for any domain decision categories exist.

Language: Английский

---

The secret life of crime labs

Proceedings of the National Academy of Sciences, Journal Year: 2023, Volume and Issue: 120(41)

Published: Oct. 2, 2023

Houston TX experienced a widely known failure of its police forensic laboratory. This gave rise to the Forensic Science Center (HFSC) as separate entity provide services City Houston. HFSC is very large laboratory and has made significant progress at remediating past failures improving public trust in testing. robust blind testing program, which provided many insights into challenges laboratories face. HFSC's journey from notoriously failed lab model also gives perspective resource faced by all labs country. Challenges for include pervasive reality poor-quality evidence. Also that are necessarily part much wider system interdependent functions criminal justice making something parts have role. interconnectedness highlights need an array oversight regulatory frameworks function properly. The major essential databases forensics be programs work needed ensure results these indeed producing correct those being correctly used. Last, reports "inconclusive" challenge better understand when appropriate, necessary most importantly used rest system.

Language: Английский

---

On Coping in a Non-Binary World: Rejoinder to Biedermann and Kotsoglou

Statistics and Public Policy, Journal Year: 2024, Volume and Issue: 11(1)

Published: Jan. 5, 2024

Language: Английский

---

Inconclusive conclusions in forensic science: rejoinders to Scurich, Morrison, Sinha and Gutierrez

Law Probability and Risk, Journal Year: 2022, Volume and Issue: 21(3-4), P. 175 - 177

Published: Dec. 1, 2022

To the Editor, We thank professors Scurich, Morrison, Sinha and Mr Gutierrez for their thoughtful comments on our article (Arkes Koehler, 2021). agree with Scurich (2023) that when an examiner knows he or she is being tested, results of such a test are highly suspect. If can avoid making errors by deeming comparison to be inconclusive if inconclusives never deemed indicative error, then 'strategic' inflate accuracy levels rendering decision any difficult test. Such will not provide unbiased measure examiner's accuracy. But this reason enough change way measured. In support different view matter, provides analogy offered Kaye et al. (2022) in which student answers 'I don't know' true–false question. should know answer, say answer counted as error. think Kaye's apt context. Teachers charged determining whether For example, topic was covered required reading lecture, answer. forensic test, one cannot Dror (2020) suggested strategies might help determine but response 2022) we reasons why those inadequate.

Language: Английский

---

More unjustified inferences from limited data in

Law Probability and Risk, Journal Year: 2024, Volume and Issue: 23(1)

Published: Jan. 1, 2024

Abstract In recent years, multiple scholars have criticized the design of studies exploring accuracy firearms examination methods. Rosenblum et al. extend those criticisms to work Guyll on practitioner performance when comparing fired cartridge cases. But while thoroughly dissect issues regarding equiprobability bias and positive predictive values in study, they do not delve as deeply into other areas such variability participant performance, well sampling participants test samples, that further undercut ability generalize al.’s results. This commentary extends what began explores how low rates error reported by likely underestimate potential for misidentifications casework. Ultimately, given convenience authors should gone beyond descriptive statistics instead draw conclusive inferences classify “a highly valid forensic technique.”

Language: Английский

---

Methodological problems in every black-box study of forensic firearm comparisons

Law Probability and Risk, Journal Year: 2024, Volume and Issue: 23(1)

Published: Jan. 1, 2024

Abstract Reviews conducted by the National Academy of Sciences (2009) and President’s Council Advisors on Science Technology (2016) concluded that field forensic firearm comparisons has not been demonstrated to be scientifically valid. Scientific validity requires adequately designed studies examiner performance in terms accuracy, repeatability, reproducibility. Researchers have performed “black-box” with goal estimating these measures. As statisticians expertise experimental design, we a literature search such date then evaluated design statistical analysis methods used each study. Our conclusion is all our methodological flaws are so grave they render invalid, is, incapable establishing scientific firearms examination. Notably, error rates among examiners, both collectively individually, remain unknown. Therefore, statements about common origin bullets or cartridge cases based examination “individual” characteristics do basis. We provide some recommendations for future studies.

Language: Английский

---

How signature complexity affects expert and lay ability to distinguish genuine, disguised and simulated signatures

Journal of Forensic Sciences, Journal Year: 2024, Volume and Issue: 69(6), P. 2159 - 2170

Published: Aug. 26, 2024

Abstract This study examined how variations in signature complexity affected the ability of forensic document examiners (FDEs) and laypeople to determine whether signatures are authentic or simulated (forged), as well they disguised. Forty‐five FDEs from nine countries evaluated different comparisons this online study. Receiver Operating Characteristic (ROC) analyses revealed that performed excess chance levels, but performance varied a function complexity: Sensitivity (the true‐positive rate) did not differ much between levels (i.e., 65% vs. 79% for low vs medium high complexity), specificity true‐negative was highest (95%) lowest (73%) signatures. The high‐complexity (83%) these values. sensitivity disguised only 11% vary across levels. One hundred‐one novices also completed A comparison area under ROC curve (AUCs) outperformed low‐complexity Novices struggled detect While findings elucidate role lay expert evaluations, error rates observed here may those practice due differences experimental stimuli circumstances which were evaluated. investigation evaluation process intended estimate practice.

Language: Английский

---

Inconclusive Conclusions in Forensic Science: Rejoinders to Scurich, Morrison, Sinha & Gutierrez

SSRN Electronic Journal, Journal Year: 2023, Volume and Issue: unknown

Published: Jan. 1, 2023

We agree with Scurich (2023) that when an examiner knows he or she is being tested, the results of such a test highly suspect. If can avoid making errors by deeming comparison to be inconclusive, and if inconclusives are never deemed indicative error, then “strategic” inflate accuracy levels rendering inconclusive decision for any difficult test. Such will not provide unbiased measure examiner’s accuracy. But this reason enough change way measured. support view expressed in Morrison (2023), but our paper we accepted world as it currently exists, one which examiners use categorical conclusions. Finally, Sinha Gutierrez blind testing resolve all issues mentioned their final sentence. think would major step right direction.

Language: Английский

---