Accuracy of comparison decisions by forensic firearms examiners

Journal of Forensic Sciences, Journal Year: 2022, Volume and Issue: 68(1), P. 86 - 100

Published: Oct. 1, 2022

Abstract This black box study assessed the performance of forensic firearms examiners in United States. It involved three different types and 173 volunteers who performed a total 8640 comparisons both bullets cartridge cases. The overall false‐positive error rate was estimated as 0.656% 0.933% for cases, respectively, while false negatives 2.87% 1.87% respectively. majority errors were made by limited number examiners. Because chi‐square tests independence strongly suggest that probabilities are not same each examiner, these maximum‐likelihood estimates based on beta‐binomial probability model do depend an assumption equal examiner‐specific rates. Corresponding 95% confidence intervals (0.305%, 1.42%) (0.548%, 1.57%) positives (1.89%, 4.26%) (1.16%, 2.99%) results this consistent with prior studies, despite its comprehensive design challenging specimens.

Language: Английский

---

Inconclusives, errors, and error rates in forensic firearms analysis:Three statistical perspectives

Forensic Science International Synergy, Journal Year: 2022, Volume and Issue: 5, P. 100273 - 100273

Published: Jan. 1, 2022

Error rates that have been published in recent open black box studies of forensic firearms examiner performance very low, typically below one percent. These low error challenged, however, as not properly taking into account the categories, "Inconclusive", examiners can reach comparing a pair bullets or cartridges. challenges themselves challenged; how to consider inconclusives and their effect on is currently matter sharp debate. We review several viewpoints put forth, then examine impact from three fresh statistical perspectives: (a) an ideal perspective using objective measurements combined with algorithms, (b) basic sampling theory practice, (c) standards experimental design human studies. Our conclusions vary perspective: be simple errors (or, other hand, simply correct at least well justified); need counted bring doubt assessments rates; are potential errors, more explicitly, necessarily equivalent casework mask casework. From all these perspectives, it impossible read out trustworthy estimates those which carried date. At most, reasonable bounds rates. much larger than nominal reported To get straightforward, sound requires challenging but critical improvement A proper study-one yields direct, rates-will require new measures blind proficiency testing embedded ordinary

Language: Английский

---

The false promise of firearms examination validation studies: Lay controls, simplistic comparisons, and the failure to soundly measure misidentification rates

Journal of Forensic Sciences, Journal Year: 2024, Volume and Issue: 69(4), P. 1334 - 1349

Published: April 29, 2024

Abstract Several studies have recently attempted to estimate practitioner accuracy when comparing fired ammunition. But whether this research has included sufficiently challenging comparisons dependent upon expertise for accurate conclusions regarding source remains largely unexplored in the literature. Control groups of lay people comprise one means vetting question, assessing comparison samples were at least enough distinguish between experts and novices. This article therefore utilizes such a group, specifically 82 attorneys, as post hoc control juxtaposes their performance on set cartridge case images from commonly cited study (Duez et al. J Forensic Sci. 2018;63:1069–1084) with that original participant pool professionals. Despite lacking kind formalized training experience common latter, our participants displayed an ability, generally, cases by same versus different guns 327 they performed. And while rates lagged substantially behind those professionals same‐source comparisons, different‐source was essentially indistinguishable trained examiners. indicates although we vetted may provide useful information about professional performing it little offer terms measuring examiners' ability guns. If similar issues pervade other studies, then there is reason rely false‐positive generated.

Language: Английский

---

The Hawthorne effect in studies of firearm and toolmark examiners

Journal of Forensic Sciences, Journal Year: 2025, Volume and Issue: unknown

Published: April 10, 2025

Abstract The Hawthorne effect refers to the tendency of individuals behave differently when they know are being studied. In forensic science domain, concerns have been raised about “strategic examiner,” where examiner uses different decision thresholds depending on whether in a test situation or working an actual case. blind testing conducted by Houston Forensic Science Center (“HFSC”) firearms examination presents unique opportunity hypothesis that rate inconclusive calls differs for discovered vs. undiscovered tests firearm examination. Over 5 years, 529 item comparisons were filtered into casework at HFSC. items was 56.4%, while 39.3%. Thus, percentage 43.5% higher among than items. This pattern results held bullet (83% 59%) and cartridge case (29% 20%) both same‐source different‐source comparisons. These findings corroborate examiners tested demonstrate necessity if research goal is evaluate performance conducting casework.

Language: Английский

---

Repeatability and reproducibility of comparison decisions by firearms examiners

Journal of Forensic Sciences, Journal Year: 2023, Volume and Issue: 68(5), P. 1721 - 1740

Published: July 2, 2023

Abstract In a comprehensive study to assess various aspects of the performance qualified forensic firearms examiners, volunteer examiners compared both bullets and cartridge cases fired from three different types firearms. They rendered opinions on each comparison according Association Firearm & Tool Mark Examiners (AFTE) Range Conclusions, as Identification, Inconclusive (A, B, or C), Elimination, Unsuitable. this part study, sets used previously characterize overall accuracy were blindly resubmitted repeatability (105 examiners; 5700 comparisons cases) reproducibility (191 bullets, 193 cases; 5790 comparisons) examinations. Data gathered using prevailing AFTE also recategorized into two hypothetical scoring systems. Consistently positive differences between observed agreement expected indicate that exceed chance agreement. When averaged over cases, decisions (involving all five levels Range) was 78.3% for known matches 64.5% nonmatches. Similarly 67.3%% 36.5% For reproducibility, many disagreements definitive inconclusive category. Examiner are reliable trustworthy in sense identifications unlikely when comparing non‐matching items, eliminations they matching items.

Language: Английский

---

The influence of perceived difficulty, availability of marks, and examination time on the conclusions of firearms examiners

Journal of Forensic Sciences, Journal Year: 2025, Volume and Issue: unknown

Published: Feb. 14, 2025

Abstract Concurrent with studies on the accuracy, repeatability, and reproducibility of decisions based comparisons fired bullet cartridge cases, we also collected opinions participating examiners as to characteristics specimens provided difficulty making comparisons. Examiners rated ease which they determined every conclusion (easy, average, hard) estimated qualitatively amount visual information available them in determining a (limited, some, extensive). Comparisons deemed hard were perceived generally have somewhat fewer markings conducive for assessment, while where limited produced larger number inconclusive determinations. Perceived increased wider separation firing order (within or between three defined segments 700–850 total firings). The repeatability these qualitative assessments exceeded 60% their average was ~50%. Examination times did not vary significantly when rendering identification, elimination, inconclusive, although identifications appear taken slightly longer than those cases. Hard comparisons, limited, treated substantially differently from any other types comparison. No correlation found attempted. These results tend contradict assertions by critics that are tempted declare save time avoid an elimination identification conclusion, non‐representative casework, affected degree examiner participation.

Language: Английский

---

Shining a Light on Forensic Black-Box Studies

Statistics and Public Policy, Journal Year: 2023, Volume and Issue: 10(1)

Published: May 23, 2023

Forensic science plays a critical role in the United States criminal justice system. For decades, many feature-based fields of forensic science, such as firearm and toolmark identification, developed outside scientific community's purview. The results these studies are widely relied on by judges nationwide. However, this reliance is misplaced. Black-box to date suffer from inappropriate sampling methods high rates missingness. Current black-box ignore both problems arriving at error rate estimates presented courts. We explore impact each type limitation using available data court materials. show that rely non-representative samples examiners. Using case study popular ballistics study, we find evidence may commit fewer errors than wider population which they came. also missingness non-ignorable. recent latent print ignoring likely systematic underestimates rates. Finally, offer concrete steps overcome limitations.

Language: Английский

---

Hierarchical Bayesian non-response models for error rates in forensic black-box studies

Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences, Journal Year: 2023, Volume and Issue: 381(2247)

Published: March 27, 2023

Forensic science plays a critical role in the United States criminal legal system. Historically, however, most feature-based fields of forensic science, including firearms examination and latent print analysis, have not been shown to be scientifically valid. Recently, black-box studies proposed as means assessing whether these disciplines are valid, at least terms accuracy, reproducibility repeatability. In studies, examiners frequently either do respond every test item or select an answer equivalent 'don't know'. Current account for high levels missingness statistical analyses. Unfortunately, authors typically share data necessary meaningfully adjust estimates proportion missing responses. Borrowing from work context small area estimation, we propose use hierarchical Bayesian models that require auxiliary non-response. Using models, offer first formal exploration impact is playing error rate estimations reported studies. We show rates currently low 0.4% could actually 8.4% accounting non-response where inconclusive decisions counted correct, over 28% when inconclusives These problem But with release information, they can foundation new methodologies estimations. This article part theme issue 'Bayesian inference: challenges, perspectives, prospects'.

Language: Английский

---

Three-Way ROCs for Forensic Decision Making

Statistics and Public Policy, Journal Year: 2023, Volume and Issue: 10(1)

Published: July 21, 2023

Firearm examiners use a comparison microscope to judge whether bullets or cartridge cases were fired by the same gun. Examiners can reach one of three possible conclusions: Identification (a match), Elimination (not Inconclusive. Numerous error rate studies report that firearm commit few errors when they conduct these examinations. However, also many inconclusive judgments (>50%), and how score responses is controversial. There have recently been Signal Detection Theory (SDT) primers in this domain. Unfortunately, analyses rely on hypothetical data fail address response issue adequately. This article reports an SDT analysis using from large study practicing examiners. First, we demonstrate problem relying traditional two-way model, which either drops combines responses; addition lacking ecological validity, approach leads implausible results. Second, introduce readers three-way model. We forensic firearms While statistically complicated, it well suited evaluate performance for any domain decision categories exist.

Language: Английский

---

The use and abuse of the elusive construct of inconclusive decisions

Law Probability and Risk, Journal Year: 2022, Volume and Issue: 21(1), P. 85 - 87

Published: March 1, 2022

Inconclusives1 is an elusive conclusion in any domain, and particular the forensic sciences. Until recently, it has been relatively neglected, if not ignored, as viewed incorrectly a 'non-decision' with little implication or importance (in contrast to decisive conclusion, e.g., 'match' vs. 'exclusion', 'homicide' 'accident'). However, inconclusives simply matter of 'not decide', but actually involves making important decision 'decide decide' (Dror Langenburg, 2019). Inconclusive decisions have critical implications casework error rate studies. In casework, fingerprinting, deciding inconclusive rather than exclusion may prevent police from eliminating suspect investigation. pathology, that death undetermined homicide full This be especially problematic there systematic disproportional conclusions when people die while custody.

Language: Английский

---