Accuracy of comparison decisions by forensic firearms examiners

Journal of Forensic Sciences, Journal Year: 2022, Volume and Issue: 68(1), P. 86 - 100

Published: Oct. 1, 2022

Abstract This black box study assessed the performance of forensic firearms examiners in United States. It involved three different types and 173 volunteers who performed a total 8640 comparisons both bullets cartridge cases. The overall false‐positive error rate was estimated as 0.656% 0.933% for cases, respectively, while false negatives 2.87% 1.87% respectively. majority errors were made by limited number examiners. Because chi‐square tests independence strongly suggest that probabilities are not same each examiner, these maximum‐likelihood estimates based on beta‐binomial probability model do depend an assumption equal examiner‐specific rates. Corresponding 95% confidence intervals (0.305%, 1.42%) (0.548%, 1.57%) positives (1.89%, 4.26%) (1.16%, 2.99%) results this consistent with prior studies, despite its comprehensive design challenging specimens.

Language: Английский

---

The most consistent finding in forensic science is inconsistency

Journal of Forensic Sciences, Journal Year: 2023, Volume and Issue: 68(6), P. 1851 - 1855

Published: Sept. 2, 2023

The most consistent finding in many forensic science domains is inconsistency (i.e., lack of reliability, reproducibility, repeatability, and replicability). consistency a major problem, both from scientific criminal justice point view. Examining conclusion data, across domains, highlights the underlying cognitive issues offers better understanding challenges. Such insights enable development ways to minimize these inconsistencies move forward. aim highlight so that it can be minimized reliability evidence improved.

Language: Английский

---

Inconclusives, errors, and error rates in forensic firearms analysis:Three statistical perspectives

Forensic Science International Synergy, Journal Year: 2022, Volume and Issue: 5, P. 100273 - 100273

Published: Jan. 1, 2022

Error rates that have been published in recent open black box studies of forensic firearms examiner performance very low, typically below one percent. These low error challenged, however, as not properly taking into account the categories, "Inconclusive", examiners can reach comparing a pair bullets or cartridges. challenges themselves challenged; how to consider inconclusives and their effect on is currently matter sharp debate. We review several viewpoints put forth, then examine impact from three fresh statistical perspectives: (a) an ideal perspective using objective measurements combined with algorithms, (b) basic sampling theory practice, (c) standards experimental design human studies. Our conclusions vary perspective: be simple errors (or, other hand, simply correct at least well justified); need counted bring doubt assessments rates; are potential errors, more explicitly, necessarily equivalent casework mask casework. From all these perspectives, it impossible read out trustworthy estimates those which carried date. At most, reasonable bounds rates. much larger than nominal reported To get straightforward, sound requires challenging but critical improvement A proper study-one yields direct, rates-will require new measures blind proficiency testing embedded ordinary

Language: Английский

---

Planning, design and logistics of a decision analysis study: The FBI/Ames study involving forensic firearms examiners

Forensic Science International Synergy, Journal Year: 2022, Volume and Issue: 4, P. 100221 - 100221

Published: Jan. 1, 2022

This paper describes design and logistical aspects of a decision analysis study to assess the performance qualified firearms examiners working in accredited laboratories United States terms accuracy (error rate), repeatability, reproducibility decisions involving comparisons fired bullets cartridge cases. The purpose was validate current practice forensic discipline firearms/toolmarks (F/T) examination. It elicited error rate data by counting number false positive negative conclusions. Preceded experimental design, decisions, logistics described herein, testing ultimately administered 173 qualified, practicing F/T public private crime laboratories. first round evaluated accuracy, while two subsequent rounds repeatability examiner project expands on previous studies many challenging executing recommended double-blind format.

Language: Английский

---

Validity of forensic cartridge-case comparisons

Proceedings of the National Academy of Sciences, Journal Year: 2023, Volume and Issue: 120(20)

Published: May 8, 2023

This article presents key findings from a research project that evaluated the validity and probative value of cartridge-case comparisons under field-based conditions. Decisions provided by 228 trained firearm examiners across US showed forensic comparison is characterized low error rates. However, inconclusive decisions constituted over one-fifth all rendered, complicating evaluation technique's ability to yield unambiguously correct decisions. Specifically, restricting only conclusive identification elimination yielded true-positive true-negative rates exceeding 99%, but incorporating inconclusives caused these values drop 93.4% 63.5%, respectively. The asymmetric effect on two occurred because were rendered six times more frequently for different-source than same-source comparisons. Considering value, which decision's usefulness determining comparison's ground-truth state, predicted their corresponding states with near perfection. Likelihood ratios (LRs) further greatly increase odds state matching asserted decision. Inconclusive also possessed predicting status having LR indicating they status. study manipulated difficulty using models produce dissimilar markings. model chosen being difficult received comparisons, resulting in lower rate compared less model. Relatedly, exhibited strongly predictive

Language: Английский

---

The false promise of firearms examination validation studies: Lay controls, simplistic comparisons, and the failure to soundly measure misidentification rates

Journal of Forensic Sciences, Journal Year: 2024, Volume and Issue: 69(4), P. 1334 - 1349

Published: April 29, 2024

Abstract Several studies have recently attempted to estimate practitioner accuracy when comparing fired ammunition. But whether this research has included sufficiently challenging comparisons dependent upon expertise for accurate conclusions regarding source remains largely unexplored in the literature. Control groups of lay people comprise one means vetting question, assessing comparison samples were at least enough distinguish between experts and novices. This article therefore utilizes such a group, specifically 82 attorneys, as post hoc control juxtaposes their performance on set cartridge case images from commonly cited study (Duez et al. J Forensic Sci. 2018;63:1069–1084) with that original participant pool professionals. Despite lacking kind formalized training experience common latter, our participants displayed an ability, generally, cases by same versus different guns 327 they performed. And while rates lagged substantially behind those professionals same‐source comparisons, different‐source was essentially indistinguishable trained examiners. indicates although we vetted may provide useful information about professional performing it little offer terms measuring examiners' ability guns. If similar issues pervade other studies, then there is reason rely false‐positive generated.

Language: Английский

---

The influence of perceived difficulty, availability of marks, and examination time on the conclusions of firearms examiners

Journal of Forensic Sciences, Journal Year: 2025, Volume and Issue: unknown

Published: Feb. 14, 2025

Abstract Concurrent with studies on the accuracy, repeatability, and reproducibility of decisions based comparisons fired bullet cartridge cases, we also collected opinions participating examiners as to characteristics specimens provided difficulty making comparisons. Examiners rated ease which they determined every conclusion (easy, average, hard) estimated qualitatively amount visual information available them in determining a (limited, some, extensive). Comparisons deemed hard were perceived generally have somewhat fewer markings conducive for assessment, while where limited produced larger number inconclusive determinations. Perceived increased wider separation firing order (within or between three defined segments 700–850 total firings). The repeatability these qualitative assessments exceeded 60% their average was ~50%. Examination times did not vary significantly when rendering identification, elimination, inconclusive, although identifications appear taken slightly longer than those cases. Hard comparisons, limited, treated substantially differently from any other types comparison. No correlation found attempted. These results tend contradict assertions by critics that are tempted declare save time avoid an elimination identification conclusion, non‐representative casework, affected degree examiner participation.

Language: Английский

---

Repeatability and reproducibility of comparison decisions by firearms examiners

Journal of Forensic Sciences, Journal Year: 2023, Volume and Issue: 68(5), P. 1721 - 1740

Published: July 2, 2023

Abstract In a comprehensive study to assess various aspects of the performance qualified forensic firearms examiners, volunteer examiners compared both bullets and cartridge cases fired from three different types firearms. They rendered opinions on each comparison according Association Firearm & Tool Mark Examiners (AFTE) Range Conclusions, as Identification, Inconclusive (A, B, or C), Elimination, Unsuitable. this part study, sets used previously characterize overall accuracy were blindly resubmitted repeatability (105 examiners; 5700 comparisons cases) reproducibility (191 bullets, 193 cases; 5790 comparisons) examinations. Data gathered using prevailing AFTE also recategorized into two hypothetical scoring systems. Consistently positive differences between observed agreement expected indicate that exceed chance agreement. When averaged over cases, decisions (involving all five levels Range) was 78.3% for known matches 64.5% nonmatches. Similarly 67.3%% 36.5% For reproducibility, many disagreements definitive inconclusive category. Examiner are reliable trustworthy in sense identifications unlikely when comparing non‐matching items, eliminations they matching items.

Language: Английский

---

A Re-Analysis of Repeatability and Reproducibility in the Ames-USDOE-FBI Study

Statistics and Public Policy, Journal Year: 2022, Volume and Issue: 9(1), P. 175 - 184

Published: Sept. 6, 2022

Forensic firearms identification, the determination by a trained examiner as to whether or not bullets cartridges came from common weapon, has long been mainstay in criminal courts. Reliability of forensic identification challenged general scientific community, and, response, several studies have carried out aimed at showing that examination is accurate, is, low error rates. Less studied question consistency, two examinations same cartridge cases come conclusion, an on separate occasions—intrarater reliability repeatability—or examiners—interrater reproducibility.One important study, described 2020 Report Ames Laboratory-USDOE Federal Bureau Investigation, went beyond considerations accuracy investigate repeatability and reproducibility. The Report's conclusions were paradoxical. observed agreement examiners with themselves other appears mediocre. However, study concluded reproducibility are satisfactory, grounds exceeds quantity called expected agreement. We find appropriately employing it was intended does suggest satisfactory reproducibility, but opposite.

Language: Английский

---

Inconclusives in firearm error rate studies are not ‘a pass’

Law Probability and Risk, Journal Year: 2022, Volume and Issue: 21(2), P. 123 - 127

Published: June 1, 2022

One question Professors Arkes and Koehler (2022) (hereinafter 'A&K') ask in their thoughtful paper is 'What role should "inconclusives" play the computation of error rates?' (p. 5) The answer to this vital because number inconclusives firearm rate studies staggering. For example, examiners FBI/Ames Laboratory study made 8,640 comparisons, which 3922 (45%) were deemed inconclusive (Bajic et al., 2020, Table V). most recent reported that 51% all comparisons (Best Gardner, 2022). Determining how count half responses critical—perhaps even decisive—factor interpreting rates from study. A&K assert no computing rates. They 'argue not be coded as either correct or incorrect when tabulating forensic rates' 1). Furthermore, write, 'in more usual sense meaning error, an error. It a pass. An means examiner offers judgement about whether two [cartridge cases bullets] do share common source' 9, citing Koehler, 2008).

Language: Английский

---