Inconclusive decisions and error rates in forensic science

Forensic Science International Synergy, Journal Year: 2024, Volume and Issue: 8, P. 100472 - 100472

Published: Jan. 1, 2024

In recent years, there has been discussion and controversy relating to the treatment of inconclusive decisions in forensic feature comparison disciplines when considering reliability examination methods results. this article, we offer a brief review various viewpoints suggestions that have recently put forth, followed by solution believe addresses decisions. We consider issues context method conformance performance as two distinct concepts, both which are necessary for determination reliability. Method relates an assessment whether outcome is result analyst's adherence procedures define method. reflects capacity discriminate between different propositions interest (e.g., mated non-mated comparisons). then discuss implications these science community.

Language: Английский

---

The most consistent finding in forensic science is inconsistency

Journal of Forensic Sciences, Journal Year: 2023, Volume and Issue: 68(6), P. 1851 - 1855

Published: Sept. 2, 2023

The most consistent finding in many forensic science domains is inconsistency (i.e., lack of reliability, reproducibility, repeatability, and replicability). consistency a major problem, both from scientific criminal justice point view. Examining conclusion data, across domains, highlights the underlying cognitive issues offers better understanding challenges. Such insights enable development ways to minimize these inconsistencies move forward. aim highlight so that it can be minimized reliability evidence improved.

Language: Английский

---

Validity of forensic cartridge-case comparisons

Proceedings of the National Academy of Sciences, Journal Year: 2023, Volume and Issue: 120(20)

Published: May 8, 2023

This article presents key findings from a research project that evaluated the validity and probative value of cartridge-case comparisons under field-based conditions. Decisions provided by 228 trained firearm examiners across US showed forensic comparison is characterized low error rates. However, inconclusive decisions constituted over one-fifth all rendered, complicating evaluation technique's ability to yield unambiguously correct decisions. Specifically, restricting only conclusive identification elimination yielded true-positive true-negative rates exceeding 99%, but incorporating inconclusives caused these values drop 93.4% 63.5%, respectively. The asymmetric effect on two occurred because were rendered six times more frequently for different-source than same-source comparisons. Considering value, which decision's usefulness determining comparison's ground-truth state, predicted their corresponding states with near perfection. Likelihood ratios (LRs) further greatly increase odds state matching asserted decision. Inconclusive also possessed predicting status having LR indicating they status. study manipulated difficulty using models produce dissimilar markings. model chosen being difficult received comparisons, resulting in lower rate compared less model. Relatedly, exhibited strongly predictive

Language: Английский

---

The false promise of firearms examination validation studies: Lay controls, simplistic comparisons, and the failure to soundly measure misidentification rates

Journal of Forensic Sciences, Journal Year: 2024, Volume and Issue: 69(4), P. 1334 - 1349

Published: April 29, 2024

Abstract Several studies have recently attempted to estimate practitioner accuracy when comparing fired ammunition. But whether this research has included sufficiently challenging comparisons dependent upon expertise for accurate conclusions regarding source remains largely unexplored in the literature. Control groups of lay people comprise one means vetting question, assessing comparison samples were at least enough distinguish between experts and novices. This article therefore utilizes such a group, specifically 82 attorneys, as post hoc control juxtaposes their performance on set cartridge case images from commonly cited study (Duez et al. J Forensic Sci. 2018;63:1069–1084) with that original participant pool professionals. Despite lacking kind formalized training experience common latter, our participants displayed an ability, generally, cases by same versus different guns 327 they performed. And while rates lagged substantially behind those professionals same‐source comparisons, different‐source was essentially indistinguishable trained examiners. indicates although we vetted may provide useful information about professional performing it little offer terms measuring examiners' ability guns. If similar issues pervade other studies, then there is reason rely false‐positive generated.

Language: Английский

---

The Hawthorne effect in studies of firearm and toolmark examiners

Journal of Forensic Sciences, Journal Year: 2025, Volume and Issue: unknown

Published: April 10, 2025

Abstract The Hawthorne effect refers to the tendency of individuals behave differently when they know are being studied. In forensic science domain, concerns have been raised about “strategic examiner,” where examiner uses different decision thresholds depending on whether in a test situation or working an actual case. blind testing conducted by Houston Forensic Science Center (“HFSC”) firearms examination presents unique opportunity hypothesis that rate inconclusive calls differs for discovered vs. undiscovered tests firearm examination. Over 5 years, 529 item comparisons were filtered into casework at HFSC. items was 56.4%, while 39.3%. Thus, percentage 43.5% higher among than items. This pattern results held bullet (83% 59%) and cartridge case (29% 20%) both same‐source different‐source comparisons. These findings corroborate examiners tested demonstrate necessity if research goal is evaluate performance conducting casework.

Language: Английский

---

The influence of perceived difficulty, availability of marks, and examination time on the conclusions of firearms examiners

Journal of Forensic Sciences, Journal Year: 2025, Volume and Issue: unknown

Published: Feb. 14, 2025

Abstract Concurrent with studies on the accuracy, repeatability, and reproducibility of decisions based comparisons fired bullet cartridge cases, we also collected opinions participating examiners as to characteristics specimens provided difficulty making comparisons. Examiners rated ease which they determined every conclusion (easy, average, hard) estimated qualitatively amount visual information available them in determining a (limited, some, extensive). Comparisons deemed hard were perceived generally have somewhat fewer markings conducive for assessment, while where limited produced larger number inconclusive determinations. Perceived increased wider separation firing order (within or between three defined segments 700–850 total firings). The repeatability these qualitative assessments exceeded 60% their average was ~50%. Examination times did not vary significantly when rendering identification, elimination, inconclusive, although identifications appear taken slightly longer than those cases. Hard comparisons, limited, treated substantially differently from any other types comparison. No correlation found attempted. These results tend contradict assertions by critics that are tempted declare save time avoid an elimination identification conclusion, non‐representative casework, affected degree examiner participation.

Language: Английский

---

Shining a Light on Forensic Black-Box Studies

Statistics and Public Policy, Journal Year: 2023, Volume and Issue: 10(1)

Published: May 23, 2023

Forensic science plays a critical role in the United States criminal justice system. For decades, many feature-based fields of forensic science, such as firearm and toolmark identification, developed outside scientific community's purview. The results these studies are widely relied on by judges nationwide. However, this reliance is misplaced. Black-box to date suffer from inappropriate sampling methods high rates missingness. Current black-box ignore both problems arriving at error rate estimates presented courts. We explore impact each type limitation using available data court materials. show that rely non-representative samples examiners. Using case study popular ballistics study, we find evidence may commit fewer errors than wider population which they came. also missingness non-ignorable. recent latent print ignoring likely systematic underestimates rates. Finally, offer concrete steps overcome limitations.

Language: Английский

---

Understanding ‘error’ in the forensic sciences: A primer

Forensic Science International Synergy, Journal Year: 2024, Volume and Issue: 8, P. 100470 - 100470

Published: Jan. 1, 2024

This paper distils seven key lessons about 'error' from a collaborative webinar series between practitioners at Victoria Police Forensic Services Department and academics.It aims to provide the common understanding of error necessary foster interdisciplinary dialogue, collaboration research.The underscore inevitability, complexity subjectivity error, as well opportunities for learning growth.Ultimately, we argue that can be potent tool continuous improvement accountability, enhancing reliability forensic sciences public trust.

Language: Английский

---

Hierarchical Bayesian non-response models for error rates in forensic black-box studies

Philosophical Transactions of the Royal Society A Mathematical Physical and Engineering Sciences, Journal Year: 2023, Volume and Issue: 381(2247)

Published: March 27, 2023

Forensic science plays a critical role in the United States criminal legal system. Historically, however, most feature-based fields of forensic science, including firearms examination and latent print analysis, have not been shown to be scientifically valid. Recently, black-box studies proposed as means assessing whether these disciplines are valid, at least terms accuracy, reproducibility repeatability. In studies, examiners frequently either do respond every test item or select an answer equivalent 'don't know'. Current account for high levels missingness statistical analyses. Unfortunately, authors typically share data necessary meaningfully adjust estimates proportion missing responses. Borrowing from work context small area estimation, we propose use hierarchical Bayesian models that require auxiliary non-response. Using models, offer first formal exploration impact is playing error rate estimations reported studies. We show rates currently low 0.4% could actually 8.4% accounting non-response where inconclusive decisions counted correct, over 28% when inconclusives These problem But with release information, they can foundation new methodologies estimations. This article part theme issue 'Bayesian inference: challenges, perspectives, prospects'.

Language: Английский

---

Scientific guidelines for evaluating the validity of forensic feature-comparison methods

Proceedings of the National Academy of Sciences, Journal Year: 2023, Volume and Issue: 120(41)

Published: Oct. 2, 2023

When it comes to questions of fact in a legal context-particularly about measurement, association, and causality-courts should employ ordinary standards applied science. Applied sciences generally develop along path that proceeds from basic scientific discovery some natural process the formation theory how works what causes fail, development an invention intended assess, repair, or improve process, specification predictions instrument's actions and, finally, empirical validation determine instrument achieves effect. These elements are salient deeply embedded cultures medicine engineering, both which primarily grew sciences. However, inventions underlie most forensic science disciplines have few roots science, they do not sound theories justify their predicted results tests prove work as advertised. Inspired by "Bradford Hill Guidelines"-the dominant framework for causal inference epidemiology-we set forth four guidelines can be used establish validity comparison methods generally. This is checklist establishing threshold minimum validity, no magic formula determines when particular hypotheses passed necessary threshold. We illustrate these considering discipline firearm tool mark examination.

Language: Английский

---