The most consistent finding in forensic science is inconsistency

Journal of Forensic Sciences, Journal Year: 2023, Volume and Issue: 68(6), P. 1851 - 1855

Published: Sept. 2, 2023

The most consistent finding in many forensic science domains is inconsistency (i.e., lack of reliability, reproducibility, repeatability, and replicability). consistency a major problem, both from scientific criminal justice point view. Examining conclusion data, across domains, highlights the underlying cognitive issues offers better understanding challenges. Such insights enable development ways to minimize these inconsistencies move forward. aim highlight so that it can be minimized reliability evidence improved.

Language: Английский

---

The false promise of firearms examination validation studies: Lay controls, simplistic comparisons, and the failure to soundly measure misidentification rates

Journal of Forensic Sciences, Journal Year: 2024, Volume and Issue: 69(4), P. 1334 - 1349

Published: April 29, 2024

Abstract Several studies have recently attempted to estimate practitioner accuracy when comparing fired ammunition. But whether this research has included sufficiently challenging comparisons dependent upon expertise for accurate conclusions regarding source remains largely unexplored in the literature. Control groups of lay people comprise one means vetting question, assessing comparison samples were at least enough distinguish between experts and novices. This article therefore utilizes such a group, specifically 82 attorneys, as post hoc control juxtaposes their performance on set cartridge case images from commonly cited study (Duez et al. J Forensic Sci. 2018;63:1069–1084) with that original participant pool professionals. Despite lacking kind formalized training experience common latter, our participants displayed an ability, generally, cases by same versus different guns 327 they performed. And while rates lagged substantially behind those professionals same‐source comparisons, different‐source was essentially indistinguishable trained examiners. indicates although we vetted may provide useful information about professional performing it little offer terms measuring examiners' ability guns. If similar issues pervade other studies, then there is reason rely false‐positive generated.

Language: Английский

---

The Hawthorne effect in studies of firearm and toolmark examiners

Journal of Forensic Sciences, Journal Year: 2025, Volume and Issue: unknown

Published: April 10, 2025

Abstract The Hawthorne effect refers to the tendency of individuals behave differently when they know are being studied. In forensic science domain, concerns have been raised about “strategic examiner,” where examiner uses different decision thresholds depending on whether in a test situation or working an actual case. blind testing conducted by Houston Forensic Science Center (“HFSC”) firearms examination presents unique opportunity hypothesis that rate inconclusive calls differs for discovered vs. undiscovered tests firearm examination. Over 5 years, 529 item comparisons were filtered into casework at HFSC. items was 56.4%, while 39.3%. Thus, percentage 43.5% higher among than items. This pattern results held bullet (83% 59%) and cartridge case (29% 20%) both same‐source different‐source comparisons. These findings corroborate examiners tested demonstrate necessity if research goal is evaluate performance conducting casework.

Language: Английский

---

Shifting decision thresholds can undermine the probative value and legal utility of forensic pattern-matching evidence

Proceedings of the National Academy of Sciences, Journal Year: 2023, Volume and Issue: 120(41)

Published: Oct. 2, 2023

Forensic pattern analysis requires examiners to compare the patterns of items such as fingerprints or tool marks assess whether they have a common source. This article uses signal detection theory model examiners' reported conclusions (e.g., identification, inconclusive, exclusion), focusing on connection between examiner's decision threshold and probative value forensic evidence. It Bayesian network explore how shifts in thresholds may affect rates ratios true false convictions hypothetical legal system. demonstrates that small thresholds, which arise from contextual bias, can dramatically pattern-matching evidence its utility

Language: Английский

---

A guide to measuring expert performance in forensic pattern matching

Behavior Research Methods, Journal Year: 2024, Volume and Issue: 56(6), P. 6223 - 6247

Published: March 14, 2024

Abstract Decisions in forensic science are often binary. A firearms expert must decide whether a bullet was fired from particular gun or not. face comparison photograph matches suspect fingerprint examiner crime scene belongs to Researchers who study these decisions have therefore quantified performance using measurement models derived largely signal detection theory. Here we demonstrate that the design and choices researchers make can dramatic effect on conclusions drawn about of examiners. We introduce several – proportion correct, diagnosticity ratio, parametric non-parametric measures apply them decisions. use data novice along with resampling method how experimental results change as function task, case materials, model chosen. also graphically show response bias, prevalence, inconclusive responses, floor ceiling effects, sampling, number trials might affect one’s interpretation forensics. Finally, discuss considerations for diagnostic accuracy studies: (1) include an equal same-source different-source trials; (2) record responses separately forced choices; (3) control group; (4) counterbalance randomly sample each participant; (5) present many participants is practical.

Language: Английский

---

Inconclusives in firearm error rate studies are not ‘a pass’

Law Probability and Risk, Journal Year: 2022, Volume and Issue: 21(2), P. 123 - 127

Published: June 1, 2022

One question Professors Arkes and Koehler (2022) (hereinafter 'A&K') ask in their thoughtful paper is 'What role should "inconclusives" play the computation of error rates?' (p. 5) The answer to this vital because number inconclusives firearm rate studies staggering. For example, examiners FBI/Ames Laboratory study made 8,640 comparisons, which 3922 (45%) were deemed inconclusive (Bajic et al., 2020, Table V). most recent reported that 51% all comparisons (Best Gardner, 2022). Determining how count half responses critical—perhaps even decisive—factor interpreting rates from study. A&K assert no computing rates. They 'argue not be coded as either correct or incorrect when tabulating forensic rates' 1). Furthermore, write, 'in more usual sense meaning error, an error. It a pass. An means examiner offers judgement about whether two [cartridge cases bullets] do share common source' 9, citing Koehler, 2008).

Language: Английский

---

Signal detection theory fails to account for real-world consequences of inconclusive decisions

Law Probability and Risk, Journal Year: 2022, Volume and Issue: 21(2), P. 131 - 135

Published: June 1, 2022

Language: Английский

---

Three-Way ROCs for Forensic Decision Making

Statistics and Public Policy, Journal Year: 2023, Volume and Issue: 10(1)

Published: July 21, 2023

Firearm examiners use a comparison microscope to judge whether bullets or cartridge cases were fired by the same gun. Examiners can reach one of three possible conclusions: Identification (a match), Elimination (not Inconclusive. Numerous error rate studies report that firearm commit few errors when they conduct these examinations. However, also many inconclusive judgments (>50%), and how score responses is controversial. There have recently been Signal Detection Theory (SDT) primers in this domain. Unfortunately, analyses rely on hypothetical data fail address response issue adequately. This article reports an SDT analysis using from large study practicing examiners. First, we demonstrate problem relying traditional two-way model, which either drops combines responses; addition lacking ecological validity, approach leads implausible results. Second, introduce readers three-way model. We forensic firearms While statistically complicated, it well suited evaluate performance for any domain decision categories exist.

Language: Английский

---

LARGE-SCALE SUPER-RECOGNIZER IDENTIFICATION IN THE BERLIN POLICE

Published: Sept. 22, 2023

International law enforcement agencies are tasked with processing increasing amounts of security-relevant digital information, particularly images and videos showing physical persons' facial identity. The permissible use technology is subject to certain legal conditions, which vary across systems. But even if automatic assistance permissible, neither ubiquitous nor infallible. Therefore, police the world require novel solutions enabling both efficient accurate individuals' Over past years, so-called Super-Recognizers (SRs), humans an apparently innate superiority for identity, have received interest from enforcement. For good reason: they can be found among professionals already safeguarding public. Contrary surging media in this topic, little attention has been directed toward scrutinizing methods that used identify these highly anticipated "super-heroes". This problematic as available tests developed lab by researchers, cannot expected represent variety real-life tasks, SRs' could perform. We report data obtained within entire body 18K officers Berlin Police, who were invited participate applied research study April 2021. With unique cohort, we first-ever validation lab-based procedures SR identification, using only existing tool involving authentic material: Test Super-Recognizer Identification (beSure®). results demonstrate a group SRs identified via more proficient real-life, professionally relevant However, at individual level insufficiently sensitive identifying exceptional ability. Consequently, well advised consider two main aspects. First, (existing) bespoke methods, such beSure®, specifically identification. Second, systematically document professional identification deployment. not crucial evaluation It prerequisite development needed human-centered, technology-assisted approach identity

Language: Английский

---

Commentary on: Monson KL, Smith ED, Peters EM. Accuracy of comparison decisions by forensic firearms examiners. J Forensic Sci. 2022;68(1):86–100. https://doi.org/10.1111/1556‐4029.15152.

Journal of Forensic Sciences, Journal Year: 2023, Volume and Issue: 68(3), P. 1097 - 1101

Published: April 21, 2023

Language: Английский

---