Hierarchical Decision Ensembles- An inferential framework for uncertain Human-AI collaboration in forensic examinations.

Ganesh Krishnan,

Heike Hofmann

arXiv (Cornell University), Journal Year: 2021, Volume and Issue: unknown

Published: Oct. 31, 2021

Forensic examination of evidence like firearms and toolmarks, traditionally involves a visual therefore subjective assessment similarity two questioned items. Statistical models are used to overcome this subjectivity allow specification error rates. These generally quite complex produce abstract results at different levels the analysis. Presenting such metrics complicated examiners is challenging, as do not have substantial statistical training accurately interpret results. This creates distrust in modelling lowers rate acceptance more objective measures that discipline large striving for. We present an inferential framework for assessing model its output. The designed calibrate trust forensic experts by bridging gap between domain specific knowledge predictive results, allowing validate claims while critically

Language: Английский

Match me if you can: Evidence for a domain-general visual comparison ability DOI Creative Commons
Bethany Growns, James D. Dunn, Erwin J.A.T. Mattijssen

et al.

Psychonomic Bulletin & Review, Journal Year: 2022, Volume and Issue: 29(3), P. 866 - 881

Published: Jan. 7, 2022

Visual comparison-comparing visual stimuli (e.g., fingerprints) side by and determining whether they originate from the same or different source (i.e., "match")-is a complex discrimination task involving many cognitive perceptual processes. Despite real-world consequences of this task, which is often conducted forensic scientists, little understood about psychological processes underpinning ability. There are substantial individual differences in comparison accuracy amongst both professionals novices. The variation unknown, but may reflect domain-general naturally varying Here, we investigate comparing (N = 248 across two studies) four domains: faces, fingerprints, firearms, artificial prints. Accuracy on all tasks was significantly correlated accounted for portion variance 42% Exp. 1) performance tasks. Importantly, relationship cannot be attributed to participants' intrinsic motivation skill other visual-perceptual (visual search statistical learning). This paper provides novel evidence reliable,

Language: Английский

Citations

25

Planning, design and logistics of a decision analysis study: The FBI/Ames study involving forensic firearms examiners DOI Creative Commons
Keith L. Monson,

Erich D. Smith,

Stanley J. Bajic

et al.

Forensic Science International Synergy, Journal Year: 2022, Volume and Issue: 4, P. 100221 - 100221

Published: Jan. 1, 2022

This paper describes design and logistical aspects of a decision analysis study to assess the performance qualified firearms examiners working in accredited laboratories United States terms accuracy (error rate), repeatability, reproducibility decisions involving comparisons fired bullets cartridge cases. The purpose was validate current practice forensic discipline firearms/toolmarks (F/T) examination. It elicited error rate data by counting number false positive negative conclusions. Preceded experimental design, decisions, logistics described herein, testing ultimately administered 173 qualified, practicing F/T public private crime laboratories. first round evaluated accuracy, while two subsequent rounds repeatability examiner project expands on previous studies many challenging executing recommended double-blind format.

Language: Английский

Citations

23

The influence of perceived difficulty, availability of marks, and examination time on the conclusions of firearms examiners DOI Open Access
Keith L. Monson,

Erich D. Smith,

Eugene M. Peters

et al.

Journal of Forensic Sciences, Journal Year: 2025, Volume and Issue: unknown

Published: Feb. 14, 2025

Abstract Concurrent with studies on the accuracy, repeatability, and reproducibility of decisions based comparisons fired bullet cartridge cases, we also collected opinions participating examiners as to characteristics specimens provided difficulty making comparisons. Examiners rated ease which they determined every conclusion (easy, average, hard) estimated qualitatively amount visual information available them in determining a (limited, some, extensive). Comparisons deemed hard were perceived generally have somewhat fewer markings conducive for assessment, while where limited produced larger number inconclusive determinations. Perceived increased wider separation firing order (within or between three defined segments 700–850 total firings). The repeatability these qualitative assessments exceeded 60% their average was ~50%. Examination times did not vary significantly when rendering identification, elimination, inconclusive, although identifications appear taken slightly longer than those cases. Hard comparisons, limited, treated substantially differently from any other types comparison. No correlation found attempted. These results tend contradict assertions by critics that are tempted declare save time avoid an elimination identification conclusion, non‐representative casework, affected degree examiner participation.

Language: Английский

Citations

0

Inconclusives and error rates in forensic science: a signal detection theory approach DOI
Hal R. Arkes,

Jonathan J. Koehler

Law Probability and Risk, Journal Year: 2021, Volume and Issue: 20(3), P. 153 - 168

Published: Sept. 1, 2021

Abstract There are times when a forensic scientist may not be comfortable drawing firm conclusion about whether questioned sample that appears to contain useful identifying information did or come from particular known source. In such cases, the call pair ‘inconclusive’. We suggest signal detection theory (SDT), which is concerned with of weak signals in noisy environments, provides framework for understanding role inconclusives play various feature-matching sciences. SDT shows ‘inconclusive’ often an appropriate response depending on both strength samples and thresholds adopted by examiner. also argue should coded as either correct incorrect tabulating error rates.

Language: Английский

Citations

30

Repeatability and reproducibility of comparison decisions by firearms examiners DOI Creative Commons
Keith L. Monson,

Erich D. Smith,

Eugene M. Peters

et al.

Journal of Forensic Sciences, Journal Year: 2023, Volume and Issue: 68(5), P. 1721 - 1740

Published: July 2, 2023

Abstract In a comprehensive study to assess various aspects of the performance qualified forensic firearms examiners, volunteer examiners compared both bullets and cartridge cases fired from three different types firearms. They rendered opinions on each comparison according Association Firearm & Tool Mark Examiners (AFTE) Range Conclusions, as Identification, Inconclusive (A, B, or C), Elimination, Unsuitable. this part study, sets used previously characterize overall accuracy were blindly resubmitted repeatability (105 examiners; 5700 comparisons cases) reproducibility (191 bullets, 193 cases; 5790 comparisons) examinations. Data gathered using prevailing AFTE also recategorized into two hypothetical scoring systems. Consistently positive differences between observed agreement expected indicate that exceed chance agreement. When averaged over cases, decisions (involving all five levels Range) was 78.3% for known matches 64.5% nonmatches. Similarly 67.3%% 36.5% For reproducibility, many disagreements definitive inconclusive category. Examiner are reliable trustworthy in sense identifications unlikely when comparing non‐matching items, eliminations they matching items.

Language: Английский

Citations

11

An overview of log likelihood ratio cost in forensic science – Where is it used and what values can we expect? DOI Creative Commons

Stijn van Lierop,

Daniel Ramos, Marjan Sjerps

et al.

Forensic Science International Synergy, Journal Year: 2024, Volume and Issue: 8, P. 100466 - 100466

Published: Jan. 1, 2024

There is increasing support for reporting evidential strength as a likelihood ratio (LR) and interest in (semi-)automated LR systems. The log-likelihood cost (Cllr) popular metric such systems, penalizing misleading LRs further from 1 more. Cllr = 0 indicates perfection while an uninformative system. However, beyond this, what constitutes "good" unclear. Aiming to provide handles on when "good", we studied 136 publications Results show use heavily depends the field, e.g., being absent DNA analysis. Despite more automated systems over time, proportion remains stable. Noticeably, values lack clear patterns depend area, analysis dataset. As become prevalent, comparing them becomes crucial. This hampered by different studies using datasets. We advocate public benchmark datasets advance field.

Language: Английский

Citations

4

Forensic comparison of fired cartridge cases: Feature-extraction methods for feature-based calculation of likelihood ratios DOI Creative Commons
Nabanita Basu, Rachel Bolton-King, Geoffrey Stewart Morrison

et al.

Forensic Science International Synergy, Journal Year: 2022, Volume and Issue: 5, P. 100272 - 100272

Published: Jan. 1, 2022

We describe and validate a feature-based system for calculation of likelihood ratios from 3D digital images fired cartridge cases. The includes database the bases 10 cartridges per firearm approximately 300 firearms same class (semi-automatic pistols that fire 9 mm diameter centre-fire Luger-type ammunition, have hemispherical firing pins parallel breech-face marks). were captured using Evofinder®, an imaging is commonly used by operational forensic laboratories. A key component research reported comparison different feature-extraction methods. Feature sets compared include those previously proposed in literature, plus Zernike-moment based features. Comparisons are also made feature extracted firing-pin impression, region, whole region interest (firing-pin impression + flowback if present). Likelihood calculated statistical modelling pipeline standard voice comparison. Validation conducted results assessed validation procedures metrics graphics

Language: Английский

Citations

13

Forensic Science Decision-Making DOI
Bethany Growns, Tess M. S. Neal

Cambridge University Press eBooks, Journal Year: 2024, Volume and Issue: unknown, P. 193 - 210

Published: Feb. 22, 2024

This chapter focuses on the expertise of forensic science practitioners who make legally relevant decisions from both a skills-based and vulnerabilities-based perspective. We bring together research to examiners' decision-making inform development empirically based solutions for some issues we raise. begin by describing general model bias in judgment, explaining how experts can be vulnerable systematic errors biases decision-making, but also protect against circumstances. then explore cognitive mechanisms underpinning unique abilities examiners develop, important role these their implications selection training. describe what done now enhance experts' strengths mitigate negative effects bias. Finally, conclude exploring future continue vulnerabilities professional performance.

Language: Английский

Citations

2

De la police scientifique à la traçologie DOI
Olivier Ribaux

EPFL Press eBooks, Journal Year: 2023, Volume and Issue: unknown

Published: Aug. 31, 2023

Le rôle de la police scientifique est d’abord d’exploiter les traces laissées lors d’activités criminelles. Elle aujourd’hui équipée technologies traçabilité si puissantes que celles-ci ont, en peu temps, démultiplié quantité et variété données mises à disposition l’enquête judiciaire du renseignement criminel. Or cette évolution rapide a paradoxalement eu pour conséquence une remise question rôle, statut l’action : qu’attend-on ces services ? Que sont-ils supposés conclure partir devenues aussi considérables spécialisées fragmentées L’auteur décrit comment évolue vers nouvelle discipline appelée « traçologie ». Celle-ci s’oppose l’hyper-spécialisation encourageant professionnels adopter vision d’ensemble essentielle résoudre des enquêtes complexes, analyser criminalité sérielle renseigner sécurité. Un ouvrage manifeste, principalement destiné aux criminalistes criminologues concernés par l’avenir scientifique, mais tous sécurité, qui trouveront dans pages méthodes modèles directement applicables, étudiants sciences criminelles, chercheurs quête d’interdisciplinarité au public intéressé d’investigation curieux d’en découvrir arcanes.

Citations

5

Striated toolmarks comparison and reporting methods: Review and perspectives DOI Creative Commons

Jean-Alexandre Patteet,

Christophe Champod

Forensic Science International, Journal Year: 2024, Volume and Issue: 357, P. 111997 - 111997

Published: March 18, 2024

Forensic toolmark examiners have been comparing features observed in toolmarks to help determine their source for over a century. However, the past decade, holistic process of and presenting findings court faced intense scrutiny. This paper provides summary voiced criticisms, primarily concerning scientific reliability validity comparison methods employed by conclusions they testify to. The focus this review is specifically on examination striated toolmarks. We assess reporting practices currently use, while also delving deeper into research aligned with current recommendations, such as PCAST (The President's Council Advisors Science Technology). Throughout review, we examine both strengths weaknesses existing practices, aiming assist practitioners identifying key needs addressing concerns raised critics. By doing so, seek enhance credibility effectiveness analysis field forensic science.

Language: Английский

Citations

1