We
have
outlined
several
problems
with
the
state
of
error
rate
studies
on
firearm
and
toolmark
examination.
Fundamentally,
we
do
not
know
what
is
for
these
types
comparisons.
This
a
failure
scientific
study
toolmarks,
rather
than
examiners
themselves,
but
until
this
corrected
multiple
that
meet
criteria
described
in
Section
3,
cannot
support
use
evidence
criminal
proceedings.
Journal of Forensic Sciences,
Journal Year:
2022,
Volume and Issue:
68(1), P. 86 - 100
Published: Oct. 1, 2022
Abstract
This
black
box
study
assessed
the
performance
of
forensic
firearms
examiners
in
United
States.
It
involved
three
different
types
and
173
volunteers
who
performed
a
total
8640
comparisons
both
bullets
cartridge
cases.
The
overall
false‐positive
error
rate
was
estimated
as
0.656%
0.933%
for
cases,
respectively,
while
false
negatives
2.87%
1.87%
respectively.
majority
errors
were
made
by
limited
number
examiners.
Because
chi‐square
tests
independence
strongly
suggest
that
probabilities
are
not
same
each
examiner,
these
maximum‐likelihood
estimates
based
on
beta‐binomial
probability
model
do
depend
an
assumption
equal
examiner‐specific
rates.
Corresponding
95%
confidence
intervals
(0.305%,
1.42%)
(0.548%,
1.57%)
positives
(1.89%,
4.26%)
(1.16%,
2.99%)
results
this
consistent
with
prior
studies,
despite
its
comprehensive
design
challenging
specimens.
Journal of Forensic Sciences,
Journal Year:
2025,
Volume and Issue:
unknown
Published: Feb. 14, 2025
Abstract
Concurrent
with
studies
on
the
accuracy,
repeatability,
and
reproducibility
of
decisions
based
comparisons
fired
bullet
cartridge
cases,
we
also
collected
opinions
participating
examiners
as
to
characteristics
specimens
provided
difficulty
making
comparisons.
Examiners
rated
ease
which
they
determined
every
conclusion
(easy,
average,
hard)
estimated
qualitatively
amount
visual
information
available
them
in
determining
a
(limited,
some,
extensive).
Comparisons
deemed
hard
were
perceived
generally
have
somewhat
fewer
markings
conducive
for
assessment,
while
where
limited
produced
larger
number
inconclusive
determinations.
Perceived
increased
wider
separation
firing
order
(within
or
between
three
defined
segments
700–850
total
firings).
The
repeatability
these
qualitative
assessments
exceeded
60%
their
average
was
~50%.
Examination
times
did
not
vary
significantly
when
rendering
identification,
elimination,
inconclusive,
although
identifications
appear
taken
slightly
longer
than
those
cases.
Hard
comparisons,
limited,
treated
substantially
differently
from
any
other
types
comparison.
No
correlation
found
attempted.
These
results
tend
contradict
assertions
by
critics
that
are
tempted
declare
save
time
avoid
an
elimination
identification
conclusion,
non‐representative
casework,
affected
degree
examiner
participation.
Journal of Forensic Sciences,
Journal Year:
2023,
Volume and Issue:
68(5), P. 1721 - 1740
Published: July 2, 2023
Abstract
In
a
comprehensive
study
to
assess
various
aspects
of
the
performance
qualified
forensic
firearms
examiners,
volunteer
examiners
compared
both
bullets
and
cartridge
cases
fired
from
three
different
types
firearms.
They
rendered
opinions
on
each
comparison
according
Association
Firearm
&
Tool
Mark
Examiners
(AFTE)
Range
Conclusions,
as
Identification,
Inconclusive
(A,
B,
or
C),
Elimination,
Unsuitable.
this
part
study,
sets
used
previously
characterize
overall
accuracy
were
blindly
resubmitted
repeatability
(105
examiners;
5700
comparisons
cases)
reproducibility
(191
bullets,
193
cases;
5790
comparisons)
examinations.
Data
gathered
using
prevailing
AFTE
also
recategorized
into
two
hypothetical
scoring
systems.
Consistently
positive
differences
between
observed
agreement
expected
indicate
that
exceed
chance
agreement.
When
averaged
over
cases,
decisions
(involving
all
five
levels
Range)
was
78.3%
for
known
matches
64.5%
nonmatches.
Similarly
67.3%%
36.5%
For
reproducibility,
many
disagreements
definitive
inconclusive
category.
Examiner
are
reliable
trustworthy
in
sense
identifications
unlikely
when
comparing
non‐matching
items,
eliminations
they
matching
items.
Law Probability and Risk,
Journal Year:
2020,
Volume and Issue:
19(3-4), P. 317 - 364
Published: Dec. 1, 2020
Abstract
In
the
past
decade,
and
in
response
to
recommendations
set
forth
by
National
Research
Council
Committee
on
Identifying
Needs
of
Forensic
Sciences
Community
(2009),
scientists
have
conducted
several
black-box
studies
that
attempt
estimate
error
rates
firearm
examiners.
Most
these
resulted
vanishingly
small
rates,
at
least
one
them
(D.
P.
Baldwin,
S.
J.
Bajic,
M.
Morris,
D.
Zamzow.
A
Study
False-Positive
False-Negative
Error
Rates
Cartridge
Case
Comparisons.
Technical
report,
Ames
Lab
IA,
Performing,
Fort
Belvoir,
VA,
April
2014.)
was
cited
President’s
Advisors
Science
Technology
(PCAST)
during
Obama
administration,
as
an
example
a
well-designed
experiment.
What
has
received
little
attention,
however,
is
actual
calculation
particular,
effect
inconclusive
findings
those
estimates.
The
treatment
inconclusives
assessment
errors
far-reaching
implications
legal
system.
Here,
we
revisit
area
firearms
examination,
investigating
their
results.
It
clear
there
are
stark
differences
rate
results
regions
with
different
norms
for
training
reporting
conclusions.
More
surprisingly,
decisions
materials
from
sources
notably
higher
than
same-source
some
regions.
To
mitigate
effects
this
difference
propose
unifying
approach
directly
applicable
forensic
laboratories
settings.
Forensic Science International Synergy,
Journal Year:
2022,
Volume and Issue:
4, P. 100228 - 100228
Published: Jan. 1, 2022
A
sample
(n
=
79)
of
practicing
firearm
and
toolmark
examiners
was
queried
about
casework
as
well
their
views
the
potential
role
that
statistics
might
play
in
future
examinations
expert
witness
testimony.
Principal
findings
include:
The
modal
response
for
time
spent
conducting
bullet
is
2–4
hours,
cartridge
casings
1–2
hours.
average
participant
(median)
makes
an
identification
65%
casework,
elimination
12%
reports
examination
inconclusive
20%
calls.
vast
majority
work
at
laboratories
permit
eliminations
when
class
characteristics
agree.
reported
industry-wide
false
positive
error
rate
1%,
though
very
few
participants
could
name
a
study
or
give
citation
estimate.
Qualitative
responses
were
mixed.
Journal of Forensic Sciences,
Journal Year:
2024,
Volume and Issue:
69(6), P. 2028 - 2040
Published: Aug. 22, 2024
Abstract
Traditionally,
firearm
and
toolmark
examiners
manually
evaluate
the
similarity
of
features
on
two
bullets
using
comparison
microscopy.
Advances
in
microscopy
have
made
it
possible
to
collect
3D
topographic
data,
several
automated
algorithms
been
introduced
for
bullet
striae
these
data.
In
this
study,
open‐source
approaches
cross‐correlation,
congruent
matching
profile
segments,
consecutive
striations,
a
random
forest
model
were
evaluated.
A
statistical
characterization
was
performed
four
datasets
consecutively
manufactured
firearms
provide
challenging
scenario.
Each
approach
applied
all
samples
pairwise
fashion,
classification
performance
compared.
Based
findings,
Bayesian
network
empirically
learned
constructed
leverage
strengths
each
individual
approach,
relationship
between
results,
combine
them
into
posterior
probability
given
comparison.
The
evaluated
similarly
approaches,
results
developed
classified
99.6%
correctly,
resultant
distributions
significantly
separated
more
so
than
when
used
isolation.
Multimedia Tools and Applications,
Journal Year:
2022,
Volume and Issue:
82(13), P. 20655 - 20681
Published: Oct. 21, 2022
Abstract
A
crime
scene
can
provide
valuable
evidence
critical
to
explain
reason
and
modality
of
the
occurred
crime,
it
also
lead
arrest
criminals.
The
type
collected
by
investigators
or
law
enforcement
may
accordingly
effective
involved
cases.
Bullets
cartridge
cases
examination
is
paramount
importance
in
forensic
science
because
they
contain
traces
microscopic
striations,
impressions
markings,
which
are
unique
reproducible
as
“ballistic
fingerprints”.
analysis
bullets
a
complicated
challenging
process,
typically
based
on
optical
comparison,
leading
identification
employed
firearm.
New
methods
have
recently
been
proposed
for
more
accurate
comparisons,
rely
three-dimensionally
reconstructed
data.
This
paper
aims
at
further
advancing
recent
introducing
novel
immersive
technique
ballistics
comparison
means
Virtual
Reality.
Users
examine
shapes
through
intuitive
natural
gestures,
from
any
vantage
viewpoint
(including
internal
iper-magnified
views),
while
having
their
disposal
sets
visual
aids
could
not
be
easily
implemented
desktop-based
applications.
user
study
was
conducted
assess
viability
performance
our
solution,
fourteen
individuals
acquainted
with
standard
procedures
used
agencies.
Results
clearly
indicated
that
approach
faster
adaptation
users
UI/UX
explainable
results.