Test
smells
attempt
to
capture
design
issues
in
test
code
that
reduce
their
maintainability.
Previous
work
found
such
be
highly
common
automatically
generated
test-cases,
but
based
this
result
on
specific
static
detection
rules;
although
these
are
the
original
definition
of
"test
smells",
a
recent
empirical
study
showed
developers
perceive
as
overly
strict
and
non-representative
maintainability
quality
suites.
This
leads
us
investigate
how
effective
smell
tools
In
paper,
we
build
dataset
2,340
cases
by
EVOSUITE
for
100
Java
classes.
We
performed
multi-stage,
cross-validated
manual
analysis
identify
six
types
label
instances.
benchmark
performance
two
tools:
one
widely
used
prior
work,
recently
introduced
with
express
goal
match
developer
perceptions
smells.
Our
results
show
strategies
poorly
characterized
suites;
older
tool's
strategies,
especially,
misclassified
over
70%
smells,
both
missing
real
instances
(false
negatives)
marking
many
smell-free
tests
smelly
positives).
patterns
can
improve
tools,
refine
update
certain
highlight
yet
uncharacterized
issues.
findings
suggest
need
(i)
more
appropriate
metrics
development
practice;
(ii)
accurate
evaluated
primarily
industrial
contexts.
EvoSuite
is
a
search-based
tool
that
automatically
generates
unit
tests
for
Java
code.
This
paper
summarizes
the
results
and
experiences
of
EvoSuite's
participation
at
fourth
testing
competition
SBST
2016,
where
Evo-Suite
achieved
highest
overall
score.
Application
Programming
Interfaces
(APIs)
typically
come
with
(implicit)
usage
constraints.
The
violations
of
these
constraints
(API
misuses)
can
lead
to
software
crashes.
Even
though
there
are
several
tools
that
detect
API
misuses,
most
them
suffer
from
a
very
high
rate
false
positives.
We
introduce
Catcher,
novel
misuse
detection
approach
combines
static
exception
propagation
analysis
automatic
search-based
test
case
generation
effectively
and
efficiently
pinpoint
crash-prone
misuses
in
client
applications.
validate
Catcher
against
21
Java
applications,
targeting
the
platform's
API.
Our
results
indicate
is
able
generate
cases
uncover
243
(unique)
result
empirical
evaluation
shows
large
number
(77
cases)
would
remain
undetected
by
traditional
coverage-based
generator
EvoSuite.
Additionally,
on
average,
eight
times
faster
than
EvoSuite
generating
for
identified
misuses.
Finally,
we
find
majority
exceptions
triggered
unexpected
developers,
i.e.,
not
only
unhandled
source
code
but
also
listed
documentation
Test
smells
attempt
to
capture
design
issues
in
test
code
that
reduce
their
maintainability.
Previous
work
found
such
be
highly
common
automatically
generated
test-cases,
but
based
this
result
on
specific
static
detection
rules;
although
these
are
the
original
definition
of
"test
smells",
a
recent
empirical
study
showed
developers
perceive
as
overly
strict
and
non-representative
maintainability
quality
suites.
This
leads
us
investigate
how
effective
smell
tools
In
paper,
we
build
dataset
2,340
cases
by
EVOSUITE
for
100
Java
classes.
We
performed
multi-stage,
cross-validated
manual
analysis
identify
six
types
label
instances.
benchmark
performance
two
tools:
one
widely
used
prior
work,
recently
introduced
with
express
goal
match
developer
perceptions
smells.
Our
results
show
strategies
poorly
characterized
suites;
older
tool's
strategies,
especially,
misclassified
over
70%
smells,
both
missing
real
instances
(false
negatives)
marking
many
smell-free
tests
smelly
positives).
patterns
can
improve
tools,
refine
update
certain
highlight
yet
uncharacterized
issues.
findings
suggest
need
(i)
more
appropriate
metrics
development
practice;
(ii)
accurate
evaluated
primarily
industrial
contexts.
