Off-the-Shelf Large Language Models for Causality Assessment of Individual Case Safety Reports: A Proof-of-Concept with COVID-19 Vaccines
Drug Safety,
Journal Year:
2025,
Volume and Issue:
unknown
Published: March 12, 2025
This
study
evaluated
the
feasibility
of
ChatGPT
and
Gemini,
two
off-the-shelf
large
language
models
(LLMs),
to
automate
causality
assessments,
focusing
on
Adverse
Events
Following
Immunizations
(AEFIs)
myocarditis
pericarditis
related
COVID-19
vaccines.
We
assessed
150
COVID-19-related
cases
reported
Vaccine
Event
Reporting
System
(VAERS)
in
United
States
America
(USA).
Both
LLMs
human
experts
conducted
World
Health
Organization
(WHO)
algorithm
for
vaccine
inter-rater
agreement
was
measured
using
percentage
agreement.
Adherence
WHO
by
comparing
LLM
responses
expected
sequence
algorithm.
Statistical
analyses,
including
descriptive
statistics
Random
Forest
modeling,
explored
case
complexity
(e.g.,
string
length
measurements)
factors
affecting
performance
adherence.
showed
higher
adherence
(34%)
compared
Gemini
(7%)
had
moderate
(71%)
with
experts,
whereas
fair
(53%).
often
failed
recognize
listed
AEFIs,
incorrectly
identifying
6.7%
13.3%
respectively.
inconsistencies
8.0%
46.7%.
For
ChatGPT,
associated
lower
prompt
sections.
The
random
forest
analysis
achieved
an
accuracy
55%
(95%
confidence
interval:
35.7–73.5)
predicting
ChatGPT.
Notable
limitations
have
been
identified
their
use
aiding
assessments
safety.
performed
better,
experts.
In
investigated
scenario,
both
are
better
suited
as
complementary
tools
expertise.
Language: Английский
Cutting-edge AI tools revolutionizing scientific research in life sciences
Katarzyna Lorenc-Kukula
No information about this author
BioTechnologia,
Journal Year:
2025,
Volume and Issue:
unknown
Published: March 7, 2025
Artificial
intelligence
(AI)
is
becoming
a
transformative
force
in
the
life
sciences,
pushing
boundaries
of
possibility.
Imagine
AI
automating
time-consuming
tasks,
uncovering
hidden
patterns
vast
datasets,
designing
proteins
minutes
instead
years,
and
even
predicting
disease
outbreaks
before
they
occur.
This
review
explores
latest
tools
revolutionizing
scientific
fields,
including
research
data
analysis,
healthcare,
supporting
writing.
Beyond
processing,
reshaping
how
scientists
draft
share
their
findings,
enhancing
processes
ranging
from
literature
reviews
to
citation
management.
However,
with
great
power
comes
responsibility.
Are
we
prepared
for
this
leap?
delves
into
forefront
where
innovation
meets
Language: Английский