CLEVER: Clinical Large Language Model Evaluation by Expert Review (Preprint)
Published: Feb. 4, 2025
BACKGROUND
The
proliferation
of
both
general-purpose
and
healthcare-specific
Large
Language
Models
(LLMs)
has
intensified
the
challenge
effectively
evaluating
comparing
them.
Data
contamination
plagues
validity
public
benchmarks;
self-preference
distorts
LLM-as-a-judge
approaches;
there’s
a
gap
between
tasks
used
to
test
models
those
in
clinical
practice.
OBJECTIVE
In
response,
we
propose
CLEVER:
A
methodology
for
blind,
randomized,
preference-based
evaluation
by
practicing
medical
doctors
on
specific
tasks.
METHODS
We
demonstrate
GPT-4o
against
two
LLMs,
with
8B
70B
parameters,
over
three
tasks:
text
summarization,
information
extraction,
question
answering
biomedical
research.
RESULTS
Medical
prefer
Small
LLM
45%
92%
more
often
dimensions
factuality,
relevance,
conciseness.
CONCLUSIONS
show
comparable
performance
open-ended
Q&A,
suggesting
that
LLMs
can
outperform
much
larger
require
understanding
context.
CLEVER
evaluations
conducting
inter-annotator
agreement,
inter-class
correlation,
washout
period
analysis.
CLINICALTRIAL
n/a
Language: Английский
Metastatic Head and Neck Cancer: Immunotherapy and Beyond
Seminars in Radiation Oncology,
Journal Year:
2025,
Volume and Issue:
35(2), P. 259 - 270
Published: March 14, 2025
Head
and
neck
cancer
is
estimated
to
result
in
71,000
new
diagnoses
16,000
deaths
2024.
Of
these
cases,
approximately
14%
will
be
metastatic.
Recent
changes
treatment
paradigms
have
established
immunotherapy
as
a
cornerstone
of
the
metastatic
recurrent
setting.
While
has
undoubtedly
improved
outcomes
can
lead
long
term
durable
responses
select
patients,
overall
response
rates
remain
suboptimal,
with
13%-20%
patients
responding
most
studies.
This
review
aims
provide
an
overview
current
landscape
immunotherapies
head
malignancies.
Additionally,
we
aim
discuss
future
immunotherapy,
well
novel
targets
therapeutic
platforms
that
may
continue
change
paradigm
this
disease.
Language: Английский
UV1 vaccination in pembrolizumab-treated patients with recurrent or metastatic head and neck cancer: A randomized multicenter phase 2 trial
Med,
Journal Year:
2025,
Volume and Issue:
unknown, P. 100647 - 100647
Published: April 1, 2025
Language: Английский
Breaking Ground in Recurrent or Metastatic Head and Neck Squamous Cell Carcinoma: Novel Therapies Beyond PD-L1 Immunotherapy
American Society of Clinical Oncology Educational Book,
Journal Year:
2024,
Volume and Issue:
44(3)
Published: May 8, 2024
The
treatment
for
recurrent/metastatic
(R/M)
head
and
neck
squamous
cell
carcinoma
(HNSCC)
with
immune
checkpoint
inhibitors
(anti-PD1)
or
without
chemotherapy
has
led
to
an
improvement
in
survival.
Yet,
despite
this
therapeutic
advancement,
only
15%-19%
of
patients
remain
alive
at
four
years,
highlighting
the
poor
survival
unmet
need
improved
therapies
patient
population.
Some
key
evolving
novel
therapeutics
beyond
anti-PD1
R/M
HNSCC
have
included
vaccine
therapies,
bispecific
antibodies/fusion
proteins
multitargeted
kinase
inhibitors,
antibody-drug
conjugates
(ADCs).
Multiple
concurrent
investigations
anti-PD(L)1
inhibition
are
currently
underway
some
promising
early
results.
Beyond
inhibition,
immunotherapeutic
strategies
including
vaccines
ranging
from
targeting
human
papillomavirus-specific
epitopes
personalized
neoantigen
ongoing
efficacy
signals
large,
randomized
trials.
Other
weapons
antibodies,
fusion
proteins,
leverage
multiple
targets
modulation
tumor
microenvironment
harness
antitumor
immunity
protumorigenic
signaling
pathways
emerging
Finally,
as
other
solid
tumors,
ADCs
a
intervention
either
alone
combination
immunotherapy
HNSCC.
With
enthusiasm
across
HNSCC,
results
larger
trials
eagerly
awaited.
Language: Английский
UV1 cancer vaccine in pembrolizumab-treated patients with recurrent or metastatic PD-L1 positive head and neck squamous cell carcinoma: results from the randomized phase 2 FOCUS trial
medRxiv (Cold Spring Harbor Laboratory),
Journal Year:
2024,
Volume and Issue:
unknown
Published: Oct. 24, 2024
Abstract
Background
Human
telomerase
reverse
transcriptase
(hTERT)
is
highly
expressed
(75-100%)
in
head
and
neck
squamous
cell
carcinoma
(HNSCC).
The
FOCUS
study
examines
the
role
of
hTERT-directed
vaccine
UV1
combination
with
pembrolizumab
patients
recurrent
or
metastatic
(R/M)
HNSCC.
Methods
trial,
a
two-armed,
open-label,
non-comparative,
randomized,
multicenter
phase
2
study,
was
designed
to
assess
efficacy
feasibility
as
an
add-on
first-line
treatment
R/M
PD-L1
positive
A
progression-free
survival
rate
at
6
months
(PFSR@6)
40%
deemed
promising
for
further
development
3
setting.
trial
conducted
10
centers
Germany.
Results
From
August
2021
July
2023,
25
were
enrolled
calibration
arm
50
B.
Median
age
65
years
18%
had
ECOG
performance
score
2.
PFSR@6
30%
arm.
No
specific
safety
signals
observed
apart
from
reversible
allergic
reaction
that
appeared
one
patient.
At
median
follow-up
almost
year
(11.3
months),
overall
13.1
12.6
Clinical
identification
number
NCT05075122
.
Conclusions
addition
safe
but
did
not
show
signal
this
population.
Language: Английский
Head and neck cancer care: advanced molecular and genetic diagnostics.
PubMed,
Journal Year:
2024,
Volume and Issue:
23, P. 1229 - 1231
Published: Jan. 1, 2024
Language: Английский
Preclinical in vitro models of HNSCC and their role in drug discovery - an emphasis on the cancer microenvironment and microbiota
Expert Opinion on Drug Discovery,
Journal Year:
2024,
Volume and Issue:
unknown, P. 1 - 21
Published: Dec. 15, 2024
Head
and
neck
squamous
cell
carcinoma
(HNSCC)
is
the
seventh
most
common
cancer
worldwide.
Treatment
options
patient
outcomes
have
not
improved
significantly
over
past
decades,
increasing
need
for
better
preclinical
models.
Holistic
approaches
that
include
an
intact
functional
immune
compartment
along
with
patient's
individual
tumor
microbiome
will
help
improve
predictive
value
of
novel
drug
efficacy.
Language: Английский