Meta-Reinforcement Learning reconciles surprise, value, and control in the anterior cingulate cortex
PLoS Computational Biology,
Год журнала:
2025,
Номер
21(4), С. e1013025 - e1013025
Опубликована: Апрель 22, 2025
The
role
of
the
dorsal
anterior
cingulate
cortex
(dACC)
in
cognition
is
a
frequently
studied
yet
highly
debated
topic
neuroscience.
Most
authors
agree
that
dACC
involved
either
cognitive
control
(e.g.,
voluntary
inhibition
automatic
responses)
or
monitoring
comparing
expectations
with
outcomes,
detecting
errors,
tracking
surprise).
A
consensus
on
which
theoretical
perspective
best
explains
contribution
to
behaviour
still
lacking,
as
two
distinct
sets
studies
report
activation
tasks
requiring
surprise
for
performance
and
without
involving
monitoring,
respectively.
This
creates
impasse,
no
single
current
account
can
reconcile
these
findings.
Here
we
propose
novel
hypothesis
function
integrates
both
perspectives
unifying,
meta-Reinforcement
Learning
framework,
optimized
by
meta-learning
based
Bayesian
surprise.
We
tested
quantitative
predictions
from
our
theory
three
different
functional
neuroimaging
experiments
at
basis
crisis.
show
successfully
captures
all
results
predicting
functions,
proposing
solution
crisis
about
within
an
integrative
framework.
In
sum,
suggest
be
framed
optimisation
control,
providing
its
roles
tracking,
monitoring.
Язык: Английский
Understanding disrupted motivation in Parkinson’s disease through a value-based decision-making lens
Trends in Neurosciences,
Год журнала:
2025,
Номер
unknown
Опубликована: Март 1, 2025
Язык: Английский
The Interplay Between Experimental Heat Pain and Noninvasive Stimulation of the Medial Prefrontal Cortex on Reinforcement Learning With Manipulated Outcome Contingencies
European Journal of Neuroscience,
Год журнала:
2025,
Номер
61(7)
Опубликована: Март 31, 2025
ABSTRACT
Pain
negatively
affects
several
cognitive
abilities,
but
knowledge
about
its
effect
on
reinforcement
learning
(RL)
is
limited.
During
RL,
instrumental
choices
can
be
influenced
by
heuristic
tendencies
to
approach
rewards
or
inhibit
actions
when
facing
potentially
aversive
events,
introducing
“Pavlovian
bias”
in
behavior.
Recent
studies
suggest
that
compromised
outcome
controllability
enhances
Pavlovian
bias,
a
phenomenon
may
linked
suboptimal
decision‐making
learned
helplessness
(LH).
Since
LH
common
chronic
pain
syndromes,
this
study
sought
establish
link
between
experimental
heat
(EHP),
disrupted
reward/loss
contingencies,
and
RL
performance
healthy
adults.
In
addition,
we
investigated
if
intermittent
theta
burst
stimulation
(iTBS)
above
the
medial
prefrontal/dorsal
anterior
cingulate
cortex
(mPFC/dACC)
alleviates
deleterious
effects
of
EHP
choice
preregistered,
2
×
between‐group,
double‐blind
(
N
=
100),
adult
participants
underwent
three
blocks
an
orthogonalized
Go/NoGo
task
with
two
interleaved
bouts
active
sham
iTBS,
either
warm
skin
combined
manipulated
response–outcome
contingency
during
task.
Although
did
not
impact
response
accuracy,
it
invigorated
for
rewards,
reflecting
enhanced
bias.
Whereas
iTBS
attenuated
tendencies,
was
counteracted
EHP,
indicating
antagonistic
iTBS‐modulated
mPFC
activity
Pavlovian–instrumental
interactions.
Surprisingly,
exerted
largely
similar
other
latent
parameters
(go‐bias,
rate,
exploration)
manner
resembled
LH.
These
findings
shed
light
role
mPFC/dACC
LH‐like
Язык: Английский
A neurometabolic mechanism involving dmPFC/dACC lactate in physical effort-based decision-making
bioRxiv (Cold Spring Harbor Laboratory),
Год журнала:
2024,
Номер
unknown
Опубликована: Май 4, 2024
Abstract
Motivation
levels
vary
across
individuals,
yet
the
underlying
mechanisms
driving
these
differences
remain
elusive.
The
dorsomedial
prefrontal
cortex/dorsal
anterior
cingulate
cortex
(dmPFC/dACC)
and
insula
(aIns)
play
crucial
roles
in
effort-based
decision-making.
Here,
we
investigate
influence
of
lactate,
a
key
metabolite
involved
energy
metabolism
signaling,
on
decisions
involving
both
physical
mental
effort,
as
well
its
effects
neural
activation.
Using
proton
magnetic
resonance
spectroscopy
functional
MRI
63
participants,
find
that
higher
lactate
dmPFC/dACC
are
associated
with
reduced
motivation
for
relationship
mediated
by
activity
within
this
region.
Additionally,
plasma
correlate,
suggesting
systemic
brain
metabolism.
Supported
path
analysis,
our
results
highlight
lactate’s
role
modulator
activity,
hinting
at
neurometabolic
mechanism
integrates
peripheral
central
metabolic
states
function
Язык: Английский
Unifying Effort
Опубликована: Май 12, 2024
At
first
glance,
the
type
of
effort
required
to
solve
a
chess
puzzle
and
run
marathon
seem
fundamentally
different.
This
paper
argues
that’s
wrong.
I
present
novel
"mental-effort
first"
account
in
which
all
is
mental
nature.
outline
psychological
mechanisms
behind
choice
execution,
emphasizing
role
cognitive
control,
process
by
efforts–mental
bodily–are
selected
enacted.
five
arguments
that
convergently
provide
strong
support
for
mental-effort
account.
Additionally,
examine
implications
this
unified
theory
philosophical
discussions
ethics,
philosophy
mind,
action
theory.
Язык: Английский
A neurometabolic mechanism involving dmPFC/dACC lactate in physical effort-based decision-making
Molecular Psychiatry,
Год журнала:
2024,
Номер
unknown
Опубликована: Авг. 30, 2024
Motivation
levels
vary
across
individuals,
yet
the
underlying
mechanisms
driving
these
differences
remain
elusive.
The
dorsomedial
prefrontal
cortex/dorsal
anterior
cingulate
cortex
(dmPFC/dACC)
and
insula
(aIns)
play
crucial
roles
in
effort-based
decision-making.
Here,
we
investigate
influence
of
lactate,
a
key
metabolite
involved
energy
metabolism
signaling,
on
decisions
involving
both
physical
mental
effort,
as
well
its
effects
neural
activation.
Using
proton
magnetic
resonance
spectroscopy
functional
MRI
63
participants,
find
that
higher
lactate
dmPFC/dACC
are
associated
with
reduced
motivation
for
relationship
mediated
by
activity
within
this
region.
Additionally,
plasma
correlate,
suggesting
systemic
brain
metabolism.
Supported
path
analysis,
our
results
highlight
lactate's
role
modulator
activity,
hinting
at
neurometabolic
mechanism
integrates
peripheral
central
metabolic
states
function
Язык: Английский