Emergence of belief-like representations through reinforcement learning
PLoS Computational Biology,
Journal Year:
2023,
Volume and Issue:
19(9), P. e1011067 - e1011067
Published: Sept. 11, 2023
To
behave
adaptively,
animals
must
learn
to
predict
future
reward,
or
value.
do
this,
are
thought
reward
predictions
using
reinforcement
learning.
However,
in
contrast
classical
models,
estimate
value
only
incomplete
state
information.
Previous
work
suggests
that
partially
observable
tasks
by
first
forming
"beliefs"-optimal
Bayesian
estimates
of
the
hidden
states
task.
Although
this
is
one
way
solve
problem
partial
observability,
it
not
way,
nor
most
computationally
scalable
solution
complex,
real-world
environments.
Here
we
show
a
recurrent
neural
network
(RNN)
can
directly
from
observations,
generating
prediction
errors
resemble
those
observed
experimentally,
without
any
explicit
objective
estimating
beliefs.
We
integrate
statistical,
functional,
and
dynamical
systems
perspectives
on
beliefs
RNN's
learned
representation
encodes
belief
information,
but
when
capacity
sufficiently
large.
These
results
illustrate
how
explicitly
beliefs,
yielding
useful
for
with
limited
capacity.
Language: Английский
Fragmentation and Multithreading of Experience in the Default-Mode Network
Fahd Yazin,
No information about this author
Gargi Majumdar,
No information about this author
Neil Bramley
No information about this author
et al.
bioRxiv (Cold Spring Harbor Laboratory),
Journal Year:
2024,
Volume and Issue:
unknown
Published: Oct. 25, 2024
Abstract
Reliance
on
internal
predictive
models
of
the
world
is
central
to
many
theories
human
cognition.
Yet
it
unknown
whether
humans
acquired
multiple
separate
models,
each
evolved
for
a
specific
domain,
or
maintain
globally
unified
representation.
Using
fMRI,
we
show
that
during
naturalistic
experiences
(during
movie
watching
narrative
listening),
adult
participants
selectively
engage
three
topographically
distinct
midline
prefrontal
cortical
regions,
different
forms
predictions.
Regions
responded
abstract
spatial,
referential
(social),
and
temporal
domains
model
updates
implying
representations
each.
Prediction-error-driven
neural
transitions
in
these
indicative
updates,
preceded
subjective
belief
changes
domain-specific
manner.
We
find
parallel
top-down
predictions
are
integrated
with
sensory
streams
Precuneus,
shaping
participants’
ongoing
experience.
Results
generalized
across
modalities
content,
suggesting
recruit
abstract,
modular
both
vision
language.
Our
results
highlight
key
feature
modeling:
fragmenting
information
into
before
global
integration.
Language: Английский
Dorsolateral prefrontal cortex drives strategic aborting by optimizing long-run policy extraction
bioRxiv (Cold Spring Harbor Laboratory),
Journal Year:
2024,
Volume and Issue:
unknown
Published: Nov. 28, 2024
Abstract
Real
world
choices
often
involve
balancing
decisions
that
are
optimized
for
the
short-vs.
long-term.
Here,
we
reason
apparently
sub-optimal
single
trial
in
macaques
may
fact
reflect
long-term,
strategic
planning.
We
demonstrate
freely
navigating
VR
sequentially
presented
targets
will
strategically
abort
offers,
forgoing
more
immediate
rewards
on
individual
trials
to
maximize
session-long
returns.
This
behavior
is
highly
specific
individual,
demonstrating
about
their
own
long-run
performance.
Reinforcement-learning
(RL)
models
suggest
this
algorithmically
supported
by
modular
actor-critic
networks
with
a
policy
module
not
only
optimizing
long-term
value
functions,
but
also
informed
of
state-action
values
allowing
rapid
optimization.
The
artificial
suggests
changes
matched
offer
ought
be
evident
as
soon
offers
made,
even
if
aborting
occurs
much
later.
confirm
prediction
units
and
population
dynamics
macaque
dorsolateral
prefrontal
cortex
(dlPFC),
parietal
area
7a
or
dorsomedial
superior
temporal
(MSTd),
upcoming
reward-maximizing
upon
presentation.
These
results
cast
dlPFC
specialized
module,
stand
contrast
recent
work
distributed
recurrent
nature
belief-networks.
Language: Английский
Context-invariant beliefs are supported by dynamic reconfiguration of single unit functional connectivity in prefrontal cortex
bioRxiv (Cold Spring Harbor Laboratory),
Journal Year:
2023,
Volume and Issue:
unknown
Published: July 31, 2023
Abstract
Natural
behaviors
occur
in
closed
action-perception
loops
and
are
supported
by
dynamic
flexible
beliefs
abstracted
away
from
our
immediate
sensory
milieu.
How
this
real-world
flexibility
is
instantiated
neural
circuits
remains
unknown.
Here
we
have
macaques
navigate
a
virtual
environment
primarily
leveraging
(optic
flow)
signals,
or
more
heavily
relying
on
acquired
internal
models.
We
record
single-unit
spiking
activity
simultaneously
the
dorsomedial
superior
temporal
area
(MSTd),
parietal
7a,
dorso-lateral
prefrontal
cortex
(dlPFC).
Results
show
that
while
animals
were
able
to
maintain
adaptive
task-relevant
regardless
of
context,
fine-grain
statistical
dependencies
between
neurons,
particularly
7a
dlPFC,
dynamically
remapped
with
changing
computational
demands.
In
but
not
destroying
these
abolished
area’s
ability
for
cross-context
decoding.
Lastly,
correlation
analyses
suggested
unit-to-unit
couplings
less
they
did
so
MSTd,
population
codes
behavior
impacted
loss
evidence.
conclude
functional
connectivity
neurons
maintains
stable
code
context-invariant
during
naturalistic
loops.
Language: Английский