Exploring the Noise Resilience of Successor Features and Predecessor Features Algorithms in One and Two-Dimensional Environments

Research Square (Research Square), Год журнала: 2024, Номер unknown

Опубликована: Фев. 12, 2024

Abstract Based on the predictive map theory of spatial learning in animals, this study delves into dynamics Successor Feature (SF) and Predecessor (PF) algorithms within noisy environments. Utilizing Q-learning Q($\lambda$) as benchmarks for comparative analysis, our investigation yielded unexpected outcomes. Contrary to prevailing expectations previous literature where PF demonstrated superior performance, findings reveal that environments, did not surpass SF. In a one-dimensional grid world, SF exhibited adaptability, maintaining robust performance across varying noise levels. This trend diminishing with increasing was consistent all examined algorithms, indicating linear degradation pattern.The scenario shifted two-dimensional impact algorithm non-linear relationship, influenced by $\lambda$ parameter eligibility trace. complexity suggests interaction between efficacy is tied environmental dimensionality specific algorithmic parameters.Furthermore, research contributes bridging discourse computational neuroscience reinforcement (RL), exploring neurobiological parallels navigation. Despite unforeseen trends, enrich comprehension strengths weaknesses inherent RL algorithms. knowledge pivotal advancing applications robotics, gaming AI, autonomous vehicle navigation, underscoring imperative continued exploration how process learn from inputs.

Язык: Английский

---

The successor representation subserves hierarchical abstraction for goal-directed behavior

PLoS Computational Biology, Год журнала: 2024, Номер 20(2), С. e1011312 - e1011312

Опубликована: Фев. 20, 2024

Humans have the ability to craft abstract, temporally extended and hierarchically organized plans. For instance, when considering how make spaghetti for dinner, we typically concern ourselves with useful “subgoals” in task, such as cutting onions, boiling pasta, cooking a sauce, rather than particulars many cuts onion, or exactly which muscles contract. A core question is decomposition of more abstract task into logical subtasks happens first place. Previous research has shown that humans are sensitive form higher-order statistical learning named “community structure”. Community structure common feature tasks characterized by ordering subtasks. This can be captured model where learn predictions upcoming events multiple steps future, discounting further away time. One “successor representation”, been argued hierarchical abstraction. As yet, no study convincingly this abstraction put use goal-directed behavior. Here, investigate whether participants utilize learned community informed action plans Participants were asked search paintings virtual museum, grouped together “wings” representing museum. We find participants’ choices accord museum their response times best predicted successor representation. The degree reflect correlates several measures performance, including These results suggest representation subserves abstractions relevant

Язык: Английский

---

Neural representations of predicted events: Evidence from time-resolved EEG decoding

Опубликована: Апрель 29, 2024

Through statistical learning, humans are able to extract temporal regularities, using the past predict future. Evidence suggests that learning relational structures makes it possible anticipate imminent future; yet, neural dynamics of predicting future and its time-course remain elusive. To examine whether representations denoted in a temporally discounted fashion, we used high-temporal-resolution electroencephalography (EEG). Observers were exposed fixed sequence events at four unique spatial positions within display. Using multivariate pattern analyses trained on independent estimators, decode position dots full sequences, randomly intermixed partial sequences wherein only single dot was presented. Crucially, these subsequent could be reliably decoded their expected moment time. These findings highlight dynamic weight changes assumed priority map mark first implementation EEG predicted, yet critically omitted events.Utilizing EEG, visualized by decoding expected, omitted,

Язык: Английский

---

Automated construction of cognitive maps with visual predictive coding

Nature Machine Intelligence, Год журнала: 2024, Номер 6(7), С. 820 - 833

Опубликована: Июль 18, 2024

Abstract Humans construct internal cognitive maps of their environment directly from sensory inputs without access to a system explicit coordinates or distance measurements. Although machine learning algorithms like simultaneous localization and mapping utilize specialized inference procedures identify visual features spatial odometry data, the general nature in brain suggests unified algorithmic strategy that can generalize auditory, tactile linguistic inputs. Here we demonstrate predictive coding provides natural versatile neural network algorithm for constructing using data. We introduce framework which an agent navigates virtual while engaging self-attention-equipped convolutional network. While next-image prediction task, automatically constructs representation quantitatively reflects distances. The map enables pinpoint its location relative landmarks only information.The generates vectorized encoding supports vector navigation, where individual latent space units delineate localized, overlapping neighbourhoods environment. Broadly, our work introduces as naturally extend sensorimotor

Язык: Английский

---

Barcode activity in a recurrent network model of the hippocampus enables efficient memory binding

bioRxiv (Cold Spring Harbor Laboratory), Год журнала: 2024, Номер unknown

Опубликована: Сен. 13, 2024

Abstract Forming an episodic memory requires binding together disparate elements that co-occur in a single experience. One model of this process is neurons representing different components bind to “index” — subset unique memory. Evidence for has recently been found chickadees, which use hippocampal store and recall locations cached food. Chickadee hippocampus produces sparse, high-dimensional patterns (“barcodes”) uniquely specify each caching event. Unexpectedly, the same participate barcodes also exhibit conventional place tuning. It unknown how barcode activity generated, what role it plays formation retrieval. unclear index (e.g. barcodes) could function neural population represents content place). Here, we design biologically plausible generates uses them experiential content. Our from inputs through chaotic dynamics recurrent network Hebbian plasticity as attractor states. The matches experimental observations indices (barcodes) signals (place tuning) are randomly intermixed neurons. We demonstrate reduce interference between correlated experiences. show tuning complementary barcodes, enabling flexible, contextually-appropriate Finally, our compatible with previous models generating predictive map. Distinct indexing functions achieved via adjustment global gain. results suggest may resolve fundamental tensions specificity (pattern separation) flexible completion) general systems.

Язык: Английский

---