pymdp: A Python library for active inference in discrete state spaces

The Journal of Open Source Software, Journal Year: 2022, Volume and Issue: 7(73), P. 4098 - 4098

Published: May 4, 2022

Active inference is an account of cognition and behavior in complex systems which brings together action, perception, learning under the theoretical mantle Bayesian inference. has seen growing applications academic research, especially fields that seek to model human or animal behavior. While recent years, some code arising from active literature been written open source languages like Python Julia, to-date, most popular software for simulating agents DEM toolbox SPM, a MATLAB library originally developed statistical analysis modelling neuroimaging data. Increasing interest inference, manifested both terms sheer number as well diversifying across scientific disciplines, thus created need generic, widely-available, user-friendly open-source computing Python. The package we present here, pymdp (see https://github.com/infer-actively/pymdp), represents significant step this direction: namely, provide first with partially-observable Markov Decision Processes POMDPs. We review package's structure explain its advantages modular design customizability, while providing in-text blocks along way demonstrate how it can be used build run processes ease. increase accessibility exposure framework researchers, engineers, developers diverse disciplinary backgrounds. In spirit software, also hope spurs new innovation, development, collaboration community.

Language: Английский

---

World models and predictive coding for cognitive and developmental robotics: frontiers and challenges

Advanced Robotics, Journal Year: 2023, Volume and Issue: 37(13), P. 780 - 806

Published: June 26, 2023

Creating autonomous robots that can actively explore the environment, acquire knowledge and learn skills continuously is ultimate achievement envisioned in cognitive developmental robotics. Importantly, if aim to create develop through interactions with their learning processes should be based on physical social world manner of human development. Based this context, paper, we focus two concepts models predictive coding. Recently, have attracted renewed attention as a topic considerable interest artificial intelligence. Cognitive systems better predict future sensory observations optimize policies, i.e. controllers. Alternatively, neuroscience, coding proposes brain predicts its inputs adapts model own dynamics control behavior environment. Both ideas may considered underpinning development humans capable continual or lifelong learning. Although many studies been conducted robotics neurorobotics, relationship between model-based approaches AI has rarely discussed. Therefore, clarify definitions, relationships, status current research these topics, well missing pieces conjunction crucially related such free-energy principle active inference context Furthermore, outline frontiers challenges involved toward further integration robotics, creation real capabilities future.

Language: Английский

---

Introducing ActiveInference.jl: A Julia Library for Simulation and Parameter Estimation with Active Inference Models

Entropy, Journal Year: 2025, Volume and Issue: 27(1), P. 62 - 62

Published: Jan. 12, 2025

We introduce a new software package for the Julia programming language, library ActiveInference.jl. To make active inference agents with Partially Observable Markov Decision Process (POMDP) generative models available to growing research community using Julia, we re-implemented pymdp Python. ActiveInference.jl is compatible cutting-edge libraries designed cognitive and behavioural modelling, as it used in computational psychiatry, science neuroscience. This means that POMDP can now be easily fit empirically observed behaviour sampling, well variational methods. In this article, show how makes building straightforward, enables researchers use them simulation, fitting data or performing model comparison.

Language: Английский

---

Epistemic Communities under Active Inference

Entropy, Journal Year: 2022, Volume and Issue: 24(4), P. 476 - 476

Published: March 29, 2022

The spread of ideas is a fundamental concern today’s news ecology. Understanding the dynamics information and its co-option by interested parties critical importance. Research on this topic has shown that individuals tend to cluster in echo-chambers are driven confirmation bias. In paper, we leverage active inference framework provide an silico model bias effect echo-chamber formation. We build based inference, where agents sample order justify their own view reality, which eventually leads them have high degree certainty about beliefs. show that, once reached certain level beliefs, it becomes very difficult get change views. This system self-confirming beliefs upheld reinforced evolving relationship between agent’s observations, over time will continue evidence for ingrained world. epistemic communities consolidated these shared turn, produce perceptions reality reinforce those account community formation mechanism. postulate value they obtain from sampling or observing behaviours other agents. Inspired digital social networks like Twitter, generative generate observable claims posts (e.g., ‘tweets’) while reading socially lend support one two mutually exclusive abstract topics. Agents can choose agent pay attention at each timestep, crucially who attend what read influences also assess local network’s perspective, influencing kinds expect see making. was built simulated using freely available Python package pymdp. proposed reproduce networks, gives us insight into cognitive processes lead phenomenon.

Language: Английский

---

Active Inference and Behavior Trees for Reactive Action Planning and Execution in Robotics

IEEE Transactions on Robotics, Journal Year: 2023, Volume and Issue: 39(2), P. 1050 - 1069

Published: Jan. 2, 2023

In this article, we propose a hybrid combination of active inference and behavior trees (BTs) for reactive action planning execution in dynamic environments, showing how robotic tasks can be formulated as free-energy minimization problem. The proposed approach allows handling partially observable initial states improves the robustness classical BTs against unexpected contingencies while at same time reducing number nodes tree. work, specify nominal offline, through BTs. However, contrast to previous approaches, introduce new type leaf node desired state achieved rather than an execute. decision which execute reach is performed online inference. This results continual hierarchical deliberation. By doing so, agent follow predefined offline plan still keeping ability locally adapt take autonomous decisions runtime, respecting safety constraints. We provide proof convergence analysis, validate our method two different mobile manipulators performing similar tasks, both simulated real retail environment. showed improved runtime adaptability with fraction hand-coded compared

Language: Английский

---

Equilibrium in the Computing Continuum through Active Inference

Future Generation Computer Systems, Journal Year: 2024, Volume and Issue: 160, P. 92 - 108

Published: May 30, 2024

Computing Continuum (CC) systems are challenged to ensure the intricate requirements of each computational tier. Given system's scale, Service Level Objectives (SLOs), which expressed as these requirements, must be disaggregated into smaller parts that can decentralized. We present our framework for collaborative edge intelligence, enabling individual devices (1) develop a causal understanding how enforce their SLOs and (2) transfer knowledge speed up onboarding heterogeneous devices. Through collaboration, they (3) increase scope SLO fulfillment. implemented evaluated use case in CC system is responsible ensuring Quality (QoS) Experience (QoE) during video streaming. Our results showed required only ten training rounds four SLOs; furthermore, underlying structures were also rationally explainable. The addition new types done posteriori; allowed them reuse existing models, even though device type had been unknown. Finally, rebalancing load within cluster recover compliance after network failure from 22% 89%.

Language: Английский

---

Intrinsic Motivation as Constrained Entropy Maximization

Entropy, Journal Year: 2025, Volume and Issue: 27(4), P. 372 - 372

Published: March 31, 2025

“Intrinsic motivation” refers to the capacity for intelligent systems be motivated endogenously, i.e., by features of agential architecture itself rather than learned associations between action and reward. This paper views active inference, empowerment, other formal accounts intrinsic motivation as variations on theme constrained maximum entropy providing a general perspective complementary existing frameworks. The connection free energy empowerment noted in previous literature is further explored, it argued that maximum-occupancy approach practice incorporates an implicit model-evidence constraint.

Language: Английский

---

A New Look at Young Children’s Referential Informativeness

Perspectives on Psychological Science, Journal Year: 2022, Volume and Issue: 18(3), P. 624 - 648

Published: Sept. 28, 2022

In this article, I review experimental evidence for the dependence of 2- to 5-year-olds’ linguistic referential informativeness on cues common ground (CG) and propose a process model. Cues CG provide CG, that is, shared knowledge, beliefs, attitudes interlocutors. The presence (e.g., unimpeded listener line regard or prior mention) is shown be associated with less informative reference pronouns). contrast, absence impeded new more nouns). Informativeness sensitive before nonlinguistic (i.e., 2.0 vs. 2.5 years old, respectively). Reference cast as active inference, formulation Bayesian belief-guided control in biological systems. Child speakers are hierarchical generative models that, characteristically, expect sensory evolved, belief interlocutor mental states aligned exists). Referential emerges an embodied tool gather belief. Bottom-up elicited by action drive updates beliefs about CG. turn, guide efficient control.

Language: Английский

---

An active inference approach to interpersonal differences in depression

New Ideas in Psychology, Journal Year: 2024, Volume and Issue: 74, P. 101092 - 101092

Published: April 26, 2024

Depression is characterized by different distortions in interpersonal experience and behavior, ranging from social withdrawal to overt hostility. However, clinical psychological research has largely neglected the need for an integrative framework operationalize these phenomena their dynamic change more accurately depression. In this article, we draw on active inference theory, a comprehensive theory of perception, action, learning, provide formal model explaining how variations patients' internal belief-systems lead differences behavior. context, assume that individuals cannot directly grasp characteristics environment. Instead, they must infer them indirectly ambiguous observations, which themselves generate alter through actions. Differences behavior arise interplay prior expectations, propensity particular states certain beliefs ability influence situations specific We then use concrete examples demonstrate future can take our approach identify systematic experiences behaviors among depressed patients (or patient subgroups) investigate changes response new experiences. also discuss potential applications diagnosing treating This work move towards understanding aspects depression detail, recognizing importance etiology, diagnosis, treatment.

Language: Английский

---

Active Inference on the Edge: A Design Study

2022 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops), Journal Year: 2024, Volume and Issue: unknown, P. 550 - 555

Published: March 11, 2024

Every year, the amount of data created by Internet Things (IoT) devices increases; therefore, processing is carried out edge in close proximity. To ensure Quality Service (QoS) throughout these operations, systems are supervised and adapted with help Machine Learning (ML). However, as long ML models not retrained, they fail to capture gradual shifts variable distribution, leading an inaccurate view system state poor inference. In this paper, we present a novel paradigm that constructed upon Active Inference (ACI) – concept from neuroscience describes how brain constantly predicts evaluates sensory information decrease long-term surprise. We implemented use case, which ACI-based agent continuously optimized operation on smart manufacturing engine according QoS requirements. The used causal knowledge gradually develop understanding its actions related requirements fulfillment, configurations favor. As result, our required 5 cycles converge optimal solution.

Language: Английский

---