Open-Ethical AI: Advancements in Open-Source Human-Centric Neural Language Models DOI Creative Commons
Sabrina Sicari, Jesús F. Cevallos M., Alessandra Rizzardi

et al.

ACM Computing Surveys, Journal Year: 2024, Volume and Issue: 57(4), P. 1 - 47

Published: Nov. 6, 2024

This survey summarises the most recent methods for building and assessing helpful, honest, harmless neural language models, considering small, medium, large-size models. Pointers to open-source resources that help align pre-trained models are given, including use parameter-efficient techniques, specialized prompting frameworks, adapter modules, case-specific knowledge injection, adversarially robust training techniques. Special care is given evidencing progress on value alignment, commonsense reasoning, factuality enhancement, abstract reasoning of Most reviewed works in this publicly shared their code related data were accepted world-leading Machine Learning venues. work aims at helping researchers practitioners accelerate entrance into field human-centric which might be a cornerstone contemporary near-future industrial societal revolution.

Language: Английский

A non-Hebbian code for episodic memory DOI Creative Commons
Rich Pang, Stefano Recanatesi

Science Advances, Journal Year: 2025, Volume and Issue: 11(8)

Published: Feb. 21, 2025

Hebbian plasticity has long dominated neurobiological models of memory formation. Yet, rules operating on one-shot episodic timescales rarely depend both pre- and postsynaptic spiking, challenging theory in this crucial regime. Here, we present an model governed by a simpler rule depending only presynaptic activity. We show that rule, capitalizing high-dimensional neural activity with restricted transitions, naturally stores episodes as paths through complex state spaces like those underlying world model. The resulting traces, which term path vectors, are highly expressive decodable odor-tracking algorithm. vectors robust alternatives to support sequential associative recall, along policy learning, shed light specific hippocampal rules. Thus, non-Hebbian is sufficient for flexible learning well-suited encode policies

Language: Английский

Citations

0
Toward the Emergence of Intelligent Control: Episodic Generalization and Optimization DOI Creative Commons
Tyler Giallanza,

Declan Campbell,

Jonathan Cohen

et al.

Open Mind, Journal Year: 2024, Volume and Issue: 8, P. 688 - 722

Published: Jan. 1, 2024

Abstract Human cognition is unique in its ability to perform a wide range of tasks and learn new quickly. Both abilities have long been associated with the acquisition knowledge that can generalize across flexible use execute goal-directed behavior. We investigate how this emerges neural network by describing testing Episodic Generalization Optimization (EGO) framework. The framework consists an episodic memory module, which rapidly learns relationships between stimuli; semantic pathway, more slowly stimuli map responses; recurrent context maintains representation task-relevant information, integrates over time, uses it both recall context-relevant memories (in memory) bias processing favor features responses pathway). address empirical phenomena reinforcement learning, event segmentation, category showing simulations same set underlying mechanisms accounts for human performance all three domains. results demonstrate components EGO efficiently be flexibly generalized tasks, furthering our understanding humans quickly tasks—a capability fundamental intelligence.

Language: Английский

Citations

2
ASAP: Automatic Synthesis of Attack Prototypes, an online-learning, end-to-end approach DOI Creative Commons
Jesús F. Cevallos M., Alessandra Rizzardi, Sabrina Sicari

et al.

Computer Networks, Journal Year: 2024, Volume and Issue: unknown, P. 110828 - 110828

Published: Sept. 1, 2024

Language: Английский

Citations

1
Toward the Emergence of Intelligent Control: Episodic Generalization and Optimization DOI Open Access
Tyler Giallanza,

Declan Campbell,

Jonathan D. Cohen

et al.

Published: Nov. 22, 2023

Human cognition is unique in its ability to perform a wide range of tasks and learn new quickly. Both abilities have long been associated with the acquisition knowledge that can generalize across flexible use execute goal-directed behavior. We investigate how this emerges neural network by describing testing Episodic Generalization Optimization (EGO) framework. The framework consists an episodic memory module, which rapidly learns relationships between stimuli; semantic pathway, more slowly stimuli map responses; recurrent context maintains representation task-relevant information, integrates over time, uses it both recall context-relevant memories (in memory) bias processing favor features responses pathway). address empirical phenomena reinforcement learning, event segmentation, category showing simulations same set underlying mechanisms accounts for human performance all three domains. results demonstrate components EGO efficiently be flexibly generalized tasks, furthering our understanding humans quickly — capability fundamental intelligence.

Language: Английский

Citations

2
Is human compositionality meta-learned? DOI
Jacob Russin, Sam Whitman McGrath, Ellie Pavlick

et al.

Behavioral and Brain Sciences, Journal Year: 2024, Volume and Issue: 47

Published: Jan. 1, 2024

Abstract Recent studies suggest that meta-learning may provide an original solution to enduring puzzle about whether neural networks can explain compositionality – in particular, by raising the prospect be understood as emergent property of inner-loop learning algorithm. We elaborate on this hypothesis and consider its empirical predictions regarding mechanisms development human compositionality.

Language: Английский

Citations

0
Improving Systematic Generalization of Linear Transformer Using Normalization Layers and Orthogonality Loss Function DOI Creative Commons
Tae-Won Park, Hyun‐Chul Kim

Mathematics, Journal Year: 2024, Volume and Issue: 12(21), P. 3390 - 3390

Published: Oct. 30, 2024

A Linear Transformer linearizes the attention mechanism of vanilla architecture, significantly improving efficiency and achieving linear theoretical complexity with respect to sequence length. However, few studies have explored capabilities beyond its efficiency. In this work, we investigate systematic generalization capability Transformer, a crucial property for strong unseen data. Through preliminary experiments, identify two major issues contributing unstable performance: (i) unconstrained norms Queries Keys, (ii) high correlation among Values across sequence. To address these issues, propose simple yet effective methods: normalization layers an orthogonality loss function applied during training. demonstrate that applying methods improves stability performance several well-known tasks. Furthermore, our proposed outperform on specific tasks, such as sort-of-CLEVR SCAN

Language: Английский

Citations

0
Open-Ethical AI: Advancements in Open-Source Human-Centric Neural Language Models DOI Creative Commons
Sabrina Sicari, Jesús F. Cevallos M., Alessandra Rizzardi

et al.

ACM Computing Surveys, Journal Year: 2024, Volume and Issue: 57(4), P. 1 - 47

Published: Nov. 6, 2024

This survey summarises the most recent methods for building and assessing helpful, honest, harmless neural language models, considering small, medium, large-size models. Pointers to open-source resources that help align pre-trained models are given, including use parameter-efficient techniques, specialized prompting frameworks, adapter modules, case-specific knowledge injection, adversarially robust training techniques. Special care is given evidencing progress on value alignment, commonsense reasoning, factuality enhancement, abstract reasoning of Most reviewed works in this publicly shared their code related data were accepted world-leading Machine Learning venues. work aims at helping researchers practitioners accelerate entrance into field human-centric which might be a cornerstone contemporary near-future industrial societal revolution.

Language: Английский

Citations

0