Overcoming catastrophic forgetting in neural networks

Proceedings of the National Academy of Sciences, Journal Year: 2017, Volume and Issue: 114(13), P. 3521 - 3526

Published: March 14, 2017

The ability to learn tasks in a sequential fashion is crucial the development of artificial intelligence. Until now neural networks have not been capable this and it has widely thought that catastrophic forgetting an inevitable feature connectionist models. We show possible overcome limitation train can maintain expertise on they experienced for long time. Our approach remembers old by selectively slowing down learning weights important those tasks. demonstrate our scalable effective solving set classification based hand-written digit dataset several Atari 2600 games sequentially.

Language: Английский

---

Continual lifelong learning with neural networks: A review

Neural Networks, Journal Year: 2019, Volume and Issue: 113, P. 54 - 71

Published: Feb. 10, 2019

Humans and animals have the ability to continually acquire, fine-tune, transfer knowledge skills throughout their lifespan. This ability, referred as lifelong learning, is mediated by a rich set of neurocognitive mechanisms that together contribute development specialization our sensorimotor well long-term memory consolidation retrieval. Consequently, learning capabilities are crucial for computational systems autonomous agents interacting in real world processing continuous streams information. However, remains long-standing challenge machine neural network models since continual acquisition incrementally available information from non-stationary data distributions generally leads catastrophic forgetting or interference. limitation represents major drawback state-of-the-art deep typically learn representations stationary batches training data, thus without accounting situations which becomes over time. In this review, we critically summarize main challenges linked artificial compare existing approaches alleviate, different extents, forgetting. Although significant advances been made domain-specific with networks, extensive research efforts required robust on robots. We discuss well-established emerging motivated factors biological such structural plasticity, replay, curriculum intrinsic motivation, multisensory integration.

Language: Английский

---

Hybrid computing using a neural network with dynamic external memory

Nature, Journal Year: 2016, Volume and Issue: 538(7626), P. 471 - 476

Published: Oct. 12, 2016

Language: Английский

---

Neuroscience-Inspired Artificial Intelligence

Neuron, Journal Year: 2017, Volume and Issue: 95(2), P. 245 - 258

Published: July 1, 2017

Language: Английский

---

Mechanisms of systems memory consolidation during sleep

Nature Neuroscience, Journal Year: 2019, Volume and Issue: 22(10), P. 1598 - 1610

Published: Aug. 26, 2019

Language: Английский

---

Toward an Integration of Deep Learning and Neuroscience

Frontiers in Computational Neuroscience, Journal Year: 2016, Volume and Issue: 10

Published: Sept. 14, 2016

Neuroscience has focused on the detailed implementation of computation, studying neural codes, dynamics and circuits. In machine learning, however, artificial networks tend to eschew precisely designed or circuits in favor brute force optimization a cost function, often using simple relatively uniform initial architectures. Two recent developments have emerged within learning that create an opportunity connect these seemingly divergent perspectives. First, structured architectures are used, including dedicated systems for attention, recursion various forms short- long-term memory storage. Second, functions training procedures become more complex varied across layers over time. Here we think about brain terms ideas. We hypothesize (1) optimizes functions, (2) diverse differ locations development, (3) operates pre-structured architecture matched computational problems posed by behavior. Such heterogeneously optimized system, enabled series interacting serves make data-efficient targeted needs organism. suggest directions which neuroscience could seek refine test hypotheses.

Language: Английский

---

Bridging Biological and Artificial Neural Networks with Emerging Neuromorphic Devices: Fundamentals, Progress, and Challenges

Advanced Materials, Journal Year: 2019, Volume and Issue: 31(49)

Published: Sept. 24, 2019

As the research on artificial intelligence booms, there is broad interest in brain-inspired computing using novel neuromorphic devices. The potential of various emerging materials and devices for has attracted extensive efforts, leading to a large number publications. Going forward, order better emulate brain's functions, its relevant fundamentals, working mechanisms, resultant behaviors need be re-visited, understood, connected electronics. A systematic overview biological neural systems given, along with their related critical mechanisms. Recent progress reviewed and, more importantly, existing challenges are highlighted hopefully shed light future directions.

Language: Английский

---

Reinforcement Learning, Fast and Slow

Trends in Cognitive Sciences, Journal Year: 2019, Volume and Issue: 23(5), P. 408 - 422

Published: April 17, 2019

Recent AI research has given rise to powerful techniques for deep reinforcement learning. In their combination of representation learning with reward-driven behavior, would appear have inherent interest psychology and neuroscience. One reservation been that procedures demand large amounts training data, suggesting these algorithms may differ fundamentally from those underlying human While this concern applies the initial wave RL techniques, subsequent work established methods allow systems learn more quickly efficiently. Two particularly interesting promising center, respectively, on episodic memory meta-learning. Alongside as leveraging meta-learning direct implications subtle but critically important insight which bring into focus is fundamental connection between fast slow forms Deep (RL) driven impressive advances in artificial intelligence recent years, exceeding performance domains ranging Atari Go no-limit poker. This progress drawn attention cognitive scientists interested understanding However, raised be too sample-inefficient – is, it simply provide a plausible model how humans learn. present review, we counter critique by describing recently developed operate nimbly, solving problems much than previous methods. Although were an context, propose they rich A key insight, arising methods, concerns slower, incremental Over just past few revolutionary occurred (AI) research, where resurgence neural network or 'deep learning' [1LeCun Y. et al.Deep learning.Nature. 2015; 521: 436Crossref PubMed Scopus (42113) Google Scholar, 2Goodfellow I. Learning. Vol. 1. MIT Press, 2016Google Scholar] fueled breakthroughs image [3Krizhevsky A. al.Imagenet classification convolutional networks.Adv. Neural Inf. Process. Syst. 2012; : 1097-1105Google 4Eslami S.M.A. al.Neural scene rendering.Science. 2018; 360: 1204-1210Crossref (264) Scholar], natural language processing [5Bahdanau D. machine translation jointly align translate.arXiv. 2014; 1409.0473Google 6Van Den Oord al.Wavenet: generative raw audio.arXiv. 2016; 1609.03499Google many other areas. These developments attracted growing psychologists, psycholinguists, neuroscientists, curious about whether might suggest new hypotheses concerning cognition brain function [7Marblestone A.H. al.Toward integration neuroscience.Front. Comput. Neurosci. 10: 94Crossref (316) 8Song H.F. al.Reward-based recurrent networks value-based tasks.eLife. 2017; 6: e21492Crossref (1) 9Yamins D.L.K. DiCarlo J.J. Using goal-driven models understand sensory cortex.Nat. 19: 356Crossref (650) 10Sussillo al.A finds naturalistic solution production muscle activity.Nat. 18: 1025Crossref (229) 11Khaligh-Razavi S.-M. Kriegeskorte N. supervised, not unsupervised, explain cortical representation.PLoS Biol. e1003915Crossref (554) Scholar]. area appears inviting perspective (Box 1). marries modeling (see Glossary) learning, set rewards punishments rather explicit instruction [12Sutton R.S. Barto A.G. Reinforcement Learning: An Introduction. 2018Google After decades aspirational practical idea, within 5 years exploded one most intense areas generating super-human tasks video games [13Mnih V. al.Human-level control through 518: 529Crossref (13741) poker [14Moravčík M. al.Deepstack: expert-level heads-up poker.Science. 356: 508-513Crossref (431) multiplayer contests [15Jaderberg first-person population-based learning.arXiv. 1807.01281Google complex board games, including go chess [16Silver al.Mastering game tree search.Nature. 529: 484Crossref (8554) 17Silver shogi self-play general algorithm.arXiv. 1712.01815Google 18Silver without knowledge.Nature. 550: 354Crossref (4476) 19Silver algorithm masters chess, shogi, self-play.Science. 362: 1140-1144Crossref (1270) Scholar].Box 1Deep LearningRL centers problem behavioral policy, mapping states situations actions, maximizes cumulative long-term reward simple settings, policy can represented look-up table, listing appropriate action any state. richer environments, however, kind infeasible, must instead encoded implicitly parameterized function. Pioneering 1990s showed could approximated using multilayer (or deep) ([78Tesauro G. Temporal difference td-gammon.Commun. ACM. 1995; 38: 58-68Crossref (964) L.J. Lin, PhD Thesis, Carnegie Melon University, 1993), allowing gradient-descent discover rich, nonlinear mappings perceptual inputs actions panel below). technical challenges prevented until 2015, when breakthrough demonstrated made such Figure IB Since then, rapid toward improving scaling [79Hessel al.Rainbow: combining improvements 1710.02298Google its application task Capture Flag [80Jaderberg al.Population based networks.arXiv. 1711.09846Google cases, later involved integrating architectural algorithmic complements, search slot-based, episodic-like [52Graves al.Hybrid computing dynamic external memory.Nature. 538: 471Crossref (801) IC Other focused goal speed, make observations, reviewed main text.The figure illustrates evolution starting Tesauro's groundbreaking backgammon-playing system 'TD-gammon' [78Tesauro centered took input learned output estimate 'state value,' defined expected future rewards, here equal estimated probability eventually winning current position. Panel B shows Atari-playing DQN reported Mnih colleagues Here, Scholar]) takes screen pixels learns joystick actions. C schematic state-of-the art Wayne [51Wayne al.Unsupervised predictive goal-directed agent.arXiv. 1803.10760Google full description detailed 'wiring' agent beyond scope paper (but found Scholar]). indicates, architecture comprises multiple modules, leverages predict upcoming events, 'speaks' reinforcement-learning module selects predictor module's The learns, among tasks, perform navigation maze-like shown text. Beyond topic, hold special mechanisms drive originally inspired animal conditioning [20Sutton Toward modern theory adaptive networks: expectation prediction.Psychol. Rev. 1981; 88: 135Crossref (924) are believed relate closely reward-based centering dopamine [21Schultz W. substrate prediction reward.Science. 1997; 275: 1593-1599Crossref (5895) At same time, representations support generalization transfer, abilities biological brains. Given connections, offer source ideas researchers both at neuroscientific levels. And indeed, started take notice commentary first also sounded note caution. On blush fashion quite different humans. hallmark difference, argued, lies sample efficiency versus RL. Sample refers amount data required attain chosen target level performance. measure, indeed drastically learners. To expert human-level orders magnitude experts themselves [22Tsividis P.A. al.Human Atari.2017 AAAI Spring Symposium Series. short, RL, least incarnation, Or so argument gone [23Lake B.M. concept probabilistic program induction.Science. 350: 1332-1338Crossref (1414) 24Marcus learning: critical appraisal.arXiv. 1801.00631Google applicable beginning around 2013 (e.g., [25Mnih al.Playing atari 2013; 1312.5602Google even short time since innovations show dramatically increased. mitigate original demands huge effectively fast. emergence computational revives candidate consider two problem: meta-RL. We examine enable potential point considering why fact slow. describe primary sources inefficiency. end paper, will circle back constellations issues described concepts connected. slowness requirement parameter adjustment. Initial (which still very widely used research) employed gradient descent sculpt connectivity outputs As discussed only [26Kumaran al.What do intelligent agents need? complementary updated.Trends Cogn. Sci. 20: 512-534Abstract Full Text PDF (278) adjustments during form small, order maximize [27Hardt al.Train faster, generalize better: stability stochastic descent.arXiv. 1509.01240Google avoid overwriting effects earlier (an effect sometimes referred 'catastrophic interference'). small step-sizes proposed second weak inductive bias. basic lesson procedure necessarily faces bias–variance trade-off: stronger assumptions makes patterns (i.e., bias procedure) less accomplished (assuming matches what's data!). able master wider range (greater variance), sample-efficient [28Bishop C.M. Pattern Recognition Machine Learning (information science statistics). Springer-Verlag, 2006Google effect, strong what allows considers narrow interpreting incoming will, perforce, hone correct hypothesis rapidly weaker biases (again, assuming falls range). Importantly, generic extremely low-bias systems; parameters (connection weights) capable fit wide data. dictated trade-off, means networks, 1) tend sample-inefficient, requiring Together, factors—incremental adjustment bias—explain first-generation models. clear factors mitigated, proceed manner. follows, specific confronts problem, addition field, bear suggestive links neuroscience, shall detail. If then way faster updating. Naively increasing rate governing optimization leads catastrophic interference. there another accomplish goal, keep record use directly reference making decisions. [29Pritzel control.arXiv. 1703.01988Google 30Gershman S.J. Daw N.D. animals: integrative framework.Annu. Psychol. 68: 101-128Crossref (47) 42Lengyel Dayan P. Hippocampal contributions control: third way.Adv. 2008; 889-896Google parallels 'non-parametric' approaches resembles 'instance-' 'exemplar-based' theories [31Logan G.D. instance automatization.Psychol. 1988; 95: 492Crossref (2336) 32Smith E.E. Medin D.L. Categories Concepts. 9. Harvard University 1981Google When situation encountered decision take, compare internal stored situations. associated highest value, outcomes similar present. state computed network, refer resulting 'episodic RL'. explanation mechanics presented Box 2.Box 2Episodic RLEpisodic value memories [30Gershman 43Bornstein A.M. al.Reminders choices decisions humans.Nat. Commun. 8: 15958Crossref (106) 44Bornstein Norman K.A. Reinstated context guides sampling-based reward.Nat. 997Crossref (82) Consider, example, valuation depicted I, wherein stores each along discounted sum obtained next n steps. items comprise followed. state, computes weighted similarity (sim.) extended values recording taken sums store, querying store find to-be-evaluated was taken. fact, [81Blundell C. al.Model-free 1606.04460Google achieve games.The success depends compute similarity. follow-up Pritzel al. improved gradually shaping results 57 Environment showcasing benefits (representation) (value) Episodic games. unlike standard approach, information gained experienced event leveraged immediately guide behavior. whereas 'fast' went 'slow,' twist story:

Language: Английский

---

Navigating cognition: Spatial codes for human thinking

Science, Journal Year: 2018, Volume and Issue: 362(6415)

Published: Nov. 8, 2018

A framework for cognitive spaces Ever since Tolman's proposal of maps in the 1940s, question how spatial representations support flexible behavior has been a contentious topic. Bellmund et al. review and combine concepts from science philosophy with findings neurophysiology navigation rodents to propose neuroscience. They argue that spatial-processing principles hippocampalentorhinal region provide geometric code map information domains high-level cognition discuss recent evidence this proposal. Science , issue p. eaat6766

Language: Английский

---

Neural scene representation and rendering

Science, Journal Year: 2018, Volume and Issue: 360(6394), P. 1204 - 1210

Published: June 14, 2018

Scene representation-the process of converting visual sensory data into concise descriptions-is a requirement for intelligent behavior. Recent work has shown that neural networks excel at this task when provided with large, labeled datasets. However, removing the reliance on human labeling remains an important open problem. To end, we introduce Generative Query Network (GQN), framework within which machines learn to represent scenes using only their own sensors. The GQN takes as input images scene taken from different viewpoints, constructs internal representation, and uses representation predict appearance previously unobserved viewpoints. demonstrates learning without labels or domain knowledge, paving way toward autonomously understand world around them.

Language: Английский

---