Continual Deep Learning for Time Series Modeling DOI Creative Commons

Sio-Iong Ao,

Haytham M. Fayek

Sensors, Journal Year: 2023, Volume and Issue: 23(16), P. 7167 - 7167

Published: Aug. 14, 2023

The multi-layer structures of Deep Learning facilitate the processing higher-level abstractions from data, thus leading to improved generalization and widespread applications in diverse domains with various types data. Each domain data type presents its own set challenges. Real-world time series may have a non-stationary distribution that lead models facing problem catastrophic forgetting, abrupt loss previously learned knowledge. Continual learning is paradigm machine handle situations when stationarity datasets no longer be true or required. This paper systematic review recent sensor series, need for advanced preprocessing techniques some environments, as well summaries how deploy modeling while alleviating forgetting continual methods. selected case studies cover wide collection can illustrate tailor-made Learning, techniques, algorithms practical, real-world application aspects.

Language: Английский

Three types of incremental learning DOI Creative Commons
Gido M. van de Ven, Tinne Tuytelaars, Andreas S. Tolias

et al.

Nature Machine Intelligence, Journal Year: 2022, Volume and Issue: 4(12), P. 1185 - 1197

Published: Dec. 5, 2022

Incrementally learning new information from a non-stationary stream of data, referred to as 'continual learning', is key feature natural intelligence, but challenging problem for deep neural networks. In recent years, numerous methods continual have been proposed, comparing their performances difficult due the lack common framework. To help address this, we describe three fundamental types, or 'scenarios', learning: task-incremental, domain-incremental and class-incremental learning. Each these scenarios has its own set challenges. illustrate provide comprehensive empirical comparison currently used strategies, by performing Split MNIST CIFAR-100 protocols according each scenario. We demonstrate substantial differences between in terms difficulty effectiveness different strategies. The proposed categorization aims structure field, forming foundation clearly defining benchmark problems.

Language: Английский

Citations

264
A Comprehensive Survey of Continual Learning: Theory, Method and Application DOI
Liyuan Wang, Xingxing Zhang, Hang Su

et al.

IEEE Transactions on Pattern Analysis and Machine Intelligence, Journal Year: 2024, Volume and Issue: 46(8), P. 5362 - 5383

Published: Feb. 26, 2024

To cope with real-world dynamics, an intelligent system needs to incrementally acquire, update, accumulate, and exploit knowledge throughout its lifetime. This ability, known as continual learning, provides a foundation for AI systems develop themselves adaptively. In general sense, learning is explicitly limited by catastrophic forgetting, where new task usually results in dramatic performance drop of the old tasks. Beyond this, increasingly numerous advances have emerged recent years that largely extend understanding application learning. The growing widespread interest this direction demonstrates realistic significance well complexity. work, we present comprehensive survey seeking bridge basic settings, theoretical foundations, representative methods, practical applications. Based on existing empirical results, summarize objectives ensuring proper stability-plasticity trade-off adequate intra/inter-task generalizability context resource efficiency. Then provide state-of-the-art elaborated taxonomy, extensively analyzing how strategies address they are adapted particular challenges various Through in-depth discussion promising directions, believe such holistic perspective can greatly facilitate subsequent exploration field beyond.

Language: Английский

Citations

191
Battery safety: Machine learning-based prognostics DOI Creative Commons
Jingyuan Zhao,

Xuning Feng,

Quanquan Pang

et al.

Progress in Energy and Combustion Science, Journal Year: 2024, Volume and Issue: 102, P. 101142 - 101142

Published: Jan. 19, 2024

Lithium-ion batteries play a pivotal role in wide range of applications, from electronic devices to large-scale electrified transportation systems and grid-scale energy storage. Nevertheless, they are vulnerable both progressive aging unexpected failures, which can result catastrophic events such as explosions or fires. Given their expanding global presence, the safety these potential hazards serious malfunctions now major public concerns. Over past decade, scholars industry experts intensively exploring methods monitor battery safety, spanning materials cell, pack system levels across various spectral, spatial, temporal scopes. In this Review, we start by summarizing mechanisms nature failures. Following this, explore intricacies predicting evolution delve into specialized knowledge essential for data-driven, machine learning models. We offer an exhaustive review spotlighting latest strides fault diagnosis failure prognosis via array approaches. Our discussion encompasses: (1) supervised reinforcement integrated with models, apt faults/failures probing causes protocols at cell level; (2) unsupervised, semi-supervised, self-supervised learning, advantageous harnessing vast data sets modules/packs; (3) few-shot tailored gleaning insights scarce examples, alongside physics-informed bolster model generalization optimize training data-scarce settings. conclude casting light on prospective horizons comprehensive, real-world prognostics management.

Language: Английский

Citations

66
A CNN-BiGRU-AM neural network for AI applications in shale oil production prediction DOI

Guangzhao Zhou,

Zanquan Guo,

Simin Sun

et al.

Applied Energy, Journal Year: 2023, Volume and Issue: 344, P. 121249 - 121249

Published: May 22, 2023

Language: Английский

Citations

52
Multi-source information fusion: Progress and future DOI Creative Commons
Xinde Li, Fir Dunkin, Jean Dezert

et al.

Chinese Journal of Aeronautics, Journal Year: 2023, Volume and Issue: 37(7), P. 24 - 58

Published: Dec. 12, 2023

Multi-Source Information Fusion (MSIF), as a comprehensive interdisciplinary field based on modern information technology, has gained significant research value and extensive application prospects in various domains, attracting high attention interest from scholars, engineering experts, practitioners worldwide. Despite achieving fruitful results both theoretical applied aspects over the past five decades, there remains lack of systematic review articles that provide an overview recent development MSIF. In light this, this paper aims to assist researchers individuals interested gaining quick understanding relevant techniques trends MSIF, which conducts statistical analysis academic reports related achievements MSIF two provides brief theories, methodologies, well key issues challenges currently faced. Finally, outlook future directions are presented.

Language: Английский

Citations

52
Brain-inspired learning in artificial neural networks: A review DOI Creative Commons
Samuel Schmidgall,

Rojin Ziaei,

Jascha Achterberg

et al.

APL Machine Learning, Journal Year: 2024, Volume and Issue: 2(2)

Published: May 9, 2024

Artificial neural networks (ANNs) have emerged as an essential tool in machine learning, achieving remarkable success across diverse domains, including image and speech generation, game playing, robotics. However, there exist fundamental differences between ANNs’ operating mechanisms those of the biological brain, particularly concerning learning processes. This paper presents a comprehensive review current brain-inspired representations artificial networks. We investigate integration more biologically plausible mechanisms, such synaptic plasticity, to improve these networks’ capabilities. Moreover, we delve into potential advantages challenges accompanying this approach. In review, pinpoint promising avenues for future research rapidly advancing field, which could bring us closer understanding essence intelligence.

Language: Английский

Citations

38
End-Edge-Cloud Collaborative Computing for Deep Learning: A Comprehensive Survey DOI
Yingchao Wang, Chen Yang, Shulin Lan

et al.

IEEE Communications Surveys & Tutorials, Journal Year: 2024, Volume and Issue: 26(4), P. 2647 - 2683

Published: Jan. 1, 2024

The booming development of deep learning applications and services heavily relies on large models massive data in the cloud. However, cloud-based encounters challenges meeting application requirements responsiveness, adaptability, reliability. Edge-based end-based enables rapid, near real-time analysis response, but edge nodes end devices usually have limited resources to support models. This necessitates integration end, edge, cloud computing technologies combine their different advantages. Despite existence numerous studies edge-cloud collaboration, a comprehensive survey for end-edge-cloud computing-enabled is needed review current status point out future directions. Therefore, this paper: 1) analyzes collaborative elements within system learning, proposes training, inference, updating methods mechanisms under collaboration framework. 2) provides systematic investigation key enabling including model compression, partition, knowledge transfer. 3) highlights six open issues stimulate continuous research efforts field learning.

Language: Английский

Citations

23
Concepts and applications of digital twins in healthcare and medicine DOI Creative Commons
Kang Zhang, Hong-Yu Zhou, Daniel T. Baptista‐Hon

et al.

Patterns, Journal Year: 2024, Volume and Issue: 5(8), P. 101028 - 101028

Published: Aug. 1, 2024

The digital twin (DT) is a concept widely used in industry to create replicas of physical objects or systems. dynamic, bi-directional link between the entity and its counterpart enables real-time update entity. It can predict perturbations related object's function. obvious applications DTs healthcare medicine are extremely attractive prospects that have potential revolutionize patient diagnosis treatment. However, challenges including technical obstacles, biological heterogeneity, ethical considerations make it difficult achieve desired goal. Advances multi-modal deep learning methods, embodied AI agents, metaverse may mitigate some difficulties. Here, we discuss basic concepts underlying DTs, requirements for implementing medicine, their current uses. We also provide our perspective on five hallmarks DT system advance research this field.

Language: Английский

Citations

19
Two-Dimensional Materials for Brain-Inspired Computing Hardware DOI
Shreyash Hadke, Min‐A Kang,

Vinod K. Sangwan

et al.

Chemical Reviews, Journal Year: 2025, Volume and Issue: unknown

Published: Jan. 2, 2025

Recent breakthroughs in brain-inspired computing promise to address a wide range of problems from security healthcare. However, the current strategy implementing artificial intelligence algorithms using conventional silicon hardware is leading unsustainable energy consumption. Neuromorphic based on electronic devices mimicking biological systems emerging as low-energy alternative, although further progress requires materials that can mimic function while maintaining scalability and speed. As result their diverse unique properties, atomically thin two-dimensional (2D) are promising building blocks for next-generation electronics including nonvolatile memory, in-memory neuromorphic computing, flexible edge-computing systems. Furthermore, 2D achieve biorealistic synaptic neuronal responses extend beyond logic memory Here, we provide comprehensive review growth, fabrication, integration van der Waals heterojunctions optoelectronic devices, circuits, For each case, relationship between physical properties device emphasized followed by critical comparison technologies different applications. We conclude with forward-looking perspective key remaining challenges opportunities applications leverage fundamental heterojunctions.

Language: Английский

Citations

5
The neurobench framework for benchmarking neuromorphic computing algorithms and systems DOI Creative Commons
Jason Yik,

Korneel Van den Berghe,

Douwe den Blanken

et al.

Nature Communications, Journal Year: 2025, Volume and Issue: 16(1)

Published: Feb. 11, 2025

Language: Английский

Citations

3