Domain Generalization: A Survey

IEEE Transactions on Pattern Analysis and Machine Intelligence, Journal Year: 2022, Volume and Issue: unknown, P. 1 - 20

Published: Jan. 1, 2022

Generalization to out-of-distribution (OOD) data is a capability natural humans yet challenging for machines reproduce. This because most learning algorithms strongly rely on the i.i.d.~assumption source/target data, which often violated in practice due domain shift. Domain generalization (DG) aims achieve OOD by using only source model learning. Over last ten years, research DG has made great progress, leading broad spectrum of methodologies, e.g., those based alignment, meta-learning, augmentation, or ensemble learning, name few; also been studied various application areas including computer vision, speech recognition, language processing, medical imaging, and reinforcement In this paper, first time comprehensive literature review provided summarize developments over past decade. Specifically, we cover background formally defining relating it other relevant fields like adaptation transfer Then, conduct thorough into existing methods theories. Finally, conclude survey with insights discussions future directions.

Language: Английский

---

Hands-On Bayesian Neural Networks—A Tutorial for Deep Learning Users

IEEE Computational Intelligence Magazine, Journal Year: 2022, Volume and Issue: 17(2), P. 29 - 48

Published: April 13, 2022

Modern deep learning methods constitute incredibly powerful tools to tackle a myriad of challenging problems. However, since operate as black boxes, the uncertainty associated with their predictions is often quantify. Bayesian statistics offer formalism understand and quantify neural network predictions. This tutorial provides an overview relevant literature complete toolset design, implement, train, use evaluate Neural Networks, i.e. Stochastic Artificial Networks trained using methods.

Language: Английский

---

Generalizing to Unseen Domains: A Survey on Domain Generalization

IEEE Transactions on Knowledge and Data Engineering, Journal Year: 2022, Volume and Issue: unknown, P. 1 - 1

Published: Jan. 1, 2022

Machine learning systems generally assume that the training and testing distributions are same. To this end, a key requirement is to develop models can generalize unseen distributions. Domain generalization (DG), i.e., out-of-distribution generalization, has attracted increasing interests in recent years. deals with challenging setting where one or several different but related domain(s) given, goal learn model an test domain. Great progress been made area of domain for This paper presents first review advances area. First, we provide formal definition discuss fields. We then thoroughly theories carefully analyze theory behind generalization. categorize algorithms into three classes: data manipulation, representation learning, strategy, present popular detail each category. Third, introduce commonly used datasets, applications, our open-sourced codebase fair evaluation. Finally, summarize existing literature some potential research topics future.

Language: Английский

---

A Survey of Ensemble Learning: Concepts, Algorithms, Applications, and Prospects

IEEE Access, Journal Year: 2022, Volume and Issue: 10, P. 99129 - 99149

Published: Jan. 1, 2022

Ensemble learning techniques have achieved state-of-the-art performance in diverse machine applications by combining the predictions from two or more base models. This paper presents a concise overview of ensemble learning, covering three main methods: bagging, boosting, and stacking, their early development to recent algorithms. The study focuses on widely used algorithms, including random forest, adaptive boosting (AdaBoost), gradient extreme (XGBoost), light (LightGBM), categorical (CatBoost). An attempt is made concisely cover mathematical algorithmic representations, which lacking existing literature would be beneficial researchers practitioners.

Language: Английский

---

A comprehensive review on ensemble deep learning: Opportunities and challenges

Journal of King Saud University - Computer and Information Sciences, Journal Year: 2023, Volume and Issue: 35(2), P. 757 - 774

Published: Feb. 1, 2023

In machine learning, two approaches outperform traditional algorithms: ensemble learning and deep learning. The former refers to methods that integrate multiple base models in the same framework obtain a stronger model outperforms them. success of an method depends on several factors, including how baseline are trained they combined. literature, there common building successfully applied domains. On other hand, learning-based have improved predictive accuracy across wide range Despite diversity architectures their ability deal with complex problems extract features automatically, main challenge is it requires lot expertise experience tune optimal hyper-parameters, which makes tedious time-consuming task. Numerous recent research efforts been made approach overcome this challenge. Most these focus simple some limitations. Hence, review paper provides comprehensive reviews various strategies for especially case Also, explains detail or factors influence methods. addition, presents accurately categorized used

Language: Английский

---

Social physics

Physics Reports, Journal Year: 2022, Volume and Issue: 948, P. 1 - 148

Published: Jan. 11, 2022

Language: Английский

---

A Survey on Curriculum Learning

IEEE Transactions on Pattern Analysis and Machine Intelligence, Journal Year: 2021, Volume and Issue: unknown, P. 1 - 1

Published: Jan. 1, 2021

Curriculum learning (CL) is a training strategy that trains machine model from easier data to harder data, which imitates the meaningful order in human curricula. As an easy-to-use plug-in, CL has demonstrated its power improving generalization capacity and convergence rate of various models wide range scenarios such as computer vision natural language processing etc. In this survey article, we comprehensively review aspects including motivations, definitions, theories, applications. We discuss works on curriculum within general framework, elaborating how design manually predefined or automatic curriculum. particular, summarize existing designs based framework Difficulty Measurer $+$+ Training Scheduler further categorize methodologies for into four groups, i.e., Self-paced Learning, Transfer Teacher, RL Other Automatic CL. also analyze principles select different may benefit practical Finally, present our insights relationships connecting other concepts transfer learning, meta-learning, continual active etc., then point out challenges well potential future research directions deserving investigations.

Language: Английский

---

Intelligent fault diagnosis of machines with small & imbalanced data: A state-of-the-art review and possible extensions

ISA Transactions, Journal Year: 2021, Volume and Issue: 119, P. 152 - 171

Published: March 8, 2021

Language: Английский

---

Learning the protein language: Evolution, structure, and function

Cell Systems, Journal Year: 2021, Volume and Issue: 12(6), P. 654 - 669.e3

Published: June 1, 2021

Language models have recently emerged as a powerful machine-learning approach for distilling information from massive protein sequence databases. From readily available data alone, these discover evolutionary, structural, and functional organization across space. Using language models, we can encode amino-acid sequences into distributed vector representations that capture their structural properties, well evaluate the evolutionary fitness of variants. We discuss recent advances in modeling applications to downstream property prediction problems. then consider how be enriched with prior biological knowledge introduce an encoding learned representations. The distilled by allows us improve function through transfer learning. Deep are revolutionizing biology. They suggest new ways therapeutic design. However, further developments needed strong priors increase accessibility broader community.

Language: Английский

---

Deep Learning for Visual Tracking: A Comprehensive Survey

IEEE Transactions on Intelligent Transportation Systems, Journal Year: 2021, Volume and Issue: 23(5), P. 3943 - 3968

Published: Jan. 28, 2021

Visual target tracking is one of the most sought-after yet challenging research topics in computer vision. Given ill-posed nature problem and its popularity a broad range real-world scenarios, number large-scale benchmark datasets have been established, on which considerable methods developed demonstrated with significant progress recent years -- predominantly by deep learning (DL)-based methods. This survey aims to systematically investigate current DL-based visual methods, datasets, evaluation metrics. It also extensively evaluates analyzes leading First, fundamental characteristics, primary motivations, contributions are summarized from nine key aspects of: network architecture, exploitation, training for tracking, objective, output, exploitation correlation filter advantages, aerial-view long-term online tracking. Second, popular benchmarks their respective properties compared, metrics summarized. Third, state-of-the-art comprehensively examined set well-established OTB2013, OTB2015, VOT2018, LaSOT, UAV123, UAVDT, VisDrone2019. Finally, conducting critical analyses these trackers quantitatively qualitatively, pros cons under various common scenarios investigated. may serve as gentle use guide practitioners weigh when what conditions choose method(s). facilitates discussion ongoing issues sheds light promising directions.

Language: Английский

---