Understanding and Mitigating Overfitting in Prompt Tuning for Vision-Language Models DOI
Chengcheng Ma, Yang Liu, Jiankang Deng

et al.

IEEE Transactions on Circuits and Systems for Video Technology, Journal Year: 2023, Volume and Issue: 33(9), P. 4616 - 4629

Published: Feb. 16, 2023

Pretrained vision-language models (VLMs) such as CLIP have shown impressive generalization capability in downstream vision tasks with appropriate text prompts. Instead of designing prompts manually, Context Optimization (CoOp) has been recently proposed to learn continuous using task-specific training data. Despite the performance improvements on tasks, several studies reported that CoOp suffers from overfitting issue two aspects: (i) test accuracy base classes first improves and then worsens during training; (ii) novel keeps decreasing. However, none existing can understand mitigate problems. In this study, we explore cause by analyzing gradient flow. Comparative experiments reveal favors generalizable spurious features early later stages, respectively, leading non-overfitting phenomena. Given those observations, propose Subspace Prompt Tuning (Sub PT) project gradients back-propagation onto low-rank subspace spanned early-stage flow eigenvectors entire process successfully eliminate problem. addition, equip a Novel Feature Learner (NFL) enhance ability learned categories beyond set, needless image Extensive 11 classification datasets demonstrate Sub PT+NFL consistently boost outperform state-of-the-art CoCoOp approach. Experiments more challenging including open-vocabulary object detection zero-shot semantic segmentation, also verify effectiveness method. Codes be found at https://tinyurl.com/mpe64f89 .

Language: Английский

A Survey on Evaluation of Large Language Models DOI Open Access
Yupeng Chang, Xu Wang, Jindong Wang

et al.

ACM Transactions on Intelligent Systems and Technology, Journal Year: 2024, Volume and Issue: 15(3), P. 1 - 45

Published: Jan. 23, 2024

Large language models (LLMs) are gaining increasing popularity in both academia and industry, owing to their unprecedented performance various applications. As LLMs continue play a vital role research daily use, evaluation becomes increasingly critical, not only at the task level, but also society level for better understanding of potential risks. Over past years, significant efforts have been made examine from perspectives. This paper presents comprehensive review these methods LLMs, focusing on three key dimensions: what evaluate , where how . Firstly, we provide an overview perspective tasks, encompassing general natural processing reasoning, medical usage, ethics, education, social sciences, agent applications, other areas. Secondly, answer ‘where’ ‘how’ questions by diving into benchmarks, which serve as crucial components assessing LLMs. Then, summarize success failure cases different tasks. Finally, shed light several future challenges that lie ahead evaluation. Our aim is offer invaluable insights researchers realm evaluation, thereby aiding development more proficient point should be treated essential discipline assist We consistently maintain related open-source materials at: https://github.com/MLGroupJLU/LLM-eval-survey

Language: Английский

Citations

731
Towards Out-Of-Distribution Generalization: A Survey DOI Creative Commons

Zheyan Shen,

Jiashuo Liu,

Yue He

et al.

arXiv (Cornell University), Journal Year: 2021, Volume and Issue: unknown

Published: Jan. 1, 2021

Traditional machine learning paradigms are based on the assumption that both training and test data follow same statistical pattern, which is mathematically referred to as Independent Identically Distributed ($i.i.d.$). However, in real-world applications, this $i.i.d.$ often fails hold due unforeseen distributional shifts, leading considerable degradation model performance upon deployment. This observed discrepancy indicates significance of investigating Out-of-Distribution (OOD) generalization problem. OOD an emerging topic research focuses complex scenarios wherein distributions differ from those data. paper represents first comprehensive, systematic review generalization, encompassing a spectrum aspects problem definition, methodological development, evaluation procedures, implications future directions field. Our discussion begins with precise, formal characterization Following that, we categorize existing methodologies into three segments: unsupervised representation learning, supervised optimization, according their positions within overarching process. We provide in-depth representative for each category, further elucidating theoretical links between them. Subsequently, outline prevailing benchmark datasets employed studies. To conclude, overview body work domain suggest potential avenues generalization. A summary surveyed can be accessed at http://out-of-distribution-generalization.com.

Language: Английский

Citations

187
AdaRNN DOI
Yuntao Du, Jindong Wang, Wenjie Feng

et al.

Published: Oct. 26, 2021

Time series has wide applications in the real world and is known to be difficult forecast. Since its statistical properties change over time, distribution also changes temporally, which will cause severe shift problem existing methods. However, it remains unexplored model time perspective. In this paper, we term as Temporal Covariate Shift (TCS). This paper proposes Adaptive RNNs (AdaRNN) tackle TCS by building an adaptive that generalizes well on unseen test data. AdaRNN sequentially composed of two novel algorithms. First, propose Distribution Characterization better characterize information TS. Second, Matching reduce mismatch TS learn model. a general framework with flexible distances integrated. Experiments human activity recognition, air quality prediction, financial analysis show outperforms latest methods classification accuracy 2.6% significantly reduces RMSE 9.0%. We temporal matching algorithm can extended Transformer structure boost performance.

Language: Английский

Citations

133
Deep imbalanced domain adaptation for transfer learning fault diagnosis of bearings under multiple working conditions DOI
Yifei Ding, Minping Jia, Jichao Zhuang

et al.

Reliability Engineering & System Safety, Journal Year: 2022, Volume and Issue: 230, P. 108890 - 108890

Published: Oct. 12, 2022

Language: Английский

Citations

133
Federated learning for medical image analysis: A survey DOI
Hao Guan, Pew‐Thian Yap, Andrea Bozoki

et al.

Pattern Recognition, Journal Year: 2024, Volume and Issue: 151, P. 110424 - 110424

Published: March 12, 2024

Language: Английский

Citations

84
A Review of Practical AI for Remote Sensing in Earth Sciences DOI Creative Commons

Bhargavi Janga,

Gokul Prathin Asamani,

Ziheng Sun

et al.

Remote Sensing, Journal Year: 2023, Volume and Issue: 15(16), P. 4112 - 4112

Published: Aug. 21, 2023

Integrating Artificial Intelligence (AI) techniques with remote sensing holds great potential for revolutionizing data analysis and applications in many domains of Earth sciences. This review paper synthesizes the existing literature on AI sensing, consolidating analyzing methodologies, outcomes, limitations. The primary objectives are to identify research gaps, assess effectiveness approaches practice, highlight emerging trends challenges. We explore diverse including image classification, land cover mapping, object detection, change hyperspectral radar analysis, fusion. present an overview technologies, methods employed, relevant use cases. further challenges associated practical such as quality availability, model uncertainty interpretability, integration domain expertise well solutions, advancements, future directions. provide a comprehensive researchers, practitioners, decision makers, informing at exciting intersection sensing.

Language: Английский

Citations

68
Domain augmentation generalization network for real-time fault diagnosis under unseen working conditions DOI
Yaowei Shi,

Aidong Deng,

Minqiang Deng

et al.

Reliability Engineering & System Safety, Journal Year: 2023, Volume and Issue: 235, P. 109188 - 109188

Published: Feb. 24, 2023

Language: Английский

Citations

63
Automated detection and forecasting of COVID-19 using deep learning techniques: A review DOI
Afshin Shoeibi, Marjane Khodatars, Mahboobeh Jafari

et al.

Neurocomputing, Journal Year: 2024, Volume and Issue: 577, P. 127317 - 127317

Published: Jan. 26, 2024

Language: Английский

Citations

58
Deep Mixed Domain Generalization Network for Intelligent Fault Diagnosis Under Unseen Conditions DOI
Zhenhua Fan, Qifa Xu, Cuixia Jiang

et al.

IEEE Transactions on Industrial Electronics, Journal Year: 2023, Volume and Issue: 71(1), P. 965 - 974

Published: Feb. 16, 2023

Emerging intelligent fault diagnosis models based on domain adaptation can resolve shift problems produced by different working conditions. However, the prerequisite of obtaining target data in advance limits application these to practical engineering scenarios. To address this challenge, a deep mixed generalization network (DMDGN) is proposed for diagnosis. In novel model, augmentation applied both class and spaces, adversarial learning employed introduce perturbations, domain-based discrepancy metric used balance intra- interdomain distances. The model effectively learn more domain-invariant discriminative features from multiple source domains perform tasks loads machines. feasibility DMDGN verified two public datasets one private dataset collected production processes. Empirical results show that outperforms several state-of-the-art models.

Language: Английский

Citations

54
Deep causal factorization network: A novel domain generalization method for cross-machine bearing fault diagnosis DOI
Sixiang Jia, Yongbo Li, Xinyue Wang

et al.

Mechanical Systems and Signal Processing, Journal Year: 2023, Volume and Issue: 192, P. 110228 - 110228

Published: Feb. 27, 2023

Language: Английский

Citations

54