Wild Patterns Reloaded: A Survey of Machine Learning Security against Training Data Poisoning

ACM Computing Surveys, Journal Year: 2023, Volume and Issue: 55(13s), P. 1 - 39

Published: March 1, 2023

The success of machine learning is fueled by the increasing availability computing power and large training datasets. data used to learn new models or update existing ones, assuming that it sufficiently representative will be encountered at test time. This assumption challenged threat poisoning, an attack manipulates compromise model’s performance Although poisoning has been acknowledged as a relevant in industry applications, variety different attacks defenses have proposed so far, complete systematization critical review field still missing. In this survey, we provide comprehensive learning, reviewing more than 100 papers published past 15 years. We start categorizing current then organize accordingly. While focus mostly on computer-vision argue our also encompasses state-of-the-art for other modalities. Finally, discuss resources research shed light limitations open questions field.

Language: Английский

---

Dataset Distillation: A Comprehensive Review

IEEE Transactions on Pattern Analysis and Machine Intelligence, Journal Year: 2023, Volume and Issue: 46(1), P. 150 - 170

Published: Oct. 10, 2023

Recent success of deep learning is largely attributed to the sheer amount data used for training neural networks. Despite unprecedented success, massive data, unfortunately, significantly increases burden on storage and transmission further gives rise a cumbersome model process. Besides, relying raw per se yields concerns about privacy copyright. To alleviate these shortcomings, dataset distillation (DD), also known as condensation (DC), was introduced has recently attracted much research attention in community. Given an original dataset, DD aims derive smaller containing synthetic samples, based which trained models yield performance comparable with those dataset. In this paper, we give comprehensive review summary recent advances its application. We first introduce task formally propose overall algorithmic framework followed by all existing methods. Next, provide systematic taxonomy current methodologies area, discuss their theoretical interconnections. present challenges through extensive empirical studies envision possible directions future works.

Language: Английский

---

FIBA: Frequency-Injection based Backdoor Attack in Medical Image Analysis

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2022, Volume and Issue: unknown, P. 20844 - 20853

Published: June 1, 2022

In recent years, the security of AI systems has drawn increasing research attention, especially in medical imaging realm. To develop a secure image analysis (MIA) system, it is must to study possible backdoor attacks (BAs), which can embed hidden malicious behaviors into system. However, designing unified BA method that be applied various MIA challenging due diversity modalities (e.g., X-Ray, CT, and MRI) tasks classification, detection, segmentation). Most existing methods are designed attack natural classification models, apply spatial triggers training images inevitably corrupt semantics poisoned pixels, leading failures attacking dense prediction models. address this issue, we propose novel Frequency-Injection based Backdoor Attack (FIBA) capable delivering tasks. Specifically, FIBA leverages trigger function frequency domain inject low-frequency information by linearly combining spectral amplitude both images. Since preserves perform on Experiments three benchmarks (i.e., ISIC-2019 [4] for skin lesion KiTS-19 [17] kidney tumor segmentation, EAD-2019 [1] endoscopic artifact detection), validate effectiveness its superiority over stateof-the-art models bypassing defense. Source code will available at code.

Language: Английский

---

BppAttack: Stealthy and Efficient Trojan Attacks against Deep Neural Networks via Image Quantization and Contrastive Adversarial Learning

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2022, Volume and Issue: unknown, P. 15054 - 15063

Published: June 1, 2022

Deep neural networks are vulnerable to Trojan attacks. Existing attacks use visible patterns (e.g., a patch or image transformations) as triggers, which human inspection. In this paper, we propose stealthy and efficient attacks, BppAttack. Based on existing biology literature visual systems, quantization dithering the trigger, making imperceptible changes. It is attack without training auxiliary models. Due small changes made images, it hard inject such triggers during training. To alleviate problem, contrastive learning based approach that leverages adversarial generate negative sample pairs so learned trigger precise accurate. The proposed method achieves high success rates four benchmark datasets, including MNIST, CIFAR-10, GTSRB, CelebA. also effectively bypasses defenses Our code can be found in https://github.com/RU-System-Software-and-Security/BppAttack.

Language: Английский

---

DEFEAT: Deep Hidden Feature Backdoor Attacks by Imperceptible Perturbation and Latent Representation Constraints

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2022, Volume and Issue: unknown

Published: June 1, 2022

Backdoor attack is a type of serious security threat to deep learning models. An adversary can provide users with model trained on poisoned data manipulate prediction behavior in test stage using backdoor. The backdoored models behave normally clean images, yet be activated and output incorrect if the input stamped specific trigger pattern. Most existing backdoor attacks focus manually defining imperceptible triggers space without considering abnormality triggers' latent representations model. These are susceptible detection algorithms even visual inspection. In this paper, We propose novel stealthy - DEFEAT. It poisons adaptive perturbation restricts representation during training process strengthen our attack's stealthiness resistance defense algorithms. conduct extensive experiments multiple image classifiers real-world datasets demonstrate that 1) hold against state-of-the-art defenses, 2) deceive victim high success jeopardizing utility, 3) practical data.

Language: Английский

---

Backdoor Defense with Machine Unlearning

IEEE INFOCOM 2022 - IEEE Conference on Computer Communications, Journal Year: 2022, Volume and Issue: unknown

Published: May 2, 2022

Backdoor injection attack is an emerging threat to the security of neural networks, however, there still exist limited effective defense methods against attack. In this paper, we propose BAERASER, a novel method that can erase backdoor injected into victim model through machine unlearning. Specifically, BAERASER mainly implements in two key steps. First, trigger pattern recovery conducted extract patterns infected by model. Here, problem equivalent one extracting unknown noise distribution from model, which be easily resolved entropy maximization based generative Subsequently, leverages these recovered reverse procedure and induce polluted memories newly designed gradient ascent unlearning method. Compared with previous solutions, proposed approach gets rid reliance on full access training data for retraining shows higher effectiveness erasing than existing fine-tuning or pruning methods. Moreover, experiments show averagely lower success rates three kinds state-of-the-art attacks 99% four benchmark datasets.

Language: Английский

---

An Overview of Backdoor Attacks Against Deep Neural Networks and Possible Defences

IEEE Open Journal of Signal Processing, Journal Year: 2022, Volume and Issue: 3, P. 261 - 287

Published: Jan. 1, 2022

Together with impressive advances touching every aspect of our society, AI technology based on Deep Neural Networks (DNN) is bringing increasing security concerns. While attacks operating at test time have monopolised the initial attention researchers, backdoor attacks, exploiting possibility corrupting DNN models by interfering training process, represents a further serious threat undermining dependability techniques. In attack, attacker corrupts data so to induce an erroneous behaviour time. Test errors, however, are activated only in presence triggering event corresponding properly crafted input sample. this way, corrupted network continues work as expected for regular inputs, and malicious occurs when decides activate hidden within network. last few years, been subject intense research activity focusing both development new classes proposal possible countermeasures. The goal overview paper review works published until now, classifying different types defences proposed far. classification guiding analysis amount control that has capability defender verify integrity used training, monitor operations As such, particularly suited highlight strengths weaknesses reference application scenarios they in.

Language: Английский

---

Color Backdoor: A Robust Poisoning Attack in Color Space

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2023, Volume and Issue: unknown

Published: June 1, 2023

Backdoor attacks against neural networks have been intensively investigated, where the adversary compromises integrity of victim model, causing it to make wrong predictions for inference samples containing a specific trigger. To trigger more imperceptible and human-unnoticeable, variety stealthy backdoor proposed, some works employ perturbations as triggers, which restrict pixel differences triggered image clean image. Some use special styles (e.g., reflection, Instagram filter) triggers. However, these sacrifice robustness, can be easily defeated by common preprocessing-based defenses. This paper presents novel color attack, exhibit robustness stealthiness at same time. The key insight our attack is apply uniform space shift all pixels global feature robust transformation operations maintain natural-looking. find optimal trigger, we first define naturalness restrictions through metrics PSNR, SSIM LPIPS. Then Particle Swarm Optimization (PSO) algorithm searchfor that achieve high effectiveness while satisfying restrictions. Extensive experiments demonstrate superiority PSO different main-stream

Language: Английский

---

Backdoor Defense via Adaptively Splitting Poisoned Dataset

2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Journal Year: 2023, Volume and Issue: unknown

Published: June 1, 2023

Backdoor defenses have been studied to alleviate the threat of deep neural networks (DNNs) being backdoor attacked and thus maliciously altered. Since DNNs usually adopt some external training data from an untrusted third party, a robust defense strategy during stage is importance. We argue that core training-time select poisoned samples handle them properly. In this work, we summarize unified framework as splitting dataset into two pools. Under our framework, propose adaptively dataset-based (ASD). Concretely, apply loss-guided split meta-learning-inspired dynamically update With clean pool polluted pool, ASD successfully defends against attacks training. Extensive experiments on multiple benchmark datasets DNN models six state-of-the-art demonstrate superiority ASD. Our code available at https://github.com/KuofengGao/ASD.

Language: Английский

---

A survey on vulnerability of federated learning: A learning algorithm perspective

Neurocomputing, Journal Year: 2024, Volume and Issue: 573, P. 127225 - 127225

Published: Jan. 8, 2024

Federated Learning (FL) has emerged as a powerful paradigm for training Machine (ML), particularly Deep (DL) models on multiple devices or servers while maintaining data localized at owners' sites. Without centralizing data, FL holds promise scenarios where integrity, privacy and security are critical. However, this decentralized process also opens up new avenues opponents to launch unique attacks, it been becoming an urgent need understand the vulnerabilities corresponding defense mechanisms from learning algorithm perspective. This review paper takes comprehensive look malicious attacks against FL, categorizing them perspectives attack origins targets, providing insights into their methodology impact. In survey, we focus threat targeting of systems. Based source target attack, categorize existing four types, Data Model (D2M), (M2D), (M2M) composite attacks. For each type, discuss strategies proposed, highlighting effectiveness, assumptions potential areas improvement. Defense have evolved using singular metric excluding clients, employing multifaceted approach examining client various phases. survey paper, our research indicates that to-learn gradients, learned model different stages all can be manipulated initiate range undermining performance, reconstructing private local inserting backdoors. We seen these more insidious. While earlier studies typically amplified recent endeavors subtly alter least significant weights in bypass measures. literature provides holistic understanding current landscape highlights importance developing robust, efficient, privacy-preserving defenses ensure safe trusted adoption real-world applications. The categorized bibliography found at: https://github.com/Rand2AI/Awesome-Vulnerability-of-Federated-Learning.

Language: Английский

---