Your Privacy My Cloak: Backdoor Attacks on Differentially Private Federated Learning

Source: arXiv:2606.17035 · Published 2026-06-15 · By Xiaolin Li, Ning Wang, Ninghui Li, Wenhai Sun

TL;DR

This paper challenges the conventional assumption that applying differential privacy (DP) to federated learning (FL) inherently strengthens robustness against backdoor attacks. Through empirical analysis of two baseline attack strategies—DP-opt-in (where attackers comply with DP perturbation) and DP-opt-out (where attackers bypass DP)—the authors find a fundamental tension: DP noise masks the statistical signatures of poisoned updates, causing existing defenses to lose detection power, but it also suppresses backdoor effectiveness. To overcome this, the authors propose RING, a novel coordinated backdoor attack that explicitly leverages this masking effect. RING crafts malicious model updates that individually resemble DP-perturbed benign updates to evade detection yet collectively sum to reconstruct a strong backdoor signal upon aggregation. This perturbation approach is agnostic to the underlying backdoor technique, making it widely applicable.

Extensive experiments on four image and text datasets under non-iid data distributions demonstrate that RING achieves an average attack success rate (ASR) of 90.3% against six state-of-the-art defenses, exceeding baseline strategies by up to 26.08 times under moderate privacy budgets. Ablations show robust performance across attacker/defender parameters and data settings. The paper also evaluates potential countermeasures, revealing substantial utility and privacy trade-offs, underscoring a fundamental security gap in deploying DP-protected federated learning systems.

Key findings

• DP-opt-out baseline attack achieves ASR close to 100% in undefended settings but is more readily detected by six SOTA defenses after bypassing DP noise.
• DP-opt-in baseline attack reduces ASR significantly but simultaneously masks malicious update signatures, causing existing defenses to fail to further reduce ASR.
• RING attack achieves an average ASR of 90.3% across four benchmark datasets with non-iid distributions against six defenses (DeepSight, Krum, Flame, MESAS, FreqFed, FLShield).
• RING improves ASR over baseline attacks by up to 26.08× under sample-level DP-SGD with privacy budget ϵ=1, clipping bound C=10.
• RING’s perturbation layer ensures individual updates resemble DP-noised benign updates (stealthiness), while perturbations sum to zero within subgroups, recovering backdoor signal on aggregation (effectiveness).
• The variance of RING’s perturbation noise matches DP noise as subgroup size increases, improving stealthiness (Theorem 1).
• Partial filtering of malicious updates reduces RING's effectiveness but attack remains robust unless a large fraction of malicious updates are removed (Theorem 2).
• Candidate countermeasures reduce attack effectiveness but incur significant trade-offs in utility and privacy guarantees.

Threat model

The adversary controls a fraction β of the FL clients and can arbitrarily manipulate local training and model updates including applying backdoors. They have full knowledge of local training data and parameters but are unaware of the exact server-side defenses, treating them as black-boxes. The attacker cannot control the server or other benign clients and does not intercept or influence their data or updates. The server is semi-honest and applies standard DP-SGD for privacy and may deploy various in-training backdoor defenses.

Methodology — deep read

1. Threat Model & Assumptions:

• The adversary controls a fraction β of clients in a federated learning setting with a semi-honest server applying DP-SGD for privacy.
• Malicious clients can fully control local training and updates, including launching backdoor attacks and manipulating gradients.
• The attacker has no knowledge of server defenses, treating them as a black box.

2. Data and Experimental Setup:

• Experiments run on four benchmark datasets: MNIST, CIFAR-10, Fashion-MNIST (image), and a text dataset (unspecified in excerpt).
• Data distributions are non-iid with varying partition stratagems among clients.
• In typical rounds, 30 out of 120 clients participate; 20% are malicious.
• Poisoned data ratio per malicious client is 0.5; visible-trigger backdoor attacks used for image datasets.

3. Architecture and Algorithm:

• FL uses FedAvg aggregation with DP-SGD noise added per client gradients.
• Existing defenses evaluated include six state-of-the-art methods with diverse detection mechanisms.
• Two baseline attacks implemented:
  • DP-opt-in: adversaries apply DP noise like benign clients.
  • DP-opt-out: adversaries bypass DP noise for maximal attack strength.
• Proposed RING constructs malicious updates as ˜g_j = ḡ_j + ζ_j with ζ_j perturbations collaboratively designed such that ∑ ζ_j = 0 within subgroups.
• Each ζ_j is constructed by sampling noise vector zj from Gaussian and adjusting it to ensure zero-sum cancellation across subgroup.

4. Training Regime:

• Local clients train for five epochs per round with batch-level DP enforced with noise multiplier σm to meet privacy budget ϵ=1.
• Learning rate η=0.05, clipping bound C=10, momentum 0.9.
• Multiple independent runs (5) for statistical averaging.

5. Evaluation Protocol:

• Metrics: Attack Success Rate (ASR) on backdoor test samples, model accuracy on clean samples, and retention rate (fraction of malicious updates surviving defense filtering).
• Baselines: no-defense, DP-opt-in and DP-opt-out attacks.
• Defenses evaluated: DeepSight, Krum, Flame, MESAS, FreqFed, FLShield.
• Ablation studies consider subgroup sizes, number of malicious clients, data heterogeneity, privacy budgets, and clipping bounds.
• Statistical confidence intervals reported.

6. Reproducibility:

• Code and data release status is not explicitly stated; datasets used are standard benchmarks.
• Mathematical proofs provided for theoretical claims.

Concrete example: For MNIST under iid setting with ϵ=1, baseline DP-opt-in attack showed ASR ~12.81% under Flame defense, while RING achieved ~99.91% ASR by applying coordinated zero-sum perturbations that preserve backdoor strength while mimicking DP noise to evade detection.

Technical innovations

• Identification and empirical characterization of the fundamental tension in DP-FL between attack effectiveness and stealthiness due to DP noise masking malicious update signatures.
• Design of RING, a coordinated perturbation attack that adds adversarial noise to malicious updates that statistically resemble DP-perturbed benign updates to evade detection while exactly canceling out in aggregate to preserve the backdoor signal.
• Mathematical formulation and closed-form construction of perturbations using zero-sum Gaussian noise shares within attacker subgroups inspired by secret sharing schemes.
• Theoretical analysis quantifying the impact of subgroup size and partial removal of malicious updates on RING's effectiveness and stealthiness.

Datasets

• MNIST — standard benchmark image dataset — public
• CIFAR-10 — standard benchmark image dataset — public
• Fashion-MNIST — standard benchmark image dataset — public
• Text dataset (unnamed, presumably public benchmark) — size unspecified

Baselines vs proposed

• DP-opt-out attack: ASR ~100% without defenses; drops under some defenses but detectable (retention rate for malicious updates lower than benign).
• DP-opt-in attack: ASR around ~12.8% under Flame [20] with DP noise (ϵ=1), but defense retention rates for malicious and benign are similar, showing poor detection.
• RING attack: ASR up to ~99.91% vs Flame at ϵ=1, outperforming DP-opt-in by up to 26.08× while maintaining stealth via retention rates comparable to benign updates.
• Clean accuracy trade-offs: Defenses like Flame, MESAS, FLShield degrade clean accuracy to achieve backdoor suppression under DP-opt-out attacks.

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.17035.

Fig 3: Performance of existing defenses against DP-opt-in and DP-opt-out attacks.

Limitations

• RING requires multiple malicious clients to be selected in the same communication round to perform coordinated perturbations, limiting applicability when attacker client participation is sparse.
• Theoretical analysis assumes idealized equal aggregation weights; in practice, weight heterogeneity or adaptive defenses may reduce the attack's cancellation effectiveness.
• Evaluations focus on sample-level DP with privacy budgets around ϵ=1; scalability to very tight privacy regimes or example-level DP variants is unexplored.
• Attack effectiveness has not been tested against adversarially adaptive defenses designed specifically to detect coordinated perturbations or against post-training detection techniques.
• Datasets and backdoor triggers mainly involve relatively simple visible triggers or edge cases; performance on complex semantic backdoors or real-world tasks remains to be studied fully.
• Code release and full reproducibility are not clearly stated, limiting external validation.

Open questions / follow-ons

• How can defenses be designed that simultaneously preserve model utility, enforce strong differential privacy guarantees, and effectively detect or mitigate coordinated perturbation backdoor attacks like RING?
• What are the implications of RING-like coordinated attacks in broader settings such as cross-device FL with sparse or unreliable client participation?
• Can adaptive adversaries use more sophisticated secret-sharing or cryptographic noise sharing to further weaken defenses under other privacy mechanisms beyond sample-level DP-SGD?
• How do post-training detection and mitigation methods affect RING and similar attacks, and can holistic defense strategies be developed?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this paper highlights a subtle yet critical vulnerability in deploying differential privacy within federated learning paradigms, especially when relying on DP as an intrinsic backdoor defense. The findings emphasize that DP noise—while intended to limit data leakage—can paradoxically help attackers hide malicious updates, rendering existing detection defenses ineffective.

Practitioners should be wary of assuming privacy protections inherently provide robustness to adversarial backdoors. The RING attack illustrates how attackers can coordinate to mask their signals while retaining attack potency, defeating anomaly and clustering-based defenses. This suggests that CAPTCHAs or bot detection approaches relying on anomaly detection of user behavior or update patterns may fail if adversaries exploit similar collusion and masking strategies. Therefore, bot-defense engineers need to consider defenses that account for collusion and noise masking, incorporate multi-faceted detection beyond statistical heuristics, and be prepared for utility-privacy trade-offs in securing privacy-preserving collaborative systems.

Cite

@article{arxiv2606_17035,
  title={ Your Privacy My Cloak: Backdoor Attacks on Differentially Private Federated Learning },
  author={ Xiaolin Li and Ning Wang and Ninghui Li and Wenhai Sun },
  journal={arXiv preprint arXiv:2606.17035},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.17035}
}

Read the full paper

• arXiv:2606.17035 (abstract)
• arXiv:2606.17035 (PDF)