Detecting Audio Deepfakes on the Edge:Lightweight SSL-Based Detection in a Browser Plugin

Source: arXiv:2606.30780 · Published 2026-06-29 · By Octavian Pascu, Dan Oneata, Horia Cucu, Nicolas M. Muller

TL;DR

This paper addresses the pressing challenge of detecting audio deepfakes in a manner that respects user privacy and is computationally efficient for on-device deployment. Prior solutions often depend on cloud-based processing, which raises privacy concerns and impedes practical, real-time use, especially for journalists and fact-checkers dealing with sensitive audio sources. To overcome these issues, the authors propose a lightweight audio deepfake detection approach that leverages truncated self-supervised learning (SSL) features from the Wav2Vec2 XLS-R-300M model combined with a simple logistic regression classifier. By truncating the SSL network to its first seven transformer layers, they strike an effective trade-off between accuracy, efficiency, and model size.

Key findings

• Truncating Wav2Vec2-300M at layer 7 yields the best out-of-domain (OOD) average Equal Error Rate (EER) of 8.4%, outperforming deeper (layer 24) usage (16.9% EER) on multiple evaluation datasets.
• The truncated model improves generalization by 25% relative error reduction on average across six diverse out-of-domain datasets compared to baselines trained on ASVspoof19.
• Inference speed of the truncated model on a single CPU core is approximately 3.4 seconds per 5-second audio clip; 40% faster than the full model, enabling real-time detection feasibility.
• Memory footprint for the truncated model (layer 7, 101M parameters) remains under the 2GB storage limit required for deployment in a Chrome extension, compared to 300M parameters for the full model.
• The truncated SSL-based model outperforms state-of-the-art architectures such as AASIST (11.3% OOD EER), RawNet2 (23.9% EER), RawNet3 (39.6% EER), and RawGAT (27.0% EER) on generalization tests.
• Mid-level layers (especially 5 to 7) of the Wav2Vec2 transformer produce more discriminative features for detecting deepfake speech artifacts than early or very deep layers, which tend to overfit or degrade performance on OOD data.
• The authors successfully integrated the model into a privacy-preserving Chrome browser plugin that performs all detection locally without any audio uploading.

Threat model

The adversary is assumed to deploy audio deepfake generation techniques (text-to-speech and voice conversion) to produce synthetic speech mimicking bona fide speakers. The adversary cannot access or manipulate the local detection model running on the user's device or intercept data, as detection is performed entirely on-device without transmitting any audio externally. Thus, the defender assumes white-box access only to pretrained feature extractors but no remote adversarial perturbation or model evasion attacks.

Methodology — deep read

The authors target the problem of robust and efficient audio deepfake detection deployable on consumer devices without cloud reliance, addressing privacy concerns for sensitive users such as journalists. The threat model assumes an adversary who produces synthetic speech designed to mimic real speakers, but the defender's detection runs on-device without sharing data externally, so attacks cannot manipulate detection models remotely or via adversarial perturbations beyond natural deepfake generation.

The approach uses self-supervised Wav2Vec2 representations as the feature extractor. The chosen model is XLS-R-300M variant, pretrained on large multilingual speech datasets (CommonVoice, Babel, Multilingual LibriSpeech, VoxPopuli, VoxLingual107). It has a convolutional frontend (0.5M params) and 24 Transformer layers (about 13M params each), totaling 310 million parameters.

Instead of using all Transformer layers, the authors extract frozen features from intermediary layers—specifically focusing on the first seven layers—without any fine-tuning of the SSL model. On top of these extracted features (768-dimensional vectors), they train a simple logistic regression classifier (binary cross-entropy loss with L2 regularization, max 5,000 iterations) to classify audio as bona fide or spoofed.

Training is done exclusively on the ASVspoof19 dataset (121k utterances, English). To test generalization, evaluation occurs on six diverse out-of-domain datasets: ASVspoof21 DF, Fake Or Real (FoR), MLAAD (multi-language), In The Wild (ITW), TIMIT-TTS, and WaveFake totaling over half a million utterances. They employ RawBoost augmentation during training, which applies noise, reverberation, equalization, and compression simulating real-world distortions.

Performance is measured by Equal Error Rate (EER) on each dataset, with detailed layer-wise analysis identifying which truncated layer optimally balances OOD detection accuracy and computational efficiency.

Computational profiling measures inference time and memory footprint for each truncation choice, targeting practical deployment constraints such as maximum 2GB storage and real-time processing on a single CPU core.

Comparisons are made against leading baselines including RawNet2, RawNet3 (CNN on raw audio), RawGAT (graph attention), and AASIST (spectro-temporal graph attention with full Wav2Vec2 fine-tuning). Results demonstrate the truncated SSL model at layer 7 outperforms all these baselines in OOD EER while requiring fewer parameters and significantly less compute.

Finally, the model is integrated into a Chrome browser extension leveraging ONNX runtime for inference. The extension processes the first 5 seconds of locally loaded audio entirely on-device, respecting privacy and enabling end users like journalists to detect audio deepfakes efficiently and securely. Performance tests on consumer CPU hardware (Ryzen 7 and Intel i3) confirm inference latency of 3.4-4.2 seconds per 5-second audio sample.

The paper provides detailed dataset statistics, layer-wise results, and efficiency tradeoff curves. However, no fine-tuning of Wav2Vec2 was performed; study focuses on frozen representations and the simple linear classifier for deployment feasibility. Code and browser plugin are publicly available to encourage reproducibility and real-world adoption.

Technical innovations

• Proposal to truncate self-supervised Wav2Vec2 features at an intermediate transformer layer (layer 7) for optimal tradeoff between detection accuracy, generalization, and computational efficiency.
• Demonstration that simple logistic regression on frozen SSL features from early-to-mid layers outperforms complex architectures like AASIST and RawGAT for out-of-domain audio deepfake detection.
• Integration of an on-device audio deepfake detection pipeline into a privacy-preserving browser extension using the ONNX runtime, enabling local inference without audio data uploading.
• Systematic evaluation across six diverse out-of-domain datasets showing mid-layer SSL representations retain discriminative deepfake artifacts better than deeper layers.

Datasets

• ASVspoof19 — 121,000 utterances — public
• ASVspoof21 DF — 593,000 utterances — public
• Fake Or Real (FoR) — 195,000 utterances — public
• MLAAD — 154,000 utterances — public
• In the Wild (ITW) — 31,000 utterances — public
• TIMIT-TTS — 20,000 utterances — public
• WaveFake — 136,000 utterances — public

Baselines vs proposed

• RawNet2: OOD EER = 23.9% vs Wav2Vec2-layer7: 8.4%
• RawNet3: OOD EER = 39.6% vs Wav2Vec2-layer7: 8.4%
• RawGAT: OOD EER = 27.0% vs Wav2Vec2-layer7: 8.4%
• AASIST: OOD EER = 11.3% vs Wav2Vec2-layer7: 8.4%
• Wav2Vec2 full model (layer 24): OOD EER = 16.9% vs truncated layer 7: 8.4%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.30780.

Fig 1: Audio Deepfake Detection Chrome Extension: (a)

Fig 2 (page 1).

Fig 2: Tradeoff between Out-Of-Domain performance, infer-

Fig 3: Out-of-domain EER vs model parameters for tested

Limitations

• Evaluation is based on frozen SSL features without fine-tuning; fine-tuning might improve accuracy but increase computational costs.
• Experiments focus on English and several other languages but may not cover all languages or rare dialects in real-world scenarios.
• Robustness to adversarially crafted audio deepfakes was not explicitly tested; only natural deepfakes from datasets were considered.
• The deployed browser extension currently processes the first 5 seconds of audio only, which may limit detection on shorter or longer samples.
• The study uses classical EER metric without reporting other uncertainty or calibration metrics relevant to high-stakes deployments.
• Real-time CPU-only inference was demonstrated on desktop CPUs; further evaluation on mobile or embedded hardware is needed.

Open questions / follow-ons

• How would fine-tuning the truncated Wav2Vec2 layers affect the trade-off between accuracy and efficiency for on-device detection?
• Can the approach generalize to more low-resource or underrepresented languages beyond those in the evaluation datasets?
• How robust is the truncated SSL-based model to adversarial audio attacks designed specifically to evade on-device detectors?
• What optimizations are necessary to port the model to mobile or embedded devices with more stringent resource constraints?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this work underscores the feasibility of accurate audio deepfake detection completely on-device using truncated self-supervised features with minimal computational resources. The demonstrated privacy-preserving approach using a lightweight linear classifier on truncated SSL embeddings offers a blueprint for deploying real-time audio authenticity checks directly within browsers or mobile apps without relying on cloud servers. This mitigates risks associated with data exposure or latency that cloud-based detection solutions can incur.

Moreover, the detailed analysis of layer-wise feature efficacy in SSL models informs exploitations of mid-level representations for robust generalization to unseen deepfake generation methods. The practical integration as a user-friendly Chrome extension further exemplifies how secure, transparent, and accessible audio verification tools can be embedded into end-user workflows—important for hostile environment bot-defense tasks that require trustworthy media verification under constrained compute budgets.

Cite

@article{arxiv2606_30780,
  title={ Detecting Audio Deepfakes on the Edge:Lightweight SSL-Based Detection in a Browser Plugin },
  author={ Octavian Pascu and Dan Oneata and Horia Cucu and Nicolas M. Muller },
  journal={arXiv preprint arXiv:2606.30780},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.30780}
}

Read the full paper

• arXiv:2606.30780 (abstract)
• arXiv:2606.30780 (PDF)