An Enhanced Source-Free Unsupervised Domain Adaptation Framework for Cross-Dataset EEG Emotion Recognition via Predictive Coding and Test-Time Training
Source: arXiv:2606.28202 · Published 2026-06-26 · By Md Niaz Imtiaz, Naimul Khan
TL;DR
This paper addresses the challenge of cross-dataset EEG-based emotion recognition under domain shifts caused by inter-subject variability and dataset differences. Standard unsupervised domain adaptation (UDA) approaches require access to source data during adaptation, which is often unavailable in practice due to privacy constraints. To overcome this, the authors propose an enhanced source-free unsupervised domain adaptation (SF-UDA) framework tailored for EEG emotion recognition that does not require source data during adaptation. Their framework incorporates a novel non-contrastive predictive coding self-supervised pretraining to capture temporal dependencies in EEG, a dual-stage adaptation process (Multi-Loss Adaptive Regularization and Localized Consistency Learning) to stabilize and refine pseudo-labels, and a lightweight uncertainty-aware test-time training scheme that selectively updates model components on ambiguous target samples during inference. Evaluated on three benchmark EEG emotion datasets (DEAP, SEED, DREAMER) in cross-dataset transfer scenarios, their method consistently outperforms prior SF-UDA baselines, achieving up to 69.56% accuracy and demonstrating improved class-wise consistency, robustness against noisy pseudo-labels, and efficient online adaptation.
Key findings
- Proposed SF-UDA framework achieves 69.56% accuracy on DEAP → SEED and 61.38% on SEED → DEAP for binary emotion classification, surpassing prior SF-UDA baselines including the authors' own previous method by 3-4%.
- On DREAMER as target dataset, method obtains 68.90% (SEED → DREAMER) and 63.03% (DEAP → DREAMER) accuracy, exceeding prior SF-UDA results by up to 4.16%.
- Multi-Loss Adaptive Regularization (MLAR) reduces classifer disagreement and enforces entropy minimization and balanced class predictions under noisy pseudo-labels.
- Localized Consistency Learning (LCL) improves local neighborhood prediction consistency by intersecting feature-space and class probability similarity.
- Uncertainty-based test-time training selectively adapts normalization and attention gate layers online only on high-entropy, high-reconstruction-error samples, improving final inference robustness with minimal computation.
- Ablation studies show removing MLAR or test-time training degrades performance by approximately 3-5% accuracy, confirming their individual contributions.
- Power Spectral Density (PSD) features outperform Differential Entropy (DE) features in most settings for EEG representation.
- t-SNE visualization demonstrates improved feature cluster compactness and separation between emotion classes after adaptation.
Threat model
Adversary model assumes no access to source domain raw data due to privacy constraints but observes only unlabeled target EEG data and a pretrained source model. The adversary cannot manipulate source domain data or labels and is challenged by substantial domain shifts preventing direct alignment. The framework assumes the attacker does not perform active adversarial attacks such as input perturbations, focusing on unsupervised adaptation under natural shifts.
Methodology — deep read
The authors consider cross-dataset EEG emotion recognition as a source-free unsupervised domain adaptation problem, where a model pre-trained on labeled source domain data (source EEG samples and labels) must adapt to unlabeled target domain data without access to source samples during adaptation.
Data originates from three public EEG emotion datasets (DEAP, SEED, DREAMER) preprocessed to extract Power Spectral Density (PSD) features from 2-second overlapping windows across five frequency bands and common EEG channels (32 for DEAP/SEED, 14 for DREAMER). Features are normalized to [-1,1]. Labels are mapped for binary (positive/negative) and three-class tasks.
The proposed neural architecture comprises: a feature generator applying learnable input normalization, axis-wise attention gates, Conv2D and multiscale convolutional blocks to extract hierarchical EEG features; a reconstructor module (decoder) used during pretraining and test-time training to perform predictive coding by reconstructing future PSD representations; and two parallel classifiers producing averaged predictions, with discrepancy loss between them to enhance robustness.
Pretraining optimizes a composite loss combining weighted cross-entropy for emotion classification, classifier discrepancy via Euclidean distance between classifier outputs, and a predictive coding loss that reconstructs next-segment PSD features from current input latent features. This self-supervised predictive coding encourages temporally predictive, domain-invariant representations without requiring contrastive learning or negative samples.
After pretraining, the model adapts to unlabeled target data in two stages: Multi-Loss Adaptive Regularization (MLAR) and Localized Consistency Learning (LCL). MLAR generates pseudo-labels for target samples by assigning each to the nearest class centroid computed with prediction confidence weighting. A reliable subset of samples satisfying label agreement and confidence thresholds is used to optimize a combined loss of pseudo-label agreement, Jensen-Shannon divergence to reduce classifier discrepancy, entropy minimization, and target distribution class balance, with centroids updated iteratively each epoch. LCL enforces local consistency by selecting reliable neighbors for each sample based on intersection of feature-space and classifier-output nearest neighbors, minimizing prediction discrepancies within the neighborhood.
During inference, a lightweight uncertainty-based test-time training mechanism identifies uncertain samples via prediction entropy and predictive reconstruction error statistics within each batch. Only these uncertain samples are used to adapt model normalization layers and attention gates online by minimizing the sum of entropy and reconstruction loss for a few gradient steps, enabling dynamic refinement of the model on ambiguous target data while reducing unnecessary computation on confident samples.
Evaluation is conducted on cross-dataset transfer scenarios across DEAP, SEED, and DREAMER for both binary and multi-class emotion classification. Metrics include accuracy, precision, recall, F1 score, and confusion matrices. Multiple SF-UDA baselines including the authors' prior method and several recent SF-UDA approaches from EEG and computer vision domains are reimplemented for fair comparison. Ablation studies examine the effects of removing MLAR, LCL, and test-time training components.
Experiments run on a standard Linux server with NVIDIA Tesla T4 GPU using PyTorch. Hyperparameters governing loss weights, confidence thresholds, and update iterations are tuned empirically. The source code and trained model details were not specified as publicly released in this version.
A representative end-to-end example: Given source-labeled PSD EEG data from DEAP, the model is pretrained with classification, classifier discrepancy, and predictive reconstruction losses. Then, source data is discarded. Target data from SEED (unlabeled) is passed to compute centroids and pseudo-labels. MLAR uses confident pseudo-labeled samples to optimize multi-loss functions and update centroids. LCL refines neighborhood consistency. During test-time inference on SEED, uncertain samples identified by high entropy and reconstruction loss selectively update normalization and attention weights for improved predictions. This pipeline leads to higher accuracy and stable adaptation despite the large domain gap and no access to source data.
Technical innovations
- Non-contrastive predictive coding–based self-supervised pretraining module that models temporal EEG dynamics without requiring negative pairs or contrastive loss.
- Multi-Loss Adaptive Regularization combining pseudo-label agreement, classifier discrepancy Jensen-Shannon divergence, entropy minimization, and class distribution balancing to stabilize noisy pseudo-labels during target adaptation.
- Localized Consistency Learning using intersection of feature-space and classifier-output nearest neighbors to enforce robust local prediction consistency under domain shift.
- Uncertainty-guided lightweight test-time training that selectively adapts normalization and attention gate layers online for uncertain target samples using combined entropy and predictive reconstruction losses.
Datasets
- DEAP — 32 subjects, 40 trials each — public
- SEED — 15 subjects, 3 sessions each — public
- DREAMER — 23 subjects, 18 trials each — public
Baselines vs proposed
- Ahmed et al. [12]: DEAP → SEED binary accuracy = 61.33% vs proposed: 69.56%
- Du et al. [13]: SEED → DEAP binary accuracy = 48.74% vs proposed: 61.38%
- Ragab et al. [18]: SEED → DREAMER binary accuracy = 58.26% vs proposed: 68.90%
- Zhao et al. [19]: DEAP → DREAMER binary accuracy = 53.75% vs proposed: 63.03%
- Authors' previous SF-UDA [21]: SEED → DEAP binary accuracy = 58.99% vs proposed: 61.38%
- Proposed method achieves highest F1-scores with 72.06% for positive emotion (SEED → DEAP) and 70.98% (DEAP → SEED).
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.28202.
Fig 1: Overall architecture of the proposed network. The feature generator integrates learnable input normalization,
Fig 6: Distributions of SHAP values for representative subjects under cross-dataset settings: (a) SEED →DEAP
Fig 7: Multi-domain visualization of the top 12 EEG electrodes: (a) temporal signal evolution, (b) frequency-
Fig 4: Confusion matrices comparing the proposed method with the strongest SF-UDA baselines on SEED →
Fig 5: t-SNE visualization of feature distributions before and after adaptation: (a) SEED →DEAP and (b) DEAP
Limitations
- No explicit adversarial robustness evaluation against adaptive attackers or intentional perturbations.
- Performance improvements moderate for multi-class tasks on some transfer directions, e.g. DEAP → SEED multi-class accuracy slightly lower than prior method.
- Framework tested only on three public EEG datasets; generalization to clinical or real-world wearable device data not validated.
- Source code and pretrained models are not explicitly stated as publicly released, limiting reproducibility.
- Inferences rely on handcrafted confidence thresholds and hyperparameters possibly dataset-specific; no reported automatic tuning.
- Test-time training adapts only normalization and attention modules, leaving other layers fixed, which may limit adaptation potential under severe shifts.
Open questions / follow-ons
- How does the method perform under adversarially perturbed EEG signals or intentional spoofing attacks?
- Can predictive coding pretraining be extended to multimodal affective signals beyond EEG for enhanced cross-domain generalization?
- What is the impact of adapting more model components, or all layers, during test-time training in terms of performance and stability?
- Can the hyperparameters for confidence thresholds and pseudo-label filtering be dynamically or automatically tuned to improve adaptation robustness?
Why it matters for bot defense
For bot-defense and CAPTCHA practitioners working with EEG or other biosignal modalities, this paper provides a rigorous source-free unsupervised domain adaptation pipeline that enables privacy-preserving adaptation across heterogeneous data domains without source access. The integration of self-supervised predictive coding for robust temporal feature learning, combined with noise-robust pseudo-label refinement and selective test-time online updates, offers a promising template for designing adaptive authentication or liveness detection systems resilient to dataset shifts and user variability. The uncertainty-based selective test-time training mechanism is particularly useful for balancing inference efficiency with robustness, a key consideration for real-time security systems. However, the approach is tailored to EEG emotion recognition, so practitioners should consider domain-specific feature extraction and shift characteristics when applying similar strategies to behavioral biometrics or brain-computer interface-based CAPTCHA defenses. Finally, rigorous adversarial evaluations remain an open area to better understand robustness against malicious attempts to subvert adaptation.
Cite
@article{arxiv2606_28202,
title={ An Enhanced Source-Free Unsupervised Domain Adaptation Framework for Cross-Dataset EEG Emotion Recognition via Predictive Coding and Test-Time Training },
author={ Md Niaz Imtiaz and Naimul Khan },
journal={arXiv preprint arXiv:2606.28202},
year={ 2026 },
url={https://arxiv.org/abs/2606.28202}
}