Assessing Distribution Shift in Human Activity Recognition for Domain Generalization

Source: arXiv:2606.24781 · Published 2026-06-23 · By Rebecca Adaimi, Edison Thomaz

TL;DR

This paper addresses the critical challenge in Human Activity Recognition (HAR) of domain generalization under diverse distribution shifts inherent in real-world sensor data. Key data heterogeneities arise from differences in device type, sensor placement, sampling rate, and user behavior changes such as skill improvement. The authors systematically characterize and quantify these four distribution shifts through a two-dimensional metric comprising diversity shift and correlation shift, revealing that diversity shifts—representing unique features absent in some domains—dominate all types of distribution changes. They contribute a unified benchmark platform composed of multiple public HAR datasets and a newly collected juggling skill progression dataset to enable realistic evaluation of domain generalization methods.

The authors then conduct a comprehensive empirical study evaluating 28 state-of-the-art domain generalization algorithms spanning data augmentation, representation learning, and learning strategies, using their benchmark. Across all considered distribution shifts and datasets, these methods achieve only marginal improvements upon the strong empirical risk minimization (ERM) baseline. The results expose fundamental limitations in current approaches when applied to sensor-based HAR data exhibiting complex real-world heterogeneity. This is the first systematic exploration of domain generalization methods specifically in the context of sensor heterogeneity and contextual user behavior shifts for HAR, accompanied by an open-source benchmark to stimulate future research.

Key findings

• All four studied distribution shifts (device type, sensor placement, sampling rate, user behavior) are predominantly characterized by diversity shift rather than correlation shift, indicating presence of domain-unique features.
• Downsampling experiments simulate sampling rate shifts effectively, with PAMAP2 dataset creating domains at 10Hz, 50Hz, and 100Hz, and HHAR dataset creating domains ranging from 10Hz to 200Hz.
• The juggling dataset captures user behavior shifts quantitatively by varying number of juggling balls (3,4,5,7), reflecting skill progression as a domain shift.
• Across up to 28 domain generalization algorithms evaluated, ERM baseline performance is only marginally improved, highlighting limited existing method effectiveness on sensor-based domain shifts.
• Benchmark datasets used include PAMAP2 (100Hz, 3 sensor locations), DSADS (25Hz, 5 body locations), Opportunity (30Hz, 6 body locations), RealWorld (50Hz, 7 locations), HHAR (50-200Hz, multiple devices), and the newly collected juggling dataset.
• Neural network-based domain classifier trained to quantify diversity and correlation shifts leveraged feature extractor inspired by VGGish architecture with m=16 embedding dimensions.
• Multi-domain splits in benchmarks reflect realistic domain shifts with k≥3 domains per dataset to approximate real-world heterogeneity settings.
• Model evaluation averaged over 5 random seeds with stratified splitting and reported macro F1-scores on held-out target domains.

Threat model

The threat model assumes an unknown real-world domain shift manifested as changes in device type, sensor placement, sampling rate, or user behavior that alter the distribution of sensor data seen by the model at test time. The adversary in this context is the natural heterogeneity and variability of sensor data in deployment environments. The learner has access to multiple source domains during training but no data or labels from the target shifted domain, making adaptation impossible. The adversary cannot directly manipulate training data or inject malicious inputs but causes covariate and conditional distribution shifts inherent in HAR sensing scenarios.

Methodology — deep read

1. Threat Model & Assumptions: The paper considers a domain generalization setting where multiple labeled source domains (sensor-device or user environments) are available during training, but no access to any data from the target (test) domain is allowed. The adversary is an unknown real-world distribution shift embodied by domain heterogeneity in device type, sensor placement, sampling rate, or user behavior, with no possibility of target data adaptation or fine-tuning.

2. Data:

• Public datasets: PAMAP2 (9 subjects, 100Hz, 3 IMU sensor locations, multiple activities), DSADS (8 subjects, 25Hz, 5 body locations), Opportunity (4 subjects, 30Hz, 6 body locations), RealWorld (15 subjects, 50Hz, 7 locations), HHAR (multiple devices, 50-200Hz).
• Newly collected Juggling dataset: 9 participants wearing Apple Watches on both wrists, performing juggling sessions at increasing ball counts (3,4,5,7) to simulate skill improvement, plus non-juggling activities.
• Data is normalized per axis and segmented into sliding windows of 100 samples with 50% overlap.

3. Architecture/Algorithm: For shift quantification, a neural network with a VGGish-inspired feature extractor produces 16-dimensional embeddings from accelerometer windows. A domain classifier uses these embeddings plus labels to predict domain membership, enabling estimation of diversity and correlation shifts numerically.

For generalization experiments, classification models are trained on multiple source domains to predict activities. A wide range of 28 domain generalization algorithms are implemented via the DomainBed framework. These encompass empirical risk minimization (ERM), invariant risk minimization (IRM), group distributionally robust optimization (GroupDRO), Mixup augmentation, Fish gradient matching, and other recent representation and learning strategy methods. Models use cross-entropy loss, optimized with Adam.

4. Training regime: Networks are trained for 100 epochs with batch size 128, learning rate 1e-5 for shift quantification, with random stratified splits for train/val/test domains. Generalization experiments use DomainBed defaults, running 5 random seeds per setup to average results. Hyperparameter details per method are based on DomainBed recommendations.

5. Evaluation protocol: Domain generalization is evaluated by training on k-1 source domains and testing on the kth target domain across all splits, measuring macro F1 scores on activity classification. Diversity and correlation shifts are measured between domain pairs. Ablations examine each shift type separately. Baselines include ERM and full supervised training within-domain. Statistical variation is reported across random seeds.

6. Reproducibility: The authors provide an open-source benchmark platform for sensor-based HAR domain generalization with code and datasets where applicable, including the juggling dataset collected by authors. Exact hyperparameters and split seeds are documented. The HHAR dataset is public. Exact frozen weights are not detailed in the text.

Example end-to-end: Considering PAMAP2 sensor location shift, accelerometer data from wrist, ankle, and chest form three domains. The data is segmented and normalized. A model is trained on two sensor-location domains and tested on the third, evaluating macro F1 on held-out location. Diversity and correlation shifts are computed via domain classifiers trained to discriminate sensor location from features plus labels, quantifying how dissimilar domains are in feature-label space. The domain generalization algorithms are benchmarked on this split to assess robustness to sensor placement variability.

Technical innovations

• Systematic characterization of four distinct types of distribution shifts (device type, sensor placement, sampling rate, user behavior) in HAR via two-dimensional diversity and correlation shift metrics.
• Development of a unified benchmark platform combining multiple public HAR datasets and a novel sensor-based juggling dataset capturing user skill progression as a domain shift.
• Comprehensive empirical evaluation of 28 domain generalization algorithms from multiple methodological categories specifically in the sensor-based HAR setting.
• Identification and demonstration that current domain generalization methods only marginally outperform ERM on realistic sensor-based distribution shifts, revealing a critical gap in robustness.

Datasets

• PAMAP2 — 9 subjects, 18 activities, 3 sensor locations (wrist, ankle, chest), 100 Hz — Public
• DSADS — 8 subjects, 19 activities, 5 body locations, 25 Hz — Public
• Opportunity — 4 subjects, 5 daily activities, 6 sensor locations, 30 Hz — Public
• RealWorld — 15 subjects, 8 activities, 7 sensor locations, 50 Hz — Public
• HHAR — multiple subjects/devices, 6 locomotion activities, sensor data at 50-200 Hz, four smartphones and two smartwatches — Public
• Juggling dataset — 9 participants, accelerometer from Apple Watches on both wrists, juggling 3-7 balls plus non-juggling activities, 100 Hz — Collected by authors

Baselines vs proposed

• ERM baseline: macro F1 ranges from 0.6 to 0.8 depending on distribution shift and dataset.
• Invariant Risk Minimization (IRM): marginal improvements over ERM (~1-2% increase in macro F1) across shifts.
• Fish (gradient matching): small to negligible gains vs ERM (less than 2% improvement).
• GroupDRO: slight robustness gain over ERM on certain shifts but overall marginal.
• Mixup augmentation: similarly marginal gains, rarely exceeding 3% improvement.
• Overall: no evaluated method consistently outperforms ERM baseline by more than 3-5% macro F1 across all domains and shifts.

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.24781.

Fig 1: Accelerometer signals captured with varying (a) sensor location (wrist, ankle, chest), (b) Device type

Fig 2 (page 4).

Fig 3 (page 4).

Fig 4 (page 4).

Fig 2: Screenshots from video recorded sessions of a participant juggling 5 and 7 balls.

Fig 3: Domain Classifier model overview. The neural network extracts features from the accelerometer data

Limitations

• Distribution shifts studied cover key but not all realistic shifts; occlusions, environment/context changes could further impact HAR.
• Benchmark scenarios use accelerometer data only, ignoring multimodal sensor fusion which may aid generalization.
• User behavior shift analysis limited to juggling skill progression with 9 participants, relatively small sample size.
• Evaluation focuses on macro F1 but lacks detailed analysis of per-class or per-domain performance variations.
• No adversarial or deliberate attack evaluation of domain generalization methods presented.
• Statistical significance testing details are limited, reproducibility partly depends on external datasets.

Open questions / follow-ons

• How can multimodal sensor fusion be integrated to improve domain generalization across heterogeneous wearable data?
• Can meta-learning or continual learning approaches better address gradual user behavior or skill-based distribution shifts?
• What are the effects of adversarial or targeted distribution shifts on HAR domain generalization methods?
• How do distribution shifts impact specific HAR tasks beyond locomotion, such as fine motor activities or complex gestures?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this paper offers a detailed empirical investigation into domain generalization challenges on sensor-derived time-series data, a modality increasingly relevant due to the rise in wearable and mobile sensors. The finding that existing domain generalization approaches provide minimal improvement over naive baselines underscores the difficulty in building truly robust models for dynamic, heterogeneous data sources—analogous to challenges in detecting bot behavior across diverse devices and usage contexts. The benchmark and diversity/correlation shift metrics introduced could inspire more granular evaluation of robustness in other domains where distribution shifts pose risks to system security and reliability.

Practitioners could apply similar metrics to quantify shift types in biometric or behavioral signals used in bot-detection CAPTCHAs or sensor-based authentication. The systematic benchmarking approach also highlights the importance of evaluating across multiple realistic shift types when certifying models against evasion or spoofing attacks that exploit domain-specific sensor or user behavior variations.

Cite

@article{arxiv2606_24781,
  title={ Assessing Distribution Shift in Human Activity Recognition for Domain Generalization },
  author={ Rebecca Adaimi and Edison Thomaz },
  journal={arXiv preprint arXiv:2606.24781},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.24781}
}

Read the full paper

• arXiv:2606.24781 (abstract)
• arXiv:2606.24781 (PDF)