DriveFace: A Cross-Spectral Through-Glass Face Dataset for On-the-Move Vehicular Border Control

Source: arXiv:2607.13515 · Published 2026-07-15 · By Anjith George, Luis Luevano, Alain Komaty, Zeina Al Amine, Vidit Vidit, Sebastien Marcel

TL;DR

DriveFace addresses the challenge of face recognition for on-the-move vehicular border control where subjects are captured through automotive glass, often under difficult conditions such as motion blur, varying illumination, spectral differences (visible enrollment vs near-infrared probes), window tint, and occlusions. Existing face datasets largely focus on controlled environments and do not capture this realistic and operationally critical scenario. DriveFace provides a new publicly available benchmark dataset that pairs smartphone-acquired visible-spectrum reference images with near-infrared video probes captured through vehicle windows under varying conditions (indoor, outdoor, simulation of tint levels), along with a presentation attack subset for anti-spoofing research. Through comprehensive baseline experiments, the authors show that state-of-the-art face recognition models suffer significant performance degradation in the most challenging protocols involving tinted glass and spectral mismatch, highlighting the need for domain adaptation and robust model design. Similarly, presentation attack detection models perform well on known attacks but generalize poorly to previously unseen print and mask attacks under NIR through-glass capture. Overall, DriveFace fills an important gap by providing a large-scale, richly annotated dataset with realistic cross-spectral, through-glass conditions and adversarial attacks representative of real operational border biometrics.

Key findings

• DriveFace dataset contains 70 subjects with paired RGB smartphone reference and NIR through-glass probe videos, collected over two sessions to capture natural variability.
• AdaFace achieves best face recognition performance with 99.45% AUC and 2.69% EER in outdoor protocol, but drops to 96.23% AUC and 8.26% EER in the simulated tint scenario.
• xEdgeFace cross-spectral adaptation improves EER from 5.37% to 4.18% in outdoor, and VR@FAR=1% from 91.35% to 93.12%, showing domain adaptation benefits.
• Face recognition performance improves with higher visible light transmission (VLT) through glass; clear glass yields near 100% AUC while darkest tint (T05) yields ~98% AUC.
• Presentation attack detection models achieve very low Average Classification Error Rate (ACER) ~0.5% in known-attack (grandtest) protocol but ACER rises to 18.4-45.8% for unseen print and unseen mask attack protocols.
• Mask attack detection under unseen setting is particularly challenging; DinoV2 achieves best unseen mask ACER of 26.20% but overall poor generalization is noted.
• Lightweight models (EdgeFace, xEdgeFace) perform competitively with much larger backbones (AdaFace, LVFace), indicating a practical trade-off for deployment in resource-constrained border control systems.
• Rich metadata including tint level, illumination, pose, vehicle speed support fine-grained performance analyses and understanding of degradation factors.

Threat model

The adversary is an impostor or attacker attempting to deceive the vehicular border control biometric system either through presenting fake biometric traits (print, replay, or mask attacks) or by matching incorrect identities under natural cross-spectral and environmental domain shifts. The attacker can only present physical attacks visible or partially visible in NIR imaging through vehicle glass. They cannot realistically alter enrolled reference data or manipulate the biometric system internals.

Methodology — deep read

The study sets out to create and benchmark a cross-spectral face recognition dataset simulating on-the-move vehicular border control capture conditions.

1. Threat Model & Assumptions: The adversary is a potential impostor or presentation attacker attempting to circumvent face recognition or anti-spoofing in realistic border scenarios. The models must handle cross-spectral enrollment (visible RGB) vs probe (NIR) biometric samples captured with through-glass degradations, motion blur, variable lighting, and occlusions, under uncontrolled head poses and viewing angles. Attackers may present print, replay, or mask-based presentation attacks.

2. Data Collection: The DriveFace dataset contains 70 consenting volunteers, each recorded in two sessions approximately two months apart to capture natural appearance variability (e.g. facial hair, makeup). Pre-enrollment data were acquired with front-facing smartphone RGB cameras (iPhone 12, Samsung Galaxy S9) capturing videos with controlled head motion to provide enrollment/reference.

Probes were captured using a VIDAR PAX near-infrared sensor positioned outside vehicles recording subjects through automotive glass. Three probe protocols were used: outdoor real vehicle (multiple viewpoints 0°, 45°, 90°; stationary and moving vehicles), indoor real vehicle (controlled illumination and position), and indoor simulated car captures with automotive glass panels of varying tint levels (Visible Light Transmission 5-45%) and illumination powers. Motion blur, reflections, and spectral distortions are present.

Presentation attacks were recorded in both indoor and outdoor vehicle settings with print, replay, and mask attack instruments applied to bona fide or external identities. Variations include printer type, paper finish, mask materials (silicone, resin, paper, plastic), poses, and illuminations.

3. Architecture / Algorithms: For face recognition, four open-source baseline models were evaluated: AdaFace (ResNet-100, adaptive margin loss, quality-aware training), LVFace (Vision Transformer ViT-L), EdgeFace (lightweight hybrid CNN-Transformer), and xEdgeFace (contrastive self-distillation cross-modal adaptation built on EdgeFace). xEdgeFace was further fine-tuned on DriveFace training data to address domain adaptation.

For presentation attack detection (PAD), multiple models were evaluated including DeepPixBiS (DenseNet with pixel-wise binary supervision), CLIP vision encoders (ViT-B/32 frozen and fully fine-tuned), DinoV2 (ViT-B/14 with frozen backbone), ConvNeXtV2-Tiny, and EfficientNet-B0.

4. Training Regime: FR pretrained models were mostly used out-of-the-box for baseline comparisons, except xEdgeFace which was fine-tuned on the training split (~60% of subjects). For PAD, models were trained from scratch on DriveFace-PAD with balanced sampling and data augmentation (horizontal flip, RandAugment, rotations, color jitter) for 100 epochs with learning rate 1e-4 and weight decay 1e-6 on NVIDIA RTX 3090.

5. Evaluation Protocol: Data was split into disjoint train (60% IDs) and test (40% IDs) sets with cross-session evaluation (enrollment and probe from different sessions). Three protocols evaluated: outdoor (real vehicle external NIR), indoor car, and indoor simulation with tint variation. Metrics reported include AUC, EER, rank-1 identification rate, and verification rates at FAR=1%.

For PAD, three protocols evaluate known attacks (grandtest) and unseen print or mask attacks to assess generalization. Metrics per ISO/IEC 30107-3 include APCER, BPCER, ACER, and EER.

6. Reproducibility: The dataset, protocol definitions, and baseline code are publicly released, enabling replication and further research on cross-spectral through-glass face biometrics under operational vehicular border control conditions.

Concrete example: In the outdoor protocol, subjects seated inside a moving vehicle were recorded from 0°, 45°, and 90° angles with NIR sensor through clear glass. Enrollment used frontal RGB images from smartphones collected 2 months prior. Face detection and alignment produced 112x112 crops fed into AdaFace pretrained model for embedding extraction. Cosine similarity verified probe-face vs enrolled identity. AdaFace yielded 99.45% AUC and 2.69% EER despite cross-spectral and through-glass artifacts.

In contrast, in the simulation protocol with dark tint (T05) glass, the performance dropped to 96.23% AUC and 8.26% EER due to severe attenuation of NIR signal and increased distortions emphasizing the challenge.

Technical innovations

• DriveFace introduces a large-scale cross-spectral face dataset specifically for through-glass, on-the-move vehicular border control, capturing complex operational factors absent in prior datasets.
• Comprehensive pairing of high-quality RGB pre-enrollment smartphone videos with NIR probe videos under multiple conditions supports realistic heterogeneous face recognition benchmarking.
• Design of three distinct probe acquisition protocols, including realistic controlled and uncontrolled outdoor and indoor vehicle scenarios with simulated glass tint levels, enabling systematic degradation analysis.
• Development of a multi-modal presentation attack subset covering print, replay, and diverse mask attacks under varying illumination and view angles in the NIR domain through automotive glass.
• Adaptation of a lightweight contrastive self-distillation method (xEdgeFace) for effective cross-spectral domain alignment between visible enrollment and NIR probe face images, improving recognition robustness.
• Provision of rich metadata annotations per acquisition (tint level, illumination, pose, vehicle speed) facilitating detailed performance attribution and analysis of degradation factors.

Datasets

• DriveFace — 70 subjects, 3,870+ NIR video clips paired with multiple smartphone RGB enrollment videos — Publicly available at Idiap Research Institute repository
• DriveFace-PAD — 3,500+ presentation attack NIR videos spanning print, replay, and mask-based attacks with controlled lighting/tint conditions — Publicly available

Baselines vs proposed

• AdaFace: Outdoor AUC = 99.45% vs Simulation AUC = 96.23%
• xEdgeFace: EER outdoor 4.18% vs EdgeFace 5.37%, VR@FAR=1% 93.12% vs 91.35%
• AdaFace: Indoor car EER 3.11% vs Simulation EER 8.26%
• EfficientNet-B0 PAD grandtest ACER = 0.50% vs unseen mask ACER = 27.20%
• DeepPixBiS PAD grandtest ACER = 1.60% vs unseen print ACER = 18.40%
• DinoV2 PAD unseen mask ACER = 26.20% vs grandtest ACER = 2.20%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.13515.

Fig 1: Overview of the acquisition setup in DriveFace , includ-

Fig 2: Samples from a single subject in DriveFace . The fig-

Fig 3: Example presentation attack samples from DriveFace-

Fig 4 (page 1).

Fig 5 (page 1).

Fig 4: Age group distribution labeled by Fitzpatrick skin color

Fig 7 (page 3).

Fig 8 (page 3).

Limitations

• Dataset size is moderate (70 subjects) and may limit modeling of wider demographic or cross-device variability.
• Evaluation focuses on cross-session and cross-spectral variation but does not analyze long-term aging effects beyond two-month span.
• No adversarial attacker explicitly simulating spoof attempts beyond presentation attacks (e.g. morphing, makeup alteration) were tested.
• Presentation attack detection generalization to unseen attacks remains weak, highlighting unaddressed robustness gaps.
• Some protocols (especially simulated tint indoor) may not fully capture the complexity of real-world vehicular glass conditions and lighting variability.
• Potential demographic imbalance exists with only 6 dark skin tone subjects, limiting study of fairness across ethnicities.

Open questions / follow-ons

• How can face recognition models be further improved for robustness under extreme glass tint and illumination changes beyond current domain adaptation approaches?
• What are effective strategies for improving generalization of presentation attack detection models to unseen and novel spoof instruments in the NIR through-glass setting?
• Can multi-modal biometric fusion (e.g. face + iris or depth sensing) further improve recognition and anti-spoofing under vehicular border control constraints?
• How do demographic factors like skin tone and age impact cross-spectral recognition and PAD under through-glass conditions, and how to mitigate any biases?

Why it matters for bot defense

For bot-defense and CAPTCHA engineers, DriveFace offers valuable insights into the challenges of cross-spectral biometric recognition through optical degradations akin to adversarial noise or signal degradation scenarios encountered in automated attacks. It underscores the difficulty of robustly handling heterogeneous sensing modalities and domain shifts, a problem analogous to recognizing or discriminating challenging bot or spoof behaviors under varied environmental conditions. The inclusion of extensive metadata and controlled variations provides a blueprint for benchmarking model degradation and adaptation strategies, which can inspire improved robustness evaluations and defenses in CAPTCHA systems. Additionally, the demonstrated limitations in unseen presentation attack detection highlight the fundamental challenge of generalizing spoof detection beyond trained attack types—a concern relevant for maintaining bot detection efficacy against novel evasion techniques. Overall, this work encourages deeper, domain-specific evaluation and adaptation rather than reliance on generic, out-of-domain pretrained features when building real-world biometric or bot-defense systems.

Cite

@article{arxiv2607_13515,
  title={ DriveFace: A Cross-Spectral Through-Glass Face Dataset for On-the-Move Vehicular Border Control },
  author={ Anjith George and Luis Luevano and Alain Komaty and Zeina Al Amine and Vidit Vidit and Sebastien Marcel },
  journal={arXiv preprint arXiv:2607.13515},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.13515}
}

Read the full paper

• arXiv:2607.13515 (abstract)
• arXiv:2607.13515 (PDF)