Sign in the Air to Unlock: An Interface for authentication in Virtual and Augmented Reality Powered by Point-Voxel Cross-Attention Network
Source: arXiv:2607.01435 · Published 2026-07-01 · By Neda Abdolrahimi, Thiru Siddharth, Frank Sicongchen, Vir V Phoha
TL;DR
This paper addresses the challenge of user authentication within immersive Virtual and Augmented Reality (VR/AR) environments, where traditional methods like passwords or PINs disrupt immersion and require external devices. The authors propose a novel behavioral biometric method called "Sign in the Air to Unlock," which allows users to authenticate via natural, personal 3D in-air signatures captured by inertial sensors in VR controllers. To accurately model these complex 3D trajectories, they develop PV-Net, a Point-Voxel Cross-Attention Network that jointly leverages local motion dynamics from point cloud representations and global spatial structure from voxelized inputs using a dual-branch architecture and bidirectional multi-head cross-attention.
PV-Net is evaluated on two datasets: DeepAirSig, a public dataset with 1,800 signatures from 40 users collected by depth cameras, and ImmAirSig, a new dataset collected by the authors in real immersive VR using Meta Quest 2 controllers with 880 samples from 22 users. On DeepAirSig, PV-Net significantly improves over prior baselines achieving an Equal Error Rate (EER) of 2.5%, a 54.5% relative reduction from previous best. On ImmAirSig, representing the challenging, dynamic VR environment, it achieves a classification accuracy of 76%. These results demonstrate the feasibility of 3D signature behavioral biometrics for seamless, secure, and natural authentication directly integrated into VR/AR experiences, balancing usability and security without external sensors or devices.
Key findings
- PV-Net achieves an Equal Error Rate (EER) of 2.5% on the DeepAirSig dataset, a 54.5% reduction relative to the prior best baseline of 5.5% EER using PointCloud verification.
- On the ImmAirSig dataset collected with Meta Quest 2 controllers in immersive VR, PV-Net achieves 76% accuracy authenticating users.
- The dual-branch architecture combining point cloud and voxel representations outperforms single-branch variants: point cloud-only EER = 17.5%, voxel-only EER = 15.0%, fused PV-Net EER = 2.5%.
- Cross-attention mechanisms allow effective integration of local motion cues and global spatial patterns in 3D signature trajectories, improving discrimination of genuine vs forged samples.
- In-air signatures enable authentication in immersive VR/AR without requiring external hardware or breaking user presence, overcoming limitations of PINs, passwords, and wearable sensor-based methods.
- ImmAirSig dataset reveals challenges unique to VR, such as inconsistent controller grip, user orientation drift, and spatial instability, which moderately reduce accuracy vs camera-based data.
- PV-Net uses temporal context modeling via LSTM over point embeddings combined with multi-scale 3D CNN voxel encodings, capturing dynamic signatures better than prior static or single-modal approaches.
Threat model
The adversary is modeled as an impostor who attempts to impersonate a legitimate user by forging that user’s in-air signature gestures, potentially with access to observation or 3D representations of genuine signatures to practice imitation. The adversary does not have direct access to the IMU sensor signals or internal models, nor the capability to inject false sensor data. Attacks such as replay or advanced spoofing are not explicitly addressed.
Methodology — deep read
The paper addresses the problem of biometric authentication in immersive VR/AR by capturing 3D in-air signature trajectories from user hand movements detected by IMUs in headset controllers. The adversary is modeled as an impostor attempting forgery mimicking genuine signatures with no access to the inertial sensor data directly.
Data is sourced from two datasets: (1) DeepAirSig, a public depth camera-based dataset with 1,800 in-air signatures from 40 subjects, reduced to 35 after cleaning, using 10 train and 10 test samples per subject. (2) ImmAirSig, a novel dataset collected in VR using Meta Quest 2 handheld controllers from 22 subjects, each contributing 20 genuine and 20 forgery signatures. Trajectories are sequences of 3D coordinates standardized to length N=300 by padding/truncation.
PV-Net architecture is a dual-branch neural network with a point cloud branch and a voxel branch. The point cloud branch processes the raw sequence of points with a linear embedding layer (with layer norm, dropout, and ReLU), followed by a unidirectional LSTM to capture temporal dependencies in signing speed and acceleration. The voxel branch voxelizes the normalized 3D points into a 32 x 32 x 32 grid, encoding point density via a logarithmic transform, then uses two blocks of 3D CNN layers with batch normalization, ReLU, max-pooling, and adaptive average pooling to extract global spatial features. Output voxel feature maps are flattened into sequences.
A bidirectional multi-head cross-attention module models interactions between the point cloud (local) and voxel (global) feature sets. Point-to-Voxel attention lets each point attend to voxel tokens, incorporating global spatial context into local features. Voxel-to-Point attention lets voxel tokens attend to point features, enriching global representation with fine local details. Outputs are pooled (mean and max) and concatenated to form embedding vectors.
The fused feature vector passes through a linear classification layer (softmax) with cross-entropy loss and label smoothing (α=0.1). AdamW optimizer with learning rate 3e-4, weight decay 1e-4, gradient clipping (max norm 5), dynamic LR scheduler (ReduceLROnPlateau), and early stopping based on validation are used. Training and evaluation use held-out test sets with no cross-contamination.
Performance metrics include Equal Error Rate (EER) for verification tasks and classification accuracy for identification tasks. ROC curves, FAR/FRR analysis, and score distributions are analyzed. Ablations compare point-only, voxel-only, and full PV-Net fusion. Comparisons are made against prior baselines including Dynamic Time Warping (DTW) 3D, Deep One-Class SVDD, image-based verification, and prior point-cloud verification methods.
The methodology is transparent and reproducible though no code release is indicated. Data splitting strategies are clearly described. The ImmAirSig dataset is newly introduced and collected using commercial VR hardware to simulate realistic immersive authentication conditions. Overall, this method integrates spatiotemporal deep learning with cross-modal fusion specialized for the noisy, unconstrained nature of in-air VR signatures.
Technical innovations
- Development of PV-Net, a dual-branch deep neural network combining point cloud and voxel representations of 3D in-air signature trajectories with bidirectional cross-attention to fuse local and global features.
- Introduction of a cross-modal temporal learning design that integrates LSTM-based sequential modeling on point embeddings with multi-head attention between point and voxel features.
- Use of voxelization with logarithmic scaled point density encoding and multi-scale 3D CNN feature extraction to model global spatial patterns robust to intra-user variability.
- Application of a fully embedded inertial sensing-based in-air signature authentication method optimized for immersive VR/AR, requiring no external sensors or hardware beyond controllers.
Datasets
- DeepAirSig — 1,800 signatures from 40 subjects (public dataset from Malik et al.)
- ImmAirSig — 880 signatures (440 genuine + 440 forged) from 22 subjects (collected by authors in immersive VR using Meta Quest 2)
Baselines vs proposed
- DTW 3D (X,Y,Z): EER = 17.0% vs PV-Net: 2.5%
- Deep One-Class SVDD: EER = 20.7% vs PV-Net: 2.5%
- Image-based Verification: EER = 26.0% vs PV-Net: 2.5%
- PointCloud-based Verification: EER = 5.5% vs PV-Net: 2.5%
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.01435.
Fig 1: Overview of the PV-Net architecture. The input signing trajectory in 3D space is processed by a dual-branch cross-attention
Fig 2: Performance curves for the point cloud–only variant of PV-Net on DeepAirSig.
Fig 3: Performance curves for the voxel-only variant of PV-Net on DeepAirSig.
Fig 4: Performance curves for the full PV-Net model on DeepAirSig (EER = 2.5%).
Fig 5: Data collection and signing workflow in the immersive VR environment.
Fig 6: shows two 3D slots of in-air signature samples corresponding to the genuine (Figure 6a) and forged (Figure
Fig 7: Performance curves for the point cloud–only variant of PV-Net on ImmAirSig.
Fig 8: Performance curves for the voxel–only variant of PV-Net on ImmAirSig.
Limitations
- ImmAirSig dataset accuracy (76%) is substantially lower than DeepAirSig results (EER 2.5%), reflecting challenges of noisy VR data, inconsistent grips, and spatial drift.
- No adversarial robustness evaluation beyond naive forgery attempts; resistant to advanced mimicry or replay attacks remains unexplored.
- The model assumes IMU controller data with stable capture of hand motions; variation in hardware or tracking quality could degrade performance.
- No user longitudinal evaluation assessing the stability of signature biometrics over extended time or varied user states.
- Partial reliance on fixed-length trajectories (N=300 points) which may truncate or pad data, possibly losing subtle temporal nuances.
- Code and ImmAirSig dataset are not publicly released, which may limit reproducibility and external validation.
Open questions / follow-ons
- How robust is PV-Net against sophisticated imitation, replay, or adversarial attacks beyond naive forgeries?
- What is the long-term repeatability and behavioral stability of in-air signatures over days, weeks, or under varying user states?
- Can multi-modal fusion including eye-tracking or physiological signals improve authentication accuracy and spoof resistance in VR?
- How does PV-Net performance generalize across different VR hardware platforms with varying sensor qualities and user interaction styles?
Why it matters for bot defense
For bot-defense and CAPTCHA practitioners, this work highlights a promising direction of behavioral biometric authentication leveraging natural, embodied 3D gestures within immersive environments. The in-air signature modality provides a user-friendly and sensor-efficient alternative to traditional passwords or external token devices, aligning well with VR/AR interaction paradigms without breaking immersion. The dual-branch cross-attentional architecture offers a novel approach to fusing local motion features and global structure, which could inspire more robust gesture or motion-based recognition models for bot or spoof detection.
However, the moderate accuracy achieved in realistic VR settings and the lack of adversarial robustness evaluation suggest caution in deploying similar techniques without further hardening. For CAPTCHA-like challenges requiring liveness or uniqueness proofs in 3D interaction, integrating multi-modal sensors or continuous behavioral monitoring could enhance security. Overall, this paper provides useful architectural and dataset insights for building seamless, user-centric authentication in immersive platforms with potential applications in anti-bot measures leveraging gesture biometrics.
Cite
@article{arxiv2607_01435,
title={ Sign in the Air to Unlock: An Interface for authentication in Virtual and Augmented Reality Powered by Point-Voxel Cross-Attention Network },
author={ Neda Abdolrahimi and Thiru Siddharth and Frank Sicongchen and Vir V Phoha },
journal={arXiv preprint arXiv:2607.01435},
year={ 2026 },
url={https://arxiv.org/abs/2607.01435}
}