Physical surfaces make touch interactions in virtual reality precise, efficient, and bimanual
Source: arXiv:2607.02430 · Published 2026-07-02 · By Wen Ying, Seongkook Heo
TL;DR
This paper addresses the challenge of achieving precise and efficient touch interactions in virtual reality (VR), where mid-air hand gestures lack tactile feedback and physical support surfaces, limiting accuracy in tasks such as selecting, tracing, and sketching. The authors systematically investigate how different types of hand-grounded haptic feedback—namely no haptic feedback (visual only), tactile feedback (vibrotactile and pressure via a haptic glove), and a portable physical surface—affect user performance and behavior in precision-demanding VR tasks. The study finds that interacting with a physical surface significantly outperforms the other conditions, delivering substantially improved selection accuracy, tracing precision and speed, and sketching quality. Participants also exhibited greater bimanual coordination when using the physical surface, with the non-dominant hand stabilizing the surface while the dominant hand performed detailed manipulations. These behaviors correlated with subjective preferences for physical surfaces due to increased confidence and control.
The work contributes clear evidence that physically grounded surfaces provide critical proprioceptive and tactile cues that enable users to perform complex, precise hand interactions more effectively in VR, compared to purely visual or synthetic tactile feedback. The study highlights the synergy between physical surfaces and asymmetric bimanual coordination for improving touch accuracy and efficiency, offering practical insights for designing VR input modalities that better support professional and creative applications.
Key findings
- Physical surfaces improved selection accuracy by 51.2% over tactile feedback and markedly over no haptic feedback.
- Tracing precision increased by 20.3%, tracing continuity by 86.8%, and tracing speed by 18.3% with physical surfaces versus tactile feedback.
- Stroke smoothness improved by 21.6%, stroke continuity by 21.2%, and sketch clarity by 17.5% on physical surfaces compared to tactile feedback.
- Participants’ non-dominant hand moved ~100% greater distance holding and adjusting the physical surface during tracing and sketching tasks.
- Dominant hand reduced total movement but hovered ~150% higher and approached faster in the physical surface condition, indicating bimanual coordination.
- Vibrotactile plus pressure feedback on gloves outperformed no haptic feedback but was substantially less effective than physical surfaces across all metrics.
- Subjective ratings from 30 external reviewers confirmed that sketch quality was highest with physical surface interactions.
- Learning effects were controlled via counterbalancing; improvements aligned consistently with haptic feedback modality rather than practice.
Threat model
n/a — This is a human-computer interaction study focusing on performance and perceptual effects of haptic feedback modalities in VR, not a security or adversarial setting.
Methodology — deep read
The study tested three haptic feedback conditions impacting finger-based VR touch tasks: (1) Bare (no haptic feedback, visual only), (2) Tactile (vibrotactile plus pressure feedback via a modified SenseGlove with LRA and servo-actuated pulley), and (3) Physical (user holds a 255×205 mm acrylic plate as a tangible surface with capacitive touch sensing). The dominant right index finger performed interactions tracked by SenseGlove and OptiTrack system at high precision (mean error ~0.28 mm), with a virtual surface rendered visually in VR. Twelve right-handed university participants (balanced gender, limited VR/VR-sketching experience) completed within-subject experiments over three tasks: selection (Fitts’ law 2D tasks with 4 mm and 5.5 mm targets), tracing (triangle and circle shapes), and sketching (building, fruit shape, freestyle). Each participant experienced all haptic conditions in counterbalanced order.
Interaction data (finger 3D position, surface contact) were logged at 60 Hz. Selection performance was measured by Fitts’ throughput and error rate. Tracing was evaluated with mean projected deviation (precision), mean fairness deviation (stroke smoothness), number of breakpoints (continuity), and completion time. Sketching quality was assessed subjectively by 30 MTurk crowdworkers rating smoothness, continuity, and clarity on a 10-point scale and ranking sketches from each haptic condition.
The tactile feedback combined vibrotactile contact vibrations (235 Hz decay pulses) at the fingertip plus pressure via a servo-controlled pulley pulling a string on the finger joint proportional to penetration depth (maximum 13 N force). The physical surface condition allowed natural finger bending and bimanual holding with the non-dominant hand, while other conditions used a narrow bar for hand reference but no tactile contact.
Tracking data also allowed analysis of bimanual hand trajectories and coordination. Statistical comparisons quantified performance improvements between conditions, controlling for learning effects through counterbalancing. The software ran in Unity on standard consumer VR hardware (Oculus Rift S, SenseGlove DK1, OptiTrack at 360 Hz). Participants trained briefly per task-condition to familiarize. All experimental procedures were IRB-approved with monetary compensation.
One concrete example: during selection tasks, participants used their right index finger to select a sequence of circular targets spaced 50 mm apart on the virtual surface; hit/miss and timing data were recorded. Under the physical surface condition, the finger had a real acrylic plate to rest on and push against, enabling more precise tip placement compared to mid-air or glove-based pressure feedback. This difference manifested in throughput and error rate improvements by 51.2% and fewer misses respectively, highlighting how grounded surface tactile cues improve targeting accuracy.
Technical innovations
- Systematic comparison of three haptic modalities (no feedback, vibrotactile+pressure feedback glove, and portable physical surface) on precision VR tasks.
- Integration of a lightweight, portable acrylic physical surface with capacitive touch sensing for bimanual VR input, contrasting with prior bulky or fixed haptic setups.
- Measurement and analysis of asymmetric bimanual coordination facilitated by physical surface holding, including detailed hand trajectory metrics.
- Adaptation of HapThimble-style combined vibrotactile and pressure feedback into a flexible glove allowing free finger articulation rather than rigid cylinders.
Datasets
- User interaction logs (finger and hand trajectories, contact points) from 12 participants across 183 trials per participant — proprietary to study, not publicly released.
- Sketch quality ratings from 30 Amazon Mechanical Turk workers — proprietary, aggregated ratings not publicly available.
Baselines vs proposed
- No Haptic Feedback (Bare): Fitts’ throughput = baseline (TP not numerically specified); Error rate higher than others.
- Tactile Feedback (vibrotactile + pressure): Selection accuracy improved by approx. 32% over Bare; Tracing precision improved by approx. 15% over Bare.
- Physical Surface: Selection accuracy improved by 51.2% over Tactile; Tracing precision improved by 20.3%, tracing continuity by 86.8%, and speed by 18.3% compared to Tactile; Sketch stroke smoothness improved by 21.6%, continuity by 21.2%, and clarity by 17.5% over Tactile.
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.02430.
Fig 1: This study investigates the effects of visual feedback, tactile feedback,
Fig 2: System Design Overview.
Fig 3: The experimental setup (a) and three haptic feedback conditions: (b) no haptic feedback (Bare); (c) tactile feedback (Tactile); (d) physical surface
Fig 4: Virtual haptic feedback activation: (a) The LRA generates contact
Fig 5: Control signals to generate (a) contact and (b) grain vibrations, and the corresponding acceleration measurements for (c) contact and (d) grain vibrations.
Fig 6: Screenshots of the two selection tasks performed in this study: (a)
Fig 7: Screenshots of the two tracing tasks performed in this study: (a) a
Fig 8: Screenshots of the three sketching tasks performed in this study: (a)
Limitations
- Small sample size (N=12), all right-handed university students with limited prior VR sketching experience.
- The physical surface was a single rigid acrylic plate; results may differ with other surface materials or sizes.
- The tactile feedback implementation, while adapted from prior work, may not represent the full range or quality of haptic glove technologies available.
- No long-term adaptation or fatigue effects were analyzed beyond brief training per condition.
- Tasks were limited to selection, tracing simple shapes, and freeform sketching; other VR interactions or 3D manipulations were not tested.
- The experimental setup fixed participants seated and did not explore mobile or standing usage environments.
Open questions / follow-ons
- How do different portable surface materials and textures impact touch precision and user preference in VR?
- Can similar performance benefits of physical surfaces be replicated for complex 3D object manipulations or multi-finger gestures in VR?
- How does prolonged use of physical surfaces affect fatigue, comfort, and long-term learning/adaptation compared to mid-air or tactile-feedback-only conditions?
- Can the principles of asymmetric bimanual coordination observed be extended to inform design of VR input devices supporting richer multi-hand collaboration?
Why it matters for bot defense
For bot-defense and CAPTCHA practitioners aiming to design human verification systems leveraging VR or touch interactions, this study highlights the substantial impact of haptic grounding on precision and user behavior. Systems that rely purely on mid-air gestures without tactile feedback may suffer from reduced user accuracy and slower interaction times, potentially leading to degraded human experience or false rejection of legitimate users. Incorporating physical surfaces (or realistic haptic proxies) as part of the interaction modality can improve the reliability and naturalness of touch inputs. Moreover, the demonstrated bimanual coordination enabled by physical surfaces suggests that tasks designed to require coordinated two-hand input may be a stronger behavioral signal distinguishing humans from bots, which commonly struggle with nuanced multi-finger/haptic coordination. Thus, integrating haptically grounded, bimanual touch tasks in VR CAPTCHA or bot-defense systems could enhance robustness and user satisfaction compared to visual-only or vibrotactile-only approaches. However, practical tradeoffs of requiring physical proxies versus purely glove-based or mid-air input need careful consideration.
Cite
@article{arxiv2607_02430,
title={ Physical surfaces make touch interactions in virtual reality precise, efficient, and bimanual },
author={ Wen Ying and Seongkook Heo },
journal={arXiv preprint arXiv:2607.02430},
year={ 2026 },
url={https://arxiv.org/abs/2607.02430}
}