Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Users

Source: arXiv:2606.20482 · Published 2026-06-18 · By Haw-Shiuan Chang, Jeffrey Gomez, Mehul Patwari, Aryan Sajith, Hamed Zamani

TL;DR

This paper addresses the challenge of aligning Large Language Models (LLMs) to human preferences without relying heavily on scarce explicit feedback. Existing approaches depend on users providing explicit feedback, which is infrequent and costly to collect, and largely ignore implicit feedback signals. To explore new implicit signals, the authors build IFLLM, a novel dataset of mouse trajectories and eye gaze data recorded via webcams from 59 Mechanical Turk workers while interacting with LLM-generated responses across multi-turn question-answering tasks on diverse Wikipedia topics. The dataset couples these implicit feedback signals with explicit user preferences.

Using IFLLM, the authors demonstrate that incorporating implicit feedback—especially mouse movements that often reflect scrolling behavior—into reward models markedly improves preference prediction accuracy from roughly 55% to 64%. Reward models trained with these signals produce stronger alignment improvements via Direct Preference Optimization (DPO), nearly tripling relative gains in response quality across eight different LLM families compared to text-only reward models. Diverse user reading and interaction behaviors are also characterized, showing how response length and interface layout impact gaze and mouse patterns. The work highlights implicit mouse and gaze data as promising scalable signals for LLM alignment beyond explicit annotations.

This represents a first step toward multimodal user feedback-driven LLM alignment, with publicly released data and code encouraging future research in integrating implicit behaviors accessible through ordinary interfaces. The study bridges gaps between classic implicit feedback methods from search/recommender systems and LLM preference learning, showing improved reward modeling and resulting aligned LLM output quality in realistic user conversations.

Key findings

• The IFLLM dataset contains 1336 multi-turn questions from 59 distinct Mechanical Turk workers across 275 topics.
• In pairwise preference prediction, a random forest reward model using implicit feedback features (mouse + gaze) achieves 64.15% accuracy (±0.97%), outperforming text-only models capped around 55.5%.
• Removing mouse features reduces accuracy more than removing gaze features, indicating mouse movement is the stronger implicit feedback signal for longer responses.
• User behavior analysis shows mouse and gaze trajectories correlate strongly on longer responses but weakly on short ones.
• Applying rDPO + NLL alignment training using implicit feedback-based reward models improves relative response quality improvements from 0.12 (text-only baseline) to 0.35 (RF+IF) across 8 LLMs, nearly tripling quality gain.
• Improvements are especially prominent on recent open LLMs such as the Qwen series.
• Long responses elicit more mouse-based feedback reflecting scrolling, while gaze is more informative on short responses.
• Users disproportionately focus on early portions of responses, and mouse movement reflects the need to scroll through longer answers.

Threat model

The paper does not primarily focus on an adversarial threat model. It assumes benign users whose implicit feedback (mouse and gaze patterns) are used to improve LLM alignment. The adversary is outside scope; no threat from malicious manipulation, poisoning, or privacy attacks is considered. Users consent to data collection under IRB protocols. Implicit feedback signals are treated as honest but noisy indicators of user preference.

Methodology — deep read

Threat Model and Assumptions: The paper assumes a user-facing LLM system where the adversary is not the focus; rather, the goal is to improve LLM alignment by interpreting implicit user feedback signals during normal usage. It assumes access to user mouse movements and eye gaze via webcam, with user consent, but does not address adversarial manipulation of feedback.

Data Collection: The authors develop a publicly released crowdsourcing website that records mouse trajectories and webcam-based eye-gazing points using WebGazer.js during interactions with LLM responses. 59 qualified MTurk workers interacted with multiple popular LLMs (including Claude Sonnet 4.6, GPT-4o Mini, Llama 3.3 70B, DeepSeek V3) over 275 Wikipedia-derived topics. Users asked at least three questions per topic, with multi-turn conversations allowed. Data includes time-stamped gaze and mouse coordinates sampled every ~0.1 seconds, preference annotations (pointwise rating or pairwise comparisons), and post-task questionnaires. Data quality control filtered low-attention sessions based on gaze calibration and behavioral heuristics.

Reward Model Architecture: The main model is a Random Forest (RF) classifier trained to predict user preference between LLM responses. Features include text metadata (query length, response length), detailed summary statistics extracted from gaze and mouse trajectories (e.g., max character index seen, total gazing records, reviewing time), and ratios comparing left/right or current/previous responses. The RF model ranks features by importance and uses the top 50. The choice of RF over logistic regression is justified by better performance and ability to capture non-linear feature interactions.

Training Regime: Models trained and validated via five-fold cross-validation on the 695 pairwise comparison tasks. Baseline text-only models use embeddings from modern BERT variants (mBERT base/large) as features. The implicit feedback reward model includes additional features from eye gaze and mouse movements. Training focused on supervised prediction of user preferences with standard RF hyperparameters. No adversarial training reported.

Evaluation Protocol: Accuracy and F1 scores over pairwise and pointwise preference prediction tasks measure how well the reward model predicts explicit human preferences. Improvements are assessed relative to random guessing and text-only baselines. Additionally, the reward models' outputs are used to fine-tune 8 LLMs (ranging 1B–70B parameters) via Direct Preference Optimization (DPO) with negative log-likelihood regularization (rDPO + NLL). The final aligned LLM responses are evaluated by GPT4.1-mini scoring across 300 held-out pointwise queries, compared by win rates against supervised fine-tuning baselines. Human annotation validates model judgments in an appendix.

Reproducibility: The dataset IFLLM, data collection website with open source code (Apache 2 license), and all model training code are publicly released at https://github.com/themehulpatwari/llm-implicit-feedback/. Detailed feature descriptions and evaluation setups are provided. However, some raw gaze data quality varies due to webcam diversity, controlled by calibration and filtering. Data collection protocol received IRB approval. Evaluation uses well-known LLM benchmarks and official Mechanical Turk workers.

Example Pipeline End-to-End: A user logs into the platform, selects topics from Wikipedia, calibrates eye gaze via webcam in a green-box alignment step. They converse with LLMs for at least 3 questions, mouse and gaze coordinates are recorded every 0.1s while they read response text and rate preference via pointwise or pairwise annotation. Features from trajectories and text metadata are extracted and fed into a Random Forest model trained to predict preference. The predictions then guide rDPO-based alignment fine-tuning of the LLMs, which are evaluated by LLM judges and human raters on held-out queries, showing improved alignment quality when reward models incorporate implicit feedback.

Technical innovations

• Construction of IFLLM, the first multi-turn LLM interaction dataset combining webcam-based eye gaze and mouse trajectory implicit feedback paired with explicit preference labels in the wild.
• Use of a Random Forest reward model leveraging detailed aggregated features from temporally dense gaze and mouse signals to predict user LLM response preferences, surpassing text-only approaches.
• Application of rDPO with NLL to align multiple LLMs using reward models trained on implicit feedback, resulting in substantial output quality improvements versus baselines.
• Comprehensive behavioral analysis documenting the interaction effects of response length on correlation and informativeness of mouse and gaze trajectories.

Datasets

• IFLLM — 1336 multi-turn question-answering interactions — collected from 59 Mechanical Turk workers via a custom crowdsourcing website

Baselines vs proposed

• mBERT base + Text only: accuracy = 0.5549 ± 0.0347 vs RF + IF: accuracy = 0.6415 ± 0.0097 for pairwise preference prediction
• RF + (IF - Mouse): accuracy = 0.6089 ± 0.0130 vs RF + (IF - Gaze): accuracy = 0.6267 ± 0.0097 shows mouse features contribute more strongly
• Explicit feedback reward model + DPO: DPO-SFT win rate difference = 0.1079 vs RF + (IF - Gaze) reward model + rDPO: 0.3450, nearly tripling the relative improvement
• Zero-shot Claude Sonnet 4.6: accuracy = 0.5303 vs reward model trained with RF + IF: 0.6415 accuracy
• Improvement gap increases with response length — longer responses benefit more from mouse movement features

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.20482.

Fig 1: IFLLM records the trajectories of eye gazing

Fig 2: Diagram of webpage navigation for a worker. 1 cycle of the webpages correlates to 1 task, equivocally

Fig 3 (page 1).

Fig 3: Average fixation weight over the response text

Fig 4: indicates the average reading trajectory

Fig 6: suggests that reading speed further de-

Fig 7: are full of back-and-forth movement, and

Fig 8 (page 3).

Limitations

• Reward model requires implicit feedback input at inference/training time, limiting applicability in fully offline fine-tuning scenarios.
• Eye-gazing data quality depends on webcam calibration and ambient conditions, introducing noise and requiring extensive filtering.
• Multi-turn context is split into single-turn sessions simplifying analysis but potentially losing cross-turn dependencies in preferences.
• No adversarial evaluation of potential manipulation of implicit feedback signals was conducted.
• Dataset size (59 workers) is relatively small and may limit generalizability to broader user populations.
• Implicit feedback features may be noisier or less indicative in non-text interface settings or detached from mouse/gaze input devices.

Open questions / follow-ons

• How to better model multi-turn dependencies using implicit feedback signals over the full conversation rather than treating turns independently?
• Can implicit feedback signals be combined with explicit feedback in a unified, continuous alignment framework to further improve LLM quality?
• What robustness guarantees can be designed to ensure implicit feedback reward models cannot be gamed or manipulated?
• How generalizable are implicit feedback models across diverse UI platforms, devices, and non-English languages?

Why it matters for bot defense

This paper is highly relevant to bot-defense and CAPTCHA practitioners interested in implicit user signals as a form of behavioral feedback indicating genuine human presence and engagement. While the study focuses on LLM alignment, the methodologies for capturing and modeling mouse and eye gaze trajectories can inform bot detection by highlighting nuanced user interactions that differ from automated behavior. The dataset and feature extraction methods provide new avenues for modeling implicit behavioral biometrics that may enhance challenge-response systems or continuous authentication.

For CAPTCHA engineers, the demonstrated correlation between mouse/gaze patterns and user attention/preference suggests such signals could augment traditional explicit challenge tasks with passive signals to reduce user friction. However, practical deployment requires robust real-time capture and privacy-preserving processing. Moreover, the findings stress that user behaviors are highly diverse, dependent on content length and interface layout, which calls for adaptive bot-defense modeling sensitive to these factors.

Cite

@article{arxiv2606_20482,
  title={ Your Mouse and Eyes Secretly Leak Your Preference: LLM Alignment using Implicit Feedback from Users },
  author={ Haw-Shiuan Chang and Jeffrey Gomez and Mehul Patwari and Aryan Sajith and Hamed Zamani },
  journal={arXiv preprint arXiv:2606.20482},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.20482}
}

Read the full paper

• arXiv:2606.20482 (abstract)
• arXiv:2606.20482 (PDF)