Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning

Source: arXiv:2607.28478 · Published 2026-07-30 · By Zheng Wu, Chenhao Xue, Shijie Zheng, Yijie Lu, Cheng Yang, Zhuosheng Zhang

TL;DR

This paper identifies and thoroughly investigates a surprising failure mode in large language models (LLMs) called Salience Bias, where models overly focus on explicit but irrelevant input details (e.g., numerical distractors) and consequently ignore implicit commonsense or physical prerequisites necessary to reason correctly about a task. To study this phenomenon, the authors create the SaliTrap Benchmark, a diverse set of 1,145 carefully constructed examples across four categories of physical and commonsense violations, each embedding impossible premises camouflaged with distracting computations. The authors evaluate 12 state-of-the-art LLMs on this benchmark, revealing pervasive vulnerabilities: even the best model avoids traps only 54.8% of the time, and several models fall well below 30% trap avoidance. Intriguingly, failures are driven not by missing knowledge but by knowledge suppression triggered by salient distractors, confirmed by context-free re-probing that recovers over 90% of ignored knowledge. Finally, the paper demonstrates that lightweight, inference-time prompt interventions can substantially mitigate salience bias without retraining. Overall, the work shifts the blame for certain commonsense reasoning failures in LLMs from inherent knowledge deficits to elicitation bottlenecks caused by misleading input framing.

Key findings

• Across all 12 evaluated LLMs, the top trap avoidance rate (TAR) is 54.8%, with 8 models below 30%, showing widespread susceptibility to salience bias (Table 1).
• Increasing the number of numerical distractors monotonically decreases TAR and increases Chain-of-Thought (CoT) hijacking, confirming distractor density exacerbates bias (Figure 6).
• Trap detection and trap avoidance are decoupled: GLM-5.1 and Kimi-K2 comply with traps 86.2% and 81.8% of the time even after explicitly recognizing the trap (Table 2, Sycophantic Compliance Rate and Sycophancy Index).
• Items involving missing prerequisite traps are the hardest across all models, while rule mismatch traps are the easiest (IRT difficulty β distributions, Figure 4).
• Hierarchical clustering reveals failure sets cluster by model origin and capability (Chinese vs Western clusters), indicating training provenance shapes blind spots (Figure 5).
• Re-eliciting knowledge with context-free prompts recovers over 90% of previous sycophantic compliance failures, proving failures arise from knowledge suppression rather than absence (Figure 7).
• Simple system-level prompt prefixes that encourage premise verification substantially increase trap avoidance rates over the baseline, especially for weaker models (Figure 8).
• SaliTrap dataset contains 1,145 items spanning missing prerequisite, environmental mismatch, temporal/physiological violation, and rule mismatch trap dimensions.

Threat model

The adversary is the misleading task input consisting of explicit but irrelevant or impossible conditions (distractors), deliberately designed to cause LLMs to incorrectly prioritize salient surface details over implicit commonsense reasoning. The model is assumed to possess the underlying knowledge but can be 'attacked' by these distractors due to elicitation failures. The adversary cannot forcibly remove or retrain the model but can only craft input queries. External manipulation or access to model internals is out-of-scope.

Methodology — deep read

1. Threat Model and Assumptions: The adversary is essentially the inherent bias in LLMs themselves when prompted with natural language queries containing salient but irrelevant explicit distractors like numerical values. The adversary's capability is the model's tendency to over-attribute importance to these distractors and ignore commonsense prerequisites. The study does not consider external attackers but focuses on testing models’ intrinsic reasoning failures under adversarially designed inputs.

2. Data: The authors construct the SaliTrap benchmark with 1,145 natural language queries embedding physically impossible premises hidden within computation-laden traps. These are carefully generated and validated through expert seed annotations followed by an LLM-assisted three-stage pipeline: seed generation and scaling, candidate validation, and iterative refinement. The dataset spans four trap dimensions—missing prerequisite, environmental mismatch, temporal/physiological violation, and rule mismatch—with distractors uniformly distributed and rigorous deduplication protocols.

3. Architecture / Algorithm: The work is empirical rather than proposing a new model, but evaluation leverages 12 LLMs including Claude-Opus series, GPT-5 variants, Gemini, DeepSeek, GLM, and others. Three LLM models (Claude-Opus-4.7, GPT-5.5, DeepSeek-R1, Gemini-2.5-Pro) act in rotating roles during evaluation as solver (answerer), checker, and judge to assign behavioral labels such as 'Hard Fail', 'CoT Hijacked', 'Sycophantic Compliance', or 'Strict Pass'. This evaluates not only correct answer production but reasoning trajectories and trap awareness.

4. Training Regime: Models are evaluated zero-shot without task-specific fine-tuning or prompt engineering to fairly measure intrinsic salience bias. For the benchmark construction, the judge/checker models use Claude-Opus-4.7 with rewriting and generation models also built on it. Solver is implemented as a round-robin ensemble of four strong models. Hyperparameters for generation and rewriting are not exhaustively detailed but include multi-round rewriting loops with seed-based exclusion and acceptance criteria.

5. Evaluation Protocol: Key metrics measured include Trap Avoidance Rate (TAR), Hard Fail Rate (HFR), Sycophantic Compliance Rate (SCR), and Sycophancy Index (SI), the last two discerning overshooting compliance despite trap awareness. Item Response Theory (IRT) models estimate per-item difficulty (β) and per-model ability (θ) to disentangle factors. Statistical tests are not detailed. The evaluation protocol includes re-elicitation experiments testing if knowledge is absent or suppressed and prompt intervention experiments applying system-level prefixes at inference time to measure remediation of bias.

6. Reproducibility: The code and dataset SaliTrap are publicly released at https://github.com/Wuzheng02/SaliTrap. Model weights are not released, but all evaluated models are publicly known LLMs. Full iterative construction pipeline with multi-stage vetting and rewriting is described transparently, supporting reproducibility of benchmark creation.

Concrete end-to-end example: A prompt querying whether one should walk or drive 50 meters to a car wash embeds a physically impossible premise if treated literally (one must drive the car, not walk it). Models tend to fixate on the salient numeric '50 meters' and advise walking, ignoring the implicit prerequisite that the car must be driven. This exemplar highlights the salience bias: the distractor cues suppress commonsense, leading to incorrect compliance. Stripping the task framing and probing with a context-free query recovers correct knowledge that driving is required. Prompt interventions encouraging explicit feasibility checks boost correct trap avoidance.

Technical innovations

• Formalization of Salience Bias as a distinct failure mode where LLMs prioritize explicit distractors over implicit commonsense prerequisites.
• Construction of SaliTrap, a high-quality large-scale benchmark with rigorously validated physically impossible premises camouflaged by computation distractors spanning four trap categories.
• Decoupling trap detection from trap avoidance through a novel tri-checker and solver-judge multi-label evaluation framework that identifies behavioral failure modes like sycophantic compliance.
• Demonstration that salience bias results from knowledge suppression due to misleading input framing, proven by re-eliciting suppressed knowledge with context-free probes achieving over 90% recovery.
• Introduction and evaluation of lightweight, inference-time prompting strategies that substantially improve trap avoidance without retraining, highlighting elicitation bottlenecks.

Datasets

• SaliTrap Benchmark — 1,145 examples — publicly released at https://github.com/Wuzheng02/SaliTrap

Baselines vs proposed

• Claude-Opus-4.7: TAR = 54.8% overall vs DeepSeek-V4-Flash: TAR = 12.1% (Table 1)
• GLM-5.1: SI (Sycophancy Index) = 86.2% vs Claude-Opus-4.7 SI = 53.1% (Table 2)
• Increasing numeric distractors from 0 to 5 reduces average TAR from ~58% to ~25% and increases CoT hijacking from ~10% to ~50% (Figure 6)
• Prompt intervention P1 boosts TAR from ~55% to ~75% on Claude-Opus-4.7, and greater improvements on weaker models (Figure 8)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.28478.

Fig 1: All LLMs suffer from salience bias, which stems

Fig 2: Exemplifying salience bias in LLMs. Driven by

Fig 3: The construction of the SaliTrap benchmark is divided into three stages: (i) Seed generation and scaling stage, (ii)

Fig 4: IRT-estimated item difficulty β distribution across the four trap dimensions (12 evaluated models). Missing prerequisite

Fig 5: Hierarchical clustering of the 12 models by Jaccard

Fig 6: TAR and CoT-Hijacked rate versus the number of

Fig 7: Liberation Rate of Sycophantic Compliance cases

Fig 8: TAR under Control and three system-level prompt

Limitations

• Focuses primarily on zero-shot prompting evaluation; does not explore fine-tuning or supervised training to mitigate salience bias.
• Relies on LLMs for generation, checking, and judging during benchmark construction, which could bias dataset content and evaluation.
• Does not cover attacks or adversarial settings beyond salience-based distractors; broader adversarial threat models untested.
• Limited detail on statistical significance testing or confidence intervals for reported metrics.
• While publicly released, details on computational resource requirements for benchmark construction and evaluation are not provided.

Open questions / follow-ons

• How can training paradigms be adapted to reduce salience bias by penalizing over-reliance on irrelevant explicit details?
• Can fine-tuning or reinforcement learning with salience-aware objectives further improve trap avoidance beyond prompting?
• What are the effects of salience bias in multi-modal or agentic LLM settings involving real-world tool use?
• How can salience bias detection be integrated into real-time system monitoring for safer LLM deployments?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this research highlights a nuanced blind spot in large language models whereby irrelevant but salient distractors in challenge prompts can cause the system to ignore essential implicit commonsense constraints. This has direct implications for designing CAPTCHAs or proofof-human tests that rely on commonsense reasoning: adversarially crafted inputs laden with distracting numeric or irrelevant details can hijack model reasoning and cause incorrect compliance or failures. Understanding salience bias informs how to frame CAPTCHA challenges to reduce exploitability by LLM-powered bots, or conversely how to design detection signals that identify when a bot overly focuses on superficial cues rather than task-critical logic. Furthermore, the findings that lightweight prompt engineering can mitigate bias suggests avenues for runtime defenses or filtering layers. Overall, practitioners should test candidate LLM defenses or attacks against salience bias benchmarks like SaliTrap to ensure robustness against subtle reasoning exploits. The benchmark itself can be used for stress-testing automated CAPTCHA solvers or evaluating defense prompt designs.

Cite

@article{arxiv2607_28478,
  title={ Would You Walk to the Car Wash? Revealing the Salience Bias of Large Language Models in Commonsense Reasoning },
  author={ Zheng Wu and Chenhao Xue and Shijie Zheng and Yijie Lu and Cheng Yang and Zhuosheng Zhang },
  journal={arXiv preprint arXiv:2607.28478},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.28478}
}

Read the full paper

• arXiv:2607.28478 (abstract)
• arXiv:2607.28478 (PDF)