When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space

Source: arXiv:2607.15218 · Published 2026-07-16 · By Weimeng Wang, Ziqiang Wang, Zihang Zhan, Chuanpu Fu, Qi Li, Ke Xu

TL;DR

This paper addresses the distinct safety challenge presented when large language models (LLMs) are used as high-level planners for embodied agents—where instructions that appear linguistically safe can lead to physical danger once executed in the real world. The authors distinguish between two types of safety risks: content danger (CD), where unsafe instructions contain overt harmful language cues, and physical danger (PD), where instructions are linguistically benign but cause unsafe physical outcomes. They show that these two danger types form separable signals in the hidden states of various LLMs, including Qwen2.5 models and other architectures. Leveraging this insight, they propose PRISM, a single-layer L2-regularized logistic probe over frozen hidden states, to detect unsafe instructions with a joint focus on both CD and PD. PRISM achieves balanced detection performance with substantially lower false positives than zero-shot LLM judge methods on benchmarks including SafeAgentBench and a newly introduced PhysicalSafetyBench-1K (PSB-1K) that isolates physically grounded risk without explicit harm cues.

Key findings

• Content danger (CD) and physical danger (PD) directions in LLM hidden states maintain angles between 71.6° and 75.9° probe-weight separation across Qwen2.5-3B/7B/14B/32B models, exceeding random-split null baselines (p < 0.0001).
• PRISM achieves 86.2–87.7% accuracy with 11.7–13.7% false positive rate (FPR) on SafeAgentBench 600-example benchmark across Qwen2.5 scales, detecting physical danger with ~80% recall and content danger above 93.8%.
• Zero-shot Qwen2.5-32B judge detects 95.7% of physical danger but rejects 37.0% of safe tasks (high FPR), illustrating over-blocking compared to PRISM’s calibrated operating point.
• On PSB-1K, containing 1,000 paired physically risky and safe instructions without overt harm keywords, PRISM reaches 99.6% accuracy and 0.7% FPR, while Qwen2.5-3B judge rejects 67.8% of safe tasks to achieve 96.5% physical-danger recall.
• CDD-only and LPM prototype-based methods detect 9.6–32.6% of out-of-domain physical danger tasks, confirming that physical danger is not captured by content-oriented representations alone.
• Scaling PRISM from 3B to 32B parameters yields stable accuracy (~86–88%) and FPR (~12–14%), whereas scaling LLM judges improves detection recall but increases FPR substantially, indicating decreased calibration.
• Cross-architecture experiments on Phi-3.5-mini and SmolLM2-1.7B replicate the CD/PD directional separation, showing generality beyond Qwen models.
• Latency benchmarks show PRISM doubles inference speed compared to LLM judges on 7B to 32B scale models while providing lower false positives.

Threat model

The threat is posed by unsafe instructions to embodied agents that can cause physical harm upon execution. The adversary is modeled implicitly as any instruction or planner output that results in unsafe physical actions either via explicit harmful content (content danger) or through context-dependent physical causality (physical danger). The probe must detect unsafe instructions before execution, without relying solely on linguistic surface cues. The adversary cannot manipulate the internal hidden-state space or probe parameters directly.

Methodology — deep read

The authors formulate the problem as detecting unsafe instructions that can cause either linguistic content danger (CD) or physically grounded danger (PD). The adversary here is implicit—the system must preemptively detect unsafe plans before execution, differentiating between the two danger types based on representations internal to frozen LLMs.

They use datasets of household instructions annotated as safe, CD, or PD from SafeAgentBench (600 examples), SafeText, EARBench, and introduce PhysicalSafetyBench-1K (PSB-1K), which contains 1,000 minimally contrasted safe-risk instruction pairs focused exclusively on physical dangers without explicit harm keywords.

Hidden states from the selected transformer layers of frozen LLMs (e.g., Qwen2.5-3B/7B/14B/32B, Phi-3.5-mini, SmolLM2-1.7B) are extracted for the last token. Two key mean-difference vectors are computed over these hidden states: Content Danger Direction (CDD = mean_CD - mean_safe) and Physical Danger Direction (PDD = mean_PD - mean_safe). Angular separability between these vectors is statistically compared against a random-split null distribution obtained by randomly partitioning the unsafe set and fitting separate probes.

Building on the stable CD/PD separation, they design PRISM, a single-layer logistic regression probe with L2 regularization over standardized full hidden states at the selected layer, trained to classify safe versus unsafe instructions jointly considering both danger subtypes. The layer is selected by cross-validation from all transformer layers, independently per model. Standard scikit-learn LogisticRegression with max_iter=2000 and C=1.0 is used. Training incorporates safe, CD, and PD labeled examples simultaneously.

Evaluation includes 5-fold cross-validation or pair-wise grouped splits to avoid leakage on contrastive pairs. Metrics are accuracy, F1, AUC, false positive rate (FPR), and CD/PD detection rates reported separately. Comparisons are made to directional probes trained on only CD or PD, nearest centroid classifiers, prototype methods (LPM), non-linear baselines (SVM-RBF, MLP), and zero-shot LLM judge methods that output safe/unsafe decisions based on prompted generation.

For latency, wall-clock inference times of PRISM forward passes versus judge model generation are measured on 600 warmup + 580 test samples. Ablations include layer choices, probe weight angles, and safety subtype splits. Cross-architecture and cross-dataset replications demonstrate generality. Code and data availability are not explicitly stated, but substantial experimental detail supports reproducibility.

As a concrete example, on SafeAgentBench with Qwen2.5-3B, hidden states at layer 27 are extracted for 600 instructions. PRISM is trained using logistic regression on standardized layer 27 hidden states, achieving 86.2% accuracy, 13.7% FPR, and >80% recall on physical danger, while zero-shot judge rejects 27.3% of safe instructions to achieve higher recall. The random-split null angle test confirms CDD and PDD probe directions are well-separated beyond chance.

Overall, the methodology centers on hidden-state probing with clear statistical controls, dataset contrastive design, and multi-scale evaluation to isolate physical vs content risks beyond textual moderation.

Technical innovations

• Identification of two statistically separable hidden-state directions (CDD and PDD) encoding content danger and physical danger respectively, beyond chance baselines.
• Development of PRISM, a single-layer L2-regularized logistic probe over full hidden states to jointly detect both CD and PD with a low false-positive rate.
• Introduction of PhysicalSafetyBench-1K (PSB-1K), a novel 1,000 paired household instruction benchmark isolating physical danger detection without relying on explicit harm keywords.
• Demonstration that physical danger is a distinct hidden-state safety concept not captured by prior content-oriented text-moderation models or prototype-based detectors.

Datasets

• SafeAgentBench — 600 examples — public AI2-THOR household instructions subset
• SafeText — 1,465 examples — adapted from prior physical commonsense safety dataset
• EARBench — 2,634 examples — multi-agent embodied safety scenarios
• PhysicalSafetyBench-1K (PSB-1K) — 2,000 examples (1,000 paired safe-risk) — newly constructed for physical-danger without explicit harm keywords

Baselines vs proposed

• LLM Judge (Qwen2.5-32B): Physical-danger recall = 95.7%, FPR = 37.0% vs PRISM: Physical-danger recall = 81.8%, FPR = 12.0% on SafeAgentBench
• LLM Judge (Qwen2.5-3B): Accuracy = ~64% vs PRISM: Accuracy = 99.6% on PSB-1K, with FPR 67.8% vs 0.7% respectively
• CDD-only probe: Physical-danger recall = 19.8–32.6% vs PRISM: 79.1–82.4% across Qwen2.5 scales on physical OOD sets
• PDD-only probe: Content-danger recall = 68.1–85.0% vs PRISM: >93.8% content-danger detection
• LPM (latent prototype method): Physical-danger recall = 9.6–19.3% vs PRISM: ~80% physical-danger recall
• SVM-RBF baseline: Accuracy ~82.8–86.0% vs PRISM: 86.2–87.7% accuracy on SafeAgentBench

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.15218.

Fig 2: Decision boundaries in the CDD/PDD projection plane on Qwen2.5-3B layer 27 (5-fold CV). CDD-

Fig 3: CDD/PDD probe-weight angles exceed random-split null angles.

Fig 4: PRISM maps hidden states from a frozen LLM to a safe/unsafe decision boundary in hidden-state

Fig 4 (page 6).

Fig 5: Cross-model scaling: PRISM accuracy and FPR stay flat while the LLM judge’s physical-danger

Fig 6: Cross-architecture universality of CDD/PDD. (a) CDD–PDD probe-weight angles cluster between

Fig 7: Per-layer PRISM 5-fold accuracy for Qwen2.5-3B/7B/14B/32B; circles mark the selected layer.

Fig 8: PRISM ROC analysis on Qwen2.5-3B (5-fold-CV out-of-fold probabilities). Left: overall safe-vs-

Limitations

• SafeAgentBench’s CD/PD split relies on keyword-based heuristics which can cause label leakage between classes.
• PRISM requires access to full hidden states from a frozen LLM, which may be unavailable in black-box deployment scenarios.
• The physical danger category is operationalized via proxy benchmarks and linguistic minimal pairs; real-world physical harm scenarios might be more complex or require multimodal grounding.
• No adversarial evaluation against intentionally crafted unsafe instructions to probe robustness is reported.
• While cross-architecture tests are shown, experiments focus primarily on Qwen2.5 and two smaller models, limiting generalization to diverse LLM families.
• Training and evaluation rely on offline datasets without online interaction or real physical execution feedback.

Open questions / follow-ons

• How can physical danger detection scales to multimodal inputs combining language with vision or environment states for grounded safety?
• Can representation-level probing methods like PRISM be extended or integrated with intervention techniques to actively steer agents away from unsafe plans?
• How robust are hidden-state danger directions under adversarial instruction generation or distributional shifts beyond household tasks?
• What is the optimal way to combine LLM decoded safety judgments with hidden-state probes to balance recall and false positives in practice?

Why it matters for bot defense

This work is directly relevant to bot-defense and CAPTCHA practitioners concerned with security in embodied AI systems or autonomous agents. It highlights that safety evaluation for instructions or high-level natural language commands must go beyond traditional text-based content moderation—a common technique in bot defense—and consider distinct physical danger signals embedded in the model’s internal representations. The PRISM approach suggests that hidden-state probing can provide a lightweight, efficient alternative to large LLM-based safety judges, reducing false positives while maintaining strong detection of subtle physical risks. Practitioners designing safety monitors or CAPTCHAs for embodied agents can leverage these findings to better separate hazardous physical plans from safe ones, especially when no overtly harmful language is present. However, deploying such hidden-state probes requires access to model internals, which may limit direct applicability to closed-source LLM APIs. Nonetheless, the distinction between content and physical danger may inform safer task design, prompt engineering, and multi-stage verification in embodied bot defense pipelines.

Cite

@article{arxiv2607_15218,
  title={ When Words Are Safe But Actions Kill: Probing Physical Danger Beyond Text Safety in Hidden-State Risk Space },
  author={ Weimeng Wang and Ziqiang Wang and Zihang Zhan and Chuanpu Fu and Qi Li and Ke Xu },
  journal={arXiv preprint arXiv:2607.15218},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.15218}
}

Read the full paper

• arXiv:2607.15218 (abstract)
• arXiv:2607.15218 (PDF)