LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agents

Source: arXiv:2608.04741 · Published 2026-08-05 · By Longtao Guo, Zelin Zhang, Kaifeng Huang, Yang Shi

TL;DR

This paper investigates a new attack surface called login inducement in LLM-based web agents, which autonomously execute browser tasks on behalf of users. Existing research showed that malicious webpage content can manipulate these web agents, but did not explore whether such content can induce agents to initiate login actions—which involve sensitive credentials—and leak private data end-to-end. Under a strong black-box threat model where the attacker controls only the webpage content but not the agent internals or user task, the authors design LoginTrap, a task-agnostic login-inducing attack. LoginTrap uses a fuzzing-inspired process to generate page-specific injection statements that induce the web agent to treat login as an essential prerequisite to continuing the task, thereby guiding it into an attacker-controlled login flow. The evaluation demonstrates that LoginTrap achieves an average end-to-end attack success rate (ASR) of 86% across diverse LLM foundations and agent architectures while remaining effective on 80 cloned webpages spanning five realistic domains. This work reveals login inducement as a systemic vulnerability boundary for LLM-based web agents, highlighting the urgent need for authentication-aware defenses.

Key findings

• LoginTrap achieves an average login entry rate (LER) of 93% and an average end-to-end attack success rate (ASR) of 86% across tested LLM backbones (GPT-4o, Gemini 3 Flash, Claude Sonnet 4, DeepSeek-V3.2).
• Across different web agent architectures, LoginTrap maintains an average ASR of 79%, showing generalization beyond a single agent design.
• Page-level exploitability rate (PER) analysis over 80 cloned webpages shows that the attack succeeds on 58% of webpages within 9 attempts (3 tasks × 3 runs each), reflecting broad domain applicability.
• LoginTrap’s fuzzing-inspired injection generation method successfully produces page-specific login-inducing statements without knowledge of user tasks or agent internals.
• Workflow completion tendency amplifies attacks: models with stronger task persistence (e.g., Gemini 3 Flash, Claude Sonnet 4) show smaller gaps between LER and ASR, implying higher likelihood to follow through the malicious login flow.
• Injected login prompts do not directly request sensitive data but frame login as a credible prerequisite step, which makes the attack subtle and widely effective.
• The attack remains effective despite existing defenses designed for prompt injection or deceptive UI patterns, indicating current solutions lack focus on login boundary risks.
• Fine-grained injection mutations guided by evaluator feedback enable effective task-agnostic exploitation across diverse webpages and domains (Travel, Service, Information, Shopping, Entertainment).

Threat model

The adversary is a black-box attacker who controls cloned webpages observed by the LLM-based web agent, including embedding login-inducing statements and managing the subsequent login page. They do not know the specific user task, agent internal policies or safety mechanisms, and cannot modify the agent's runtime environment or observe internal reasoning steps. Their sole leverage is controlling webpage content visible to the agent before login is induced. The adversary’s goal is to induce the agent to treat login as necessary and fill sensitive user information into a forged login page controlled by the attacker.

Methodology — deep read

1. Threat Model & Assumptions: The attacker is a black-box adversary controlling cloned webpages resembling benign originals, including injecting login prompts inside a controlled popup container and managing the subsequent login page. The attacker does not know the user task T, web agent internals, system prompts, or safety policies. They cannot modify agent runtime or observe internal reasoning. Attack is performed solely through webpage content visible to the agent before login is induced.

2. Data: The evaluation uses 80 cloned webpages derived from the Mind2Web benchmark, covering five domains (Travel, Service, Information, Shopping, Entertainment). The dataset includes 1,175 user task instances paired with these webpages. Pages that were unreliable to clone were removed. Tasks represent normal user objectives such as searching or comparing items.

3. Architecture / Algorithm: LoginTrap injects task-agnostic login-inducing statements into a controlled popup on the cloned page. Using an LLM-based fuzzing-inspired generation process, candidate statements are iteratively formed, filtered, and mutated based on dual shadow LLM evaluators' predictions, optimizing for inducing login clicks without explicit task or agent prior knowledge. The induced login entry redirects the agent to a controlled login page that mimics the service purpose of the original page (summary generated by LLMs) and presents a standard sensitive info form. The attack’s novelty lies in framing login as necessary prerequisite rather than direct data exfiltration.

4. Training Regime: GPT-4o is used only for injection generation steps—summary extraction, candidate generation, mutation, and evaluation. Multiple mutation techniques including crossover, expansion, rephrasing, and compression are applied guided by evaluator rewards using a Monte-Carlo Tree Search inspired policy. No typical model training occurs on the agents since these are existing LLM-based web agents.

5. Evaluation Protocol: Metrics include Login Entry Rate (LER) measuring fraction of task executions where agent clicks the forged login entry; Attack Success Rate (ASR) assessing end-to-end leakage by whether sensitive info is actually inputted into attacker login form; and Page Exploitability Rate (PER) measuring fraction of webpages exploitable within 9 evaluation attempts per page (3 tasks × 3 repeats). Evaluations are conducted across four LLM backbones and multiple web agent architectures (Browser-Use framework) with a fixed max 5-step browser action limit per task run. Logs capturing observable browser actions define metric labels.

6. Reproducibility: The cloned webpages, controlled login flows, task sets, and injection generation process are described in detail; code or datasets are not explicitly released. Agents vary in internal architecture but evaluation uses standard frameworks and public datasets (Mind2Web). The fuzzing-inspired generator relies on proprietary LLM GPT-4o, complicating exact reproduction.

Concrete Example: Starting with an original cloned shopping site webpage, LoginTrap extracts a summary via LLM and generates candidate login-inducing prompts. Through iterative mutation and evaluation, a statement is embedded in a popup urging "Login to continue" framed as prerequisite. The LLM web agent observing the phishing page revises its plan to click the fake login entry, navigates to attacker-controlled login form tailored by LLM to mimic the service, and fills in sensitive user info from its background knowledge, submitting it to the attacker, resulting in successful privacy leakage captured in a single run.

Technical innovations

• A fuzzing-inspired, task-agnostic injection generation process that produces page-specific login-inducing statements from webpage context alone, without user task or agent internals knowledge.
• Controlled popup injection surface on cloned webpages preserves realistic page context, isolating the effect of login inducement statements from unrelated page content changes.
• An end-to-end login inducement attack flow that frames login as a plausible prerequisite for continuing the task, guiding the agent to an attacker-managed login page which harvests sensitive info.
• Dual-shadow LLM evaluator feedback loop including a suspicious content reminder model to filter and refine injection statements based on predicted agent actions.

Datasets

• Mind2Web derived cloned webpages — 80 webpages — cloned from online web services with associated 1,175 user tasks

Baselines vs proposed

• Across GPT-4o: LER = ~93%, ASR = ~79% vs Gemini 3 Flash: LER = ~96%, ASR = ~92%
• Claude Sonnet 4: LER = ~91%, ASR = ~90% vs DeepSeek-V3.2: LER = ~93%, ASR = ~72%
• Browser-Use agent architecture baseline: ASR ≈ 86%; Transfer to other agent architectures reduces ASR to ~79%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2608.04741.

Fig 3: Domain-wise LER and ASR of LOGINTRAP across different LLM backbones (RQ1). Colors and hatch patterns

Fig 5: Domain-wise LER and ASR of LOGINTRAP across different web agent architectures (RQ2). Colors and hatch patterns

Limitations

• Evaluation limited to 80 cloned webpages and 1,175 tasks; larger diverse datasets may reveal additional attack nuances.
• Attack effectiveness assessed under max 5-step agent interaction limit; longer or more complex tasks might affect results.
• Injection generation relies on proprietary GPT-4o, complicating exact reproduction or generalization to other model versions.
• No comprehensive adversarial training or defense evaluation beyond basic test defenses; robustness of defenses remains an open question.
• Assumes attacker can fully control the cloned page content and login flow; partial control or detection mechanisms may reduce feasibility.
• User background information assumed accessible to the agent, enabling filling of sensitive info; scenarios with limited agent info not explored.

Open questions / follow-ons

• How can web agents be designed or trained to correctly recognize when login is genuinely required or suspicious, preventing login inducement attacks?
• What authentication-aware defense mechanisms can be integrated to detect or block task-agnostic login inductions without impairing agent task completion?
• How resilient are LoginTrap-like attacks under adversarial training or formal verification approaches for web agents?
• Can the fuzzing-inspired injection generation technique be extended to other sensitive workflow boundaries beyond login (e.g., billing, personal data forms)?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this paper uncovers a subtle and systematic risk in LLM-driven web automation agents: the login boundary can be manipulated by attackers controlling webpage content to induce sensitive credential leakage. Unlike overt prompt injection or UI deception attacks, LoginTrap operates task-agnostically and frames login as a legitimate intermediate step, making detection through heuristic filters difficult. Practitioners should consider that automated agents operating on webpages may be vulnerable to phishing-style indirect prompt injections that exploit authentication workflows. Defenses focused only on textual prompt sanitization or interface manipulation may be insufficient without more explicit management of authentication decision boundaries and verification of login necessity. Integrating authentication-aware checks, anomaly detection on unusual login triggers, or leveraging CAPTCHAs conditioned on login actions might provide valuable mitigation avenues. Overall, this work highlights that securing LLM-based web agents requires holistic consideration of authentication workflows, a critical yet understudied attack surface in bot-defense.

Cite

@article{arxiv2608_04741,
  title={ LoginTrap: Uncovering Task-Agnostic Phishing-Style Indirect Prompt Injection Attacks against LLM-based Web Agents },
  author={ Longtao Guo and Zelin Zhang and Kaifeng Huang and Yang Shi },
  journal={arXiv preprint arXiv:2608.04741},
  year={ 2026 },
  url={https://arxiv.org/abs/2608.04741}
}

Read the full paper

• arXiv:2608.04741 (abstract)
• arXiv:2608.04741 (PDF)