Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulation
Source: arXiv:2608.02518 · Published 2026-08-03 · By Natalie Isak, Matthew Dressman
TL;DR
This paper addresses a critical gap in AI misuse detection arising from cross-session statelessness in agentic AI deployments. Adversaries can decompose a harmful goal into multiple innocuous subtasks, executed across independent sessions, thereby evading conventional single-session or multi-turn detection mechanisms. Such cross-session decomposition accumulates harmful capabilities incrementally, producing concrete artifacts that collectively constitute a malicious outcome, yet remain individually harmless within any single session context.
To mitigate this detection blind spot, the authors propose Magnet, a novel detection framework that aggregates accrued attacker capabilities over time at the entity (user ID) level, rather than per conversation. Unlike prior systems that compress historical interactions into a single scalar score or intent signal, Magnet tracks explicit binary indicators of harmful capabilities demonstrated in each session and maintains a monotonic bitwise OR union across sessions. This multi-dimensional inventory enables robust detection of low-and-slow distributed attacks. Empirical evaluation across nine LLMs and two harm domains (phishing kit and Molotov cocktail assembly) shows Magnet outperforms per-session and compressed-history detectors, achieving an 84.0% F1 score and more stable recall even under adversarial noise injection and duplicated message flooding.
Key findings
- Cross-session goal decomposition attacks achieve an average attack success rate (ASR) of 37.4% versus 26.0% multi-turn and 18.7% single-turn delivery across nine LLMs and two attack domains (Fig. 2).
- Models vary in vulnerability: gemini-2.5-pro and gpt-5.4 show minimal ASR increase under decomposition, indicating model-dependent safety-capability tradeoffs.
- Failure mode analysis reveals fewer refusals (i.e. explicit guardrail blocks) in multi-session attacks, indicating evasion via distributed benign subtasks (Fig. 3).
- Magnet detector achieves 84.0% F1 score over 336 scenarios using a capability-accumulation binary vector approach versus 61.0% for per-session detector and 81.8% for compression detector (Fig. 5).
- Per-session detector attains highest precision (96.2%) but very low recall (44.6%), failing to detect cross-session composed harm.
- Compression detector recall degrades from 59% to 43% under 25× adversarial session message duplication; Magnet is more robust, maintaining recall between 62%-73% (Table 1).
- Magnet’s monotonic accumulation of a multi-dimensional capability taxonomy reduces signal dilution and avoids premature evidence discarding seen in compression approaches.
- Capability-based detection decouples harmful output grounding from prompt intent, countering prompt laundering and masking.
Threat model
A black-box adversary able to issue arbitrary prompts and tool calls across an unbounded number of independent, stateless AI agent sessions; does not have access to internal model weights, safety stack internals, logs, or correlated session states. The attacker distributes a harmful goal as benign subtasks over multiple isolated sessions linked by a single user ID, circumventing per-session safety mechanisms. Attackers cannot or do not fragment identities across multiple correlators.
Methodology — deep read
The authors defined a threat model where an attacker, acting as a black-box, submits arbitrary prompts and tool calls but with no internal access to model weights or safety logs. The core attacker capability is to distribute queries over an unbounded number of stateless, freshly instantiated 'agentic' sessions, leveraging decomposition into innocuous subtasks that individually pass safety filters but compose into harmful artifacts cumulatively.
To evaluate this, the authors constructed an LLM-based multi-agent pipeline simulating attacker–assistant interactions. Each subtask in a predefined expert-authored decomposition of a harmful goal (e.g., phishing kit or Molotov cocktail) is dispatched sequentially to a fresh agent pair until complete, never sharing conversation histories, intents, or framing across subtasks. The agent has five tool calls (send email, search web, fetch webpage, save file, start server) to mimic realistic capabilities.
The dataset comprises 1080 experimental runs (20 runs × 9 models × 3 delivery modes × 2 attack scenarios). The delivery modes were single-turn (all subtasks in one prompt), multi-turn (subtasks sequential in one session), and multi-session (each subtask in a fresh session). Each output was labeled by an LLM judge according to an additive rubric measuring what fraction of the complete harmful capability was accrued (>=50% = successful attack). Judging was done without production safety filters to avoid blocking harmful artifact generation.
For detection, three approaches using the same gemini-3.5-flash judge model were compared: (1) per-session—each session judged independently for maliciousness; (2) compression detector—which maintains a top-K (K=20) ranked subset of cross-session messages embedded and filtered by semantic similarity to capability anchors, summarized into a single verdict; and (3) Magnet—which extracts a binary vector over an expert-authored capability taxonomy from each session, aggregating across sessions using a bitwise OR operation to form an accrued capability inventory per user ID.
Evaluation metrics were precision, recall, and F1 score on a balanced dataset combining attack runs and benign sessions sampled from the WildChat-1M dataset. Robustness was assessed by injecting increasing numbers of benign sessions to simulate dilution and by adversarial duplication of malicious sessions to test compression evictions. Statistical confidence intervals appear in the appendix but are not explicitly detailed here. The authors do not mention code or model weight release; the dataset includes proprietary experimental runs plus a public conversational corpus.
Overall, the reported pipeline demonstrates that multi-session goal decomposition succeeds across multiple LLM families and that Magnet’s capability-accumulation approach enables more effective and resilient detection of low-and-slow distributed attacks than alternatives.
Technical innovations
- Formulating cross-session goal decomposition as an adversarial statelessness attack exploiting AI agents’ session isolation to accrue harmful capabilities incrementally.
- Proposing Magnet, a capability-based detection primitive that tracks demonstrated harmful artifacts as a monotonic multi-dimensional binary vector aggregated across sessions for each user entity.
- Decoupling detection of intent (prompt framing) from capability (concrete outputs and tool effects) to harden defenses against intent laundering and masking.
- Demonstrating that compression-based detection primitives suffer recall degradation under benign noise dilution and adversarial message duplication, while Magnet’s explicit capability inventory remains robust.
Datasets
- Experimental multi-session decomposition runs — 1080 sessions across 9 LLM models and 2 attack domains — proprietary
- WildChat-1M — 1 million real ChatGPT conversational sessions — public
Baselines vs proposed
- Per-session detector: F1 = 61.0%, Precision = 96.2%, Recall = 44.6%
- Compression detector: F1 = 81.8%, Precision = 94.5%, Recall = 72.0%
- Magnet detector (proposed): F1 = 84.0%, Recall = 75.0% (Precision not explicitly stated but implied competitive)
- Compression detector recall drops from 59% to 43% under 25× adversarial message duplication; Magnet recall stable around 62%-73% under same condition
- Attack success rate (ASR) for decomposed multi-session delivery averaged 37.4%, higher than multi-turn (26.0%) and single-turn (18.7%)
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2608.02518.
Fig 1: The data generation pipeline.
Fig 2: Depiction of how ASR changes depending on how the request is packaged: in a single turn, over
Fig 3: Of unsuccessful attack attempts, this graph summarizes how many were due to safety blocks versus
Fig 4: Magnet detection technique, aggregating accrued capabilities by the attacker
Fig 5: (left): F1, precision, and recall scores per detector
Fig 6: Impact of noise dilution on F1 score across detectors
Fig 7 (page 9).
Fig 7: Spoofed login page
Limitations
- Evaluation limited to two harmful domains (phishing and incendiary device assembly); generalizability to wider domains untested.
- Relatively small number of runs per model and delivery mode, insufficient to rank models or yield high-confidence statistical conclusions.
- Fixed rubric weights and 50% success threshold chosen heuristically without principled calibration.
- Non-adaptive attacker assumed, operating under single correlator (user ID); does not explore evasion by account fragmentation or adaptive behavior.
- Detection relies on subject-matter-expert authored capability taxonomies, limiting coverage to known threat models and requiring manual effort to update.
- No investigation of memory-enabled agents that would break session statelessness assumption.
- No code or dataset release currently described, limiting reproducibility.
Open questions / follow-ons
- How to effectively detect cross-model routing attacks where attacker assigns subtasks to different LLM families for maximal evasion?
- What correlators beyond user ID (e.g., device fingerprints, network addresses) remain robust against an adaptive attacker who fragments activity?
- Can dynamic or learned capability taxonomies be developed to generalize detection to novel attack decompositions without expert manual coding?
- How to extend longitudinal detection techniques like Magnet to detect psychosocial trajectory harms, e.g., escalating self-harm or delusional spirals across sessions?
Why it matters for bot defense
For bot-defense and CAPTCHA practitioners, this work highlights an important evolving adversarial technique exploiting multi-session statelessness and task decomposition that current single-session monitoring techniques fail to address. It underscores the need for system designs that track and aggregate attacker capabilities longitudinally at an identity level rather than per interaction. This represents a shift toward entity-level behavioral inventories informed by concrete outputs — rather than reliance on prompt intent signals alone, which can be masked or obfuscated.
Practitioners should consider augmenting existing per-session detection with taxonomic, multi-dimensional capability tracking mechanisms that accumulate evidence across sessions, especially in agentic or multi-modal workflows. Magnet serves as a tractable detection primitive providing interpretable, tunable thresholds to flag capability accrual indicative of composite attacks. However, systems must also consider how to link across fragmented accounts and handle evolving capabilities beyond handcrafted taxonomies, posing ongoing research and engineering challenges for scalable, robust longitudinal bot defenses.
Cite
@article{arxiv2608_02518,
title={ Magnet: Detecting Cross-Session AI Misuse Through Capability Accumulation },
author={ Natalie Isak and Matthew Dressman },
journal={arXiv preprint arXiv:2608.02518},
year={ 2026 },
url={https://arxiv.org/abs/2608.02518}
}