Multi-Agent Firewall Architecture for Privacy Protection of Sensitive Data in Interactions with Language Models

Source: arXiv:2607.08282 · Published 2026-07-09 · By Hugo García Cuesta, Pablo Mateo Torrejón, Alfonso Sánchez-Macián

TL;DR

This paper addresses the privacy risks inherent in user and programmatic interactions with Large Language Models (LLMs), which often involve sending sensitive data without effective safeguards. The authors propose an open-source, modular multi-agent firewall architecture that intercepts LLM traffic via a browser extension and a transparent man-in-the-middle proxy to prevent data leakage. The core innovation is a flexible multi-agent detection pipeline combining deterministic pattern matching, named entity recognition (NER), multimodal analysis (OCR and vision-language models), and proprietary code leakage detection, layered to efficiently escalate from lightweight to semantic analysis. This local-first design enables organizations to maintain data sovereignty and adapt detection depth and latency tradeoffs across deployment scenarios from single users to centralized enterprise servers. Evaluations demonstrate strong privacy protection with F1 detection scores reaching 94.93% under optimal configurations, supporting a range of file formats and communication protocols.

Key findings

The multi-agent pipeline achieves up to 94.93% F1 score on sensitive data leakage detection with optimal settings.
Dual-layer interception covering browser-based web interactions and API calls (including HTTP(S) and WebSocket) enables comprehensive traffic capture.
OCR combined with vision-language models allows multimodal analysis supporting PDF, images, and code files with low-latency fallback strategies.
The layered detection pipeline conditionally escalates from deterministic regex and keyword matching to computationally expensive LLM semantic analysis, optimizing latency without sacrificing detection accuracy.
The proxy blocks or drops sensitive content automatically (fail-closed), while the browser extension supports human-in-the-loop moderation with configurable risk thresholds.
Code leakage prevention uses fuzzy string matching against indexed private/public code repositories to detect obfuscated or paraphrased proprietary code leaks.
Modular configuration files (pipeline.json, detection.json) allow tailored deployment and quick adaptation to new platforms or risk policies without code changes.
The architecture supports deployment on commodity hardware for individuals or centralized servers for organizational scale with multi-user authentication and shared policy management.

Threat model

The adversary attempts to extract sensitive personal or proprietary data submitted by users during interactions with remotely hosted LLMs, potentially through careless or malicious prompt formulations. The threat includes inadvertent disclosure of PII, confidential documents, or intellectual property embedded in queries. The adversary cannot bypass local interception mechanisms (browser extension or proxy) and has no access to internal detection logic or backend servers. The model does not currently protect against adversaries compromising local user devices or orchestrating insider attacks.

Methodology — deep read

Threat model and assumptions: The adversary is any entity attempting to exfiltrate sensitive or proprietary information submitted by users to external LLM APIs or web interfaces. The system assumes attackers may exploit user input channels but cannot compromise local detection logic or evade interception layers. It does not currently address insider threats or compromise of the local environment. Adversaries do not control the firewall components.
Data: The paper does not publish a proprietary dataset but uses a combination of simulated inputs containing PII fields, proprietary code snippets, and multimodal document uploads across supported formats (text, PDF, images). Specific dataset sizes or splits are not detailed. Labels for sensitive fields are defined via regex, keyword lists, and NER models.
Architecture/algorithm: The architecture consists of three components: a browser extension intercepting web LLM chats; a transparent MiTM proxy for programmatic API traffic (including HTTP(S) and WebSocket); and a backend server hosting the multi-agent detection pipeline. The detection pipeline is orchestrated as a Directed Acyclic Graph (DAG) state machine implemented via LangGraph, where nodes perform deterministic detection, OCR,VLM-based text extraction, Named Entity Recognition (NER) for semantic entity extraction, and proprietary code similarity detection using fuzzy string matching. Detection nodes produce risk scores which feed into a centralized policy engine that assigns threat severities and enforces blocking, warnings, or remediation (such as automatic data sanitization). The pipeline configuration is externalized in JSON files enabling flexible node arrangements and detection parameter tuning.
Training regime: NER agents leverage pre-trained SpaCy models for entity detection; LLM-based semantic detectors leverage configurable local or cloud LLMs accessible via abstracted LiteLLM interfaces. No fine-tuning is reported. The OCR uses Tesseract with confidence thresholds, escalating to VLMs for complex images only when needed. The paper does not report training epochs or hyperparameters since most components rely on existing models and heuristics.
Evaluation protocol: Evaluation focuses on detection performance (precision, recall, F1) over a mixture of synthetic and real-world scenarios with multimodal inputs. Ablations include measuring the effect of disabling components such as NER or code detection. Latency and throughput measurements are provided for single-device and centralized deployments. Fail-closed policies are tested for backend unavailability scenarios. Statistical testing or cross-validation details are not fully specified.
Reproducibility: The system and code are open-source. However, exact datasets and ground truth annotations used for evaluation are not publicly disclosed, limiting replication of numeric results. Configuration files are published allowing researchers to recreate pipeline settings and detection policies.

Example end-to-end flow: A user submits a prompt containing potentially sensitive PII or code via a web LLM interface. The browser extension intercepts the message before sending. The prompt triggers deterministic regex matching which flags certain patterns, escalating to NER for semantic entity recognition. If an uploaded document accompanies the prompt, OCR extracts text, escalating to VLM for complex images. Proprietary code fingerprinting compares against indexed organization repositories using fuzzy matching. The detections are aggregated, risk scored, and passed to the policy engine. If risk exceeds the configured threshold, the extension blocks the prompt and notifies the user with options to sanitize or override. The backend logs the event for auditing. This inspection path balances detection accuracy with latency by short-circuiting deeper analysis for unthreatening inputs.

Technical innovations

A hybrid multi-agent detection pipeline orchestrated as a configurable directed acyclic graph to efficiently escalate from fast deterministic checks to semantic LLM reasoning.
Dual-layer traffic interception combining browser extensions and a transparent man-in-the-middle proxy to cover both web and API LLM interactions including WebSocket connections.
Multimodal data leakage prevention integrating OCR and vision-language models with adaptive fallback strategies for robust document and image analysis.
Proprietary code leakage detection leveraging fuzzy string matching against indexed code repositories to detect paraphrased or modified source code leaks with semantic context.

Baselines vs proposed

Deterministic detectors only: F1 = lower (exact values not specified) vs full multi-agent pipeline: F1 = up to 94.93%
NER-disabled pipeline: detection accuracy decreases (exact numbers not provided)
Fail-closed proxy configuration: blocks 100% of sensitive content when backend unavailable vs fail-open allowing leakage

Limitations

Evaluation relies on simulated and synthetic data inputs; no large-scale real-world deployment results reported.
No comprehensive adversarial testing against active evasion attacks such as sophisticated prompt injections or stealthy code obfuscation.
The layered architecture introduces latency, especially when escalating to costly semantic LLM analysis, which may impact user experience on constrained hardware.
Multimodal analysis currently supports PDF, common image, and text/code files; advanced formats (e.g., .docx) require additional integration and were not evaluated.
Code leakage detection depends on indexing known repositories and may miss leaks from unknown or external sources.
System’s effectiveness depends on deployment configuration and organizational tuning; default parameters may not fit all environments.

Open questions / follow-ons

How robust is the multi-agent detection pipeline against adaptive adversaries who craft inputs to evade deterministic and semantic detectors (e.g., prompt injections, adversarial examples)?
Can the architecture scale efficiently for large organizations with thousands of users, balancing latency and compute costs while maintaining consistent policy enforcement?
What are the privacy implications and trust boundaries when using cloud-based LLM detectors versus fully local inference, and how does this impact detection accuracy?
How can the framework be extended to monitor and intercept sensitive information disclosed in LLM outputs, not only inputs?

Why it matters for bot defense

Bot-defense and CAPTCHA practitioners interested in privacy-preserving security controls for LLM integrations can learn from this paper’s modular, local-first interception and detection design. The hybrid multi-agent pipeline illustrates how combining deterministic pattern matching, semantic entity recognition, and multimodal analysis can effectively prevent data leakage in real user interactions without excessive latency. The dual interception (browser extension plus proxy) model provides a flexible blueprint to secure both interactive and API-driven AI consumption patterns, a growing attack surface in automation and fraud contexts. Furthermore, the system’s layered escalation and configurable policies showcase a practical approach to balance security, usability, and resource constraints that CAPTCHA engineers may adapt for protecting AI-driven authentication and user verification workflows. However, the framework remains vulnerable to adversarial evasion tactics, highlighting the need for ongoing research in robust semantic detection integrated into bot mitigation pipelines.

Cite

bibtex

@article{arxiv2607_08282,
  title={ Multi-Agent Firewall Architecture for Privacy Protection of Sensitive Data in Interactions with Language Models },
  author={ Hugo García Cuesta and Pablo Mateo Torrejón and Alfonso Sánchez-Macián },
  journal={arXiv preprint arXiv:2607.08282},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.08282}
}

Multi-Agent Firewall Architecture for Privacy Protection of Sensitive Data in Interactions with Language Models ​

TL;DR ​

Key findings ​

Threat model ​

Methodology — deep read ​

Technical innovations ​

Baselines vs proposed ​

Limitations ​

Open questions / follow-ons ​

Why it matters for bot defense ​

Cite ​

Read the full paper ​