Deontic Policies for Runtime Governance of Agentic AI Systems
Source: arXiv:2606.19464 · Published 2026-06-17 · By Anupam Joshi, Tim Finin, Karuna Pande Joshi, Lalana Kagal
TL;DR
This paper addresses a critical gap in governing autonomous agentic AI systems driven by large language models (LLMs) that invoke tools, manipulate data, and coordinate with peers across organizations. Traditional policy frameworks like XACML, Rego, and Cedar support only basic permit/prohibit rules and fail to capture richer enterprise governance needs such as obligations triggered by permitted actions, meta-policy conflict resolution, dispensations (waivers) of obligations, and ontological reasoning over domain hierarchies common in healthcare, cybersecurity, and data privacy. The authors propose AgenticRei, a runtime governance system using a deontic policy language built on the Rei framework expressed in OWL (Web Ontology Language), combined with a high-performance reasoning engine (RDFox). AgenticRei enforces policies deterministically outside the LLM at every action boundary (tool calls and agent-to-agent messages), preventing unauthorized actions even under adversarial prompt injection. It supports permissions, prohibitions, obligations with lifecycle tracking, meta-policy conflict resolution, semantic domain reasoning, and cross-pillar credential trust composition.
Through detailed examples drawn from security, healthcare, and financial services domains, the paper demonstrates how AgenticRei can express and enforce governance constraints that current production engines cannot, such as conditional permissions that trigger obligations, resolving rule conflicts by meta-policy rather than evaluation order, and prohibitions applying across evolving domain class hierarchies without enumerating every resource subtype. This approach closes a critical implementation gap between mandated control objectives in agent governance standards and their runtime enforcement, and it complements industry frameworks like A2AS. Preliminary runtime measurements show sub-10ms decision latency suitable for synchronous agent interactions. Ongoing work aims to integrate with cryptographic credential verification and enterprise agent frameworks.
Key findings
- Existing policy engines like XACML, Rego, and Cedar handle only permit/prohibit rules and lack constructs for obligations, meta-policy resolution, dispensations, and ontological reasoning.
- AgenticRei achieves sub-10ms end-to-end runtime policy decision latency, with raw RDFox reasoning queries completing under 1ms.
- Deontic policy obligations can enforce behaviors triggered by permitted actions, e.g., notifying a CISO after software installation, a capability absent in current frameworks.
- Meta-policy RulePriority constructs explicitly and audibly resolve policy conflicts (e.g., project-level permission overriding org-wide prohibition) without brittle evaluation-order heuristics.
- Ontological reasoning over domain class hierarchies allows a single prohibition on reading PHI at the class level to automatically protect all current and future PHI subclasses without manual updates.
- Cross-pillar trust composition embeds trusted credential issuers within policies; only credentials from these issuers can override prohibitions, supporting dynamic multi-authority trust management.
- A financial services use case combines prohibition of autonomous high-value transactions with dispensation by treasury officer approval credentials and mandatory transaction reporting obligations, exhibiting all key governance properties.
- AgenticRei logs structured audit records including matched rules, credential issuers, and policy version hashes to enable forensic reconstruction of decisions and accountability.
Threat model
The adversary consists of an attacker or compromised LLM that attempts to cause unauthorized agent actions through non-deterministic or adversarial prompt manipulation. The enforcement layer is outside and independent of the LLM, intercepting all outbound tool invocations or agent-to-agent messages. It deterministically permits or denies actions according to governance policies, preventing any policy-violating action from executing regardless of internal agent reasoning. The system does not aim to prevent prompt injection or reasoning manipulation within the LLM but to ensure policy compliance at runtime action boundaries.
Methodology — deep read
The authors take a requirements-driven approach to address governance challenges of agentic AI systems that invoke distributed tool calls and agent-to-agent messages.
Threat Model & Assumptions: The adversary may manipulate the LLM’s reasoning (e.g., prompt injection) to attempt prohibited actions. AgenticRei enforces policies deterministically outside the LLM at action boundaries. It accepts or rejects actions based on policy evaluation independent of the LLM. Credential verification is based on matching trusted issuer identifiers simulated in this prototype, pending cryptographic integration.
Data: No large traditional datasets are used. Policies are authored in Rei’s deontic framework expressed in OWL/RDF. Domain ontologies (e.g., healthcare PHI fragment, financial instruments) provide semantic grounding. Examples illustrate policy sources and enforced queries. No public dataset is referenced; domain ontology fragments are illustrative but conceptually aligned with standards like HL7 FHIR and FIBO.
Architecture / Algorithm: The system intercepts agent actions or messages at middleware boundaries and extracts a ⟨subject, action, resource⟩ triple plus credentials. This triple is evaluated against an ontology-enhanced Rei policy knowledge base loaded into an RDFox reasoner. The engine supports deontic predicates: permissions, prohibitions, obligations (deontic:provision), dispensations (waivers), and meta-policies for conflict resolution (metapolicy:RulePriority). Reasoning includes subclass inference and constraint satisfaction.
Training Regime: N/A; system is rule- and ontology-based rather than learned. The high-performance reasoner materializes inference closures at policy KB load time for fast lookup.
Evaluation Protocol: The evaluation is qualitative and example-driven, demonstrating expressive power beyond flat allow/deny engines. Performance benchmarking shows action evaluation latency under 10ms on a RHEL 9 host using RDFox 7.5. Security evaluation focuses on guaranteed deny on errors and adversarial robustness by enforcement external to the LLM rather than on LLM robustness itself.
Reproducibility: The prototype implementation is described but no public code or data release is noted. Some domain ontologies and detailed policy examples are provided in the paper. Credential verification is simulated.
Example End-to-End: An agent invokes a software installation tool (action). The TripleExtractor converts this to a structured triple with the invoking agent and target host typed as managed. The policy engine queries the Rei policy KB in RDFox, confirming permission with the attached obligation to notify CISO. The middleware returns PERMIT with obligation instructions to the agent. The obligation manager registers the notification duty with monitoring. The invocation executes. All decisions and metadata are logged for audit. This ensures that permission and associated governance duties are enforced at runtime even if triggered by adversarial LLM instructions.
Technical innovations
- Introduction of a deontic policy language integrated with OWL/RDF ontologies for runtime governance of agentic AI, enabling first-class modeling of obligations, dispensations, and meta-policies beyond permit/prohibit.
- Use of high-performance OWL/RDFS entailment reasoning (RDFox) at action boundaries for sub-10ms deterministic policy enforcement external to LLM reasoning.
- Explicit meta-policy conflict resolution semantics replacing brittle evaluation order heuristics common in flat rule engines.
- Cross-pillar policy composition embedding trusted cryptographic issuer identifiers within policies to enable semantic trust in credentials overriding prohibitions.
- Ontological reasoning enabling policies expressed over domain class hierarchies to automatically govern dynamically evolving resource types without manual updates.
Baselines vs proposed
- Rego and Cedar policy engines: no native obligation or meta-policy constructs; AgenticRei supports obligations and meta-policy with obligation lifecycle management.
- Flat ABAC or allow/deny engines: require enumeration of all resource subclasses; AgenticRei uses OWL ontologies for semantic subclass reasoning automatically applying policies to all subclasses.
- Evaluation latency: RDFox policy engine query execution <1ms; end-to-end decision latency under 10ms, meeting production synchronous interception needs.
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.19464.
Fig 1: AgenticRei runtime architecture. Both tool-call interception and A2A
Limitations
- Credential verification is currently simulated by trusted issuer matching rather than fully integrated cryptographic signature verification.
- Paper presents prototype and example-driven qualitative evaluation without extensive large-scale or adversarial testing of the system in real deployment environments.
- Obligation discharge enforcement is currently based on monitoring and audit, requiring future automation and more transparent enforcement.
- No public code or dataset release noted, potentially limiting reproducibility.
- Focuses on policy evaluation at action boundaries; does not address adversarial manipulation or robustness of LLM internal reasoning itself.
- Semantic domain ontologies in practice require curation and updating; paper uses illustrative fragments.
Open questions / follow-ons
- How to fully integrate cryptographic credential verification and revocation with the policy trust model in production environments?
- What methods can automate obligation discharge verification beyond deadline tracking and audit, enabling automated compliance enforcement?
- How does the system perform under adversarial attempts to evade policy enforcement or with dynamically changing agent or system states?
- What are scalable strategies for managing and updating evolving domain ontologies to maintain semantic grounding as enterprise environments change?
Why it matters for bot defense
Bot-defense engineers face increasing challenges from agentic AI systems capable of autonomous actions, tool invocation, and cross-organization coordination. AgenticRei’s approach to externalizing deterministic, semantics-rich governance at the point of action execution provides a blueprint for runtime enforcement that cannot be bypassed via internal LLM manipulations or prompt injections. The use of deontic logic enables encoding not only what is permitted but also what duties arise upon permission, a nuance critical for compliance and audit trails in bot-defense systems. Semantic reasoning over ontologies allows policies to scale with evolving resource types without brittle manual rule updates. Incorporating meta-policies for conflict resolution and cryptographic credential trust also aligns with multi-authority environments typical in bot ecosystems. While not a CAPTCHA or bot detection method per se, this paper’s governance framework can enhance the control layers surrounding advanced autonomous bots, aiding bot-defense practitioners dealing with increasingly capable agentic AI.
Cite
@article{arxiv2606_19464,
title={ Deontic Policies for Runtime Governance of Agentic AI Systems },
author={ Anupam Joshi and Tim Finin and Karuna Pande Joshi and Lalana Kagal },
journal={arXiv preprint arXiv:2606.19464},
year={ 2026 },
url={https://arxiv.org/abs/2606.19464}
}