Can Trustless Agents Be Trusted? An Empirical Study of the ERC-8004 Decentralized AI Agent Ecosystem
Source: arXiv:2606.26028 · Published 2026-06-24 · By Xihan Xiong, Zelin Li, Wei Wei, Qin Wang, William Knottenbelt, Zhipeng Wang
TL;DR
This paper provides the first comprehensive empirical investigation of the ERC-8004 protocol, which implements a permissionless trust layer for decentralized AI agent ecosystems. ERC-8004 aims to support autonomous AI agents that transact and interoperate across organizational boundaries without pre-existing trust. The protocol provides on-chain registries for Identity, Reputation, and Validation, enabling agents to discover and assess counterpart trustworthiness. Despite rapid adoption across Ethereum, BNB Smart Chain (BSC), and Base networks, there was previously no empirical analysis of whether the on-chain identity and reputation data reliably reflect genuine active agents or trustworthy reputation signals.
By crawling the entire on-chain Identity and Reputation events and fetching off-chain registration and feedback files from deployment through May 2026, the authors characterize agent registration behavior, the gap between registration and true activity, reputation market dynamics, and vulnerabilities such as Sybil attacks. They find that most agent registrations are placeholders lacking valid registration files or active services, with only 3-15% of agents exposing valid registration files with service endpoints. The reputation system fails to meet core criteria for trustworthiness as feedback is often ungrounded, manipulable at minimal cost, and dominated by coordinated Sybil reviewers. Removing Sybil feedback leaves large fractions (15-89%) of agents with no valid reputation.
These findings highlight significant gaps and weaknesses in the ERC-8004 ecosystem, yielding concrete design recommendations to improve identity activation requirements, enforce verifiable feedback provenance, and mitigate Sybil attacks. The study establishes an empirical baseline and critical insights for decentralized AI agent trust protocols and markets.
Key findings
- Only 3%, 4%, and 15% of registered agents on Ethereum, BSC, and Base respectively expose valid ERC-8004 registration files with at least one live declared service endpoint.
- Batch registrations account for 48.3% of all agents on Ethereum but only 0.0% on BSC and 7.8% on Base, driving high ownership concentration (Gini 0.733 Ethereum; 0.708 Base; 0.134 BSC).
- 53% of Ethereum agents and 37% of Base agents never activate their URI registration file by the end of observation; 91% of BSC agents are active at mint.
- Median manipulation cost to fabricate or destroy reputation scores is $0.055 on Ethereum, $0.0042 on BSC, and $0.0027 on Base, enabling low-cost reputation gaming.
- Sybil-flagged reviewers constitute 73.6% of reviewers on Ethereum, 59.2% on BSC, and 90.6% on Base, affecting 26.1%, 75.8%, and 96.8% of rated agents respectively.
- After removing Sybil feedback, 15.5% (Ethereum), 72.3% (BSC), and 89.4% (Base) of rated agents have no valid reputation feedback remaining.
- Reputation feedback values are often non-commensurable and lack provable interaction evidence; feedback distributions differ significantly across chains.
- Reputation reviewer participation is highly concentrated on BSC (76 reviewers submitting avg 387 feedbacks each) compared to Ethereum and Base.
Threat model
The adversary is an unknown AI agent or entity attempting to game or subvert the decentralized trust layer by registering multiple Sybil agents, submitting fraudulent or ungrounded reputation feedback, or withholding registration file activation. The adversary can manipulate feedback at low economic cost due to minimal transaction fees and lacks a centralized gatekeeper. They cannot alter on-chain immutable registration or feedback event logs but can exploit protocol design weaknesses to fabricate reputation or obscure identity activation status.
Methodology — deep read
The study treats ERC-8004 as a deployed decentralized trust layer protocol for autonomous AI agents on three EVM-compatible chains: Ethereum, BNB Smart Chain (BSC), and Base. The threat model involves unknown counterpart AI agents needing to assess trustworthiness without centralized intermediaries or pre-existing relationships. The adversary can mount Sybil or low-cost reputation manipulation attacks but cannot compromise on-chain immutability.
The authors collected the entire on-chain historical data from Identity and Reputation Registries across all three chains from deployment (Ethereum from Jan 29, 2026) through May 13, 2026. This includes every event relevant to agent registration (Registered, URIUpdated, Transfer) and reputation (NewFeedback, FeedbackRevoked, ResponseAppended). They also fetched and parsed referenced off-chain JSON registration files and feedback payloads (typically hosted on HTTPS or IPFS).
They quantified various metrics: registration counts, URI activation status, batch minting, registration file validity against the ERC-8004 schema, declared services, and wallet ownership concentration (Gini). For reputation, they analyzed counts of feedback records, reviewers, rated agents, feedback value distributions, tag semantics, and the presence of Sybil-like reviewer behavior. Sybil detection was performed via behavioral clustering and network analysis heuristics described in the paper.
The authors also measured the gas and USD cost to submit registrations and feedback transactions to evaluate economic feasibility of attacks. By filtering out Sybil reviewers and their feedback, they assessed the robustness of the reputation to manipulation.
No mainnet deployment of the Validation Registry limited study to Identity and Reputation components. The evaluation focused on descriptive statistics, distributions, ownership concentration via Lorenz curves, and cross-chain comparisons. The authors provide detailed event crawling, off-chain data reconciliation, and integration with payment transaction data (x402) for provenance where available.
They present representative figures (e.g., Fig 2, 4, 5, 7, 8, 9, 10) illustrating trends and distributions. The study also examines submission timing distributions (activation lag), URI scheme usage, declared agent service types, and tag semantics. They release the assembled dataset upon request to encourage reproducibility.
An example end-to-end data path for a single agent includes tracking its Registered event to assign ID and owner, monitoring subsequent URIUpdated events to capture registration file URI, fetching and verifying the JSON file format and contents, collecting all NewFeedback events impacting that agent, and indexing off-chain feedback metadata and payment proofs for assessment. Aggregated across agents, this pipeline supports the systemic integrity and trust analysis presented.
Overall, the methodology rigorously combines on-chain event data, off-chain content validation, economic cost analysis, and behavioral clustering to empirically evaluate the real-world functioning and vulnerabilities of ERC-8004.
Technical innovations
- First permissionless decentralized trust layer protocol (ERC-8004) combining on-chain Identity (ERC-721-based), Reputation, and Validation registries for AI agents.
- Use of transferable NFTs for portable agent identity tokens linked to off-chain JSON registration files exposing service capabilities.
- Reputation registry design exposing compact on-chain feedback signals linked to richer off-chain evidence documents.
- Pluggable Validation Registry concept for high-assurance protocol layer combining stake-secured re-execution, zkML proofs, and TEE attestations (not yet deployed).
- Integration with machine-native payment rail (x402) enabling atomic settlement within agent request lifecycles.
Datasets
- ERC-8004 Multi-chain Dataset — 170,000+ registered agents, 150,000+ feedback records — Crawled from Ethereum, BSC, and Base mainnets, data collection from protocol deployment (Jan/Feb 2026) through May 13, 2026
Baselines vs proposed
- n/a - This is the first empirical study; no prior baselines exist for deployed ERC-8004 usage metrics or reputation trustworthiness.
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.26028.
Fig 4: Ownership concentration of agents across Ethereum, BSC, and Base (Lorenz curves).
Fig 17: Feedback records classified by the strongest
Limitations
- Validation Registry component untested due to no confirmed mainnet deployment during observation period, limiting scope to Identity and Reputation registries.
- Sybil detection relies on heuristics and clustering which may have false positives/negatives; comprehensive adversarial robustness evaluation is absent.
- Data collection limited to three EVM-compatible chains; results may not generalize to other chains or future protocol versions.
- Analysis cannot prove ground-truth agent service activity beyond metadata and endpoint declarations; off-chain identity verification remains challenging.
- Feedback files and payment proofs are off-chain and may be incomplete or unavailable, limiting trustworthiness assessment.
Open questions / follow-ons
- How can validation mechanisms (stake-secured re-execution, zkML proofs, TEE attestations) be integrated effectively to provide high-assurance trust signals beyond reputation feedback?
- What automated or cryptographic protocols can enforce verifiable linkage between feedback records and real-world interactions or payments to strengthen provenance?
- How can Sybil attack resistance be improved in permissionless trust layers without sacrificing openness and decentralization?
- What cross-chain identity and reputation portability models best preserve reputation integrity in a multi-chain AI agent economy?
Why it matters for bot defense
For practitioners in bot-defense and CAPTCHA, this study illustrates critical trust challenges when autonomous agents operate and interact in decentralized, permissionless settings without prior relationships. Much like CAPTCHAs distinguish bots from humans, ERC-8004 seeks to help AI agents evaluate unknown counterparts’ trustworthiness without centralized intermediaries. The findings emphasize that naive on-chain registries and reputation schemes can be easily manipulated by Sybil or placeholder registrations, eroding trust signals. This underscores the importance of verifiable identity activation, robust reputation provenance linked to real interactions or payments, and economically costly Sybil resistance mechanisms.
In bot-defense contexts, lessons from ERC-8004’s gaps highlight how relying solely on public feedback or self-declared metadata can be insufficient to differentiate genuine active agents from placeholders or adversaries. Incorporating cryptographic attestations or economic staking (akin to proof of work or proof of stake) may be necessary to build trustworthy decentralized identity and reputation systems. Thus, this paper provides foundational empirical baseline insights into the design and deployment pitfalls of decentralized trust protocols for autonomous agents, relevant for CAPTCHAs and bot-detection mechanisms in distributed AI ecosystems.
Cite
@article{arxiv2606_26028,
title={ Can Trustless Agents Be Trusted? An Empirical Study of the ERC-8004 Decentralized AI Agent Ecosystem },
author={ Xihan Xiong and Zelin Li and Wei Wei and Qin Wang and William Knottenbelt and Zhipeng Wang },
journal={arXiv preprint arXiv:2606.26028},
year={ 2026 },
url={https://arxiv.org/abs/2606.26028}
}