TSAI-MetaFraud: A Benchmark Dataset for Financial Fraud Transaction and Behavioral Risk Detection in Metaverse Ecosystems

Source: arXiv:2607.09528 · Published 2026-07-10 · By Refat Ishrak Hemel, Ehsan Hallaji, Roozbeh Razavi-Far

TL;DR

TSAI-MetaFraud addresses the critical gap in multimodal fraud detection datasets for metaverse ecosystems by providing a rich benchmark that integrates behavioral data, financial transactions, and graph-structured relations within a realistic virtual economy. Unlike prior datasets that focus on isolated aspects such as user behavior or transaction records, TSAI-MetaFraud encompasses avatar interactions, session biometrics, transaction flows, and fraud/bot labels in a single unified framework. The dataset is generated through a simulation built on OpenSimulator, incorporating diverse user types including benign, behavioral fraud (bot), financial fraud, hybrid fraud, and unknown categories with masked labels to emulate real-world uncertainty.

The paper defines four benchmark tasks—transaction fraud detection, cross-modal node classification, temporal link prediction, and weakly supervised fraud detection—and evaluates several classical machine learning models and graph neural networks. Results show that tree-based models like Random Forest and XGBoost perform well on behavioral fraud detection but fail on financial and hybrid fraud, whereas graph neural networks (GraphSAGE) leveraging relational structure achieve meaningful performance across all fraud types. The dataset’s complexity, class imbalance, multimodality, temporal dynamics, and relational sparsity produce a challenging environment for future research on trustworthy AI in metaverse financial security.

Key findings

• TSAI-MetaFraud contains 936 active avatars, 74,671 financial transactions, and 230,490 behavioral interaction events in a simulated metaverse environment.
• Fraud class imbalance is severe: 45 behavioral fraud, 16 financial fraud, 10 hybrid fraud accounts, plus 400 unknown, with majority benign users.
• Baseline Random Forest achieves F1-score 0.90 for behavioral fraud detection but near zero for financial and hybrid fraud in transaction classification (Table VI).
• GraphSAGE model attains balanced detection across all fraud categories, demonstrating the critical value of graph structure for detecting complex fraudulent behavior.
• Cross-modal node classification shows tabular models excel on benign and behavioral fraud nodes (F1 up to 0.98 and 0.90), but struggle on financial and hybrid fraud classes.
• Temporal link prediction task models forecast future transactions in dynamic graphs; baselines show promise but indicate challenges in non-stationary settings (details truncated).
• Weak supervision setting with 90% label masking tests fraud detection under restricted supervision, reflecting real-world investigative constraints.
• TSAI-MetaFraud enables evaluation under strict inductive splits where test accounts are unseen during training, supporting generalization assessments.

Threat model

The adversary is a malicious virtual economy participant capable of executing behavioral fraud via automated bot accounts (with varying sophistication from naive to stealth bots) and financial fraud through illicit transactions such as layering and structuring. Hybrid adversaries combine these tactics. The adversary cannot bypass the simulation controls or obfuscate all traces, but may attempt to mimic benign behavioral patterns to evade detection. The defender models assume access to avatar behavioral biometrics, transactional data, and network relations but no real-world private or external data.

Methodology — deep read

1. Threat Model & Assumptions: The adversary is modeled as a malicious actor who can control accounts in the metaverse ecosystem to conduct behavioral fraud (bots with synthetic interactions), financial fraud (transaction layering/structuring), or a hybrid of both. The model assumes no access to private data beyond the released dataset and excludes attackers capable of circumventing platform-level controls. Detection targets fraud embedded within realistic economic and social activity.

2. Data Generation & Provenance: The dataset is generated fully synthetically via simulation in OpenSimulator, an open-source multi-user virtual world platform supporting avatar interactions and virtual currency transactions. The environment includes multiple regions (commercial, trading, residential). Participants include 936 active avatars with different profiles: benign, behavioral fraud bots (naive and stealth with human-like keystroke timing distributions), financial fraudsters, hybrid fraudsters, and partially labeled unknown accounts. Behavioral data includes keystroke dynamics modeled from the public KMT keystroke dataset. Transaction data logs all payment events and fees. Data was collected continuously with session IDs partitioning temporal intervals.

3. Architecture / Algorithm: For baselines, classical ML classifiers such as Logistic Regression, Random Forest, and XGBoost operate on a 24-dimensional tabular feature vector combining transaction attributes, sender/receiver network centralities, and session biometrics. Graph-based approaches use a heterogeneous GraphSAGE model encoding avatar nodes, session nodes, and transaction edges to leverage multi-relational graph structure. GraphSAGE is applied both as a node classifier and a temporal link predictor on dynamic graphs formed by time-segmented transaction snapshots. Label propagation is evaluated for weakly supervised fraud classification.

4. Training Regime: Classical models undergo 5-fold cross-validation hyperparameter tuning on training splits. GraphSAGE models are trained for 100 epochs with cross-entropy loss on node or edge classification tasks. Hardware used includes an Apple Silicon workstation with 16GB unified memory. A strict inductive split partitions 80% avatars for training and 20% for testing, ensuring test transactions involve wholly unseen accounts.

5. Evaluation Protocol: Metrics reported include class-specific Precision, Recall, and F1-score, with emphasis on macro F1 to balance performance across majority (benign) and minority (fraud) classes. Transaction fraud detection and node classification tasks use inductive splits; temporal link prediction uses one-hour chronological graph snapshots with future interval prediction; weak supervision masks 90% of training labels simulating label scarcity. Negative samples for link prediction are randomly sampled non-existent edges. No mention of cross-validation beyond hyperparameter tuning; reproducibility enforced by dataset and code release.

6. Reproducibility: The dataset and evaluation framework are publicly released at https://github.com/tsai-unb/MetaFraud, including graph and tabular formats. Behavioral and financial data are synthetic, avoiding privacy concerns. Baseline implementations are provided in PyTorch/PyTorch Geometric and Scikit-Learn/XGBoost. Detailed data schema and processing steps are documented. However, no frozen weights or pre-trained models are described.

Concrete example: Transaction fraud detection task requires classifying each transaction edge connecting previously unseen avatars into one of four classes using transaction attributes and graph features. For instance, a payment edge with benign sender and hybrid fraud receiver will be evaluated; baseline Random Forest may classify it as benign (due to lack of graph structure), but GraphSAGE better leverages connectivity and achieves improved recall on hybrid fraud edges.

Technical innovations

• Creation of a comprehensive multimodal dataset combining avatar behavioral biometrics, financial transactions, and graph-structured interactions within a simulated metaverse environment.
• Definition of multiple benchmark tasks tailored to metaverse fraud analytics: transaction fraud detection, cross-modal node classification, temporal link prediction, and weakly supervised fraud detection.
• Integration of human-derived keystroke timing models with virtual-world behavioral traces to realistically simulate and differentiate benign and bot activity.
• Use of strict inductive splits on avatars for evaluation, ensuring benchmark validity for generalization to unseen accounts in dynamic multi-relational graphs.

Datasets

• TSAI-MetaFraud — 936 avatars, 74,671 transactions, 230,490 behavioral interactions — simulated via OpenSimulator

Baselines vs proposed

• Random Forest (tabular): Behavioral Fraud F1 = 0.90 vs Financial Fraud F1 ≈ 0
• XGBoost (tabular): Behavioral Fraud F1 = high (exact 0.90 not specified) vs Financial Fraud F1 ≈ 0
• GraphSAGE (graph): Balanced F1 across Behavioral, Financial, Hybrid fraud (exact values not fully specified)
• Label Propagation: Baseline for weakly supervised fraud detection (performance details truncated)

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.09528.

Fig 1: Architecture of the TSAI-MetaFraud data generation pipeline. Behavioral telemetry and financial transactions are collected from the OpenSimulator

Fig 2: Class distribution of nodes and edge labels, highlighting severe class

Fig 3: Transaction flow between account categories. Ribbon widths are pro-

Fig 4: Spatial density heatmap showing avatar coordinates and active hot-

Fig 5: Network representations of TSAI-MetaFraud, including the transaction-to-transaction money flow graph, avatar interaction graph, and avatar–transaction

Limitations

• Dataset is entirely synthetic and simulated, limiting assessment on real-world metaverse fraud distributions and adversarial adaptations.
• Financial fraud labels and behaviors are designed but may not cover all novel or emergent attack vectors in production metaverse economies.
• Evaluation is limited to standard machine learning and GraphSAGE baselines; no adversarial robustness tests or performance under adaptive attackers.
• Temporal link prediction results and details are truncated, reducing visibility into performance under evolving dynamics.
• Unknown category with masked labels simulates investigation scenarios but does not provide ground truth, limiting fully supervised learning analyses.

Open questions / follow-ons

• How do models perform against adaptive adversaries actively trying to mimic benign behaviors or altering transaction patterns dynamically?
• Can multimodal fusion methods beyond baseline classical and GraphSAGE models improve detection accuracy on minority fraud classes and temporal prediction?
• What is the impact of distribution shift over longer time scales as virtual economies evolve or as new attack types emerge?
• How can weakly supervised or unsupervised learning approaches leverage TSAI-MetaFraud to improve discovery of unknown or emerging fraud patterns?

Why it matters for bot defense

TSAI-MetaFraud is highly relevant for bot-defense and CAPTCHA practitioners focused on virtual economy ecosystems or metaverse platforms where heterogeneous data sources (behavioral biometrics, financial transactions, and network relations) are available. The dataset’s multimodal and graph-structured nature encourages development of detection mechanisms that combine behavior profiling with relational fraud signals rather than relying on transactional analysis alone. The benchmark’s inclusion of stealth bots with human-like keystroke timing highlights the difficulty of detecting sophisticated automated actors, underscoring the need for advanced graph-based and temporal inference methods. Practitioners can leverage TSAI-MetaFraud to evaluate and refine CAPTCHA-triggering heuristics or bot risk scoring models under realistic cross-modal fraud scenarios and label scarcity conditions typical in production environments.

Cite

@article{arxiv2607_09528,
  title={ TSAI-MetaFraud: A Benchmark Dataset for Financial Fraud Transaction and Behavioral Risk Detection in Metaverse Ecosystems },
  author={ Refat Ishrak Hemel and Ehsan Hallaji and Roozbeh Razavi-Far },
  journal={arXiv preprint arXiv:2607.09528},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.09528}
}

Read the full paper

• arXiv:2607.09528 (abstract)
• arXiv:2607.09528 (PDF)