CoDiMAD: Diffusion-Based Privileged Distillation for Communication-Free Multi-Robot Coordination

Source: arXiv:2607.09587 · Published 2026-07-10 · By Jiyue Tao, Shunheng Xin, Tongsheng Shen, Dexin Zhao, Feitian Zhang

TL;DR

This paper addresses the challenge of decentralized multi-robot coordination under partial observability in communication-free settings. The core problem is that the same local observation can correspond to multiple distinct global states, each requiring different cooperative actions, resulting in inherently multi-modal conditional action distributions. Traditional deterministic distillation, which regresses to the mean action, collapses these modes and produces invalid or hesitant actions. The authors propose CoDiMAD, a three-stage privileged distillation framework where a MAPPO-trained oracle with full global observations generates an offline dataset of local-observation to oracle-action pairs. This data is then used to train decentralized students parameterized as conditional denoising diffusion probabilistic models. These diffusion models explicitly approximate the multi-modal conditional action distribution and sample coherent coordination modes instead of collapsing modes into invalid averages.

The paper includes a formal theoretical analysis proving the mode-averaging failure of deterministic regression-based distillation and the conditional density modeling capability of diffusion-based distillation. Experiments on three cooperative multi-robot tasks—coverage, pursuit-evasion, and box pushing—demonstrate that CoDiMAD consistently outperforms direct MAPPO trained on local observations and deterministic behavioral cloning baselines, closing the gap toward the privileged oracle policy. Results show significant improvements in task success rates and large reductions in collisions, especially in challenging multi-agent coordination requiring multi-modal actions.

Overall, CoDiMAD advances privileged policy distillation for multi-agent robotics by integrating diffusion modeling to address inherent multi-modality caused by partial observability and lack of communication.

Key findings

• Partial observability induces multi-modal conditional action distributions in decentralized multi-agent coordination where identical local observations correspond to multiple distinct valid global configurations (Proposition 1).
• Deterministic behavioral cloning minimizing MSE regresses to the conditional mean of multimodal distributions, placing predicted actions in low-probability regions that cause invalid or collision-prone coordination behaviors (Corollary 1).
• CoDiMAD's diffusion-based policy models conditional oracle action distributions and samples decisive multimodal actions, avoiding mode averaging failure inherent to deterministic distillation (supported by theory and empirical visualizations in Fig. 3).
• On the coverage task, CoDiMAD achieves 95.7% map coverage with 0.55 collisions per episode versus BC-RNN's 80.9% coverage and 10.37 collisions, a 14.8 percentage-point improvement and approx. 19x fewer collisions.
• On pursuit-evasion, CoDiMAD attains 90.6% evader capture rate with 0.80 collisions, outperforming MAPPO-Local (14.2% capture, 1.89 collisions) and BC-RNN (90.3% capture, 3.41 collisions).
• On box pushing, CoDiMAD achieves 72.2% success rate compared to BC-RNN's 6.5% and MAPPO-Local's 18%, showing the biggest margin on force-coordination-intensive tasks.
• Temporal history encoding via a GRU improves performance; removing it leads to consistent drops in success and increased collisions across tasks.
• Diffusion policies produce diverse but coherent multi-agent trajectories from the same initial observation, in contrast to deterministic policies collapsing to a single averaged trajectory (Fig. 4).

Threat model

n/a — This is a cooperative multi-agent reinforcement learning and imitation learning framework focused on decentralized control under partial observability rather than an adversarial security setting.

Methodology — deep read

The paper addresses decentralized multi-robot coordination under partial observability without communication. The adversary is not explicitly modeled as this is a cooperative multi-agent learning study. The main challenge is that each agent's limited local observation may correspond to multiple distinct global states, inducing multi-modal distributions over optimal actions, which deterministic regression-based student policies cannot represent.

Data was generated by first training a privileged oracle policy via MAPPO using full global state observations alongside local sensor maps. The oracle policy is shared across homogeneous agents, encoding dual-stream inputs (local maps and global privileged maps) via convolutional neural networks and a GRU. The centralized critic uses joint global states for value estimation. After convergence, the oracle policy is frozen and rolled out deterministically (using mean actions) to generate an offline dataset of tuples containing the student-observable local observation, oracle action, and done indicators filtered to keep only successful, collision-free episodes. Actions are normalized dimension-wise.

The student policy is a conditional denoising diffusion probabilistic model (DDPM) that is trained to model the conditional distribution of oracle actions given only local observations over time. It consists of a single-branch CNN-GRU encoder to process local observations and produce a conditioning embedding, and a noise prediction network composed of residual MLP blocks that predicts noise to iteratively denoise perturbed oracle actions. The forward diffusion corrupts oracle actions over 200 timesteps with a cosine noise schedule, and the loss trains the network to predict the noise added.

To enable real-time inference, DDIM deterministic sampling with 20 denoising steps is employed to generate multi-modal action samples. This reduces inference latency by approximately 10x compared to the full DDPM reverse process, while retaining the ability to sample from different coordination modes by varying the initial noise.

Evaluation uses three robotics coordination tasks simulating cooperative coverage, pursuit-evasion, and box pushing in continuous environments with three agents. Baselines include the privileged MAPPO oracle (upper bound), MAPPO trained only on local observations (end-to-end local RL), deterministic behavior cloning with a recurrent encoder (BC-RNN), and an ablation without temporal history encoding. Each method is run on three random seeds with 200 episodes each.

Metrics include task-specific success rates (e.g., coverage completion, evader capture, box pushing success) and collision counts (inter-agent and obstacle collisions). The theoretical framework models the conditional oracle action distribution as a Gaussian mixture over latent global states, formalizing mode averaging failure and diffusion policy advantages.

Reproducibility is partially supported by promising to release code upon acceptance. However, no mention is made of frozen weights or open datasets. Experiments leveraged an NVIDIA RTX 4090 GPU. Oracle training was 5 million environment steps; student trained for 50 epochs with batch size 128 using AdamW optimizer.

An example end-to-end flow: first train oracle MAPPO with privileged inputs; second, collect local observation and oracle actions from oracle rollouts only retaining successful episodes; third, train diffusion student with encoded local observations as conditions to learn noise prediction for reconstructing oracle actions. Finally, at inference, generate action samples via DDIM conditioned on current local observations to produce coordinated, decisive multi-agent behavior that captures multi-modal distributions induced by partial observability.

Technical innovations

• The identification and formal proof that partial observability causes inherently multi-modal conditional action distributions in multi-agent privileged distillation, causing failure of deterministic regression-based behavioral cloning.
• Integration of conditional denoising diffusion probabilistic models as decentralized student policies to model multi-modal distributions over cooperative actions purely from local observations in a multi-robot coordination context.
• Adaptation of DDIM sampling to accelerate diffusion policy inference from 200 to 20 steps without retraining, enabling practical real-time onboard multi-robot deployment.
• A three-stage framework combining MAPPO oracle training, offline dataset collection of local observations paired with oracle actions, and diffusion-based behavioral cloning for communication-free multi-agent coordination.

Datasets

• Offline dataset D: Thousands of tuples (local observation, oracle action, done indicator) collected from collision-free, successful rollouts of the MAPPO privileged oracle policy in simulated multi-robot environments; exact size unreported

Baselines vs proposed

• MAPPO-Oracle: coverage rate = 97.6%, collision = 0.42 vs CoDiMAD: coverage rate = 95.7%, collision = 0.55
• MAPPO-Oracle: capture rate = 99.1%, collision = 0.83 vs CoDiMAD: capture rate = 90.6%, collision = 0.80
• MAPPO-Oracle: success rate (box pushing) = 98.2%, collision = 1.18 vs CoDiMAD: success rate = 72.2%, collision = 2.37
• MAPPO-Local: coverage rate = 83.7%, collision = 0.71 vs CoDiMAD: 95.7%, collision = 0.55
• MAPPO-Local: capture rate = 14.2%, collision = 1.89 vs CoDiMAD: 90.6%, collision = 0.80
• MAPPO-Local: success rate (box pushing) = 18.0%, collision = 1.17 vs CoDiMAD: 72.2%, collision = 2.37
• BC-RNN (deterministic RNN distillation): coverage = 80.9%, collisions = 10.37 vs CoDiMAD: 95.7%, collisions = 0.55
• BC-RNN: capture rate = 90.3%, collisions = 3.41 vs CoDiMAD: 90.6%, collisions = 0.80
• BC-RNN: success rate = 6.5%, collisions = 10.32 vs CoDiMAD: 72.2%, collisions = 2.37
• CoDiMAD w/o RNN: coverage = 94.8% vs CoDiMAD: 95.7%, capture = 84.0% vs 90.6%, box pushing = 68.9% vs 72.2%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.09587.

Fig 1: Overview of the CoDiMAD framework. (i) Stage 1 (Oracle Policy Training): The oracle actor encodes dual-stream inputs—local maps oloc

Fig 2: Overview of the three evaluation environments. (a) Coverage:

Fig 3: Conditional action distributions under partial observability. Under

Limitations

• Experiments conducted in simulated environments with three-agent homogeneous teams; results may not generalize to larger, heterogeneous teams or complex real-world robotics.
• No evaluation against adversarial or unexpected disturbances; robustness to environment or observation noise beyond standard simulation not tested.
• The offline dataset size and diversity are not reported, limiting understanding of scalability and data requirements.
• Inference speed improvements use DDIM without retraining, but practical latency and computational footprint on embedded hardware are not detailed.
• The approach assumes oracle policies are near-optimal; if oracle training is imperfect, distillation quality may degrade but this is not explored.
• No real-world robotic deployment or field experiments to verify transfer from simulation to physical multi-robot systems.

Open questions / follow-ons

• How well does CoDiMAD scale to larger multi-agent teams and heterogeneous agent capabilities?
• Can the diffusion-based distillation approach incorporate online fine-tuning or adaptation to changing observation distributions or damaged communication?
• What are the trade-offs in inference latency, compute, and energy consumption for deploying diffusion-based policies on physical multi-robot platforms?
• How does the approach handle non-stationarities in the environment or adversarial agents if introduced?

Why it matters for bot defense

While this paper addresses decentralized multi-robot coordination rather than traditional bot detection or CAPTCHA tasks, its core insight into modeling multi-modal conditional action distributions under partial observability has broader implications for bot-defense. In bot detection or CAPTCHA-solving contest scenarios with partial or noisy observations, decision-making systems often face ambiguous inputs where multiple valid outputs exist. Deterministic regression models averaging over multiple plausible actions can produce invalid or suboptimal outputs that may be easier to detect or less effective. Using diffusion-based generative policies to better capture multi-modal action distributions could inspire new strategies in designing robust, diverse interaction policies or detection methods that consider the distributional complexity rather than point estimates.

Bot-defense engineers might apply similar privileged distillation frameworks, training privileged or oracle models with full environment information and then distilling multi-modal generative policies to decentralized or no-communication agents, improving stealthy or coordinated behavior modeling. However, the direct application requires adaptation because this paper focuses on physical robot velocity controls rather than discrete input challenges typically found in CAPTCHA and bot-detection settings.

Cite

@article{arxiv2607_09587,
  title={ CoDiMAD: Diffusion-Based Privileged Distillation for Communication-Free Multi-Robot Coordination },
  author={ Jiyue Tao and Shunheng Xin and Tongsheng Shen and Dexin Zhao and Feitian Zhang },
  journal={arXiv preprint arXiv:2607.09587},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.09587}
}

Read the full paper

• arXiv:2607.09587 (abstract)
• arXiv:2607.09587 (PDF)