Mixture-of-Experts RL for Fault-Tolerant Legged Locomotion

Source: arXiv:2606.25965 · Published 2026-06-24 · By Giulio Turrisi, Ozan Pali, Luca Oneto, Claudio Semini

TL;DR

This paper addresses the challenge of maintaining reliable locomotion for legged robots operating under actuator failures and challenging terrains, such as planetary exploration platforms. Existing monolithic reinforcement learning policies struggle to represent the heterogeneous control strategies required to compensate for different fault modes within a single network. To overcome this, the authors propose a fault-aware Mixture-of-Experts (MoE) reinforcement learning architecture, explicitly using fault diagnosis information to activate expert policies specialized for distinct actuator failure conditions. This explicit fault-conditioned expert selection reduces inference complexity and improves learning efficiency compared to monolithic policies that encode all fault modes simultaneously.

Experimental results on a Unitree Go2 quadruped in simulation and real hardware demonstrate that the fault-aware MoE policies consistently outperform monolithic PPO baselines of comparable size across multiple fault scenarios including no failure, single-leg failure, and severe double leg failures. The performance advantage grows with failure severity. Importantly, the MoE architecture maintains competitive or better locomotion performance even when network capacity is significantly reduced, illustrating suitability for compute-constrained robotic platforms. This work releases code for reproducibility and provides a modular framework for fault-tolerant legged locomotion leveraging diagnosis-informed architectural specialization.

Key findings

• Fault-aware MoE policies achieve higher cumulative episodic rewards than monolithic PPO baselines across all considered fault scenarios (No Fail, LF Fail, LH Fail, LH+RH Fail), with largest gains under severe failures (Fig 4).
• MoE policies exhibit more concentrated reward distributions, indicating improved robustness and consistency in fault scenarios compared to monolithic policies.
• Under network capacity reduction (Large: [128,128,128], Medium: [64,64,64], Small: [32,32,32]), MoE policies degrade less and small MoE policies perform comparably to large monolithic baselines across fault settings (Fig 5).
• Runtime inference cost is reduced since only one expert is activated given diagnosed fault, contrasted with dense MoE requiring all experts to execute.
• Fault diagnosis is framed as a supervised learning problem predicting failure mode from stacked proprioceptive observation history, enabling explicit expert selection without learned routing.
• State estimation of base linear velocity uses a learned model conditioned on fault state to handle degraded dynamics from failure modes instead of traditional contact-based observers.
• Training employed over 16,384 parallel simulation environments (~4,096 per fault condition), mitigating noisier gradient estimates due to reduced expert sample sizes.
• Real-robot experiments on Unitree Go2 confirm offline simulation results for nominal, single-leg, and double-leg failure scenarios on challenging terrain.

Threat model

The adversary is modeled as spontaneous actuator failures affecting one or more robot legs. The system assumes access to reliable onboard fault-diagnosis outputs identifying the current failure mode. The adversary cannot manipulate sensor inputs or fault detection, nor interfere with the robot's internal control processing. The framework focuses on fault adaptation and recovery, not fault prevention or detection robustness under adversarial manipulation.

Methodology — deep read

1. Threat model and assumptions: The adversary is modeled as unpredictable actuator failures occurring in one or more legs of a quadruped robot. The system assumes access to onboard fault-diagnosis information capable of identifying the current fault mode (which actuators or legs have failed) using proprioceptive measurements. The adversary cannot actively spoof or hide faults; fault diagnoses are assumed reliable. The system must adapt locomotion control to maintain functionality under these faults, not prevent fault occurrence.

2. Data: Training and evaluation are performed in simulation using the IsaacLab simulator focused on a Unitree Go2 quadruped platform. Four fault conditions are modeled: No Fail, Left-Front (LF) leg failure, Left-Hind (LH) leg failure, and both Hind legs (LH+RH) failure. Labels for fault conditions are directly available in simulation, facilitating supervised training of the fault detection network. Observations include proprioceptive states (velocities, joint positions/velocities), local heightmap terrain data, and desired velocity commands, stacked over a temporal horizon to capture dynamics.

3. Architecture: The overall framework consists of three components:

• Fault Detection Module: a neural network trained with cross-entropy loss to classify current fault condition from a temporal history of proprioceptive observations.
• State Estimator: a neural network that estimates base linear velocity conditioned on the diagnosed fault state, trained with mean squared error against privileged simulated ground truth velocity.
• Fault-Aware Mixture-of-Experts Policy: consists of a shared encoder network that processes observations into a latent representation, feeding into K specialized policy experts, each expert specialized and responsible for one fault condition. Expert selection is deterministic and fault-conditioned (one-hot), bypassing need for a learned routing network. Each expert outputs joint action commands. The critic uses the same inputs along with additional privileged info (contact states, timings).

Policies and critics use multilayer perceptrons with three hidden layers. Policy networks vary in size across experiments ([128,128,128] Large, [64,64,64] Medium, [32,32,32] Small). The modular architecture has linear parameter growth with number of experts, but only activates one expert at runtime reducing computational cost.

4. Training regime: Policies are trained with Proximal Policy Optimization (PPO). Large scale parallelization is used with 16,384 simultaneous simulation environments, resulting in roughly 4,096 samples per fault condition expert to overcome noisy gradient estimates caused by sample partitioning among experts. Reward functions combine base linear velocity tracking, yaw rate, base height and orientation, joint torque, acceleration and power penalties, feet air time and contact penalties, and fault-specific objectives dependent on number of failed legs. Training and evaluation details such as total epochs or seed strategies are not explicitly specified.

5. Evaluation protocol: Performance is evaluated using the cumulative episodic reward over challenging rough-terrain locomotion tasks under the four fault scenarios. Comparison baselines consist of monolithic PPO policies trained with similar capacity and input features. Ablations study the effect of reducing policy network capacity on performance degradation. Results include violin plots of reward distributions (Fig 4) and aggregate performance sensitivity to network size (Fig 5). Real hardware experiments on the Unitree Go2 robot confirm simulation trends by commanding the robot to traverse terrain with induced leg failures.

6. Reproducibility: The authors provide open source code at https://github.com/iit-DLSLab/fault-locomotion-isaaclab to reproduce training and evaluation. However, the dataset is largely simulation-generated and the exact seeds / hyperparameters for training runs are not detailed in the text, leaving some gaps for perfect replication. The fault scenarios are limited to four major classes without scaling to arbitrary faults.

End-to-end example: Observations consisting of stacked proprioceptive and terrain heightmap data along with desired velocity are encoded into a latent vector by the shared encoder. The fault detection network analyzes recent state history and classifies the current fault mode (e.g., Left-Hind leg failure). Based on this classification, the policy activates the corresponding expert specialized for this failure mode. This expert outputs joint torque commands tailored to redistribute load and adapt gait patterns for lost leg functionality to maintain locomotion. The state estimator concurrently predicts base linear velocity conditioned on the fault mode to better inform the policy. Actions are then executed by the robot to achieve fault-tolerant locomotion. This modular process contrasts with monolithic policies encoding all fault conditions simultaneously, typically resulting in interference and reduced performance.

Some training details (epochs, optimizer learning rates, exploration strategies) are not explicitly described in the paper.

Technical innovations

• Explicit fault-aware Mixture-of-Experts policy design leveraging fault diagnosis module outputs for one-hot expert selection instead of learned gating, reducing inference complexity.
• Integration of a supervised neural fault-detection network that predicts actuator failures from stacked proprioceptive data to directly inform expert selection in control policy.
• Concurrent training of a learned state estimator conditioned on fault mode to robustly estimate base velocity under abnormal locomotion dynamics due to failures.
• Demonstration that fault-specialized modular policies retain competitive locomotion performance with significantly reduced network capacity, beneficial for compute-limited robots.

Datasets

• IsaacLab simulation dataset of Unitree Go2 quadruped locomotion - size not explicitly stated - synthetic data generated during training and evaluation

Baselines vs proposed

• Monolithic PPO baseline: cumulative episodic reward = lower across fault modes vs proposed MoE policy: substantially higher rewards (Fig 4)
• Monolithic PPO Large network: performance degraded more than MoE Small network with reduced actor capacity across fault conditions (Fig 5)
• MoE Small: cumulative reward comparable to Monolithic PPO Large, showing smaller networks with specialized experts outperform monolithic large networks

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.25965.

Fig 1: Experimental validation of the proposed fault-aware locomotion framework on a Unitree Go2 traversing highly cluttered terrain. From left to

Fig 2: Block scheme of the proposed approach. State Estimation and Fault Detection Nets are queried with stacked proprioceptive information, and

Fig 3 (page 1).

Fig 4 (page 2).

Fig 3: The four different fault modalities. From left to right: No Fail, LF Fail, RH Fail, and LH+RH Fail. Red dots represent the heightmap that enters

Fig 4: reports the distribution of cumulative episodic

Fig 5: evaluates the sensitivity of the proposed approach

Fig 8 (page 5).

Limitations

• Sample efficiency decreases as number of experts (fault conditions) increases due to lower per-expert samples, requiring massive parallel simulation (16,384 envs used).
• Policy-gradient variance is higher for experts since each expert receives data only from corresponding fault subsets, potentially slowing training convergence.
• Experiments limited to four fault scenarios; scalability and generalization to arbitrary or multiple concurrent actuator failures not explored.
• Real-world evaluation is limited with only a few fault conditions and simple terrain; no extensive adversarial or external perturbations tested.
• Details on training hyperparameters, seed strategies, and complete reproducibility documentation are limited, potentially hindering exact replication.

Open questions / follow-ons

• How does the approach scale when the number of fault modes and experts increases beyond the tested four scenarios?
• Can parameter-sharing or meta-learning strategies be introduced to reduce sample complexity and improve training efficiency across experts?
• What is the robustness of the fault-detection module under noisy sensor conditions or partial observability in real deployment?
• How would the framework integrate with online adaptation or continual learning to handle previously unseen fault conditions?

Why it matters for bot defense

Though this paper is primarily focused on legged robot fault-tolerant locomotion, the principle of integrating fault diagnosis with modular control architectures could be conceptually applied to bot defense systems where distinct failure modes or attack vectors require specialized mitigation strategies. The explicit fault-informed routing to specialized experts reduces unnecessary computation and may improve robustness under distinct operational failure modes, analogous to different attack or compromise scenarios. The modular approach also suggests architectural patterns for resource-constrained systems to gracefully degrade performance while maintaining key capabilities.

Practitioners designing CAPTCHA or bot-detection defenses could draw inspiration from the fault-aware Mixture-of-Experts design to implement explicit conditionally activated modules based on detected threat intelligence or bot behavior diagnosis, rather than deploying monolithic models that must handle all cases indiscriminately. This can improve inference efficiency and adaptation speed under evolving threat landscapes. However, direct application requires translation from physical fault diagnosis to threat diagnosis in security systems, which involves different data modalities and attack models.

Cite

@article{arxiv2606_25965,
  title={ Mixture-of-Experts RL for Fault-Tolerant Legged Locomotion },
  author={ Giulio Turrisi and Ozan Pali and Luca Oneto and Claudio Semini },
  journal={arXiv preprint arXiv:2606.25965},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.25965}
}

Read the full paper

• arXiv:2606.25965 (abstract)
• arXiv:2606.25965 (PDF)