Choreographing the Way of Water: A Computational Framework for Aquatic Robotic Art

Source: arXiv:2607.02174 · Published 2026-07-02 · By Aswin Ramachandran, Christopher Golling, Sebastian Burmester, Noa Sendlhofer, Jan Kamm, Ruiheng Jiang et al.

TL;DR

This paper addresses the complex problem of choreographing autonomous surface vessels (ASVs) for artistic aquatic performances in unpredictable and nonlinear fluid environments. Traditional robotic choreography faces major challenges due to water disturbances such as waves, currents, and drift, which make precise trajectory tracking difficult. The authors present the "Way of Water" system, a vertically integrated cyber-physical framework that orchestrates a fleet of custom-built holonomic vessels equipped with laminar water nozzles and multi-zone lighting to create 3D volumetric artistic effects beyond simple surface-level displays. Central to the contribution is the Way of Water Studio, a browser-based, timeline-compositing authoring tool akin to a digital audio workstation, which abstracts complex control and optimization math (sequential convex programming for collision-free path generation, model predictive control for disturbance rejection) behind an intuitive interface accessible to non-programmer artists. The system includes tightly synchronized LTE/MQTT communications, RTK-GPS timing, onboard computation (Raspberry Pi 5), and robust state estimation via EKF for precise pose tracking. The authors validate the framework through two distinct large-scale deployments: an 18-vessel Swan Lake aquatic ballet on Lake Zurich and an 8-vessel music-responsive installation at the 2025 Venice Biennale, demonstrating high precision (RMSE 0.038 m positional, 1.4° heading) in real-world conditions with wind and waves. The Studio workflow drastically reduced authoring time and lowered technical barriers, broadening creative access. This work establishes a foundational cyber-physical and authoring reference for future aquatic swarm robotic art that combines rigorous controls with artistic expression.

Key findings

• The model predictive controller achieves per-vessel RMSE of 0.038 m in position (x,y) and 1.4° in heading (ψ) under environmental disturbances (Figure 4).
• The studio-based authoring reduced choreography creation time per song from 4-8 hours in the code-first system to 90-120 minutes, with iteration cycle dropping from 5 minutes to under 10 seconds.
• The LSAP solver ensures collision-free fleet transitions with minimum inter-robot safety gaps derived as approximately 21 cm for 50 cm craft diameter.
• Two separate deployments: an 18-vessel Swan Lake ballet on a 60 m lake area and an 8-vessel Venice Biennale show, both demonstrating operational endurance over 10+ hours and reliable position/heading tracking under wind and wave disturbances.
• The system enables sub-millisecond inter-vessel temporal synchronization using RTK-GPS pulse-per-second signals without dedicated master clocks.
• Onboard computing on Raspberry Pi 5 executes OSQP-based MPC for real-time trajectory control at 10 Hz directly on each vessel.
• Parametric primitives (e.g., circle shapes defined by global position, scale, rotation, phase offset) allow intuitive motion design, moving away from individual waypoint editing.
• The audio-reactive payload layer modulates LED lighting and laminar jet intensities through offline audio analysis (Librosa), driving visual effects synchronized to music structure.

Threat model

The primary adversary is the uncontrolled stochastic aquatic environment, including wind, waves, currents, and drift forces that degrade precise vessel positioning and synchronization. There is no assumption of malicious attackers or adversarial signal interference. Communication latency, packet loss, or temporary blackouts may occur, but vessels operate autonomously with GPS time synchronization. The system does not address intentional cyber attacks or spoofing.

Methodology — deep read

1. Threat Model & Assumptions: The adversary is the stochastic aquatic environment imposing nonlinear disturbances (wind, waves, currents) that can cause position and heading drift of autonomous surface vessels. The vessels are assumed trusted, and there is no active malicious attacker. The system must be robust to natural disturbances but does not address adversarial robotic attackers or signal spoofing. Communication delays and blackouts can occur but choreography must progress correctly via RTK-GPS synchronization.

2. Data: The evaluation used data collected during two physical deployments: an 18-vessel choreography on Lake Zurich and an 8-vessel deployment at Venice Biennale 2025. The telemetry includes RTK-GPS (10 Hz position, velocity), IMU (100 Hz acceleration & rates), and onboard MPC controller state logs. No public benchmark datasets are used. The Studio's audio analysis uses offline processing with Librosa and Scipy on pre-recorded music tracks.

3. Architecture / Algorithm: The core architecture includes (a) a custom holonomic vessel chassis with an X-drive 4-thruster propulsion system enabling 3-DOF marine motion, (b) onboard sensing (RTK-GPS + IMU) fused via an Extended Kalman Filter estimating 3-DOF pose and velocity, (c) a Model Predictive Controller running onboard Raspberry Pi 5 at 10 Hz solving Quadratic Programs via OSQP that minimize deviation from reference trajectory while compensating for disturbances, (d) Sequential Convex Programming solver providing offline collision avoidance constraints by iteratively linearizing pairwise collision constraints between vessels, (e) Linear Sum Assignment Problem (LSAP) solver optimizing fleet transitions between formations to minimize total travel distance.

The authoring interface, the Way of Water Studio, runs in a browser and abstracts physical constraints and optimization layers behind parametric timeline blocks representing motion primitives (e.g., circles, lines) and payload effects (lighting, water jets). The Studio composes choreography against a visual audio waveform timeline for music alignment.

4. Training Regime: Not applicable as the system is primarily a control and planning framework, not a learned model. MPC and SCP solve optimization problems online/offline respectively. Parameter tuning for MPC weighting prioritizes heading precision over position to maintain laminar nozzle alignment. The control loop runs at 10 Hz onboard each vessel.

5. Evaluation Protocol: Performance metrics include positional RMSE (0.038 m), heading RMSE (1.4°), operational endurance (10+ hours continuous), communication latency (80-250 ms LTE), and qualitative robustness tested in real freshwater and brackish water conditions with wind and wave disturbances. Two full live performances served as case studies. An informal authoring time comparison showed reduced creative friction moving from code to GUI. No formal cross-validation was described; deployments confirm real-world applicability.

6. Reproducibility: The paper does not mention public release of code, weights, or datasets. The system uses proprietary hardware and live deployments. The Studio appears to be closed-source and the evaluation is based on production use by the internal team.

One end-to-end example: For the Swan Lake deployment, choreographers first defined parametric motion primitives (e.g., circle formation) with global position, scale, and rotation in the Studio timeline overlaying the musical waveform. Motion transitions between primitives were automatically assigned to vessels using LSAP, ensuring collision avoidance via offline SCP optimization. The Studio generated a choreographed state file combining trajectories and payload modulation schedules. This was uploaded to vessels prior to show time. During performance, each vessel used its EKF state estimator to track pose, executing MPC commands to minimize deviation despite environmental disturbances. The entire fleet was synchronized by GPS pulse-per-second timing. Lighting and water jets were modulated to musical structure via preprocessed audio feature triggers from the Studio. Post-show MPC telemetry confirmed position errors within 3.8 cm RMSE and heading errors within 1.4 degrees.

Technical innovations

• Integration of Sequential Convex Programming for scalable collision avoidance in planar aquatic swarm trajectory planning, adapted from aerial robotics work.
• Development of a holonomic X-drive propulsion vessel designed for stable, precise 3-DOF maneuvering in nonlinear water boundary layer dynamics.
• A Model Predictive Controller running onboard resource-constrained Raspberry Pi 5 units achieves centimeter-level trajectory tracking in aquatic environments with significant wind and wave disturbances.
• A browser-based DAW-like timeline-compositing authoring interface (Way of Water Studio) that abstracts complex trajectory optimization and control theory for non-programmer artists composing music-synchronized aquatic choreography.

Datasets

• Lake Zurich Swan Lake Deployment Telemetry — 18 vessels over multiple hours — proprietary live deployment dataset
• Venice Biennale 2025 Deployment Telemetry — 8 vessels in lagoon environment — proprietary live deployment dataset
• Offline music audio tracks with extracted MIR features using Librosa — pre-recorded publicly available musical works

Baselines vs proposed

• Code-first Python scripting authoring workflow: average 4-8 hours per 5-minute scene composition vs Studio authoring reduced to 90-120 minutes per song and under 10 seconds edit-preview iteration.
• Reactive PID controllers (baseline): insufficient disturbance rejection and anticipatory motion vs proposed MPC achieved 0.038 m positional RMSE and 1.4° heading RMSE under wave disturbances.
• Naive waypoint-following without collision avoidance: potential collisions vs proposed SCP solver guaranteeing collision-free trajectories pre-deployment.

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.02174.

Fig 1: The Way of Water installation: A fleet of illuminated vessels performing synchronized motion on Lake Zurich.

Fig 2: The Swan Lake performance, where autonomous vessels enact a balletic choreography, mirroring the elegance of

Fig 3: Overview of the hardware design, showing the

Fig 6: Parametric Motion Editor. Double-clicking the timeline (bottom) creates an effect, which can be dragged and resized.

Limitations

• Scalability limited to ~24 vessels due to centralized solver; distributed or hierarchical planning needed for larger swarms.
• Current audio pipeline is offline and does not support live real-time audio-driven choreography or improvisation.
• Lack of formal usability studies with external choreographers and audience experience research limits evaluation of creative workflow efficacy.
• System depends on reliable RTK-GPS corrections and LTE connectivity; performance degrades with GPS outage or correction dropouts.
• No adversarial robustness evaluation or security analysis of communication channels was presented.
• Physical hardware designs and datasets are proprietary and not publicly released, limiting external reproducibility and benchmarking.

Open questions / follow-ons

• How can the system scale beyond ~24 vessels with distributed middleware and hierarchical planning?
• What approaches enable live, real-time audio listening and music-driven motion improvisation to complement offline authoring?
• Can local-rule emergent choreographic primitives inspired by Reynolds/Pask models be integrated alongside parametric top-down formations?
• How do external choreographers and audiences perceive and interact with the system, requiring formal user studies and evaluation?

Why it matters for bot defense

This work is primarily focused on robotics and artistic aquatic swarm choreography rather than bot-detection or CAPTCHA systems. However, it provides a rich example of managing distributed autonomous agents with nonlinear dynamics and latency constraints via MPC and convex optimization. The design of a browser-based compositional interface that abstracts complex control can inspire more user-friendly authoring tools for other distributed autonomous systems. The system’s robust synchronization approach using GPS pulse-per-second signals to maintain temporal alignment without a centralized master clock could be informative for coordinating distributed defenses or multi-agent challenges in CAPTCHA platforms. Also, the layered control framework combining offline trajectory optimization and real-time disturbance rejection with safety constraints illustrates practical multi-level security guarantees in physical agent coordination. Still, direct application is limited since the domain concerns physical aquatic robots and artistic expression rather than adversarial bot mitigation. Bot defense engineers may find the principled approach to integrating control theory, real-time state estimation, and compositional interfaces a useful reference for designing coordinated multi-agent interaction models or visual challenge orchestration.

Cite

@article{arxiv2607_02174,
  title={ Choreographing the Way of Water: A Computational Framework for Aquatic Robotic Art },
  author={ Aswin Ramachandran and Christopher Golling and Sebastian Burmester and Noa Sendlhofer and Jan Kamm and Ruiheng Jiang and Raffaello D'Andrea },
  journal={arXiv preprint arXiv:2607.02174},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.02174}
}

Read the full paper

• arXiv:2607.02174 (abstract)
• arXiv:2607.02174 (PDF)