Function-Space Priors for Bayesian Neural ODEs with Application to Vessel Trajectory Prediction

Source: arXiv:2606.06351 · Published 2026-06-04 · By Jaeyeong Lee, Wonmo Koo, Heeyoung Kim

TL;DR

This paper addresses the challenge of vessel trajectory prediction from irregularly sampled and partially missing AIS maritime data, emphasizing the importance of calibrated uncertainty estimates for safety-critical applications. The authors focus on Bayesian Neural Ordinary Differential Equations (Neural ODEs) as a continuous-time modeling framework that quantifies epistemic uncertainty by placing a prior over the neural vector field parameters. However, standard isotropic Gaussian weight priors fail to encode structural dynamics properties like smoothness and locality inherent in vessel motion. To overcome this, the paper introduces a novel function-space prior that imposes a Gaussian process (GP)-kernel-based regularizer directly on the vector field evaluated at a finite set of state-space measurement points. This approach augments the standard weight-space variational inference objective with a GP-inspired regularization, providing a tractable inductive bias that is otherwise analytically intractable when attempting to place a prior over trajectories themselves.

The proposed method is combined with probabilistic multiple shooting to handle long, irregular trajectories stably by segmenting the temporal domain and allowing local inference while maintaining global consistency. Empirical evaluation on a large real-world AIS dataset from the NY/NJ Harbor demonstrates that the function-space prior improves predictive accuracy and uncertainty calibration relative to both weight-space Bayesian Neural ODEs and Gaussian process ODE baselines. Moreover, selecting measurement points near vessel maneuvering events yields further gains, highlighting the benefit of targeted structural prior placement. Overall, the method achieves sharper and better-calibrated predictive distributions, addressing core limitations of previous Bayesian Neural ODE formulations in maritime trajectory forecasting.

Key findings

• Function-space prior with GP-kernel regularization reduces average displacement error (ADE) from 3.03 km (weight-space prior) to 2.66 km when maneuver-weighted measurement points are used.
• Final displacement error (FDE) decreases from 4.47 km (weight-space prior) to 3.82 km with FS-BayesNODE (maneuver), a 14.5% improvement.
• Negative log-likelihood (NLL) improves from 18.90 for weight-space Bayesian Neural ODEs to 18.96 with the general function-space prior, and 18.96 with maneuver-weighted points; all significantly better than 29.46 for GPODE.
• Continuous ranked probability score (CRPS) improves by about 69%, from 4.57 km (GPODE) to 1.41 km (FS-BayesNODE maneuver variant).
• GPODE baseline uncertainty expands abruptly and is overdispersed, while FS-BayesNODE produces smoother uncertainty growth with prediction horizon.
• Concentrating measurement points near turning and deceleration events yields better predictive accuracy and uncertainty calibration than generic k-means clustering-based selection.
• Combining probabilistic multiple shooting with function-space regularization stabilizes training on long irregularly sampled sequences while preserving global temporal consistency.
• The function-space prior imposes smoothness and locality on the neural vector field, complementing but distinct from standard weight-space Gaussian priors.

Threat model

n/a — The paper is focused on improving probabilistic vessel trajectory prediction and uncertainty quantification rather than adversarial attack or security threat models.

Methodology — deep read

1. Threat Model & Assumptions: The adversary is not explicitly modeled as this is a forecasting and uncertainty quantification approach rather than an adversarial setting. The focus is on learning reliable vessel trajectory distributions under irregular sampling, missing data, and complex dynamics to support maritime situational awareness.

2. Data: The dataset consists of AIS data from the NY/NJ Harbor region from January 1–7, 2024, covering 1.5 million records from 590 vessels, segmented into 1589 contiguous trajectories. Each observation vector y_i includes x, y coordinates (converted to local ENU meters), speed over ground (SOG), and course over ground (COG) encoded as sine/cosine to avoid angular discontinuities. Data splits of 70% train, 10% validation, 20% test are split by vessel identity to prevent leakage. Sliding windows with 10–20 minutes history and 10-minute prediction horizon are used.

3. Architecture / Algorithm: The core latent dynamics are modeled as a Neural ODE with vector field f_\theta: R^d x R^+ -> R^d parameterized by a single-hidden-layer MLP (32 units). Observations y_i relate to latent states via an identity likelihood with Gaussian noise. Bayesian inference places a prior over ODE parameters \theta, seeking a variational posterior q_\phi(\theta). The key innovation is augmenting the weight-space prior with a function-space prior imposed via a GP kernel-based regularizer evaluated at a finite measurement set Z of M points in state space. The vector field evaluated at these points, F_Z(\theta), is regularized to match a zero-mean GP with kernel matrix K_Z. This imposes structural smoothness and locality directly on the vector field. Additional computational scalability and stability in training long irregular trajectories are achieved via probabilistic multiple shooting, partitioning trajectories into K segments with learnable boundary states s_k and soft continuity constraints between segments.

4. Training Regime: Two-phase training protocol: Phase 1 fixes shared quantities like measurement set Z and kernel hyperparameters from the training data. Phase 2 fits the model to individual trajectories' history (5–10 points) for a fixed number of optimization steps using Adam optimizer with early stopping on validation NLL. Reparameterization trick and backpropagation through the ODE solver are used for variational inference. The function-space regularization weight \lambda_FS is tuned on validation data. Typically, K=1 segment used for the short history windows. The kernel inverse K_Z^{-1} is precomputed for efficiency.

5. Evaluation Protocol: Metrics include ADE (average displacement error), FDE (final displacement error), negative log-likelihood (NLL), and continuous ranked probability score (CRPS), computed on predicted position coordinates (km). Metrics average over 40 test trajectories with 30 Monte Carlo posterior samples each. Baselines include GPODE (Gaussian process-based vector field), WS-BayesNODE (weight-space prior Bayesian Neural ODE), and proposed FS-BayesNODE variants with k-means and maneuver-weighted measurement point selection. Ablations evaluate impact of function-space regularizer and measurement point strategies. Qualitative uncertainty bands and growth are visualized.

6. Reproducibility: The authors do not explicitly mention code or model weight release. Dataset is public AIS data with detailed preprocessing steps described. The measurement sets Z are fixed after construction, making the function-space prior reproducible. Hyperparameter selections and architecture details are provided to enable replication. Some components such as kernel choice (RBF or Matérn) and its hyperparameters are tuned on validation.

Example end-to-end: A single test trajectory is initialized with fresh weights, then fit on 5–10 historical AIS points over 10–20 minutes by maximizing the variational objective incorporating weight-space KL, multiple shooting continuity constraints, and the function-space GP kernel regularizer on the vector field at Z. Posterior samples of parameters yield predictive trajectories over the 10-minute horizon, with uncertainty quantified by sampling and Gaussian likelihood, yielding displacement error and calibrated intervals shown in Fig. 3.

Technical innovations

• Augmenting weight-space Bayesian Neural ODE variational inference with a GP kernel-based function-space regularizer imposed on the vector field at a finite set of measurement points, injecting interpretable structural bias.
• Combining probabilistic multiple shooting with function-space regularization to stabilize inference on long, irregular AIS trajectories while maintaining global temporal consistency.
• Demonstrating that maneuver-weighted sampling of measurement points near vessel turning and deceleration events improves predictive accuracy and uncertainty calibration compared to generic clustering-based selection.
• Bridging function-space Bayesian neural network methods from static mappings to the Neural ODE setting by formulating a tractable, approximate trajectory-level prior via vector field regularization rather than direct trajectory GP priors.

Datasets

• NY/NJ Harbor AIS dataset — 1.5 million AIS records from 590 vessels, segmented into 1,589 trajectories — public AIS maritime data from Jan 1–7, 2024

Baselines vs proposed

• GPODE: ADE = 4.44 km, FDE = 5.67 km, NLL = 29.46, CRPS = 4.57 km vs proposed FS-BayesNODE (maneuver): ADE = 2.66 km, FDE = 3.82 km, NLL = 18.96, CRPS = 1.41 km
• WS-BayesNODE (weight-space prior): ADE = 3.03 km, FDE = 4.47 km, NLL = 18.90, CRPS = 1.51 km vs FS-BayesNODE (k-means): ADE = 2.80 km, FDE = 3.97 km, NLL = 19.31, CRPS = 1.46 km

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.06351.

Fig 1: Model overview: probabilistic multiple shooting with a function-space (GP-kernel) prior on the neural vector field evaluated at a finite measurement

Fig 2: Left: Sample vessel trajectories in the NY/NJ Harbor region (ENU

Fig 3: Predictive uncertainty comparison for a test trajectory. Left:

Limitations

• The function-space prior is imposed on a finite set of measurement points and may not fully constrain the vector field everywhere; thus, different vector fields might satisfy the same prior values but produce different trajectories.
• The method fixes measurement points non-adaptively before training and does not update them during optimization, which could limit flexibility in some dynamic regions.
• Probabilistic multiple shooting is used with only K=1 segment for short history windows in experiments; how well the approach scales to very long or more complex sequences beyond the tested horizon is not fully explored.
• The approach relies on variational inference with a diagonal Gaussian parameter posterior, which may limit capturing complex posterior correlations.
• The AIS dataset used is limited to a single harbor region and one week of data; results may not directly generalize to other maritime environments or longer time scales without further validation.
• Computational costs of function-space regularization grow cubically with the number of measurement points M; although small M is used, scaling to very high-dimensional state spaces or large M may become expensive.

Open questions / follow-ons

• Can measurement point sets be learned or adapted dynamically during training to better capture relevant state-space regions?
• How does the function-space prior approach perform under extreme distribution shifts, e.g., vessel behaviors or environmental conditions not seen during training?
• Can the approach scale to multi-agent or interaction-aware maritime trajectory forecasting where vessel-to-vessel dependencies matter?
• What are the trade-offs between inducing-point sparse Gaussian process methods and the function-space regularizer for interpretability and scalability in very high-dimensional dynamics?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, this work provides a rigorous example of incorporating structured functional priors to impose domain-specific smoothness and locality constraints in Bayesian neural models trained from sparse, irregular data. Such function-space priors can improve uncertainty calibration and predictive sharpness, which is crucial for security systems requiring reliable confidence estimates to distinguish genuine from malicious behavior patterns. The probabilistic multiple shooting technique to stabilize training over long irregular sequences may also inspire similar segmentation strategies in temporal analysis of client interactions or threat signals. Although AIS trajectory prediction is a different domain, the core challenge of modeling noisy, irregularly sampled sequences with calibrated uncertainty is shared with bot detection. Thus, the methodologies for function-space regularization and uncertainty-aware continuous-time modeling have direct conceptual relevance for designing robust, well-calibrated ML models in bot-defense applications.

Cite

@article{arxiv2606_06351,
  title={ Function-Space Priors for Bayesian Neural ODEs with Application to Vessel Trajectory Prediction },
  author={ Jaeyeong Lee and Wonmo Koo and Heeyoung Kim },
  journal={arXiv preprint arXiv:2606.06351},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.06351}
}

Read the full paper

• arXiv:2606.06351 (abstract)
• arXiv:2606.06351 (PDF)