Deep Gaussian Processes on Directed Acyclic Graphs
Source: arXiv:2607.09645 · Published 2026-07-10 · By Federico L. Perlino, Oliver Hamelijnck, Adam M. Johansen, Theodoros Damoulas
TL;DR
This paper addresses the problem of modelling compositional latent functions arranged along general Directed Acyclic Graphs (DAGs) with noisy, heterogeneously sampled intermediate observations. Standard Deep Gaussian Processes (DGPs) focus on chain-structured compositions, but many real-world phenomena in causal modelling, multi-fidelity emulation, and biology are best represented by DAGs with multiple parents per node and partial measurements at arbitrary internal nodes. The authors introduce Deep Gaussian Processes on DAGs (DAG-DGPs), a unified probabilistic framework that generalizes chain DGPs to handle arbitrary DAG structures, fusing multiple parent inputs via kernel fusion rules, and incorporating partial noisy observations within the graph. They theoretically characterize how DAG topology, kernel choices, and intermediate observations affect information preservation through the graph, providing lower bounds that prevent prior collapse, a pathology where repeated function composition destroys input distinctions. Their structured variational approximation preserves compositional uncertainty and explaining-away posterior dependencies induced by latent colliders. Empirically, DAG-DGP models state-of-the-art performance modeling a synthetic latent-collider DAG, a protein signalling network (Sachs dataset), and a multi-fidelity heavy-ion collision task, recovering low-fidelity contributions and providing interpretable simulator hierarchies.
Key findings
- DAG-DGPs with repeated 'separating nodes' in progressive antichains prevent prior collapse almost surely (Theorem 1), ensuring persistent distinctions between different inputs downstream.
- Product fusion kernels with indegree k in nodes lead to a contraction threshold scaled linearly with k, making collapse less likely in high-indegree regions compared to chains (Fig. 3b).
- Intermediate noisy observations act as stochastic skip connections that refresh input-dependent contrasts in the posterior, quantitatively increasing the chance of input distinguishability preservation downstream (Theorem 2, Eq. 8).
- Structured variational inference (DAG-SVI) preserving posterior covariances captures explaining-away effects between co-parents better than mean-field approximations (DAG-VI), reducing RMSE from 0.114 to 0.055 in a latent-collider synthetic task (Table 1, Fig. 4).
- On the Sachs protein signalling dataset, DAG-SVI improves extrapolation RMSE from 0.737 (DAG-VI) to 0.642 and continuous ranked probability score (CRPS) from 0.325 to 0.299 (Table 1).
- On the heavy-ion collision multi-fidelity emulation problem, DAG-SVI outperforms graphical multi-fidelity Gaussian process baselines on RMSE, normalized RMSE, and CRPS metrics (Table 2), while enabling interpretable Shapley-value analysis of fidelity contributions (Fig. 7).
- Scaling experiments show that the structured variational inference with sparse chordal precision exploitation reduces ELBO evaluation time significantly on large branching-tree DAGs compared to dense implementation (Fig. 2).
Methodology — deep read
The authors develop a probabilistic framework modeling latent functions associated to each node of a DAG, where each non-root node's latent function depends on its parents via a Gaussian Process (GP) prior mapping from the product of parent latent spaces to that node's latent space. Root nodes receive deterministic inputs. Observations, possibly noisy and incomplete, are linked to arbitrary internal DAG nodes with node-specific observation masks.
The DAG-DGP prior: latent node values are formed by applying independent GP draws per node to the latent values of their parents, with kernels defined on the product spaces of parents. To fuse multiple parents, node-specific kernel fusion operations are employed, including additive, product, and ANOVA-style fusion to allow independent, multiplicative, or interaction effects.
Incorporation of intermediate observations is done via node-specific likelihood terms conditioning on the latent functions, yielding a posterior that factorizes over the DAG but induces dependencies among latent nodes due to explaining-away and observed collider structures. This posterior is generally intractable.
To enable scalable inference, two variational families are proposed: DAG-VI, a mean-field inducing-point approximation extending Salimbeni and Deisenroth (2017) to DAGs, which factorizes over latent nodes; and DAG-SVI, a structured variational family building a Gaussian distribution with precision matrix constrained by the moralized and chordal-completed graph structure, preserving posterior dependencies between latent nodes induced by shared observed descendants. The factorization leverages chordal decomposition into maximal cliques, enabling sparse Cholesky decompositions facilitating efficient evaluation and ancestral sampling that respects graph dependencies.
The ELBO estimator uses ancestral Monte Carlo sampling through the topological ordering, with analytical KL divergence. Training uses mini-batching over observed nodes and inducing points per node.
Theoretical analysis focuses on the DAG-DGP prior to characterize conditions preventing prior collapse, using metrics based on contrasts between pairs of input cases propagated across antichain decompositions of the DAG nodes. Results relate kernel properties, graph indegree/outdegree, and presence of observed intermediate nodes to lower bounds on input distinguishability at various depths.
Empirically, the authors validate the theory on synthetic DAGs with latent colliders, demonstrating that DAG-SVI captures explaining-away and compositional uncertainty better than DAG-VI. On real data, the Sachs protein signalling network (with a known causal DAG) is modeled by placing latent GPs on observed and latent confounder nodes, with experiments comparing interpolation and extrapolation performance.
On a multi-fidelity heavy-ion collision problem, the elicited simulator DAG with two complementary low-fidelity nodes and one high-fidelity output is modeled with DAG-DGP. The model's ability to capture contributions from distinct fidelities and improve predictions over baseline graphical multi-fidelity GP methods is assessed using held-out data splits and multiple evaluation metrics.
Scaling experiments measure wall-clock ELBO evaluation times across increasing DAG sizes, showing sparse structured inference is more efficient than dense.
Code release or details on seeds/hardware are not explicitly stated in the excerpt.
Technical innovations
- Definition of Deep Gaussian Processes directly on arbitrary DAGs, generalizing chain DGPs and enabling latent function composition over multiple parents with fusion kernels.
- Theoretical proof that repeated 'separating nodes' and intermediate noisy observations prevent prior collapse in DAG-DGP priors, preserving input distinguishability with positive probability.
- A structured variational approximation (DAG-SVI) that uses chordal decomposition of the moralized observed-node ancestral graph to preserve posterior dependencies including explaining-away in colliders.
- Fusion kernel formulation allowing additive, product, or ANOVA-style combination of multi-parent contributions per node, enabling flexible inductive biases tailored to domain knowledge.
- Empirical integration of DAG-DGP modeling with interpretability tools like Shapley value analysis for multi-fidelity contributions in complex simulator hierarchies.
Datasets
- Sachs protein signalling network — 902 observations on 11 variables — bnlearn DAG from (Scutari, 2010)
- Heavy-ion collision multi-fidelity emulation — ~200 low-fidelity, 25 high-fidelity observations with 75 test points — public from Ji et al. (2024)
- Synthetic latent-collider DAG — synthetic data generated with additive RBF fusion kernels
Baselines vs proposed
- DAG-VI vs DAG-SVI on COLLIDER dataset: RMSE 0.114 vs 0.055; CRPS 0.075 vs 0.060
- DAG-VI vs DAG-SVI on Sachs extrapolation: RMSE 0.737 vs 0.642; CRPS 0.325 vs 0.299
- Graphical multi-fidelity GP baseline vs DAG-VI vs DAG-SVI on heavy-ion emulation: DAG-SVI achieves lowest RMSE, normalized RMSE, and CRPS (exact values in Table 2 of paper)
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.09645.
Fig 13: Illustration of the prior-theory experiments.
Limitations
- The theoretical analysis requires assumptions like bounded-curvature kernels and monotonic radial kernels; applicability to more general kernels unclear.
- Empirical validations are on a limited number of real-world datasets: one biological signalling network and one multi-fidelity physical simulation; broader domain validation needed.
- Computational complexity of DAG-SVI scales cubically with total inducing dimension in dense settings, requiring sparse approximations or dimensionality reduction for very large DAGs.
- No explicit adversarial or robustness evaluation under model misspecification or strongly noisy observations is presented.
- Code release status and reproducibility details such as hyperparameter selection and random seeds are not clearly specified in the provided text.
Open questions / follow-ons
- How do DAG-DGPs perform under strong model misspecification or highly noisy, sparse observations across the DAG?
- Can the structured variational approximation be further scaled or approximated to handle substantially larger DAGs with thousands of nodes and high-dimensional latent spaces?
- How would the fusion kernels and theoretical guarantees extend to non-Gaussian and non-stationary kernels or kernels learned from data?
- Can DAG-DGP frameworks be integrated with automated DAG structure learning approaches to jointly infer graph and latent functions?
Why it matters for bot defense
For bot-defense or CAPTCHA systems, DAG-DGPs offer a principled way to model complex compositional relationships where multiple latent causal or generative factors combine with noisy, partial observations. The theoretical insights on preserving input distinguishability despite deep compositional hierarchy help ensure uncertainty and signal are not collapsed during inference, which is critical when distinguishing automated versus human behaviour relies on subtle latent features. The structured variational inference approach provides a scalable and probabilistically consistent way to propagate uncertainty and represent explaining-away effects, analogous to explaining away competing bot hypotheses given observed responses. Engineering NLP, vision, or behavioural CAPTCHAs could leverage DAG-DGPs to fuse multiple heterogeneous modalities or hierarchical predictors with principled Bayesian uncertainty quantification. However, practical deployment must consider computational complexity as DAG size grows, potentially limiting use to medium-scale scenarios or requiring sparse approximations. The approach notably advances beyond traditional DGP chains by incorporating graph structure and intermediate supervisory signals—common in multi-step CAPTCHA challenge-response pipelines.
Cite
@article{arxiv2607_09645,
title={ Deep Gaussian Processes on Directed Acyclic Graphs },
author={ Federico L. Perlino and Oliver Hamelijnck and Adam M. Johansen and Theodoros Damoulas },
journal={arXiv preprint arXiv:2607.09645},
year={ 2026 },
url={https://arxiv.org/abs/2607.09645}
}