Creating treatment and component hierarchies in component network meta-analysis
Source: arXiv:2605.15142 · Published 2026-05-14 · By Augustine Wigle, Audrey Béliveau, Adriani Nikolakopoulou, Lifeng Lin
TL;DR
This paper addresses the problem of creating meaningful treatment and component hierarchies in component network meta-analysis (CNMA), an extension of network meta-analysis (NMA) that models multi-component treatments by estimating effects of individual components and their combinations. While hierarchies of treatments are routinely derived in standard NMA to help interpret relative treatment performance, CNMA poses unique challenges due to issues of parameter identifiability and the ability to estimate relative effects not only for observed treatments but also for unobserved combinations. The authors develop a rigorous step-by-step workflow for forming hierarchy questions, assessing which relative effects can be uniquely estimated from the CNMA design matrix, refining hierarchy sets based on estimability, and computing standard ranking metrics (e.g., SUCRA, P-score) for the estimable set. This framework applies to both frequentist and Bayesian CNMA models. They demonstrate the workflow on two real-world networks: one for primary care depression treatments (22 treatments, connected network) and one for chronic lymphocytic leukemia (disconnected network). The key contribution is clarifying the estimability constraints in CNMA hierarchies, preventing misleading rankings based on non-identifiable parameters, and enabling hierarchies of components, observed treatments, and estimable unobserved combinations. The result is a rigorous, reproducible approach for CNMA hierarchy inference that aligns with principles established for standard NMA but adapted for the complexity of multi-component interventions.
Key findings
- The row space of the CNMA design matrix M defines the set of uniquely estimable relative effects.
- The authors provide a practical method to test if a relative effect contrast vector lies in the row space of M via matrix rank checks.
- In an example network, the relative effect of A versus D is estimable, but A+C versus D is not, illustrating limitations in constructing hierarchies involving unobserved treatment combinations.
- If matrix rank of M equals the number of parameters β, then all component effects and all relative effects (including unobserved treatments) are estimable.
- In the large depression CNMA network (93 studies, 22 treatments), the design matrix rank equaled length of β, indicating all relative effects among observed treatments are estimable.
- Ranking by median rank and point estimates produced consistent hierarchies in the depression network; ties appeared in median ranks due to uncertainty.
- Including interaction terms in CNMA models complicates component-only hierarchies because incremental effects depend on treatment combinations.
- Use of careful component and treatment set selection (S* subset of S) based on estimability is critical to avoid misleading rankings.
Threat model
n/a — the paper does not address adversarial threats but rather statistical identifiability of parameters in CNMA used for treatment ranking.
Methodology — deep read
The authors first specify the threat model as the statistical challenge of identifying uniquely estimable relative treatment effects from component-structured network meta-analysis data, rather than an adversarial security scenario.
Data provenance includes two published networks: a 93-study depression primary care network with 22 treatments (combinations of psychological and pharmacological components), and a disconnected network of chronic lymphocytic leukemia treatments. Treatment components and their multi-component combinations are labeled.
The CNMA model is parametrized by vector β of mean component effects, possibly including interaction terms. Treatments are expressed as linear combinations of components via a component matrix C and an observed contrast matrix B; the design matrix M = B*C captures the relation between observed study contrasts and parameter vector β. Anchored models fix one component parameter to zero.
Estimation approaches include both Bayesian MCMC sampling of posteriors and frequentist weighted least squares with estimated covariance matrices. Sampling enables calculation of ranking metrics like SUCRA and probability of best, incorporating uncertainty.
To determine estimability of relative effects, the procedure checks if contrast vectors representing relative effects lie in the row space of M by comparing matrix ranks (rank of M vs rank of M augmented with contrast vector). If ranks are equal, the contrast is uniquely estimable.
The workflow involves 4 main steps: 1) Define the idealized hierarchy question (i.e., which treatments/components to rank and according to which criterion/ranking metric). 2) Assess which parameters and contrasts are estimable using the design matrix method. 3) Refine the hierarchy question by restricting to a subset of treatments/components whose relative effects are estimable. 4) Calculate ranking metrics (point estimates, probability of best, median/expected rank, SUCRA, P-scores) on estimable contrasts to construct hierarchies.
The method is demonstrated concretely on the depression network, where an unanchored additive CNMA model is fit, all relative effects among 22 treatments are found estimable, and median rank hierarchies are computed and visualized with forest plots. The authors provide R code and an R function for estimability checks using matrix rank.
The paper emphasizes transparent documentation of hierarchy choices and caution when reducing the ranking set due to non-estimability. It also acknowledges that hierarchies involving unobserved treatments should be interpreted cautiously as exploratory.
Reproducibility is supported by code availability at github.com/augustinewigle/cnmaRank, and use of publicly available datasets like netmeta depression data.
Technical innovations
- Formalization of CNMA estimability of relative effects using the row space of the design matrix M and matrix rank computations.
- Stepwise workflow to create treatment and component hierarchies in CNMA addressing identifiability and estimability issues unique to CNMA.
- Extension of standard NMA hierarchy ranking metrics (e.g., SUCRA, P-scores) to CNMA with multi-component structure, including considerations for interactions.
- Provision of an R implementation to automate estimability checks and hierarchical ranking calculation based on design matrix properties.
Datasets
- Primary care depression network — 93 studies, 22 treatments — from R package netmeta (Linde et al. 2016)
- Chronic lymphocytic leukemia network — disconnected network — source not fully specified in provided text
Baselines vs proposed
- Depression network: All 22 treatment relative effects estimable (matrix rank = length of β = 19)
- Ranking using median rank metric agrees with point estimate ordering but shows ties due to uncertainty
- No direct numerical comparison to other CNMA hierarchy methods (prior methods do not handle disconnected networks or identifiability explicitly)
Limitations
- Methods depend on design matrix and model specification; misspecification or ignored network heterogeneity may affect estimability.
- Hierarchies including unobserved treatment combinations should be treated as exploratory due to lack of direct evidence.
- The approach mainly addresses parameter identifiability but does not evaluate robustness under distributional shifts or model misfit.
- Unanchored and anchored CNMA model choices can substantially affect what effects are estimable, requiring expert judgment.
- Interaction terms complicate component-level hierarchies; authors recommend caution or avoidance.
- The disconnected chronic lymphocytic leukemia network example details are less fully described; generalizability to very sparse networks may require further study.
Open questions / follow-ons
- How to best incorporate interaction effects into hierarchies beyond additive CNMA while maintaining interpretability and estimability.
- Development of Bayesian priors or regularization approaches to improve estimation stability in networks with disconnected components or sparse data.
- Methods to quantify uncertainty or sensitivity of hierarchies to model assumptions and anchor choice in CNMA.
- Extension of estimability diagnostics and hierarchies to CNMA models including patient-level covariates or network inconsistency.
Why it matters for bot defense
While not directly related to bot detection or CAPTCHAs, this paper provides a thorough methodological framework for correctly interpreting multi-component intervention effects from CNMA, which could inform research engineers working on any system involving complex treatment or defense combinations. The careful evaluation of parameter identifiability and hierarchical ranking under structural constraints may have conceptual parallels in aggregating and ranking components or features in security settings.
Practitioners focused on bot-defense might apply the principles of assessing estimability or unique identification of component contributions when combining multiple defense mechanisms or CAPTCHAs, ensuring that composite scores or rankings are based on identifiable effects rather than arbitrary combinations. The approach to hierarchical ranking with uncertainty quantification could also be informative when interpreting multi-layered defense effectiveness.
Cite
@article{arxiv2605_15142,
title={ Creating treatment and component hierarchies in component network meta-analysis },
author={ Augustine Wigle and Audrey Béliveau and Adriani Nikolakopoulou and Lifeng Lin },
journal={arXiv preprint arXiv:2605.15142},
year={ 2026 },
url={https://arxiv.org/abs/2605.15142}
}