Dynamic Frechet Regression with Feature Selection for Distributional Data

Source: arXiv:2607.09613 · Published 2026-07-10 · By Kiran Adhikari, Amrutha Dinesh, Mathew Kuttolamadom, Ying Lin

TL;DR

This paper addresses the challenge of regressing dynamic distribution-valued responses that evolve over an ordered index (e.g., time, depth) on scalar predictors. Traditional regression methods either ignore the geometry of distributional responses or inadequately model their index-dependent evolution. To overcome this, the authors propose Dynamic Fréchet Regression (DFR), which extends Global Fréchet Regression by incorporating index-aware weighting that jointly considers predictor similarity and index proximity, enabling smooth, local predictions of distribution trajectories in metric spaces such as Wasserstein space. Additionally, DFR includes a novel sparse metric learning approach for feature selection, learning a predictor relevance metric directly from the regression loss to identify important scalar inputs influencing distributional dynamics without relying on explicit coefficients. The method is computationally efficient for univariate distributions by representing responses via quantile functions and solving weighted isotonic regression problems.

Simulation studies demonstrate that DFR improves predictive accuracy and feature recovery compared to Global Fréchet Regression and Function-on-Scalar Regression baselines, especially in noisy or high-dimensional settings. An experimental case study on additive manufacturing melt pool data validates its ability to produce interpretable, layer-specific distributional predictions aligned with known process parameter effects. Overall, the paper presents a geometry-aware, dynamically adaptive regression with principled feature selection tailored for evolving distributional data, filling a key methodological gap in distributional regression literature.

Key findings

• DFR significantly improves predictive accuracy over Global Fréchet Regression (GFR) and Function-on-Scalar Regression (FoSR) in modeling dynamic distribution sequences, as shown in simulation scenarios with evolving means and variances (Fig 3).
• Sparse metric learning embedded in DFR reliably recovers relevant predictors while excluding nuisance variables, demonstrated by better feature selection accuracy under correlated predictors compared to marginal screening baselines.
• The index-aware weighting mechanism successfully balances local fidelity and information borrowing across neighboring indices, controlled by a roughness parameter \u03bb; small \u03bb emphasizes index-specific adaptivity, large \u03bb enforces smoother cross-index predictions (Eq. 4).
• For univariate distributions under 2-Wasserstein metric, representing distributions via quantile functions reduces the Fréchet regression to weighted isotonic regression, solvable efficiently with the Pool Adjacent Violators Algorithm (PAVA) (Eq. 14).
• In the additive manufacturing application (Directed Energy Deposition), DFR captures layer-specific melt pool distributional evolution sensitive to process parameters, providing interpretable dynamics beyond scalar summaries (Section 5).
• Replacing empirical precision matrix by a learnable positive semidefinite metric M in the weighting function induces sparsity and feature selection via group-lasso regularization on matrix rows (Eq. 8).
• The two-stage training approach—first learning sparse metric with roughness fixed to zero for feature selection, then refitting DFR with selected predictors while tuning \u03bb—yields stable, interpretable regression models.

Methodology — deep read

The methodological core is Dynamic Fréchet Regression (DFR) which regresses index-dependent distribution-valued responses on scalar predictors, respecting the metric geometry of the response space.

1. Problem Setup: Responses for each sample are sequences of probability distributions observed over an ordered index (e.g., layers in manufacturing). The predictor is a scalar vector x \u2208 R^p, and the goal is to predict entire distribution sequences f(x) in a metric space (\u03A9, d), e.g., Wasserstein space.

2. Limitations of prior work: Global Fréchet Regression (GFR) predicts a single distribution per x by weighted Fréchet means but treats index-dependent responses as exchangeable, ignoring smooth evolution and dependencies across indices.

3. Dynamic Fréchet Regression: DFR generalizes GFR to index-dependent prediction by introducing an index-aware weighting function w_i,l(x_0, j) that measures similarity between training sample i at index l and target predictor x_0 and target index j. Prediction at index j minimizes weighted Fréchet mean of training distributions across all samples and indices (Eq. 2). The weights depend jointly on predictor similarity and index proximity.

4. Index-aware weighting function: Drawing analogy from function-on-scalar regression (FoSR) with smoothing spline bases, weights are constructed from the smoother matrix involving B-spline basis \u03A6 and covariance of predictors \u03A3 with a roughness parameter \u03BB controlling smoothness across indices (Eq. 4). This kernel-like weighting balances local index adaptability and borrowing strength from neighboring indices.

5. Feature Selection via Sparse Metric Learning: Standard regression coefficient-based selection is not applicable. Instead, authors learn a positive semidefinite similarity metric matrix M that replaces the fixed inverse covariance in weights, so predictor influence is encoded by M. Sparse group lasso regularization on rows of M induces predictor sparsity (Eq. 8). Weight function and predictions become metric-dependent, enabling automatic variable selection.

6. Two-stage estimation: First fix \u03BB=0 (no roughness penalty) and estimate sparse metric M from reconstruction loss over training data, identifying relevant predictors. Then refit DFR with only selected predictors and cross-validate \u03BB for optimal smoothness.

7. Efficient quantile-space implementation: For univariate distributions, using 2-Wasserstein metric, distance reduces to \u2113^2 distance on quantile functions. Prediction reduces to weighted isotonic regression solvable by PAVA (Eqs. 12-14). This facilitates efficient gradient-based optimization of M.

8. Training regime details are under-specified but gradient-based optimization with parameterization M=B^TB is used for metric learning stability. Cross-validation is used for tuning \u03BB.

9. Evaluation protocol: Synthetic simulations evaluate prediction accuracy and feature selection recovery across noiseless/noisy settings, and an additive manufacturing case study shows realistic applicability.

10. Code release and reproducibility are not explicitly stated, suggesting closed datasets and possible re-implementation required.

A concrete example: given training samples with distributional responses evolving over index L (e.g., layers), DFR predicts the response distribution at layer j for new x_0 by computing index- and predictor-dependent weights (Eq. 6), then performing weighted Fréchet mean minimization (Eq. 2) in Wasserstein space via quantile-space isotonic regression. Feature relevance is identified by learning M to minimize reconstruction loss plus group-lasso penalty (Eq. 8).

Technical innovations

• Introduction of an index-aware weighting function in Fréchet regression to model dynamic, sequence-indexed distributional responses, balancing local fidelity and smoothness across indices.
• Integration of sparse metric learning into the weighting mechanism to perform feature selection without explicit coefficients by learning a predictor relevance metric matrix.
• Efficient quantile-function representation and solution of the Fréchet mean problem under 2-Wasserstein metric as weighted isotonic regression, enabling scalable and stable optimization.
• Two-stage estimation procedure separating feature selection from index smoothing allows stable identification of relevant predictors followed by tuning of smoothness hyperparameter.

Datasets

• Simulated dynamic distribution sequences — synthetic data generated under controlled scenarios (not public)
• Additive manufacturing melt pool data — experimental layer-indexed distributions from Directed Energy Deposition (DED) process (proprietary/academic dataset)

Baselines vs proposed

• Global Fréchet Regression (GFR): Prediction error higher than DFR across all simulation scenarios, demonstrating inferior modeling of index-dependent dynamics.
• Function-on-Scalar Regression (FoSR): Lower accuracy in dynamic distribution predictions than DFR due to linearization of distributions and ignoring metric geometry.
• DFR with sparse metric learning: Higher feature selection accuracy and stability than FRiSO and marginal screening methods in recovering ground-truth active predictors.
• DFR with \u03BB tuned via cross-validation outperforms versions with fixed roughness in balancing smoothness and adaptivity of distribution trajectories.

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.09613.

Fig 1: Evolution of the distributional mean and variance under the three simulation scenarios,

Fig 2 (page 24).

Fig 3 (page 24).

Fig 2: (a) DED system, (b) 25 printed samples c) Thermal Image of the meltpool for sample.

Fig 5 (page 31).

Fig 6 (page 31).

Limitations

• The method and experiments focus primarily on univariate distributions; extension to multivariate distributions is conceptually feasible but not demonstrated.
• The sparse metric learning optimization requires positive semidefinite constraint and gradient-based methods, which may face scalability or local minima issues in very high dimensions.
• Reproducibility is limited by lack of public release of code or datasets, including additive manufacturing data.
• Robustness under strong distributional shifts, adversarial predictors, or outlier distributions is not assessed.
• The influence of hyperparameters such as group lasso penalty \u03BC and roughness \u03BB lacks detailed sensitivity analysis.
• No explicit evaluation on time-series forecasting or extrapolation beyond observed indices.

Open questions / follow-ons

• How to efficiently extend Dynamic Fréchet Regression and sparse metric learning to multivariate or high-dimensional distributional responses where quantile-based closed-form distances do not exist?
• Can the index-aware weighting and sparse metric learning be integrated with deep neural architectures to model more complex, nonlinear dynamic distributional patterns?
• What are the theoretical guarantees on consistency and convergence of the two-stage estimation under various noise models and predictor correlation structures?
• How does the approach perform under distributional shifts or non-smooth dynamic evolutions, and can it be adapted for robustness?

Why it matters for bot defense

Dynamic Fréchet Regression introduces a principled framework for regressing evolving distributional responses on scalar features while preserving the intrinsic metric geometry and enabling interpretable feature selection. For bot-defense and CAPTCHA practitioners, modeling distributional trajectories of system responses (e.g., latency distributions, behavioral feature distributions over time) is critical for distinguishing automated bots from human users presenting dynamic patterns. The index-aware weighting allows capturing temporal or layered evolution of such response distributions, while sparse metric learning identifies key scalar predictors driving these dynamics.

Applying DFR concepts can enable more nuanced, geometry-respecting modeling of behavioral data that vary dynamically rather than relying solely on scalar summaries or static distributions. The weighted Fréchet mean prediction approach supports capturing distributional shape changes important for bot detection over continuous interactions or sessions. However, practical deployment would require access to appropriate metric spaces for the domain's distributions, scalable solvers for large-scale data, and consideration of multivariate or networked data. Feature selection via sparse metric learning offers a promising alternative to coefficient-based selection especially when explicit regression parameters are unavailable. Overall, DFR provides a theoretical and algorithmic foundation for modeling and interpreting dynamic distributional behavior in security-relevant settings.

Cite

@article{arxiv2607_09613,
  title={ Dynamic Frechet Regression with Feature Selection for Distributional Data },
  author={ Kiran Adhikari and Amrutha Dinesh and Mathew Kuttolamadom and Ying Lin },
  journal={arXiv preprint arXiv:2607.09613},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.09613}
}

Read the full paper

• arXiv:2607.09613 (abstract)
• arXiv:2607.09613 (PDF)