Minimax Theory of Likelihood-Based Deep Learning for Speckle Regression

Source: arXiv:2607.14064 · Published 2026-07-15 · By Soham Jana

TL;DR

This paper addresses the fundamental statistical problem of estimating smooth nonparametric regression functions from observations corrupted by multiplicative speckle noise combined with additive Gaussian noise. Speckle noise is common in coherent imaging modalities like SAR and OCT and differs from classical additive noise models by rendering the regression function unidentifiable from conditional means, thus invalidating least-squares methods typically used in deep learning-based denoising. The authors propose likelihood-based deep neural network (DNN) estimators designed specifically for this multiplicative noise model, including an ℓ1-penalized version to handle sparse high-dimensional features. They establish non-asymptotic upper bounds on the mean squared error of these estimators and prove matching minimax lower bounds up to logarithmic factors, extending prior 1D results to multivariate and sparse cases. Interestingly, the minimax rates coincide with those under additive Gaussian noise alone, showing the statistical difficulty of speckle regression is essentially equivalent. Numerical experiments corroborate the theoretical results with convergence rates aligned to the predicted minimax rates. This work provides the first minimax theory foundation for likelihood-based deep learning methods in speckle denoising, offering statistical guarantees for a problem with widespread practical applications but limited prior theoretical understanding.

Key findings

• The proposed likelihood-based deep neural network estimator achieves a mean squared error rate bounded by O((log n)^(2γ*/(2γ*+1)) · n^(-2γ*/(2γ*+1))) with probability at least 1 - 1/n^c for some constant c.
• The ℓ1-penalized likelihood estimator attains the same near-minimax convergence rate in high-dimensional sparse feature settings, adapting to unknown sparse coordinate subsets.
• A minimax lower bound on the mean squared error matching the upper bound up to logarithmic factors is established for smooth functions in the hierarchical class H(d, l, P), extending Malekian et al. (2025) from d=1 to multivariate cases.
• The minimax convergence rates for speckle regression statistically coincide with those under additive Gaussian noise despite the multiplicative speckle noise model's complexities.
• The DNN estimators do not require sample splitting and the theoretical guarantees hold for fixed design points under mild distribution assumptions.
• Numerical experiments demonstrate consistent convergence to the true function, improving with increasing sample size, depth, and training epochs, validating theoretical rates.
• In the sparse high-dimensional setting, the method successfully recovers active features, with feature detection accuracy increasing with signal strength.
• The analysis assumes the additive noise variance σ_τ bounded by a constant, ensuring numerical stability and tractability of the likelihood objective.

Threat model

n/a — This is a statistical estimation theory paper focusing on the fundamental limits of regression under multiplicative speckle noise, not involving an adversarial threat model or attacker capabilities.

Methodology — deep read

1. Threat model & assumptions: The adversary is not explicitly modeled; rather, the statistical challenge studied is estimation under the presence of multiplicative speckle noise ξ_i~N(0,1) and additive Gaussian noise τ_i~N(0,σ_τ^2) in the observed data y_i = f*(x_i)ξ_i + τ_i. The key assumption is that the data generating function f* belongs to a smooth hierarchical composition function class H(d, l, P) with dimension-adjusted smoothness parameter γ*>1/2. The additive noise level σ_τ is assumed known and bounded by a constant for technical ease. Feature vectors x_i lie in a bounded subset of R^d. The multiplicative noise model causes non-identifiability from conditional means, necessitating likelihood-based rather than least squares methods.

2. Data: The setup assumes n independent observations (x_i, y_i) with fixed or random design points x_i. For the fixed design, empirical norms are used; for random design, convergence to population norms is justified under mild assumptions (e.g., uniform grid on [0,1]^d). No explicit public dataset is used; the analysis is theoretical.

3. Architecture / algorithm: The estimator is a deep ReLU fully-connected neural network f_ϕ in the class G(L, d, 1, N, M, B) with truncation of outputs to [-M, M], depth L (constant), and width N ~ (n/log n)^{1/(4γ*+2)}. Weight magnitudes are bounded and scaling with sample size is controlled. The network minimizes the negative log-likelihood defined as ℓ_FULL(f) = (1/n) ∑ [ y_i^2 / (f(x_i)^2 + σ_τ^2) + log(f(x_i)^2 + σ_τ^2) ]. For high-dimensional sparse cases, an ℓ1-penalized problem jointly estimates a feature selection matrix Θ and network parameters ϕ, encouraging sparse feature usage.

4. Training regime: The theoretical results assume exact minimization of the likelihood over the constrained neural network class with specified hyperparameters. Empirically, networks are trained with gradient-based optimizers (details truncated), varying network depths, widths, and epochs studied.

5. Evaluation protocol: Risk is measured by squared empirical norm ∥f̂ - f*∥_n^2 or population norm in stochastic setups. Theoretical upper bounds on estimation error are derived with high probability. Minimax lower bounds are obtained by constructing suitable function classes showing any estimator must incur errors at the specified rates. Numerical experiments illustrate empirical MSE decay with sample size and validate sparsity recovery.

6. Reproducibility: Code and datasets are not publicly released as the work is primarily theoretical with simulated experiments. Details on numerical setups are summarized but exact reproducibility requires additional materials not provided.

Concrete example: For fixed dimension d, the estimator is trained by minimizing the likelihood (eq. 2) over a ReLU network with depth ~ constant and width N ~ (n/log n)^{1/(4γ*+2)}. The output is thresholded between fixed bounds to maintain stability. Under these conditions, Theorem 1 guarantees that with high probability the empirical risk ∥f̂ - f*∥_n^2 is at most O((log n)^{2γ*/(2γ*+1)} n^{-2γ*/(2γ*+1)}). This matches known minimax rates up to log factors, showing the likelihood-based DNN can optimally recover the signal corrupted by speckle noise. Similar methodology carries over to sparse high-dimensional setups with ℓ1 penalty on feature selection matrix Θ, achieving analogous rates and feature recovery properties.

Technical innovations

• Formulation of a novel likelihood-based objective function tailored for multiplicative speckle noise models, circumventing the non-identifiability issues inherent to multiplicative noise and invalidating classical least squares loss.
• Development of deep neural network estimators with provable near-minimax optimal risk bounds for nonparametric regression under multiplicative speckle noise in both low-dimensional and sparse high-dimensional settings.
• Extension of minimax lower bounds for speckle noise regression from one-dimensional cases to multivariate and sparse function classes, demonstrating rates match additive noise models up to logarithmic factors.
• Incorporation of ℓ1-penalized likelihood estimation approach enabling simultaneous function estimation and feature selection in high-dimensional sparse setups within the speckle noise framework.

Baselines vs proposed

• Classical nonparametric regression with additive Gaussian noise: minimax MSE rate O(n^{-2β/(2β+d)}) vs proposed speckle DNN estimator: MSE rate O((log n)^{2γ*/(2γ*+1)} · n^{-2γ*/(2γ*+1)}), matching rates up to log factors.
• Local polynomial and wavelet-based estimators for speckle noise (prior work): upper bounds only in fixed dimension vs proposed DNN estimator: matching upper and lower bounds established for multivariate and sparse settings.

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.14064.

Fig 1: Comparison of error across different depths with fixed width 50. (a) Test MSE curves across

Fig 2: Visualization of the estimated feature selection matrix Θ⊤, showing that the DNN estimator

Fig 3 (page 12).

Fig 4 (page 12).

Limitations

• The analysis assumes knowledge and boundedness of additive noise standard deviation σ_τ at a constant level, leaving unknown the behavior for varying or unknown noise levels.
• The theoretical guarantees rely on exact or near-exact minimization of a non-convex likelihood objective over constrained neural network classes, which may be difficult in practice.
• No adversarial robustness or explicit analysis under model misspecification or shift in noise distribution is provided.
• Empirical evaluations are limited and simulated; real-world experiments on actual coherent imaging data are not included in the current work.
• Logarithmic factors in the convergence rates arise as artifacts of empirical process bounds and may not be tight; tighter analyses or alternative estimators may reduce these.
• Reproducibility details (e.g., exact architectures, random seeds, optimization parameters) are not fully disclosed in the paper.

Open questions / follow-ons

• How do the minimax rates and estimator performance depend quantitatively on the additive noise variance σ_τ when it is unknown or varies with the sample size?
• Can the logarithmic factors in the convergence rates be eliminated or reduced by alternative estimators or refined analyses?
• How well do these likelihood-based DNN estimators perform empirically on real-world coherent imaging datasets, including various speckle noise distributions beyond Gaussian approximations?
• Can the framework and theory be extended to handle more general multiplicative noise models, e.g., non-Gaussian or correlated speckle noise?

Why it matters for bot defense

Although the paper is theoretical and focused on imaging, its core insight is that likelihood-based deep learning methods can provably denoise multiplicative noise corrupted signals up to minimax optimal rates equivalent to additive noise problems. For bot-defense and CAPTCHA practitioners, this suggests that when noise or distortions complicate the data distribution beyond common additive models, direct likelihood modeling combined with deep networks can still yield statistically efficient recovery. The proposed ℓ1-penalized likelihood approach also offers a template for sparse, high-dimensional signal recovery under complex noise, potentially useful when bots attempt to mask or distort challenge data via multiplicative perturbations. However, the work does not address adversarial or worst-case attacks typical in bot-defense but rather foundational statistical limits. CAPTCHA engineers might consider adapting or extending such noise-aware likelihood training objectives for robust signal or content extraction in noisy, multiplicatively corrupted challenge-response scenarios.

Cite

@article{arxiv2607_14064,
  title={ Minimax Theory of Likelihood-Based Deep Learning for Speckle Regression },
  author={ Soham Jana },
  journal={arXiv preprint arXiv:2607.14064},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.14064}
}

Read the full paper

• arXiv:2607.14064 (abstract)
• arXiv:2607.14064 (PDF)