Freeze, Then Select: Structured Field Adapters and Stability-Validated Weak Selection for PDE Discovery from Sparse Observations

Source: arXiv:2607.29665 · Published 2026-07-31 · By Juncheng Zhong, Chenghuang Shen, Jianfeng Liu, Zhengdong Xiao, Longjiu Luo, Qianrong Wang et al.

TL;DR

This paper addresses the problem of discovering partial differential equations (PDEs) from sparse and noisy observations, which requires both reconstructing a smooth continuous field and selecting the correct terms in the PDE governing dynamics. The authors analyze optimization paths in existing coupled neural PDE discovery methods and identify three key behaviors: the exact correct support terms may persist throughout training, appear only transiently, or never emerge at all. Motivated by this instability, they propose a novel two-stage "freeze-then-select" framework. First, a structured field adapter is trained solely to reconstruct the continuous field from data without incorporating any PDE residual loss, decoupling field reconstruction from equation selection. The adapter factorizes the field into learned spatial features and smooth temporal coefficients represented by cubic B-splines. Once trained, the field is frozen. Second, Stability-Validated Weak Selection (SVWS) identifies recurrent candidate PDE terms across multiple independently constructed weak-form systems, refits coefficients, and selects the final equation by held-out weak-form validation. This principled separate validation avoids sensitivity to checkpoint timing and threshold choice common in one-stage neural PDE discovery.

Empirically, their approach outperforms classical sparse regression, weak-form, and several state-of-the-art neural baselines on the sparse sensor regimes of the MDBench benchmark across Korteweg-de Vries, Kuramoto–Sivashinsky, and 2D advection-diffusion PDEs. The gains are especially pronounced on challenging chaotic Kuramoto–Sivashinsky dynamics. Further, by extending SVWS to symbolic regression candidates generated via genetic programming, they successfully recover the power-law form of an unknown nonlinear diffusion function from sparse noisy data, demonstrating applicability beyond fixed candidate libraries. Analysis of optimization paths substantiates the motivation to decouple reconstruction from selection for robustness.

Key findings

• Exact support recovery on all 6 sparse MDBench regimes (5/5 runs each) with freeze-then-select vs baselines with up to 0/5 on Kuramoto–Sivashinsky (KS) dynamics (Table 1).
• On KS, freeze-then-select achieves lowest median coefficient error E_\xi, e.g. 0.035 (S20) and 0.055 (T20) vs best baseline 0.512 (PDE-FIND, T20).
• Structured adapter with Fourier features outperforms coordinate MLP variants in field reconstruction and support recovery; 5/5 exact vs 0–3/5 on KS (Table 2a).
• SVWS stability-validated multi-system selection maintains exact recovery over varying sparsity thresholds, avoiding threshold sensitivity seen in single STLSQ runs (Table 2b).
• Joint coupled neural optimization paths show exact support may appear transiently or never appear, motivating selection post field freezing (Fig 3).
• Symbolic regression extension with SVWS recovers power-law nonlinear diffusion q(u)=0.1u^{1.73} exactly in all runs for 0–10% noise (5/5 each), unlike PySR baselines on full-grid q(u) (Table 3).
• Freezing the field after reconstruction enables decoupling from equation search during training, improving consistent PDE discovery from sparse/noisy data.

Threat model

The threat model consists of sparse, noisy measurements of the underlying PDE solution, limiting direct access to field values or derivatives required for equation identification. The aim is to recover the true sparse PDE support under incomplete and noisy data without direct adversarial interference. The adversary is thus environmental noise and data sparsity rather than an active attacker or data manipulation. The method assumes no direct access to ground truth PDE coefficients or residuals during reconstruction.

Methodology — deep read

1. Threat model & assumptions: The adversary here is implicit as the noise and sparsity in the observations, and the challenge is reliable PDE discovery despite limited spatial and temporal sensor coverage (20% measurements) and additive measurement noise up to 10%. The method assumes the PDE can be expressed sparsely in a fixed or symbolic candidate space.

2. Data: The experimental benchmarks use public MDBench datasets for three PDEs: Korteweg-de Vries (KdV), Kuramoto–Sivashinsky (KS), and 2D advection-diffusion (2D AD). Observations are sparsely sampled either spatially (S20: random 20% sensor locations fully observed in time) or temporally (T20: 20% random time frames fully observed spatially). Each test setting uses 5 random seeds and shared observations across methods.

3. Architecture / algorithm: The core model is a structured field adapter that reconstructs a differentiable continuous space-time field u_hat(x,t) from sparse observations using learned spatial features Φ_η(x) multiplied by temporal coefficients C in a centered cubic B-spline basis β(t). Spatial inputs are first encoded with fixed Gaussian Fourier features to enhance expressivity. The field is modeled as: u_hat_η,C(x,t) = b_η(x) + Φ_η(x)^⊤ C β̄(t), where b_η is a learned spatial background representing empirical mean, and β̄(t) is mean-centered spline basis. The adapter is trained purely by minimizing MSE observation loss with spatial feature orthogonality and temporal smoothness regularizers, intentionally without any PDE residual or physics-based loss.

For PDE selection, Stability-Validated Weak Selection (SVWS) constructs multiple independent weak-form linear systems A_rξ ≈ b_r from the frozen reconstructed field by applying integration-by-parts to shift derivatives onto compactly supported space–time test functions. Supports (candidate active PDE terms) are generated independently across generation systems via sequential threshold least squares (STLSQ) using a path of sparsity thresholds, ensuring robustness. Supports recurring frequently across systems and thresholds pass stability filtering. Then each candidate support is refit (coefficients estimated) on an independent fit system by ridge regression. Finally, the refitted supports are validated on held-out weak systems by averaging normalized residuals. SVWS selects the sparsest support within one-standard-error of the minimum validation error with tie breaks favoring lower error and higher frequency.

The same freeze-then-select framework extends to symbolic PDE discovery by genetic programming (GP). GP proposes nonlinear candidate terms q_r(u), where the outer differential operator ∂xx is fixed, and SVWS refits and validates the nonlinear functions’ continuous parameters on separate weak systems.

4. Training regime: The adapter is trained on sparse observations using gradient-based optimization with fixed Fourier feature encoding and regularization hyperparameters set as described (details in supplement). No PDE residual loss is used. The training fits parameters η and C jointly until convergence.

5. Evaluation protocol: Exact support recovery (bS = S*) on 5 test seeds per PDE / regime is the primary metric. Relative coefficient estimation error and field reconstruction error are secondary metrics. Baselines compared include PDE-FIND, WSINDy, DeepMoD, PINN-SR, Weak-PDE-LEARN, and DL-PDE — all given the same observations and candidate term libraries. Ablations analyze the structured adapter vs MLP spatial nets and Fourier features, as well as selector sensitivity to STLSQ thresholds. Optimization path behavior is measured by tracing support recovery dynamics over training iterations.

6. Reproducibility: Code and configurations will be publicly released upon acceptance. The MDBench datasets are public. Baseline hyperparameters and evaluations follow published and reimplemented protocols documented in the supplement. The approach uses standard stable optimization and weak-form integration methods described with sufficient detail for reproduction.

Example flow: From sparse observations, fit the field adapter network until convergence purely on MSE loss. Freeze this network and evaluate multiple independent weak-form linear systems on shifted test-function grids using the frozen reconstruction. Run STLSQ sparse regression across sparsity thresholds in generation systems to propose supports. Filter by stability of recurrence across systems and thresholds. Refit coefficients on an independent fit system by ridge regression. Finally, score validated residuals on held-out weak systems to select the final PDE support that is sparse and has low validation error.

Technical innovations

• Introduction of a structured field adapter representing the PDE solution as factorized spatial features times temporal cubic B-spline coefficients, trained without PDE residual loss to ensure stable field reconstruction.
• Development of Stability-Validated Weak Selection (SVWS), which separates candidate PDE support generation, coefficient refitting, and validation on independent weak-form systems constructed from the same frozen field.
• The freeze-then-select paradigm that decouples PDE discovery from neural surrogate training, avoiding checkpoint-dependent and transient support behaviors observed in coupled neural optimization.
• Extension of SVWS beyond fixed libraries to symbolic PDE discovery by genetic programming candidates with continuous parameter refitting and held-out weak-form validation.

Datasets

• MDBench — variable size, includes KdV, Kuramoto–Sivashinsky, and 2D advection-diffusion PDE datasets — public benchmark

Baselines vs proposed

• PDE-FIND: exact support recovery on KS-T20 = 3/5 vs proposed = 5/5
• WSINDy: exact support recovery on KS-T20 = 1/5 vs proposed = 5/5
• DeepMoD: exact support recovery on KS-S20 = 0/5 vs proposed = 5/5
• PINN-SR: exact support recovery on KS-S20 = 0/5 vs proposed = 5/5
• Weak-PDE-LEARN: exact support recovery on KS-S20 = 0/5 vs proposed = 5/5
• DL-PDE: exact support recovery on KS-S20 = 3/5 vs proposed = 5/5
• Symbolic regression (PySR) on full-grid q(u): power-law recovery at 0% noise = 1/5 vs proposed SVWS = 5/5

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.29665.

Fig 1: Freeze, then select. A structured adapter fits sparse

Fig 2: Structured field adapter. A spatial network produces a background and spatial features with temporal coefficients

Limitations

• No adversarial robustness evaluation against active perturbations or adversaries explicitly targeting PDE discovery.
• Field adapter training omits physics-informed loss, which may limit performance on very noisy or unstructured sparse settings.
• Experiments restricted to three PDE types and MDBench benchmark; generalization to other PDE classes or real-world noisy data remains to be seen.
• Genetic programming symbolic search evaluates a fixed nonlinear diffusion operator only; extension to richer operator grammars is future work.
• The method requires multiple independent weak-form systems and threshold paths, increasing computational cost compared to single-run sparse regressions.
• Frozen field representations assume smooth spatial and temporal decompositions, which may be challenged by highly turbulent or stochastic dynamics.

Open questions / follow-ons

• Can the freeze-then-select framework be extended to multi-physics or coupled PDE systems with vector fields?
• How does the method perform on real experimental datasets with nonuniform sensor placement and correlated noise?
• Could learned spatial features and temporal splines adapt dynamically to improve robustness to strongly transient or chaotic dynamics?
• What are the computational trade-offs and potential speedups for SVWS in large-scale 3D or high-dimensional PDE discovery?

Why it matters for bot defense

For bot-defense and CAPTCHA engineers aiming to robustly extract governing rules or continuous field dynamics from limited spatiotemporal data, this paper presents a significant methodological advance. The freeze-then-select approach demonstrates how decoupling neural field reconstruction from equation selection enhances robustness to sparse and noisy measurements, which is analogous to reliably identifying subtle user-behavioral patterns amidst partial telemetry. The SVWS multi-system stability validation offers a rigorous model selection strategy less sensitive to hyperparameter tuning or training dynamics variability, relevant for deploying reliable anomaly detection or bot-identification models on sparse activity logs. Additionally, the symbolic extension indicates potential for uncovering novel interpretable behavioral functions beyond fixed rule sets, helpful when predefined heuristics fail due to evolving automated attacks. Practitioners can draw from the structured factorization approach and weak-form systems to design PDE-inspired continuous models of user interactions with intervening layers of validation to ensure stable model interpretation and selection. Overall, the proposed validation-centric decoupling strategy could inspire more robust and interpretable classifier design in security contexts with sparse, noisy data.

Cite

@article{arxiv2607_29665,
  title={ Freeze, Then Select: Structured Field Adapters and Stability-Validated Weak Selection for PDE Discovery from Sparse Observations },
  author={ Juncheng Zhong and Chenghuang Shen and Jianfeng Liu and Zhengdong Xiao and Longjiu Luo and Qianrong Wang and Wenjun Xu and Wenlian Lu },
  journal={arXiv preprint arXiv:2607.29665},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.29665}
}

Read the full paper

• arXiv:2607.29665 (abstract)
• arXiv:2607.29665 (PDF)