ATSplat: Compact Feed-forward 3D Gaussian Splatting with Adaptive Token Expansion

Source: arXiv:2607.20417 · Published 2026-07-22 · By Cho In, Jeonghwan Cho, Mijin Yoo, Gim Hee Lee, Seon Joo Kim

TL;DR

ATSplat addresses a major inefficiency in feed-forward 3D Gaussian Splatting (3DGS) where primitives are placed densely and uniformly tied to input image pixels, leading to redundant representations in simple areas and insufficient detail in complex regions. Prior pixel-aligned methods depend on image resolution and camera viewpoints rather than scene complexity, resulting in bulky Gaussian sets and rendering inefficiencies. ATSplat restores the core advantage of per-scene optimized 3DGS — adaptive scene-dependent capacity allocation — within a feed-forward framework using a sparse set of 3D anchor tokens instead of pixel-aligned primitives. These tokens are spatially lifted from coarse depth and multi-view features to form a compact scene scaffold. An Adaptive Token Expansion (ATE) module predicts uncertainty scores per token based on intermediate rendering errors, selectively expanding tokens representing under-reconstructed regions to increase local representational capacity. Each token is decoded into local Gaussians regressed by learnable 3D offsets, decoupling Gaussian placement from input pixels. Experiments on RealEstate10K and DL3DV datasets show ATSplat achieves state-of-the-art novel-view synthesis quality while reducing the number of Gaussians by over 5.7× compared to dense pixel-aligned feed-forward methods, reconstructing from 12 input views at 512×960 resolution in under a second and rendering at over 1100 FPS with only 311K Gaussians. The method thus bridges the gap between optimization-based adaptive 3DGS and efficient feed-forward inference by learning scene-adaptive capacity allocation.

Key findings

• ATSplat reduces Gaussian primitives by more than 5.7× compared to dense pixel-aligned feed-forward 3DGS methods while matching or exceeding their rendering quality (e.g., 28.46 PSNR vs 28.65 for iLRM on RealEstate10K with only 23K vs 131K Gaussians).
• On DL3DV at 256×448 resolution, ATSplat outperforms feed-forward baselines across 2, 4, and 6 input views, e.g. PSNR of 25.67 at 4 views with 80K Gaussians vs DepthSplat’s 23.12 PSNR with 458K Gaussians.
• ATSplat completes reconstruction from 12 input images at 512×960 resolution in under 1 second on a single commercial GPU and renders novel views at 1136 FPS with only 311K Gaussians.
• Adaptive Token Expansion (ATE) learns uncertainty scores aligned with actual 2D rendering errors (measured with D-SSIM) and selectively expands tokens in challenging regions without auxiliary rendering during inference.
• Expanded tokens spatially diversify Gaussian placement beyond parent anchors, better modeling fine geometry details as visualized in Fig.3.
• Ablations show ATSplat’s anchor + local offset formulation yields better reconstruction quality (28.46 PSNR) than pixel-aligned or direct 3D center regression approaches.
• Uncertainty-guided expansion outperforms random, farthest point sampling, and fully learnable expansion strategies, improving PSNR from 27.02 (no expansion) to 28.46.
• ATSplat’s sparse-to-adaptive formulation enables compact Gaussian representations that allocate capacity based on scene complexity rather than image sampling.

Threat model

The adversary is an inefficiency and lack of adaptivity in feed-forward 3D Gaussian Splatting methods that leads to dense, redundant Gaussian primitives tied to input image grids rather than scene complexity, causing computational and representational waste as well as poor reconstruction in complex regions. ATSplat assumes access to calibrated multi-view images and seeks to mitigate this inefficiency by approximating reconstruction difficulty without iterative feedback. The threat model does not consider active attackers or adversarial input manipulation.

Methodology — deep read

The primary threat addressed is the inefficiency from feed-forward 3DGS models that allocate Gaussian primitives uniformly based on image pixels rather than scene complexity, resulting in redundancy and poor detail in complex regions. The adversary is thus inefficiency and poor generalization in feed-forward NVS pipelines rather than an explicit attacker.

ATSplat uses multi-view posed images as input. For each input image, a frozen DINOv2-B backbone extracts coarse image patches (tokens). Coarse patch-level depth is predicted per token. Each token is unprojected along its camera ray using predicted depth to localize a sparse 3D anchor token position. Patch features and local 3D context aggregated from k-nearest neighbors form the anchor token features. This sparse set of 3D anchors creates a compact scaffold approximating scene geometry.

A lightweight multi-view transformer encoder operates on coarse patch tokens from all images to produce cross-view features. A multi-block image-to-3D decoder refines anchor tokens via cross-attention to features from a higher-resolution patch-level encoding (twice resolution). Each decoder block includes an Adaptive Token Expansion (ATE) module.

The ATE module predicts a per-anchor scalar uncertainty score from anchor features via a small MLP. The scores are supervised during training by aligning 2D uncertainty maps computed by splatting intermediate Gaussian sets with rendering error maps measured by D-SSIM against ground truth images. Tokens with the highest uncertainty are expanded by learned linear projections producing multiple child tokens sharing the same 3D anchor position but with distinct features.

Each anchor token is decoded by a Gaussian head (a 2-layer MLP) into a fixed number (K=16) of 3D Gaussians parameterized by center offsets relative to the anchor, rotation quaternions, scales, opacities, and spherical harmonics. This establishes free 3D placement decoupled from input pixels.

The loss function combines a final rendering loss with MSE and perceptual components, plus weighted intermediate rendering losses and uncertainty supervision losses from each decoder block. Training occurs end-to-end on RealEstate10K (2 views) then DL3DV (6 views), with further high-resolution fine-tuning using multiple GPUs.

Evaluation uses standard novel-view metrics (PSNR, SSIM, LPIPS) on held-out views, measuring Gaussian count and runtime on a fixed RTX 3090 or higher GPU. Quantitative comparisons are made against strong feed-forward 3DGS baselines using the same splits and preprocessing. Ablations analyze anchor design variants and expansion selection strategies, showing the proposed method’s superiority.

Reproducibility is supported by detailed architecture descriptions, hyperparameters, and training setups. The authors provide a project page but no explicit public code release or pretrained weights is mentioned in the excerpt. Some prior works used for baselines have publicly available checkpoints. The paper’s approach can be instantiated end-to-end by unprojecting coarse depth tokens to sparse 3D anchors, refining with cross-attention from multi-resolution features, and selectively expanding tokens with uncertainty scoring, ultimately predicting compact sets of Gaussian primitives for real-time rendering.

Technical innovations

• Introduction of sparse 3D anchor tokens derived from coarse depth and camera cues as a compact scene scaffold for primitive placement, replacing pixel-aligned Gaussian regression.
• Adaptive Token Expansion (ATE) module that predicts per-token uncertainty scores supervised by intermediate rendering error maps to guide selective expansion of tokens in challenging regions.
• Decoupling Gaussian primitive placement from input image grids by regressing local 3D offsets from adaptive anchors, enabling free-form placement and adaptive density allocation.
• A single feed-forward framework that restores scene-adaptive capacity allocation of optimization-based 3DGS in real-time inference without iterative error feedback or expensive per-scene optimization.

Datasets

• RealEstate10K — thousands of home-tour videos from YouTube, bounded indoor scenes — used with standard train/test splits
• DL3DV — larger-scale indoor and outdoor scenes with complex geometry — standard splits from prior feed-forward 3DGS works

Baselines vs proposed

• PixelSplat: PSNR=25.89, SSIM=0.858, LPIPS=0.142, Gaussians=131K vs ATSplat: PSNR=28.46, SSIM=0.894, LPIPS=0.118, Gaussians=23K on RealEstate10K 2 views
• MVSplat: PSNR=17.54, SSIM=0.529, LPIPS=0.402, Gaussians=229K vs ATSplat: PSNR=20.07, SSIM=0.630, LPIPS=0.332, Gaussians=40K on DL3DV 2 views
• DepthSplat: PSNR=21.33, SSIM=0.740, LPIPS=0.270, Gaussians=5.9M, Inference=0.758s vs ATSplat: PSNR=24.85, SSIM=0.794, LPIPS=0.241, Gaussians=311K, Inference=0.677s on DL3DV 512x960 resolution
• iLRM (pixel-aligned downsampled): PSNR=28.65, SSIM=0.900, LPIPS=0.110, Gaussians=131K vs ATSplat: PSNR=28.46, SSIM=0.894, LPIPS=0.118, Gaussians=23K at RealEstate10K 256x256 resolution
• TokenGS: PSNR=28.41, SSIM=0.903, LPIPS=0.135, Gaussians=262K vs ATSplat: PSNR=28.46, SSIM=0.894, LPIPS=0.118, Gaussians=23K on RealEstate10K

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.20417.

Fig 1: ATSplat reconstructs compact 3D Gaussians from multi-view captures in a single forward pass. Unlike dense pixel-aligned feed-forward methods that

Fig 2: (a) illustrates an overview of the framework. Given multi-

Fig 3 (page 1).

Fig 4 (page 1).

Fig 5 (page 1).

Fig 6 (page 1).

Fig 7 (page 1).

Fig 3: Visualization of expanded tokens. The coordinate of each ex-

Limitations

• No explicit adversarial evaluation or robustness tests against input corruptions or viewpoint changes reported.
• Model evaluation uses standard PSNR/SSIM/LPIPS metrics that do not capture perceptual or semantic fidelity under challenging lighting or dynamics.
• The approach requires precomputed coarse depth estimates and camera poses, thus assuming high-quality multi-view calibration.
• The exact training hyperparameters and model weights are not publicly released in the excerpt, which may limit immediate reproducibility.
• The adaptive token expansion ratio and selection thresholds require tuning per dataset/resolution for optimal performance.
• Only static scenes with posed multi-view input are considered; dynamic or unposed scenarios remain unaddressed.

Open questions / follow-ons

• Can the adaptive token expansion mechanism be extended to handle dynamic scenes or large-scale outdoor environments beyond static indoor/outdoor settings?
• How well does ATSplat generalize to unposed or weakly posed multi-view inputs where accurate camera calibration is unavailable?
• Can uncertainty prediction be enhanced to incorporate additional cues, such as temporal consistency or semantic priors, to better guide adaptive expansion?
• What are the quantitative or qualitative impacts when combining ATSplat with compression or pruning techniques for further representation compaction?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners focused on robust novel-view synthesis or 3D content generation relevant to anti-bot visual challenges, ATSplat offers a state-of-the-art compact feed-forward 3D Gaussian Splatting framework that effectively balances scene complexity with representation size. Its adaptive token expansion enables efficient concentration of representation capacity on challenging scene parts, providing high-fidelity reconstructions with dramatically fewer primitives. This efficiency and quality balance can impact how CAPTCHA schemes generate or verify complex 3D visual patterns that resist adversarial generation or replication by bots. Furthermore, the method’s fast real-time rendering capability at high resolutions enables scalable deployment in latency-sensitive environments where bot defense requires quick verification under diverse viewpoints. However, its dependence on high-quality multi-view input and static scenes suggests it is best suited for controlled CAPTCHA generation scenarios rather than unconstrained open-world verification tasks.

Cite

@article{arxiv2607_20417,
  title={ ATSplat: Compact Feed-forward 3D Gaussian Splatting with Adaptive Token Expansion },
  author={ Cho In and Jeonghwan Cho and Mijin Yoo and Gim Hee Lee and Seon Joo Kim },
  journal={arXiv preprint arXiv:2607.20417},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.20417}
}

Read the full paper

• arXiv:2607.20417 (abstract)
• arXiv:2607.20417 (PDF)