SARLO-80: Worldwide Slant SAR Language Optic Dataset 80cm

Source: arXiv:2606.20523 · Published 2026-06-18 · By Solène Debuysère, Nicolas Trouvé, Nathan Letheule, Elise Colin, Georgia Channing

TL;DR

This paper addresses a significant gap in multimodal remote sensing datasets by introducing SARLO-80, a worldwide very-high-resolution (VHR) Synthetic Aperture Radar (SAR) and optical imagery dataset with natural-language descriptions. Unlike existing SAR-optical datasets that primarily use low-resolution, intensity-only, ground-range detected (GRD) products and often discard complex-valued SAR data and native acquisition geometries, SARLO-80 preserves complex SAR single-look complex (SLC) data in native slant-range geometry at 80cm resolution. It pairs 119,566 SAR patches with pixel-aligned high-resolution optical imagery warped into the SAR grid and three levels of natural language captions (short/mid/long). The dataset covers 257 locations across 72 countries, spanning diverse landscapes and acquisition parameters. It includes fixed train/val/test splits, full preprocessing code, and baseline experiments for cross-modal retrieval and text-to-SAR generation, demonstrating the viability of multimodal alignment with complex SAR data.

This resource is novel because it standardizes globally distributed commercial Umbra spotlight SICD SAR acquisitions via frequency-domain band-limited resampling that preserves coherent SAR physics and avoids reprojection artifacts. The local affine warping of optical tiles into SAR slant-range geometry enables pixel-level alignment between modalities. The inclusion of natural language captions derived from the optical imagery provides a multimodal anchor for vision-language modeling in SAR. Baseline experiments show that using multiple caption lengths improves text-to-SAR generation and that domain adaptation significantly boosts SAR-text cross-modal retrieval, especially for large vision transformers. The dataset thus bridges a critical gap to enable SAR-aware foundation model research at unprecedented scales and resolutions.

Key findings

• SARLO-80 contains 119,566 triplets of complex-valued SAR SLC patches (1024x1024, 0.8m resolution), aligned optical tiles, and three caption variants (short, mid, long).
• Dataset spans 257 geographic locations across 72 countries and covers diverse land types and acquisition parameters (VV/HH polarization, 10–70° incidence angles).
• Authors perform frequency-domain FFT-based band-limited resampling to standardize SAR pixel spacing to 80cm in native slant-range geometry without geometric distortions or reprojection artifacts.
• Optical images are warped to SAR slant-range grid by estimating a first-order affine transform from SAR crop corners to optical tile coordinates, enabling pixel-level alignment.
• Baseline text-to-SAR generative diffusion model using multi-caption training and late-stage timestep reweighting achieves KL divergence between generated and real SAR amplitude of 0.54 and autocorrelation of 0.38 (vs 0.21 for real SAR), indicating improved realism.
• Cross-modal retrieval experiments show frozen CLIP models yield very low recall@10 (~3–9%), but full fine-tuning on SARLO-80 boosts recall@10 to 23–28% and median rank from >200 to 29–40 on a 15K-sample test set.
• Using three caption lengths uniformly during training improves text-to-SAR generation loss and visual quality compared to single caption length.
• Large ViT-L/14 backbone achieves best cross-modal retrieval performance after fine-tuning.

Threat model

n/a — The paper introduces a research dataset and benchmarking framework rather than a security system. The adversary model is not applicable.

Methodology — deep read

The core goal is to construct a large-scale, globally distributed SAR-optical-text dataset preserving native SAR physics to advance multimodal learning, especially vision-language modeling on complex SAR data.

1. Threat model & assumptions: The dataset is aimed at enabling research on multimodal alignment and generation in SAR imagery, assuming access to complex SLCSICD data from Umbra satellite acquisitions. The adversary model is n/a since this is dataset and benchmark oriented research, not focused on active attackers.

2. Data provenance and preprocessing: They start from 2,565 Sentinel Independent Complex Data (SICD) spotlight scenes from the Umbra constellation, providing complex-valued single-look complex SAR images (magnitude + phase) with metadata. Scenes cover diverse geographic locations worldwide (72 countries, 257 locations) and diverse environments, with VV or HH polarization and incidence angles from 10° to 70°.

3. SAR Preprocessing: To ensure physically consistent multimodal learning, they standardize all SAR scenes to 80cm pixel spacing in native slant-range geometry by performing 2D FFT frequency domain band-limited resampling—cropping the spectrum if native resolution is finer, or zero-padding if coarser—and then applying inverse FT to recover complex SAR images. This avoids geometric reprojections that distort layover, shadowing, and foreshortening effects.

4. Patch tiling and geolocation: Each SAR scene is cut into overlapping 1024x1024 pixel patches (stride 512). Using SICD metadata, they compute dense pixel-to-ECEF coordinates and convert to geodetic coordinates (lat, lon, height) to produce aligned geolocation grids per patch.

5. Optical pairing and alignment: For each SAR patch, the geographic footprint defines the region to retrieve a high-resolution optical RGB tile from Google Satellite (zoom 19). A local 2D affine transform is estimated from SAR patch corners (in lat/lon) to optical tile pixels, and the optical tile is warped by inverse bilinear interpolation into the SAR slant-range grid, enabling pixel-level alignment between optical and SAR.

6. Caption generation: Three variants of natural language descriptions (short, mid, long) are automatically generated per patch using the CogVLM2 model applied to the optical tile, followed by large language model post-processing to remove vague or color references and increase lexical consistency.

7. Dataset splits: They release fixed train/validation/test splits disjoint by satellite pass to avoid overlap in acquisition geometry.

8. Baselines:

• Cross-modal retrieval: Evaluate CLIP vision transformers (ViT-B/16, ViT-L/14, etc.) with frozen and fully fine-tuned settings using the MID captions. Metrics are recall@10 and median rank on a 15K test set.
• Text-to-SAR generation: Adapt a latent diffusion model with LoRA adapters on captions, training for 8 epochs with cosine LR schedule and batch size 32 on a single H100.

9. Evaluation:

• Generation quality is quantified by KL divergence of global amplitude distributions and patch-level autocorrelation against real SAR patches.
• Retrieval is evaluated via recall@10 (higher better) and median rank (lower better).

10. Reproducibility: The authors provide the full preprocessing pipeline, metadata, and baselines code publicly. The optical imagery itself is not redistributed due to licensing; only metadata for reconstruction is provided. The dataset is hosted as sharded WebDataset tar archives on Hugging Face, totaling approx 1.42 TB.

Example end-to-end pipeline for a single patch:

• Download raw SICD scene from Umbra open catalog.
• Perform frequency-domain band-limited resampling to 80cm.
• Compute dense pixel geolocation grids from SICD metadata.
• Extract overlapping 1024x1024 slant-range patches.
• For each patch, compute geographic corner footprint.
• Retrieve optical tile at corresponding lat/lon from Google Satellite.
• Estimate affine transform from SAR patch to optical tile.
• Warp optical tile pixels into SAR grid (bilinear interpolation).
• Generate three caption variants from optical tile via VLM.
• Package complex SAR patch, amplitude PNG, metadata, and captions into dataset sample.
• Use full dataset for training vision-language models or diffusion-based SAR generation.

Overall, the methodology focuses on fully preserving native SAR data and physics, carefully aligning optical imagery via affine transforms, and producing high-quality captions to enable rich multimodal learning and benchmarking.

Technical innovations

• Frequency-domain band-limited FFT resampling of complex SAR SLC data to standardize pixel spacing at 80cm in native slant-range geometry, preserving physical SAR properties without reprojection artifacts.
• Local affine warping of optical tiles into SAR slant-range pixel grid for precise pixel-level alignment between modalities without complex sensor models.
• Generation of three caption variants (short/mid/long) per sample using CogVLM2 and large language model post-processing to support comprehensive vision-language learning scenarios.
• Release of a large-scale, complex-valued VHR SAR dataset with 119,566 samples worldwide paired with aligned optical imagery and natural language, unmatched in scale and resolution.

Datasets

• SARLO-80 — 119,566 triplets (complex SAR SLC patches, optical tiles, 3 caption variants) — Umbra SICD spotlight acquisitions, publicly available at Hugging Face Hub

Baselines vs proposed

• Text-to-SAR generation (M_C1 single-caption): KL divergence = 1.35, autocorrelation = 0.40 vs M_C3_TO multi-caption with timestep optimization: KL divergence = 0.54, autocorrelation = 0.38 (real SAR autocorrelation = 0.21).
• Cross-modal retrieval frozen ViT models: Recall@10 (Image→Text) range 2.60%–4.87%, median rank ~240–355; full fine-tuning improves Recall@10 up to 28.47% (ViT-L/14) and median rank down to 29.
• Using three caption formats uniformly (M_C3) during diffusion training lowers average loss to 0.088 vs 0.092 for single-caption (M_C1), improving generation stability and semantic fidelity.

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.20523.

Fig 1 (page 1).

Fig 1: Overview of dataset coverage and SAR acquisition geometry.

Fig 2: Overview of our final dataset (Top-N + Other): distribution of images by country (left), and

Fig 3: Detailed overview of the dataset creation pipeline. Steps

Fig 7: and Fig. 8.

Fig 4: Example of SAR–optical–Caption from our Dataset

Fig 5: Prompt labeling overview (left) and summary prompt statistics (right).

Fig 6: Comparison of generated SAR images across models and real data (1024×1024 px, 80 cm

Limitations

• Optical images are not directly redistributed due to licensing; users must download tiles on demand, which may incur latency or variability.
• Affine transform used for SAR-optical alignment is a local first-order approximation which may not perfectly capture geometric distortions especially over larger patches or complex terrain.
• Baseline models are initial proof-of-concept; optimal architectures and training strategies for SAR vision-language remain open challenges.
• The dataset currently only includes VV/HH polarization and spotlight acquisitions from one constellation (Umbra); extending to other SAR modes, sensors, or temporal sequences is future work.
• Speckle and high-frequency SAR textures remain challenging for generative modeling as suggested by remaining discrepancies in autocorrelation metrics.
• Evaluation focuses on retrieval and conditional generation tasks; more downstream SAR-specific tasks like segmentation or classification with multimodal inputs remain to be explored.

Open questions / follow-ons

• How to best exploit complex SAR phase information and derived features (e.g., interferometric or polarimetric data) in multimodal representation learning?
• Can advanced geometric transformations improve SAR-optical alignment beyond affine warping to better model terrain-induced distortions?
• What architectures and training regimes are optimal for generative modeling of complex-valued, very-high-resolution SAR imagery?
• How well do multimodal SAR–optical–text models generalize to unseen geographic regions, sensors, or temporal sequences?

Why it matters for bot defense

For bot-defense and CAPTCHA engineers, SARLO-80 presents an opportunity to study multimodal learning with extremely challenging synthetic aperture radar imagery paired with natural language descriptions and corresponding optical imagery. Although SAR data is primarily used in remote sensing, the ideas of preserving native sensor geometries and complex-valued signals can inspire more physically grounded approaches for multi-sensor fusion and multimodal learning in security-critical domains.

In CAPTCHA and bot-defense, generating or interpreting multimodal challenges that incorporate diverse sensor modalities is a growing interest. This dataset’s methodology for precise alignment and captioning in native coordinates, as well as baseline experiments in cross-modal retrieval and conditional generation, could inform design of robust multimodal challenge-response systems that leverage complex imaging data beyond natural photographs. While not directly applicable, the rigorous preprocessing and multimodal annotation strategies exemplify best practices for dataset curation that practitioners should consider when working with novel or noisy sensing modalities.

Cite

@article{arxiv2606_20523,
  title={ SARLO-80: Worldwide Slant SAR Language Optic Dataset 80cm },
  author={ Solène Debuysère and Nicolas Trouvé and Nathan Letheule and Elise Colin and Georgia Channing },
  journal={arXiv preprint arXiv:2606.20523},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.20523}
}

Read the full paper

• arXiv:2606.20523 (abstract)
• arXiv:2606.20523 (PDF)