Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization

Source: arXiv:2606.30576 · Published 2026-06-29 · By Liyao Wang, Ruipu Wu, Haojun Xu, Lei Shi, Linjiang Huang, Si Liu

TL;DR

Cross-view object geo-localization (CVOGL) involves locating a target object seen from a query perspective (such as ground or drone view) within geo-tagged satellite reference images. Existing approaches predominantly rely on 2D appearance matching and suffer from limited datasets that lack scale, geometric metadata, diverse prompt types, and standard field-of-view imagery. This paper addresses these gaps by first introducing CMA-Loc, a large-scale building-centric dataset comprising over 220,000 image pairs spanning ground-satellite and drone-satellite views, enriched with multi-modal referring prompts (points, bounding boxes, masks) and accurate camera pose metadata.

Building on this dataset, the authors propose GAGeo, a unified single-stage geometry-aware framework leveraging the permutation-equivariant 3D foundation model π3 backbone. By integrating visual features from frozen DINOv2, pretrained SAM2 prompt encodings, and learnable task tokens, GAGeo jointly predicts object bounding boxes, segmentation masks, and camera poses in one forward pass. A novel contrastive loss uses the satellite view as a universal anchor to implicitly align ground and drone embeddings, enabling zero-shot ground-to-drone geo-localization without explicit triplet data. Extensive experiments show GAGeo significantly outperforms prior state-of-the-art methods on CMA-Loc and generalizes well to unseen cities, out-of-distribution datasets, and novel cross-view setups.

Key findings

• CMA-Loc dataset contains 223,767 ground/drone-to-satellite instance pairs from 77,200 locations across 8 cities, roughly 18× larger than previous datasets like CVOGL.
• GAGeo achieves 68.48% mAcc on drone-to-satellite and 71.50% mAcc on ground-to-satellite object detection with point prompts on CMA-Loc seen test sets, outperforming the previous SOTA TROGeo (46.86% and 51.58%, respectively).
• Using mask prompts further boosts detection mAcc to 76.00% (drone) and 71.41% (ground), and segmentation mIoU to 67.04%, surpassing previous methods relying on a two-stage SAM Prompt Stage.
• GAGeo’s single-stage design jointly predicts bounding boxes, segmentation masks, and camera poses, improving efficiency and accuracy compared to decoupled multi-stage pipelines.
• Contrastive loss with satellite view as anchor enables zero-shot ground-to-drone geo-localization, achieving 49% mAcc, nearly doubling prior methods without triplet training data.
• The permutation-equivariant π3 backbone reduces viewpoint bias and enables seamless training on heterogeneous ground-to-satellite and drone-to-satellite data pairs.
• Model generalizes well to unseen cities and out-of-distribution evaluation on CVOGL and CVOGL-Seg, nearly doubling mAcc and mIoU compared to prior best approaches.
• Ablation studies confirm deep supervision, contrastive loss, and camera pose supervision each contribute over 5-10% improvements in mAcc and segmentation metrics.

Threat model

n/a — This work targets the computer vision task of cross-view object geo-localization rather than adversarial or malicious threat scenarios. The adversary model is not defined as the focus is on accurate geometric understanding and generalization across heterogeneous views.

Methodology — deep read

1. Threat model & assumptions: The problem setting assumes an adversary is not explicitly modeled as this is primarily a geo-localization and representation learning task rather than adversarial defense. The model assumes access to geo-tagged satellite imagery as a universal reference and queries from ground or drone views. The method targets robust generalization across drastically different viewpoints without requiring triplet pairing of ground, drone, and satellite views for training.

2. Data provenance & processing: The CMA-Loc dataset contains 112,063 ground-to-satellite and 111,704 drone-to-satellite instance pairs across 8 global cities with diverse architectural styles. Ground images are sampled from Google Street View panoramas and converted to monocular standard FOV views to reflect real-world conditions. Satellite images are from Google Earth at zoom level 18. Drone images are synthetically rendered from Google Photorealistic 3D Tiles via Cesium, simulating multiple yaw angles with fixed pitch. Masks are generated using SAM3 with OSM footprint prompts and refined with rule-based filtering for quality, accompanied by precise camera pose annotations for geometric supervision. A smaller set of 1,245 manually verified ground-drone-satellite triplets from University-1652 enable zero-shot evaluation.

Data splits include seen and unseen instance pairs for evaluation to test generalization. Preprocessing includes filtering occluded or low-quality masks and converting panoramas to monocular views.

3. Architecture / algorithm: The GAGeo model builds atop the π3 permutation-equivariant 3D foundation model backbone, which processes pairs of images via alternating local and global attention blocks without anchoring to any fixed viewpoint, reducing bias. Visual features are extracted using a frozen DINOv2 encoder (frozen during training) for patch-level tokens. Referring prompts are encoded via a SAM2 pretrained encoder: sparse prompts (points/boxes) yield token embeddings concatenated directly to query tokens, while dense mask embeddings modulate keys within global attention blocks.

Learnable task tokens are added only to the reference view token stream to facilitate multi-task output decoding. Outputs include bounding boxes via a DETR-style MLP, segmentation masks generated by a SAM-style dynamic convolution hypernetwork applied to upsampled reference features, and camera pose predictions via a dedicated head estimating position heatmaps and rotation on the SO(3) manifold.

4. Training regime: The model is trained end-to-end with a multi-task loss combining focal loss for classification, L1 and GIoU losses for boxes, BCE and Dice losses for mask segmentation, focal loss on heatmaps for camera position, geodesic loss for rotation, plus a contrastive loss aligning pooled object features across views using satellite images as anchor. Intermediate decoder layers receive deep supervision. Details on batch sizes, epochs, optimizer, hardware, or seeds were declared in the supplementary material.

5. Evaluation protocol: Metrics include Acc@75, Acc@50, and mAcc for object detection (averaged over IoU thresholds 0.5-0.95), and mIoU, mDice, average angular error (AAE), mean error (ME) for segmentation and pose. Baselines are existing methods such as TROGeo, DetGeo, OCGNet, Sample4Geo. Evaluations are performed on seen/unseen test splits of CMA-Loc, out-of-distribution datasets CVOGL and CVOGL-Seg, and zero-shot ground-to-drone setups via University-1652 triplets. Ablations test individual loss components and prompt modalities.

6. Reproducibility: The paper references code availability and pretrained weights for π3, DINOv2, SAM2, and SAM3. The CMA-Loc dataset is newly introduced, presumably publicly released. The supplementary material contains additional implementation details. Overall, the paper presents a clear, detailed pipeline suitable for reproduction given dataset access.

Technical innovations

• Introduction of CMA-Loc, a large-scale, high-fidelity cross-view geo-localization dataset with multi-modal prompts and explicit camera pose metadata across diverse cities and viewpoints.
• Development of GAGeo, a unified single-stage transformer framework extending the permutation-equivariant 3D foundation model π3 to integrate multi-modal prompts and learnable task tokens for joint bounding box, mask, and pose prediction.
• Novel asymmetric injection of sparse and dense prompt tokens into query streams, with mask prompt embeddings modulating keys in global attention for enhanced geometry-aware cross-view feature interaction.
• Contrastive loss formulation that uses the satellite view as a universal anchor to implicitly align ground and drone feature representations, enabling zero-shot ground-to-drone localization without explicit triplet data.

Datasets

• CMA-Loc — 223,767 ground/drone-to-satellite instance pairs, 180K images, 77,200 locations, 8 global cities — introduced by authors
• University-1652 — 1,245 manually annotated ground-drone-satellite triplets from prior work [44]
• CVOGL — ~12,000 ground-satellite pairs from prior work [32]
• CVOGL-Seg — subset of CVOGL with segmentation masks [40]

Baselines vs proposed

• TROGeo: ground-to-satellite mAcc = 51.58% (seen) vs GAGeo point prompt: 71.50%, mask prompt: 71.41%
• TROGeo: drone-to-satellite mAcc = 46.86% (seen) vs GAGeo mask prompt: 76.00%
• DetGeo: ground-to-satellite mAcc = 46.04% vs GAGeo point prompt: 71.50%
• OCGNet: ground-to-satellite mAcc = 47.08% vs GAGeo bbox prompt: 71.50%
• On unseen test splits, GAGeo maintains >=65% mAcc compared to <40% for prior SOTA
• Zero-shot ground-to-drone mAcc: prior best ~25% vs GAGeo 49%
• On CVOGL-Seg zero-shot, GAGeo nearly doubles mAcc and mIoU without fine-tuning

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.30576.

Fig 1: Overview of CMA-Loc dataset and GAGeo framework. We introduce

Fig 2 (page 1).

Fig 3 (page 1).

Fig 4 (page 1).

Fig 5 (page 1).

Fig 6 (page 1).

Fig 7 (page 1).

Fig 8 (page 1).

Limitations

• The drone-to-satellite data is synthetically rendered, which might differ from real-world drone imagery and may limit generalization.
• Precise details on training hyperparameters, hardware, and runtime performance are relegated to supplementary material, limiting immediate reproducibility insights.
• Evaluation focuses on specific urban building instances; applicability to non-building objects or rural environments remains untested.
• The model's inference is still computationally dominated by the heavy 3D backbone, potentially limiting real-time deployment scenarios.
• Zero-shot generalization is assessed on a limited manually annotated subset; wider deployment scenarios could reveal more challenges.
• The contrastive loss aligns features using satellite anchors but does not consider adversarial robustness or occlusion handling explicitly.

Open questions / follow-ons

• How well does the method generalize to other object categories beyond buildings, or non-urban environments?
• Can the framework be extended to handle temporal data or video sequences for dynamic geo-localization?
• Could incorporating real-world drone imagery (vs synthetic) alter performance or training requirements?
• What are the trade-offs between model complexity, inference time, and geo-localization accuracy in large-scale deployments?

Why it matters for bot defense

This work advances the state of cross-view object geo-localization by introducing a comprehensive dataset and a unified model architecture that jointly predicts multi-modal outputs with strong 3D geometric awareness. For bot-defense and CAPTCHA practitioners, these insights demonstrate the power of geometry-aware foundations and prompt-based inputs to robustly locate objects under extreme viewpoint changes without reliance on simple 2D appearance cues. This has implications for designing security and verification systems that require spatial reasoning or cross-view consistency checks, especially when bridging between egocentric cameras (e.g., phones, drones) and satellite imagery.

Moreover, the use of large, diverse datasets with detailed geometric metadata highlights the importance of grounded spatial context in model training to improve generalization, a principle valuable for defenses relying on multi-view verification or anti-fraud mechanisms involving geospatial evidence. However, deployment considerations related to computational resources and synthetic training data should be carefully considered in real-world CAPTCHAs or bot-detection pipelines that might leverage cross-view localization.

Cite

@article{arxiv2606_30576,
  title={ Beyond 2D Matching: A Unified Single-Stage Framework for Geometry-Aware Cross-View Object Geo-Localization },
  author={ Liyao Wang and Ruipu Wu and Haojun Xu and Lei Shi and Linjiang Huang and Si Liu },
  journal={arXiv preprint arXiv:2606.30576},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.30576}
}

Read the full paper

• arXiv:2606.30576 (abstract)
• arXiv:2606.30576 (PDF)