AeroMap3D: Anchoring Monocular UAV 6-DoF Localization to Visual-Geometric-Semantic Map Priors

Source: arXiv:2607.14009 · Published 2026-07-15 · By Zhiyun Deng, Luis Sentis

TL;DR

This paper addresses the problem of monocular 6-DoF UAV localization in GNSS-denied environments by anchoring onboard UAV imagery to publicly available map priors: satellite images, bare-earth digital elevation models (DEMs), and OpenStreetMap (OSM) semantic annotations. The main challenges tackled are the large cross-view discrepancy in scale and yaw between UAV images and satellite tiles, and the structural inconsistency of bare-earth DEMs with elevated urban structures. The authors propose AeroMap3D, which introduces a lightweight scale-yaw adapter neural module that normalizes this dominant geometric misalignment prior to dense feature matching with a pretrained generic matcher (RoMav2). Semantically unreliable correspondences on building footprints from OSM are filtered out before DEM-based 3D lifting and robust PnP pose estimation, reducing biases from unmodeled building heights. These map-based pose measurements are fused with noisy relative-motion odometry in a delayed-state EKF for continuous trajectory estimation. Using the UAV-Terra3D benchmark (55km flights over 8 urban sites), AeroMap3D achieves a 95.7% single-frame localization success and a mean 3D positional error of 5.88m over continuous trajectories, outperforming prior retrieval and Visual Odometry baselines without site-specific retraining or metric map construction.

Key findings

• RoMav2 matcher baseline geometric-verification success on UAV-to-map registration is 62.4%, raised to 99.2% with the proposed scale-yaw adapter (a 36.8-point absolute improvement).
• Single-frame dense DEM-lifted RANSAC–PnP pose estimation success is 88.24% without semantic filtering, improved to 95.69% success and mean 3D error reduced from 16.17m to 14.11m by OSM-based semantic masking of building footprints.
• Continuous trajectory localization achieves 100% success within 50m and mean 3D error of 5.88m compared to 17.17m mean error for a GeoVINS-style retrieval baseline and over 24m for visual odometry or odometry-only propagation.
• Latency overhead of the scale–yaw adapter is about 35ms on edge devices; combined with Tiny-RoMa matcher provides 92.5% registration success at 130ms per frame on Jetson Orin Nano.
• Trajectory error remains below 6m relative pose error at 2 km segment length, showing AeroMap3D bounds long-term drift better than ORB-VO or ORB-SLAM2.
• Adapter is trained fully on synthetic UAV-like renderings from satellite imagery with no real UAV data or manual labels, yet generalizes well to geographically distinct urban flights in Austin.
• Semantic filtering corrects a systematic failure mode of bare-earth DEM lifting where image matches on buildings cause consistent height bias that RANSAC alone cannot reject.

Threat model

The adversary is the GNSS-denied environment itself, disallowing access to global satellite navigation signals that a UAV might use for localization. The UAV has no prior site-specific metric maps, no onboard magnetometer outputs (due to potential bias), and only monocular camera inputs plus publicly available geospatial layers (satellite imagery, DEM, OSM). The adversary cannot corrupt these public map priors but may induce challenging cross-view appearance variations and structural mismatches through viewpoint, altitude, and urban complexity.

Methodology — deep read

The paper's core methodology unfolds in several steps. First, the threat model implicitly assumes an adversarial environment where GNSS is unavailable and the UAV must localize using only monocular imagery and publicly accessible geospatial maps (satellite imagery, bare-earth DEM, and OSM data). The system assumes a coarse initial (x,y) position but no prior learned model for the target site, and addresses cross-domain and cross-view variation challenges in matching.

Data for method development includes the UAV-Terra3D benchmark with 55 km of continuous UAV video trajectories over 8 sites in Austin, TX, equipped with calibrated camera intrinsics and pose ground truth limited to translation. Training of the scale-yaw adapter uses entirely synthetic UAV-like images generated from satellite imagery available in UAV-VisLoc (Chinese dataset), by applying known geometric scale and yaw transforms to satellite tiles. This yields scale and yaw supervision without real UAV image labels or site-specific fine-tuning.

The architecture has three key components. The Scale-Yaw Adapter is a Siamese CNN using a shared MobileNetV3-Small backbone encoding the UAV image and a north-up satellite tile, followed by a global average pooling and fused descriptor vector, regressed by an MLP to output a scale ratio and yaw offset. This normalizes the UAV and map viewpairs prior to passing them to a frozen pretrained dense image matcher (RoMav2), which extracts dense 2D–2D correspondences. These 2D correspondences are filtered through OSM-derived building footprint masks to remove matches on elevated urban structures where the bare-earth DEM elevation model is invalid. The remaining matches are lifted to 3D terrain points by mapping their 2D locations to local East-North-Up coordinates and retrieving terrain elevation from the DEM.

Pose estimation uses RANSAC–PnP on these filtered 2D–3D correspondences to solve for the UAV camera pose at each frame. Single-frame visual pose updates are fused asynchronously in a delayed-state Extended Kalman Filter (EKF) together with a stochastic relative-motion prior derived from noisy GNSS-based odometry simulation or monocular visual odometry. EKF incorporates Mahalanobis gating to discard inconsistent pose updates and continuously re-propagates the state forward for smooth trajectory estimation.

Evaluation compares multiple baselines across four questions: (Q1) baseline registration success improves by the adapter despite cross-domain training; (Q2) image matcher success improves substantially with the adapter; (Q3) semantic filtering improves single-frame pose success and mean error; (Q4) trajectory fusion with map anchoring bounds long-term drift and errors remarkably better than VO, OD, SLAM, or image retrieval alone.

The study reports success rates and 3D positional errors on the UAV-Terra3D dataset with no training or tuning on these target data, demonstrating zero-shot cross-site generalization. The dense matcher and RANSAC–PnP are standard modules; the novelty lies in the adapter, semantic filtering, and integration into a scalable monocular localization framework. The authors release UAV-Terra3D as a public benchmark with aligned RGB images, reference poses, DEM and OSM semantic layers to foster further research.

One concrete example: given a monocular UAV image at unknown scale and yaw offset relative to a satellite tile, the adapter regresses these nuisance parameters and transforms the tile accordingly. The frozen RoMav2 matcher extracts dense correspondences. Matches over building footprints are rejected, producing terrain-consistent 2D–3D matches. RANSAC–PnP estimates the pose, which is fused in EKF with a noisy odometry prior to provide a drift-bounded global pose estimate along the flight trajectory.

Technical innovations

• A lightweight scale–yaw adapter normalizes geometric scale and yaw misalignment between UAV images and satellite map tiles prior to feature matching, trained solely on synthetic data without real UAV imagery or tuning.
• Semantic conditioning via OpenStreetMap building footprint masking of UAV–map correspondences before DEM-based 3D lifting mitigates systematic height bias in bare-earth elevation models affecting pose estimation.
• Integration of delayed visual pose measurements from DEM-lifted correspondence PnP into a delayed-state EKF fusing noisy relative-motion priors enables continuous and drift-bounded 6-DoF UAV localization in GNSS-denied urban flight.
• Use of publicly available, large-scale geo-referenced priors (satellite imagery, bare-earth DEM, OSM) for metric 6-DoF UAV localization without requiring textured 3D maps, DSMs, or site-specific offline pose databases.

Datasets

• UAV-Terra3D — 55 km continuous UAV video trajectories over 22.4 km2 — public benchmark released by authors with aligned satellite imagery, DEM, and OSM data
• UAV-VisLoc — dataset used for synthetic scale–yaw adapter training — satellite imagery from Chinese urban scenes, no UAV imagery used for training
• NAIP imagery — high-resolution aerial imagery used as part of UAV-Terra3D map priors
• USGS 3DEP DEM — bare-earth elevation model used for UAV-Terra3D map priors
• OpenStreetMap (OSM) data — semantic building footprint masks used in UAV-Terra3D

Baselines vs proposed

• RoMav2 matcher baseline geometric-verification success = 62.4% vs AeroMap3D with adapter = 99.2%
• Tiny-RoMa matcher baseline = 42.1% success vs 92.5% with adapter
• Single-frame localization success: dense DEM-PnP (no semantic filter) = 88.24% vs OSM-filtered AeroMap3D = 95.69%
• Single-frame mean 3D error: dense DEM-PnP = 16.17m vs AeroMap3D = 14.11m
• Multi-frame continuous localization mean 3D error: GeoVINS-style retrieval = 17.17m vs AeroMap3D = 5.88m
• Multi-frame trajectory success rate: GeoVINS retrieval = 100% vs AeroMap3D = 100% within 50 m threshold
• Reference odometry propagation mean error = 24.2m vs AeroMap3D = 5.88m

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.14009.

Fig 1: Overview of AeroMap3D. Compared with prior 6-

Fig 2: Synthetic training data for the scale–yaw adapter.

Fig 3: AeroMap3D localization pipeline. Starting from a coarse map initialization, obtained by retrieval in prior work or by an

Fig 4: UAV-Terra3D map priors. Eight Austin sites combine WGS84-aligned NAIP imagery [11] for registration (top;

Fig 5: Cross-domain UAV-to-map registration and run-

Fig 6: Long-horizon relative drift. RPE versus segment

Fig 7: Representative 2-km UAV-Terra3D trajectory. Top:

Limitations

• Evaluation limited to eight urban sites in Austin; no multi-region or globally distributed testing reported.
• Ground truth only includes 3-DoF translational position; rotational (roll, pitch, yaw) and gimbal extrinsic accuracy not evaluated.
• Coarse initial (x,y) position needed for localization initialization; the system does not address global pose initialization.
• Use of bare-earth DEM and reliance on OSM building masking means unmapped or newly constructed structures cause pose errors or failures.
• Adapter normalization does not simulate seasonal appearance changes or full projective distortion from large UAV roll or pitch angles, constraining robustness.
• No direct evaluation with onboard IMU or raw odometry; motion prior is derived from noisy GNSS trajectories, which may differ from real odometry noise profiles.

Open questions / follow-ons

• How would AeroMap3D perform under strong seasonal, temporal, or weather-driven appearance changes in satellite and UAV images, given no domain adaptation?
• Can the approach be extended to fully automatic global initialization without coarse initial position input, potentially by integrating retrieval at scale?
• What is the impact of onboard vehicle pose noise and gimbal calibration errors on the final localization accuracy, especially for roll and pitch?
• How can OSM and DEM map validity be modeled or learned online to detect and handle missing or outdated structures that induce large localization errors?

Why it matters for bot defense

This research offers a compelling approach for robust UAV localization using only monocular imagery and publicly accessible, broadly available map data without dependence on high-fidelity textured 3D reconstructions or region-specific training. For bot-defense or CAPTCHA engineers working on UAV or aerial robot identification, navigation, or anti-spoofing, AeroMap3D introduces methods to exploit semantically filtered geographic priors and synthetic nuisance adaptation for cross-view registration, potentially informing defenses that rely on geospatial consistency checks in monocular visual streams. The scale-yaw adapter’s synthetic training approach may inspire domain adaptation techniques mitigating appearance mismatches between real and reference maps. Additionally, the semantic filtering approach emphasizes the importance of contextual map priors to handle structural inconsistencies, a principle useful for designing more reliable visual-metric anchors for UAV verification under adversarial or spoofing conditions. However, limitations in initialization requirements and map validity assumptions caution practitioners to consider robustness to unmapped dynamic scenes and to explore global pose recovery.

Cite

@article{arxiv2607_14009,
  title={ AeroMap3D: Anchoring Monocular UAV 6-DoF Localization to Visual-Geometric-Semantic Map Priors },
  author={ Zhiyun Deng and Luis Sentis },
  journal={arXiv preprint arXiv:2607.14009},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.14009}
}

Read the full paper

• arXiv:2607.14009 (abstract)
• arXiv:2607.14009 (PDF)