MVP-Nav: Multi-layer Value Map Planner Navigator

Source: arXiv:2606.31919 · Published 2026-06-30 · By Wenyuan Xie, Shaokai Wu, Yijin Zhou, Yanbiao Ji, Guodong Zhang, Bayram Bayramli et al.

TL;DR

This paper addresses the core challenge of zero-shot object goal navigation (ZSON) under RGB-only perception constraints, where no explicit depth information is available. The absence of depth leads to physical uncertainty and semantic-physical misalignment, causing agents to generate semantically plausible but physically unsafe navigation paths. Existing methods either rely on high-level semantic reasoning without geometric grounding or end-to-end learning without explicit physical constraints, both of which fall short in safely guiding agents in unseen environments. MVP-Nav introduces a novel physical-aware navigation framework that reconstructs explicit 3D physical occupancy from monocular RGB images by leveraging 3D foundation models to lift 2D semantic instances into 3D oriented bounding boxes. It then integrates these geometric reconstructions with high-level semantic reasoning from vision-language models into a unified Multi-layer Value Map (MVM). This shared cost space enables physically grounded geometric planning that respects both semantic goals and obstacle constraints simultaneously.

Key findings

• MVP-Nav improves Success Rate (SR) on HM3D from 43.5% (PanoNav) to 65.4% and SPL from 23.7% to 27.9%, a 21.9% absolute SR gain over prior best RGB-only methods.
• MVP-Nav achieves performance competitive with, and sometimes exceeding, RGB-D methods; for example, outperforming SG-Nav (54.0% SR) and UniGoal (54.5% SR) on HM3D.
• Ablation removing semantic re-projection increases SPL to 30.5% but reduces SR from 65.4% to 53.2%, showing semantic re-projection improves physical safety.
• Removing exploration memory drops SR drastically to 38.2%, indicating the critical role of exploration history for zero-shot navigation.
• Without LLM-based scoring for semantic prioritization, SR falls to 45.7%, confirming the added value of high-level reasoning.
• Full-sequence global 3D reconstruction caused out-of-memory errors, motivating MVP-Nav’s recursive local reconstruction approach to maintain fidelity and efficiency.
• The multiplicative fusion in MVM ensures midterm goals simultaneously satisfy semantic attraction and physical traversability, preventing semantically plausible but unreachable targets.
• Safety verification using semantic floor masks in the low-level controller enforces physically safe navigation trajectories.

Threat model

The adversary is the unknown and unobserved real-world environment presenting previously unseen indoor layouts and object distributions. The agent only receives monocular RGB observations and onboard pose estimates. The adversary cannot manipulate the sensor inputs but poses a challenge via environmental complexity and partial observability. No active depth or direct metric maps are available, and the agent must infer 3D occupancy and semantics passively to avoid physical collisions and reach targets.

Methodology — deep read

The threat model assumes an embodied agent navigating large-scale, previously unseen indoor environments with only monocular RGB input and onboard pose localization. The adversary is implicit — the environment’s layout and exact target position are unknown; no direct depth or metric map is available.

Data provenance includes three standard benchmarks for object-goal navigation: HM3D (~2k validation episodes, 6 object categories), MP3D (~1.8k episodes, 20 categories), and RoboTHOR (~1.8k episodes, 12 goal categories). The task involves zero-shot navigation to target objects specified by an image or language instruction.

The MVP-Nav system pipeline starts with a 360° scan to collect initial RGB observations. Physical Perception uses the VGGT 3D foundation model to regress pseudo-depth and back-project pixels into 3D point clouds forming local spatial maps. Grounded-SAM generates semantic masks for targets, objects, and floor. Semantic instances are separated and fitted with oriented bounding boxes (OBB) which are integrated into a Global Spatial Semantic List (GSSL). Scale ambiguity is resolved by aligning visual-inferred trajectories with metric localization via similarity transforms.

The Vision-Language Model (GPT-4o-mini) reasons over the GSSL entities and navigation goal to assign exploration value scores (0-1) to new entities and determine navigation modes (Explore, Find, Judge). The Multi-layer Value Map (MVM) constructs a unified 2D cost space by multiplicatively fusing three layers: semantic value (weighted spatial Gaussian contributions of entities), directional value (angular bias towards VLM-suggested direction), and traversability value (distance-based obstacle penalty). The optimal midterm goal is the global maximum in this unified map.

At the low-level execution, A* pathfinding determines discrete local waypoints, while Fast Marching Method (FMM) solves a continuous potential field for smooth and safe navigation. A safety verification step projects short-term goals onto semantic floor masks to ensure traversability and prevent collisions. The loop recursively updates the GSSL and navigation plan following goal completion or environment changes.

Training-free setup leverages pretrained 3D and vision-language foundation models with no fine-tuning. Evaluation uses Success Rate and Success-weighted Path Length on held-out challenge environments. Baselines include state-of-the-art RGB-only and RGB-D object navigation systems for comparative analysis.

One concrete episode starts with an initial panorama scan and GSSL construction, VLM reasoning to pick Explore mode and forward direction, MVM planning yields midterm goal near a cabinet, low-level loop safely navigates avoiding obstacles projecting onto traversable semantic floor, updating the GSSL with observations until final target found and confirmed by matching multi-view images via VLM judgment before termination.

Code release details were not specified, though evaluation uses standard open benchmarks enabling partial reproducibility.

Technical innovations

• Leveraging 3D foundation models to reconstruct explicit 3D oriented bounding boxes from monocular RGB images instead of relying on active depth sensing.
• Introducing the Multi-layer Value Map (MVM) that multiplicatively fuses semantic, directional, and traversability cost layers to unify high-level semantic guidance with low-level physical constraints.
• Recursive local 3D reconstruction with scale alignment to avoid global map drift and out-of-memory issues in long horizon navigation.
• Semantic re-projection based safety verification during low-level execution that projects short-term goals onto semantic floor masks to ensure physical traversability under RGB-only input.

Datasets

• HM3D — ~2000 validation episodes, 20 indoor scenes, 6 object categories — public benchmark
• MP3D — ~1800 validation episodes, 11 real-world indoor scenes, 20 categories — public benchmark
• RoboTHOR — ~1800 validation episodes, 15 scenes, 12 object categories — public benchmark

Baselines vs proposed

• PanoNav (RGB-only): SR = 43.5%, SPL = 23.7% (HM3D) vs MVP-Nav: SR = 65.4%, SPL = 27.9%
• SG-Nav (RGB-D): SR = 54.0%, SPL = 24.9% (HM3D) vs MVP-Nav (RGB-only): SR = 65.4%, SPL = 27.9%
• UniGoal (RGB-D): SR = 54.5%, SPL = 25.1% (HM3D) vs MVP-Nav: SR = 65.4%, SPL = 27.9%
• Removing semantic re-projection: SR drops to 53.2% from 65.4%, SPL rises from 27.9% to 30.5%
• Disabling exploration memory: SR drops to 38.2%, SPL to 13.4%
• Disabling LLM-based rating: SR drops to 45.7%, SPL to 20.1%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2606.31919.

Fig 1: The overview of our navigation system.

Fig 2: Overview of the MVP-Nav framework. Our system employs a recursive architecture that first transforms monocular RGB sequences

Fig 3 (page 1).

Fig 4 (page 1).

Fig 5 (page 1).

Fig 6 (page 1).

Fig 3: An example of a part of GSSL.

Fig 8 (page 3).

Limitations

• Dependence on pretrained foundation models makes the approach vulnerable to their semantic or geometric inaccuracies.
• While promising for zero-shot navigation, the method has not been evaluated under severe distribution shifts or highly dynamic environments.
• The recursive local reconstruction strategy trades off global consistency; long-term mapping performance beyond spatial scales covered is unclear.
• Safety verification relies on semantic floor masks, which could fail in visually ambiguous or cluttered scenes.
• Reported results lack ablation on adversarial scenarios or intentional physical/environmental perturbations.
• Computational overhead of 3D reconstruction and frequent VLM reasoning may limit real-time deployment on low-power robots.

Open questions / follow-ons

• How would MVP-Nav perform in dynamic or highly cluttered environments with moving obstacles or humans?
• Can the recursive local 3D reconstruction be extended to learn or refine metric scale and semantic labels over long-term operation?
• What is the impact of using alternative foundation models with differing geometry or semantic capabilities on navigation robustness?
• Could the Multi-layer Value Map formulation be generalized or learned end-to-end to adaptively balance semantic and physical constraints per environment?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners focused on embodied AI navigation, MVP-Nav's approach illustrates how monocular RGB-only agents can leverage structured physical priors and vision-language reasoning to overcome depth sensor absence. The unified value map integrating semantic priorities with geometric constraints offers a robust planner for zero-shot navigation that could inspire challenge designs centered on semantic-physical alignment. Moreover, its safety verification via semantic floor masks underscores the importance of fusing learned semantic perception with strict physical constraints to prevent plausible yet infeasible paths—an insight valuable for designing robust interactive challenges in embodied systems. The method's reliance on 3D foundation models also exemplifies the rising role of large pretrained models for grounding perception and planning in realistic 3D structure, a trend likely to affect future bot and agent assessment frameworks.

Cite

@article{arxiv2606_31919,
  title={ MVP-Nav: Multi-layer Value Map Planner Navigator },
  author={ Wenyuan Xie and Shaokai Wu and Yijin Zhou and Yanbiao Ji and Guodong Zhang and Bayram Bayramli and Qiuchang Li and Xunchu Zhou and Yue Ding and Hongtao Lu },
  journal={arXiv preprint arXiv:2606.31919},
  year={ 2026 },
  url={https://arxiv.org/abs/2606.31919}
}

Read the full paper

• arXiv:2606.31919 (abstract)
• arXiv:2606.31919 (PDF)