EAGLE-360: Embodied Active Global-to-Local Exploration in 360$^\circ$

Source: arXiv:2607.02479 · Published 2026-07-02 · By Jingtao Xu, Zizhuo Lin, Jianwen Sun, Yi Yang, Yawei Luo

TL;DR

EAGLE-360 addresses the fundamental challenge of active visual search in 360-degree panoramic environments, where conventional Multimodal Large Language Models (MLLMs) underperform due to inherent panoramic image distortions and lack of topological continuity modeling. Previous methods relied heavily on fragmented, local views initialized from constrained perspectives, resulting in inefficient exploration and poor recovery when the target is initially outside the view. EAGLE-360 introduces a novel embodied active global-to-local exploration framework that leverages holistic panoramic priors to establish a coarse global context and then perform iterative, tool-augmented visual refinements by dynamically adjusting azimuth, elevation, and field of view (FoV) parameters. Architecturally, it enhances positional encoding with RoPE Rolling to model the panorama’s continuous cylindrical topology and overcome boundary discontinuities.

Key findings

• EAGLE-360 achieves 64.44% accuracy on the large-scale 14,000+ 4K panorama EAGLE-360 dataset, representing nearly an 8-fold accuracy improvement over the strongest baseline Qwen3-VL-8B-Instruct model at 8.33%.
• The model attains a 94.72% success rate under the criterion of great circle distance (GCD) less than 50 degrees, validating effective global macroscopic spatial perception.
• Removing the dynamic FoV adjustment module reduces accuracy drastically from 64.44% to 39.44%, demonstrating the necessity of the global-to-local progressive zooming paradigm.
• Excluding RoPE Rolling positional encoding degrades accuracy from 64.44% to 46.01%, showing the critical role of cylindrical wrap-around topology modeling in addressing panoramic boundary seam issues.
• Group Relative Policy Optimization (GRPO) significantly improves accuracy from 46.94% (supervised fine-tuning alone) to 64.44%, highlighting the value of reinforcement optimization for multi-turn, tool-augmented exploration.
• EAGLE-360 achieves strong zero-shot generalization on the out-of-distribution H*Bench benchmark with an overall score of 56.1, surpassing proprietary models like Gemini-2.5-Pro and previous SOTA HVS-3B.
• Multi-turn Chain-of-Thought reasoning delays premature answer submission until deeper spatial reasoning unfolds, improving localization precision progressively over early steps (e.g., over 6 degree GCD improvement by step 2).
• The spatial accuracy on traditionally challenging back, left, and right panorama sectors improves to 60.81%, 55.56%, and 71.11%, respectively, confirming effective boundary truncation error mitigation.

Threat model

The adversary scenario is implicit: the agent operates autonomously in a 360-degree panoramic environment with no prior knowledge of target location, must robustly recover from misleading or negative initial views, and cannot externally manipulate the panorama. The adversary has limited capabilities, focusing on intrinsic perceptual and reasoning challenges rather than malicious attacks. The system assumes access to complete, unaltered panoramic input and a correct natural language query describing the target.

Methodology — deep read

EAGLE-360 formulates active panoramic object search as a Partially Observable Markov Decision Process (POMDP). At each episode, the agent receives a full equirectangular panorama image and a natural language query, then aims to predict a spherical bounding field of view (BFoV) parameterized by azimuth, elevation, and horizontal/vertical FoV angles. Success is measured by a great circle distance criterion, ensuring spatial rigor beyond 2D bounding box formulations. Architecturally, the model integrates a modality-aligned RoPE Rolling positional encoding to resolve discontinuities at the panorama’s left-right boundaries by assigning head-specific horizontal rotational offsets in the transformer attention mechanism. This preserves cylindrical wrap-around topology and allows the model to attend cyclically across panorama seams.

The exploration proceeds in two stages: initially, a global perception step assesses the entire panorama to produce a coarse estimate of the target’s angular location. Then, a tool-augmented iterative exploration uses a differentiable Equirectangular-to-Perspective projection operator invoked by predicted azimuth, elevation, and FoV parameters to generate distortion-free local views. The model applies Chain-of-Thought multi-turn spatial reasoning, progressively narrowing the FoV to zoom in and refine localization.

Training involves a two-phase progressive pipeline: first, Supervised Fine-Tuning (SFT) on 20,000 multi-turn expert trajectories enables learning foundational sequence formatting, tool-calling syntax, and basic spatial priors using next-token prediction. The vision encoder backbone is frozen, and LoRA adaptation is applied to the large language model. Second, Group Relative Policy Optimization (GRPO), a reinforcement learning method with a group-level advantage normalization, actively optimizes policy for multi-turn decision making, balancing precision, concise text generation, tool efficiency, and turn economy. The reward function components include a discrete BFoV accuracy signal, great circle distance penalties, tool-use bonuses, format correctness scores, and length penalties to ensure stable convergence.

Evaluation on the newly proposed EAGLE-360 dataset (over 14,000 panoramas, 70,000+ multi-turn reasoning dialogues) benchmarks accuracy, great circle error, failure rate, and turn count against several open-source and proprietary baselines. The experimental setup includes random initial viewpoint evaluation of competing models to fairly test true panorama-wide search capability without prior location biases. Zero-shot generalization is measured on H*Bench, an out-of-distribution embodied object and path search suite.

Ablation studies systematically remove RoPE Rolling, dynamic FoV adjustment, and GRPO to isolate their contributions. The analysis of multi-turn reasoning tracks iterative spatial refinement and submission timing shifts due to RL alignment. The entire approach is reproducible via the GitHub repository and detailed dataset/code appendices.

Overall, the methodology coherently integrates geometric panorama-aware modeling, embodied active exploration with dynamic tools, and multi-turn reinforcement learning to overcome critical limitations of prior panoramic search methods heavily dependent on fragmented local views.

Technical innovations

• Integration of RoPE Rolling positional encoding that assigns attention head-specific horizontal rotational offsets to respect cylindrical panoramic topology and resolve left-right seam discontinuities in equirectangular projections.
• Unified global-to-local active exploration paradigm that first derives a holistic panoramic spatial prior before iteratively invoking a perspective projection tool to zoom and refine localization.
• Chain-of-Thought multi-turn reasoning scaffold enabling complex spatial decision-making and error recovery within ultra-long multimodal dialogue contexts.
• Group Relative Policy Optimization (GRPO), a novel reinforcement learning approach using group-wise advantage normalization applied to multi-turn embodied navigation, stabilizing policy learning without a separate value network.

Datasets

• EAGLE-360 — 14,000+ 4K panoramas, 70,000+ multi-turn search dialogues — assembled from Kuula, 2D-3D-Semantics, Matterport3D sources with synthetic augmentation

Baselines vs proposed

• Qwen3-VL-8B-Instruct: Accuracy = 8.33% vs EAGLE-360: 64.44%
• GPT-4o (OpenAI): Accuracy = 7.50% vs EAGLE-360: 64.44%
• Gemini-2.5-Pro: Accuracy = 20.28% vs EAGLE-360: 64.44%
• HVS-3B (prior panoptic model): Accuracy = 11.11% vs EAGLE-360: 64.44%
• EAGLE-360 w/o FoV adjustment: Accuracy = 39.44% vs full EAGLE-360: 64.44%
• EAGLE-360 w/o RoPE Rolling: Accuracy = 46.01% vs full EAGLE-360: 64.44%
• EAGLE-360 w/o GRPO (SFT only): Accuracy = 46.94% vs full EAGLE-360: 64.44%
• Zero-shot on H*Bench: HOS score: HVS-3B=47.3 vs EAGLE-360=74.8

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.02479.

Fig 1: Overview of Target Object Search in Panoramic Images. Left: Conventional MLLMs struggle with

Fig 2: The overall pipeline of the proposed EAGLE-360 framework. Our method adopts an embodied active

Fig 3 (page 4).

Fig 4 (page 4).

Fig 5 (page 4).

Fig 6 (page 4).

Fig 7 (page 4).

Fig 8 (page 4).

Limitations

• Inference latency and memory overhead are significant due to multi-turn reasoning and dense high-resolution perspective crops accumulating over ultra-long dialog contexts, limiting real-time deployment.
• Localization accuracy in extreme polar regions (zenith and nadir) slightly lags behind equatorial directions due to severe non-linear pixel distortion in equirectangular projection that affects initial global perception.
• Current method is demonstrated only on static panoramic images; extension to dynamic video or 4D spatio-temporal settings remains future work.
• Policy optimization rewards balancing discrete and continuous localization metrics are challenging; hybrid reward schemes introduce gradient conflicts hampering confident final answer submission.
• While dataset covers diverse indoor and in-the-wild panoramas, full generalization to highly cluttered or novel outdoor environments requires further evaluation.

Open questions / follow-ons

• How can the approach be extended from static panoramic images to dynamic video streams for real-time embodied navigation?
• What methods can further mitigate polar distortion, such as distortion-aware patching or spherical neural representations, to improve zenith/nadir localization?
• Can the multi-turn Chain-of-Thought reasoning and tool invocation be optimized for lower latency and memory footprint to enable on-device robotic applications?
• How do privacy-preserving decentralized training methods (e.g., federated learning) integrate with embodied visual search for sensitive indoor panorama data?

Why it matters for bot defense

For bot-defense and CAPTCHA practitioners, EAGLE-360 demonstrates that incorporating geometry-aware positional encodings (RoPE Rolling) and global-to-local active exploration paradigms significantly improves robust spatial reasoning in 360-degree environments compared to fragmented local views. Techniques like multi-turn Chain-of-Thought reasoning and tool-augmented iterative projections enable precise target localization even under challenging panoramic distortions and discontinuities. These insights highlight the importance of embedding global spatial priors and avoiding naive patch-based attention when designing CAPTCHA or bot-detection systems that leverage panoramic or omnidirectional inputs. Furthermore, the advancement in reinforcement learning alignment (GRPO) to encourage deliberate exploratory behavior may inspire adaptive challenge generation that dynamically refines difficulty. However, the computational complexity and inference latency inherent in such multi-turn embodied agents suggest practical deployment remains challenging, requiring careful optimization. Additionally, limitations in polar region handling underscore known edge cases where panoramic input handling must be hardest. Overall, bot-defense systems aiming to utilize multimodal panoramic inputs can take inspiration from EAGLE-360’s architectural robustness and active exploration strategies to improve localization accuracy and resilience against adversarial search attempts.

Cite

@article{arxiv2607_02479,
  title={ EAGLE-360: Embodied Active Global-to-Local Exploration in 360$^\circ$ },
  author={ Jingtao Xu and Zizhuo Lin and Jianwen Sun and Yi Yang and Yawei Luo },
  journal={arXiv preprint arXiv:2607.02479},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.02479}
}

Read the full paper

• arXiv:2607.02479 (abstract)
• arXiv:2607.02479 (PDF)