Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning

Source: arXiv:2607.01191 · Published 2026-07-01 · By Hongxing Li, Xiufeng Huang, Dingming Li, Wenjing Jiang, Zixuan Wang, Haolei Xu et al.

TL;DR

This paper addresses the challenge of fine-grained visual reasoning in vision-language models (VLMs), particularly when critical visual details are small and embedded in high-resolution images. Prior approaches typically inject local visual evidence through repeated cropping or test-time search but do not explicitly separate perception (localizing relevant visual cues) from reasoning (answering based on these cues). The authors propose Perceive-to-Reason (P2R), a unified two-stage framework that explicitly decouples perception and reasoning: a Perceiver module first localizes question-relevant image regions, which are then fed, along with the annotated image, to a Reasoner module to produce the final answer. To effectively train this pipeline with only final-answer supervision (no bounding box annotations), they introduce Perception-Reasoning Alternating GRPO (PRA-GRPO), a role-aware reinforcement learning strategy that alternates between optimizing perception and reasoning stages, converting answer correctness into more attributable learning signals for each stage. Built on top of the Qwen3-VL-Instruct models, P2R consistently boosts performance across all model sizes on high-resolution fine-grained visual reasoning benchmarks such as V-Star and HR-Bench, with P2R-4B achieving over 11% absolute improvement compared to its backbone. Additional experiments show benefits on broader multimodal reasoning tasks, indicating the generality of the decoupled training framework.

Key findings

• P2R-4B achieves 93.2% accuracy on V-Star, 81.9% on HR-Bench-4K, and 80.5% on HR-Bench-8K, improving 11.5%, 8.1%, and 13.5% respectively over the Qwen3-VL-Instruct-4B baseline.
• Across three benchmarks, P2R-2B, -4B, and -8B improve average accuracy by 8.1%, 11.0%, and 9.7% over their respective backbones.
• In ablations, alternating PRA-GRPO training (perception followed by reasoning) outperforms training perception or reasoning alone by up to 3 percentage points on V-Star (93.2% vs 90.6%).
• P2R improves general multimodal perception and reasoning on MME-RealWorld-Lite by 4.0% to 7.1% over Qwen3-VL baselines, with gains across diverse sub-tasks.
• Role-aware alternating optimization converts final-answer correctness into role-specific rewards, allowing perception learning purely from answer supervision without ground-truth bounding boxes.
• Shared parameters for Perceiver and Reasoner allow positive transfer between roles, outperforming assembling separately trained modules.
• PRA-GRPO training scales more quickly than text-only GRPO, improving MME-RealWorld-Lite accuracy from 54.8% to 57.1% in 3 training iterations.
• On the ReasonSeg grounding task, P2R-4B outperforms Qwen3-VL-Instruct-4B by 1.1% average [email protected] without any grounding-specific training data or annotations.

Threat model

The adversary corresponds to model failures in perceiving and reasoning about fine-grained visual evidence—i.e., incorrect localization of subtle visual cues and reasoning errors due to noisy or missing perceptual input. The model operates under the assumption that only final-answer supervision is available; it cannot observe ground-truth bounding boxes or intermediate perception labels. The system is not explicitly designed to withstand adversarial attacks or manipulations, and adversarial threat considerations are outside the scope of this work.

Methodology — deep read

The core threat model is not adversarial but focuses on overcoming model failures in fine-grained visual reasoning, where subtle cues must be reliably perceived and reasoned over. The adversary, in this context, can be considered as the model's inability to correctly localize and interpret small but critical evidence in a high-resolution image. The model has access to only final-answer supervision, with no intermediate bounding box labels or explicit region information during training.

Training data is a 10K-example multimodal dataset constructed by sampling 3K examples each from DeepEyes, VisualProbe, and ZwZ benchmarks. These datasets are labeled with image-question-answer triplets but lack bounding box annotations. No bounding box labels are used for training perception.

P2R uses a single vision-language model (Qwen3-VL-Instruct at 2B, 4B, or 8B parameter scales) that assumes two roles: Perceiver and Reasoner, with shared parameters θ. Perceiver takes an image and a perception-oriented prompt Tp(Q) and outputs K bounding boxes B forming regions likely relevant to the question. These boxes produce two auxiliary inputs: an annotated image Ia (original image overlaid with boxes) and cropped evidence images Ic (image crops of localized boxes). The Reasoner then conditions on (Ia, Ic, Q) to generate the final answer sample Y.

Training uses Perception-Reasoning Alternating GRPO (PRA-GRPO), a novel role-aware reinforcement learning algorithm adapting GRPO. In each training phase, one role is actively optimized while the other's policy is frozen. For perception training, multiple rollouts sample different bounding box sets B; the fixed Reasoner produces candidate answers based on these boxes, and a binary reward r_i = I[Yi = Y] (correct answer indicator) guides updating the Perceiver's policy via policy gradient with group-relative advantage normalization. For reasoning training, bounding boxes are fixed; multiple answer rollouts Y_i are sampled and used to improve the Reasoner's policy with similar reward treatment.

This training strategy converts sparse final-answer supervision into more role-attributable learning signals, addressing credit assignment challenges without requiring explicit bounding boxes or dense intermediate supervision.

Inference first runs the Perceiver to localize bounding boxes, then builds Ia and Ic and runs the Reasoner to generate the final answer. Both stages share model parameters but operate differently conditioned on role prompts.

Evaluation includes fine-grained benchmarks V-Star, HR-Bench-4K/8K and a general multimodal benchmark MME-RealWorld-Lite. Models are compared against Qwen3-VL baselines, thinking-with-images and visual search methods. Metrics include answer accuracy and for perception stages Hit Rate (bounding box center inside GT box) and average IoU.

Ablations test the effect of PRA-GRPO role alternation, order of updates (Perceiver then Reasoner vs. the reverse), and inference strategies (P2R vs direct chain-of-thought prompting). They also analyze shared-parameter effectiveness by mixing checkpoints trained with only perception or reasoning roles.

Computational setup uses 4 Nvidia H100 GPUs, 1 epoch per role per training iteration, 8 rollouts per prompt, and KL regularization coefficient 0.01. Code and further implementation details are not explicitly stated but the model is built on publicly known Qwen3-VL-Instruct.

A concrete training iteration example: in the perception phase, given an image-question pair, the Perceiver samples several bounding box hypotheses. For each, the Reasoner (fixed) produces an answer, which is compared to ground truth yielding binary rewards. The policy gradient update improves bounding box selection focused on those leading to correct answers. Next iteration, the Reasoner is optimized similarly given fixed boxes, improving answer generation. Alternating these updates jointly improves the full system.

Technical innovations

• Explicit two-stage perceive-to-reason framework that decouples evidence localization (Perceiver) and answer generation (Reasoner) in fine-grained visual reasoning.
• Perception-Reasoning Alternating GRPO (PRA-GRPO), a role-aware reinforcement learning strategy that alternates optimization of perception and reasoning using only final-answer supervision.
• Role-conditioned behavior within a single shared vision-language model that can switch between perception and reasoning roles during inference and training.
• Use of final-answer correctness as an indirect, attributable training reward for both perception and reasoning roles, enabling end-to-end training without bounding box labels.

Datasets

• DeepEyes — 3000 examples — publicly referenced
• VisualProbe — 3000 examples — publicly referenced
• ZwZ — 4000 examples — publicly referenced
• V-Star — high-resolution visual reasoning benchmark
• HR-Bench-4K — high-resolution fine-grained visual reasoning benchmark
• HR-Bench-8K — high-resolution fine-grained visual reasoning benchmark
• MME-RealWorld-Lite — diverse multimodal reasoning benchmark

Baselines vs proposed

• Qwen3-VL-Instruct-4B: V-Star = 81.7% vs P2R-4B = 93.2%
• Qwen3-VL-Instruct-4B: HR-Bench-8K = 67.0% vs P2R-4B = 80.5%
• Qwen3-VL-Instruct-8B: HR-Bench-4K = 74.8% vs P2R-8B = 81.5%
• DeepEyes-7B: V-Star = 90.1% vs P2R-4B = 93.2%
• PixelReasoner-7B: HR-Bench-4K = 72.9% vs P2R-4B = 81.9%
• MME-RealWorld-Lite overall: Qwen3-VL-4B = 47.7% vs P2R-4B = 54.8%
• ReasonSeg grounding [email protected]: Qwen3-VL-Instruct-4B = 61.6% vs P2R-4B = 62.7%

Figures from the paper

Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.01191.

Fig 2: Overview of P2R. (a) Illustration of the proposed two-stage P2R inference pipeline. (b) Performance

Fig 7: Representative examples from the V-Star benchmark, comparing Qwen3-VL-4B and P2R-4B.

Fig 8: Performance over three iterations on MME-

Fig 6: Training dynamics of PRA-GRPO during the

Fig 5 (page 8).

Fig 6 (page 8).

Fig 7 (page 8).

Fig 8 (page 8).

Limitations

• P2R's two-stage inference pipeline increases computational cost compared to direct prompting methods.
• The PRA-GRPO training method uses only final-answer supervision, which is a sparse signal and may limit learning efficiency.
• Experiments are limited to models up to 8B parameters; the scaling behavior at larger model and dataset sizes remains unexplored.
• Evaluation focuses on fine-grained visual reasoning and some general multimodal tasks; generalization to interactive, long-horizon, or zero-shot scenarios is not demonstrated.
• No adversarial robustness evaluation or explicit assessment of error modes in perception vs reasoning components under out-of-distribution shifts was conducted.
• Intermediate outputs such as bounding boxes are not guaranteed to be fully faithful explanations of model decisions.

Open questions / follow-ons

• How does PRA-GRPO and the two-stage P2R framework perform at larger scale training and with larger vision-language models beyond 8B parameters?
• Can the decoupled perceive-to-reason approach and reinforcement learning training be extended or adapted for interactive or long-horizon multimodal reasoning tasks?
• What is the robustness of P2R to distribution shifts, adversarial perturbations, or more complex reasoning queries requiring multiple steps or external knowledge?
• Can intermediate perception outputs produced by the Perceiver be leveraged as interpretable explanations or for human-in-the-loop verification without degrading performance?

Why it matters for bot defense

Bot-defense engineers and CAPTCHA practitioners dealing with fine-grained image understanding can leverage insights from P2R to improve robustness against sophisticated bots that attempt to exploit subtle visual cues in high-resolution images for automated recognition. The explicit decoupling of perception and reasoning, along with role-aware reinforcement learning from only final-answer signals, can inspire new CAPTCHA design and evaluation strategies that require models or attackers to first correctly locate subtle evidence before reasoning over it — effectively raising the difficulty for pure holistic or brute-force visual recognition attempts. Moreover, by improving perception under weak supervision, P2R highlights the importance of intermediate localization in securing systems against attacks exploiting partial or low-level perceptions.

Cite

@article{arxiv2607_01191,
  title={ Perceive-to-Reason: Decoupling Perception and Reasoning for Fine-Grained Visual Reasoning },
  author={ Hongxing Li and Xiufeng Huang and Dingming Li and Wenjing Jiang and Zixuan Wang and Haolei Xu and Hanrong Zhang and Haiwen Hong and Longtao Huang and Hui Xue and Weiming Lu and Jun Xiao and Yueting Zhuang and Yongliang Shen },
  journal={arXiv preprint arXiv:2607.01191},
  year={ 2026 },
  url={https://arxiv.org/abs/2607.01191}
}

Read the full paper

• arXiv:2607.01191 (abstract)
• arXiv:2607.01191 (PDF)