MIRROR: Learning from the Other View for Multi-Modal Reasoning
Source: arXiv:2607.21552 · Published 2026-07-23 · By Wen Ye, Yuxiao Qu, Aviral Kumar, Xuezhe Ma
TL;DR
This paper addresses a fundamental challenge in multimodal reasoning with vision-language models (VLMs), focusing on geometry problems that can be equivalently presented in text-dominant, image-dominant, or combined image+text views. Unlike large language models, VLMs inconsistently solve the same problem across these views due to different failure modes and reasoning paths. The authors introduce ODA-Data, a carefully curated dataset of paired problems across these views, enabling systematic study of cross-modal reasoning asymmetries.
They propose MIRROR (Modality-Informed Reciprocal Reasoning Optimization), a novel reinforcement learning framework that exploits view-dependent performance asymmetries by selecting the strongest-performing view of each problem as a teacher to provide on-policy, self-supervised distillation signals to the other view(s). This approach uses an adaptive teacher selection mechanism and a reverse KL divergence loss on student-generated rollouts. Experiments demonstrate MIRROR improves accuracy and consistency across all views on ODA-Val and transfers well to external benchmarks like GeoInt and MathVerse, outperforming strong baselines with far less data. Critically, simply training jointly on all views (mixed-modality RL) underperforms single-view training, highlighting the importance of the directed knowledge transfer that MIRROR enables.
Key findings
- Text-dominant view achieves 29.58% pass@1, image-dominant 10.54%, combined image+text 26.84% on ODA-Train with base model Qwen3-VL-4B-Instruct.
- On ODA-Train, the text-dominant view is the best teacher for 51.14% of problems, image-dominant for 37.98%, combined for 10.88%, showing no single view is uniformly best.
- MIRROR improves image pass@16 on ODA-Val from 48.78% (best single-view GRPO) to 57.06%, a +14.49% absolute gain over the base model.
- MIRROR improves text pass@16 on ODA-Val from 83.16% to 86.10% (+5.88% absolute over base).
- MIRROR increases the fraction of problems solved under both image and text views on ODA-Train from 42.5% to 60.7%.
- On GeoInt benchmark, MIRROR improves pass@1 accuracy from 58.62% to 66.15%, pass@16 from 70.79% to 78.38%.
- MIRROR outperforms larger-data baseline models (Vision-R1-7B, PAPO-7B, Vero-8B) on multiple metrics despite using only 2K training samples versus 38K to 600K samples for baselines.
- Mixed-modality GRPO baseline trained on all views jointly underperforms single-view GRPO, confirming that naive multi-view training does not transfer reasoning across views.
Threat model
n/a - The paper addresses improving multimodal reasoning in VLMs rather than security adversaries or attack scenarios. The discussion centers on inherent model failure modes across input views and does not assume adversarial attacks.
Methodology — deep read
Threat model & assumptions: The focus is on vision-language models tasked with geometric reasoning problems presented in three possible views: a text-dominant view that explicitly states quantities and relations in language, an image-dominant view where essential geometric information is in the diagram and text only specifies the question, and a combined image+text view containing both. The adversary and security considerations are not the focus; rather the goal is to improve reasoning consistency across these input modalities by leveraging cross-view asymmetries.
Data: The authors construct ODA-Data by filtering and annotating a subset (~2,000 examples) of the ODA-Math-460k dataset. Each example has paired text-dominant, image-dominant, and combined views of the same underlying geometry problem. The dataset is split 85:15 into ODA-Train for post-training and ODA-Val for evaluation. The data preserves problem identity across views to measure cross-modal consistency.
Architecture / algorithm: Post-training is applied on top of the base multimodal language model Qwen3-VL-4B-Instruct. MIRROR leverages reinforcement learning with a policy gradient method GRPO. Critically, for each problem during training, MIRROR evaluates the current policy across all three views and selects the best-performing view as a teacher. The student policy samples rollouts only from its restricted view (text- or image-dominant). The teacher, parameterized as an exponential moving average (EMA) of policy weights, scores the student’s sampled token trajectories via a reverse-KL divergence loss (comparing token probabilities under teacher vs student views). This auxiliary loss guides the student towards teacher preferences without off-policy sampling.
Training regime: Training uses approximately 2,000 ODA-Train examples with maximum response length 16,384 tokens, sampling temperature 0.8, GRPO clipping ranges, KL coefficients, and entropy settings described in Section 6. The EMA parameter α=0.99 is updated after each policy step. The reverse KL coefficient λKL=0.01 balances outcome-reward RL and teacher-guided distillation losses.
Evaluation protocol: Evaluation uses pass@k metrics (pass@1 and pass@16, with k rollouts per question) under each view on held-out ODA-Val. External benchmarks GeoInt and MathVerse test transfer. Baselines include the base model, existing post-trained VLMs (Vision-R1-7B, PAPO-7B, Vero-8B), single-view GRPO, and mixed-modality GRPO trained on all views jointly. Ablations test fixed vs adaptive teachers, reverse-KL weight, and EMA vs current policies on teacher scoring. Cross-view consistency is measured by fraction of problems solved under both text and image views.
Reproducibility: Implementation is in verl. The ODA-Data dataset details are described, but it is unclear if data/code are publicly released. The core method is a policy gradient RL framework with additional on-policy distillation loss on token probabilities. Specific training hyperparameters and teacher-selection algorithms are well documented.
Example end-to-end: For a geometry problem, MIRROR first runs rollouts on all views, identifies which view yields highest success rate, sets that as the teacher. The student samples its own trajectories from a restricted single view (e.g. text-only) and receives reward signals from correctness plus reverse KL loss comparing its token-level distributions with those from the teacher (e.g. combined image+text view). The model updates parameters accordingly. Over training, this enables weaker views to learn reasoning trajectories closer to stronger views, improving both accuracy and cross-modal consistency.
Technical innovations
- Introduction of MIRROR, a reinforcement learning framework that adaptively selects the best-performing view per problem as a teacher to regularize training of student policies on other views.
- Use of an on-policy reverse-KL divergence loss that scores student-generated rollouts under a stronger teacher view to transfer token-level preferences without off-policy sampling or forcing trajectory imitation.
- Application of an exponential moving average (EMA) of policy parameters to stabilize the teacher distribution and prevent long-horizon training collapse.
- Construction of ODA-Data, a paired multimodal geometry dataset with aligned text-dominant, image-dominant, and combined views for controlled study of cross-modal reasoning asymmetry.
Datasets
- ODA-Data — ~2,000 examples — constructed from ODA-Math-460k filtered and paired with TikZ diagrams
- ODA-Math-460k — 460,000 problems — public source (OpenDataArena)
- GeoInt — size not stated — public visual mathematics benchmark
- MathVerse — size not stated — public multimodal math benchmark
Baselines vs proposed
- Base model Qwen3-VL-4B-Instruct: ODA-Val image pass@16 = 42.57%, text pass@16 = 80.22%, GeoInt pass@1 = 58.62%
- Single-view GRPO (image-dominant): ODA-Val image pass@16 = 48.78% (+6.21%), text pass@16 = 83.16% (+2.94%), GeoInt pass@1 = 62.30% (+3.68%)
- Mixed-modality GRPO: ODA-Val image pass@16 = 45.68%, text pass@16 = 81.66%, GeoInt pass@1 = 61.68%
- MIRROR: ODA-Val image pass@16 = 57.06% (+14.49% vs base), text pass@16 = 86.10% (+5.88%), GeoInt pass@1 = 66.15% (+7.53%)
- MIRROR outperforms Vision-R1-7B and PAPO-7B on ODA-Val and GeoInt metrics despite less training data.
Figures from the paper
Figures are reproduced from the source paper for academic discussion. Original copyright: the paper authors. See arXiv:2607.21552.
Fig 1: Modality-Informed Reciprocal Reasoning Optimization (MIRROR). MIRROR exploits view asymmetry by
Fig 2: Single problem analysis. Across both views, MIRROR
Fig 3 (page 4).
Fig 4 (page 4).
Fig 5 (page 5).
Fig 3: Mixed-modality GRPO does not automatically
Fig 4: Net solvability gain after training. Each bar
Fig 5: The EMA teacher prevents long-horizon collapse. Training MIRROR with the teacher scored by the current
Limitations
- ODA-Data is limited to 2,000 curated geometry problems, which is small compared to other multimodal math datasets; scalability to larger datasets and domains is untested.
- Experiments focus primarily on geometry reasoning; applicability to broader multimodal reasoning tasks is not evaluated.
- No adversarial or out-of-distribution robustness analyses are performed; model performance under deliberate input perturbations is unknown.
- Method relies on availability of paired equivalent problems across views, which may not exist in some real-world settings.
- No publicly available code or dataset release is confirmed, which may hinder immediate reproducibility.
- The tradeoff between additional FLOPs (37.5% overhead per update) and accuracy gain is noted but not studied in cost-sensitive deployment scenarios.
Open questions / follow-ons
- Can MIRROR’s adaptive teacher selection framework generalize to other multimodal reasoning domains beyond geometry, such as visual question answering or diagram understanding?
- How does the approach scale with larger datasets or models, and what is the impact on training stability and convergence?
- Could integration with explicit intermediate structured representations or symbol manipulation further enhance cross-view reasoning?
- What are the limits of self-supervised cross-view distillation when the input views differ more substantially in information content or modality?
Why it matters for bot defense
For bot-defense and CAPTCHA practitioners, MIRROR highlights the value of exploiting multiple input modalities or representations of the same challenge to improve model robustness and reasoning consistency. CAPTCHA systems that rely on combined visual and textual puzzles could benefit from training approaches that adaptively leverage the strongest modality view as a teacher signal to regularize weaker views, thereby reducing modality-dependent failure modes exploitable by bots. Furthermore, the on-policy distillation framework avoids reliance on external teacher models or curated rationales, making it applicable where labeled supervision is scarce.
However, deployment in CAPTCHA settings may require further adaptation since real-world CAPTCHAs often do not have perfectly paired multi-view inputs. Nonetheless, the idea of cross-modal self-supervision and targeted transfer from stronger input formats to weaker ones may inspire new defense techniques that systematically reduce inconsistencies and failure modes bots might exploit when perceiving visual/textual puzzles differently than humans.
Cite
@article{arxiv2607_21552,
title={ MIRROR: Learning from the Other View for Multi-Modal Reasoning },
author={ Wen Ye and Yuxiao Qu and Aviral Kumar and Xuezhe Ma },
journal={arXiv preprint arXiv:2607.21552},
year={ 2026 },
url={https://arxiv.org/abs/2607.21552}
}