Paper deep dive
On-Policy Visual Evidence Distillation
Shaohang Wei, Feifan Song, Guangyue Peng, Wenhao Yu, Wei Li, Wen Luo, Yang Xu, Yufan Shen, Luke Mao, Yang Du, Asher Qin, Houfeng Wang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Visual agents solve problems by interleaving reasoning with image operations, and on-policy distillation (OPD) provides guidance from a strong teacher on student-generated interaction trajectories. However, image operations change the evidence available for subsequent reasoning, so local errors in evidence acquisition (Acquire), reading (Read), or answer grounding (Ground) can propagate through the trajectory and lead to incorrect answers. Existing multimodal OPD methods primarily construct or contrast auxiliary views of the original image to strengthen supervision, without explicitly modeling the connections between student actions, resulting observations, and subsequent reasoning. This limits their ability to provide corrections tailored to different failure stages. We introduce Reflection on Visual Evidence (ReVuE), an on-policy distillation method for visual agents. ReVuE compares multiple student-generated trajectories for the same query, summarizes the observed visual evidence, and diagnoses the first failure across the Acquire, Read, and Ground stages. The resulting reflections provide training-time context for the teacher. We group and reweight token-level distillation losses according to how strongly these reflections affect the teacher's predictions. This design translates trajectory-level evidence diagnosis into targeted token-level supervision, guiding students to improve their visual evidence acquisition and reasoning. Across 11 benchmarks spanning the Qwen2.5-VL and InternVL3.5 model families, ReVuE outperforms all evaluated OPD baselines in weighted-average scores for perception, mathematical reasoning, and general tasks. ReVuE also reduces redundancy in reasoning and tool calls while improving tool-call accuracy and task accuracy. Code is available at this https URL
Tags
Links
- Source: https://arxiv.org/abs/2609.36838v1
- Canonical: https://arxiv.org/abs/2609.36838v1
Trouble viewing inline? Open PDF directly →
Full Text
122,429 characters extracted from source content.
Expand or collapse full text
Basic Identity 基礎部分 基礎校徽 B a s i c E m b l e m Basic Identity 基礎部分 1.01 2026-09-26 On-Policy Visual Evidence Distillation Shaohang Wei 1,‡,* , Feifan Song 1 , Guangyue Peng 1 , Wenhao Yu 3 , Wei Li 1 , Wen Luo 1 , Yang Xu 4 , Yufan Shen 2 , Luke Mao 2 , Yang Du 2 , Asher Qin 2 , Houfeng Wang 1,† 1 Peking University 2 Tencent 3 CUHK 4 Nanjing University ‡ Project leader † Corresponding authors Website§ Code Abstract Visual agents solve problems by interleaving reasoning with image operations, and on-policy distillation (OPD) provides guidance from a strong teacher on student-generated interaction tra- jectories. However, image operations change the evidence available for subsequent reasoning, so local errors in evidence acquisition (Acquire), reading (Read), or answer grounding (Ground) can propagate through the trajectory and lead to incorrect answers. Existing multimodal OPD methods primarily construct or contrast auxiliary views of the original image to strengthen supervision, without explicitly modeling the connections between student actions, resulting observations, and subsequent reasoning. This limits their ability to provide corrections tailored to different failure stages. We introduceReflection onVisualEvidence (REVUE), an on-policy distillation method for visual agents. REVUE compares multiple student-generated trajectories for the same query, summarizes the observed visual evidence, and diagnoses the first failure across the Acquire, Read, and Ground stages. The resulting reflections provide training-time context for the teacher. We group and reweight token-level distillation losses according to how strongly these reflections affect the teacher’s predictions. This design translates trajectory-level evidence diagnosis into targeted token-level supervision, guiding students to improve their visual evidence acquisition and reason- ing. Across 11 benchmarks spanning the Qwen2.5-VL and InternVL3.5 model families, REVUE outperforms all evaluated OPD baselines in weighted-average scores for perception, mathematical reasoning, and general tasks. REVUE also reduces redundancy in reasoning and tool calls while improving tool-call accuracy and task accuracy. Acquire Evidence Visual Evidence Question: What is the plate number of the blue car in the picture? <think>To determine the plate number of the blue car in the image, we need to focus on the license plate located near the bottom front of the vehicle...Here is the Python code to crop and zoom in on the license plate area:</think> <code>import cv2 ... resized_image = cv2.resize(cropped_image, ...) ... print(processed_path)</code> CoT &. → sandbox <think>Upon examining the cropped and zoomed-in image of the license plate, it becomes clear that the characters are “S QT 911”. This matches option C.</think> Sandbox Output: • Observe faithfully • Bind correctly • Integrate consistently Read Evidence <answer>Therefore, the correct answer is C.</answer> Ground Evidence • Align answer with the question • Map the fact correctly to the option • Answer consistent with the CoT trace Options: A. S OT 911 B. S TQ 119 C. S QT 911 D. B QT 119 E. This image doesn't feature the plate number. Please select the correct answer from the options above A g e n ti c V i s i o n Error Attribution 37.7% 33.8%6.3% Acquire / Read / Ground failures dominate (77.8%) Others 22.2% Reflection on Visual Evidence as supervision ReVuE: On-Policy Visual Evidence Distillation teacher policy student policy Weighted RKL Loss Previous vision-supervised OPD methods: augmented images / information as supervision O v e r v i e w Reflect on student trials Visual Interaction Augment Image Information Tool-call rate ↓Accuracy with tool ↑ Accuracy without tool ↑ 40 50 60 70 80 73.0 73.1 48.6 74.5 75.7 60.3 74.2 74.6 58.7 67.8 69.9 56.6 78.5 71.8 61.1 61.3 70.4 63.6 54.0 79.4 67.7 Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 360390420450480 Mean response length (tokens) ↓ 65 67.5 70 72.5 75 Accuracy (%) RL Expert: 71.75% ReVuE (Ours) 458 tokens Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 1.20× shorter Figure 1: REVUE overview (with HRBench-8K results) and an agentic vision example. * Work done during the internship at Tencent. Correspondence to shaohang@stu.pku.edu.cn. 1 arXiv:2609.36838v1 [cs.CV] 29 Sep 2026 On-Policy Visual Evidence Distillation 1 Introduction High-resolution image understanding, chart analysis, and document question answering often require models to actively locate task-relevant regions, inspect fine-grained details, and incorpo- rate their observations into subsequent reasoning (Wu & Xie, 2024; Qi et al., 2025). Visual agents address these demands by interleaving reasoning with image operations, actively gathering information needed to answer a query (Zheng et al., 2026b; Zhang et al., 2026b). As strong models develop these capabilities, effectively transferring their expertise to students becomes an important training objective. On-policy distillation (OPD) provides dense teacher feedback on student-generated trajectories (Agarwal et al., 2024; Gu et al., 2024; Wei et al., 2026), supervising the actions and reasoning decisions students actually make. Therefore, constructing teacher supervision tailored to visual interaction is a key step toward translating expert capabilities into reliable student behavior. Visual interactions introduce a particular challenge: image operations change the evidence available for subsequent reasoning (Qi et al., 2025; Zhang et al., 2026b). Selecting the wrong region can leave critical evidence unavailable (Hou et al., 2026); obtaining a useful observation does not guarantee that the student reads it correctly; and a correctly read fact may still be mapped to the wrong answer. We refer to these stages as evidence acquisition (Acquire), visual reading (Read), and answer grounding (Ground). Errors at any stage can propagate through later actions and reasoning, ultimately producing an incorrect answer. Effective distillation therefore requires identifying the first failed stage in the evidence chain and directing corrections to where the error begins to propagate. Representative multimodal OPD methods strengthen visual supervision through privileged visual cues or contrasts between teacher predictions under different visual conditions (Tian et al., 2026; Sun et al., 2026; Liu et al., 2026a). Vision-OPD (Yuan et al., 2026) uses evidence-centered crops to guide the teacher, while VAD (Zhang et al., 2026a) estimates the visual contribution to teacher corrections by removing relevant evidence. These methods improve how visual information informs supervision, but do not explicitly trace the links between students’ executed actions, returned observations, and subsequent reasoning. This limits their ability to identify which visual behaviors to reinforce and where the evidence chain first breaks down. This raises a central question: how can we turn students’ own interaction trajectories into more targeted teacher guidance? Multiple attempts at the same query offer a concrete basis for such guidance. Successful attempts may complement each other: one finds more focused evidence, while another reads the relevant visual facts more accurately. Failed attempts help identify the first error in Acquire, Read, or Ground. We introduce REVUE (Reflection onVisualEvidence) to turn these experiences into visual-evidence reflections. By comparing these trajectories, REVUE selects a compact set of observed images sufficient to establish the relevant visual facts. It also summarizes how to read the evidence correctly and map the resulting facts to the answer. The resulting reflections provide training-time context for a strong teacher, helping it reinforce effective behaviors already demonstrated by the student and correct errors at the first failed stage. Visual-evidence reflection changes the teacher’s assessment of student choices, with effects that vary across token positions. By comparing the teacher’s predictions on the same student trajectory with and without reflection, we identify where reflection most strongly changes the teacher’s support for student choices. In early training, we find that these changes concentrate on a small fraction of tokens. To emphasize supervision at these positions, we group tokens by the magnitude of these changes and apply grouped reweighting to give high-impact positions greater weight in the distillation loss. Our ablations show that this reweighting improves perception and math performance. Reflection changes the teacher ’s predictions and guides how 2 On-Policy Visual Evidence Distillation token-level losses are weighted. We evaluate REVUE on 11 benchmarks across the Qwen2.5-VL (Bai et al., 2025b) and In- ternVL3.5 (Wang et al., 2025a) model families. REVUE consistently outperforms all OPD baselines in performance across perception, mathematical reasoning, and general tasks. It also reduces redundant reasoning and unnecessary tool calls, while improving both tool-call accuracy and overall task accuracy. On Qwen2.5-VL-7B, masking high-impact positions in the distillation loss reduces performance more than masking an equal number of low-impact positions. Our critic’s stage diagnoses agree closely with human judgments. Together, these results support the effectiveness of REVUE and the reliability of its stage diagnoses. Our contributions are threefold: 1.Visual evidence reflection across student trajectories. We compare multiple successful student attempts on the same query to identify minimal sufficient evidence and summa- rize successful reading and grounding strategies. We also locate the first failure among Acquire, Read, and Ground, providing training-time guidance to a strong teacher. 2.Reflection-aware grouped reweighting for distillation. We partition tokens accord- ing to how strongly reflection changes the teacher’s predictions, and use group-wise normalization and reweighting to focus distillation updates. 3.Cross-model gains and validation of high-impact supervision. Across two model families and 11 benchmarks, REVUE improves task and tool-call accuracy over OPD base- lines while reducing reasoning and tool-call redundancy. On Qwen2.5-VL-7B, masking high-impact positions in the loss reduces performance more than masking equally many low-impact positions, supporting greater weight on high-impact supervision. 2 Related Work Agentic Visual Reasoning.Agentic visual reasoning interleaves reasoning with image opera- tions and the resulting observations. This capability appears in general-purpose models such as o3/o4-mini (OpenAI, 2025), Qwen3-VL (Bai et al., 2025a), and Kimi K2.5 (Kimi Team et al., 2026). Earlier systems use guided visual search or visual sketches to acquire and organize task-relevant evidence (Wu & Xie, 2024; Hu et al., 2024). CogCoM (Qi et al., 2025) and LATTE (Ma et al., 2025) learn sequences of visual operations through trajectory supervision. DeepEyes (Zheng et al., 2026b), VTool-R1 (Wu et al., 2026), and Pixel Reasoner (Su et al., 2025) use reinforcement learning to improve visual tool use and exploration. Mini-o3 (Lai et al., 2026) scales visual search to longer interactions and more diverse reasoning patterns. Thyme (Zhang et al., 2026b) and DeepEyesV2 (Hong et al., 2026) support broader reasoning through executable code, with DeepEyesV2 also incorporating web search. CodeV (Hou et al., 2026) uses process rewards on visual tool inputs and outputs to encourage evidence-consistent tool use. We study how students’ actual visual interactions can inform teacher guidance for on-policy distillation. On-Policy Distillation.On-policy distillation (OPD) supervises student-generated trajectories with teacher distributions, reducing the training–inference mismatch of offline distillation (Agar- wal et al., 2024; Gu et al., 2024). Qwen3 (Yang et al., 2025) and Qwen3-VL (Bai et al., 2025a) also use this approach to train smaller models. Privileged-context methods condition teachers on reference solutions, additional context, or peer trajectories (Zhao et al., 2026; Ye et al., 2026; Yu et al., 2026). Visual OPD builds teacher–student asymmetry through evidence-centered crops, recoverable visual cues, or stronger image augmentation for students (Yuan et al., 2026; Tian et al., 2026; Li et al., 2026b). Visual contrasts also support supervision gating and token 3 On-Policy Visual Evidence Distillation selection (Sun et al., 2026; Aniri et al., 2026), grouped loss reweighting (Liu et al., 2026a), and target reconstruction (Zhang et al., 2026a). Other methods internalize visual manipulation or generated visual thoughts to reduce inference-time computation (Cai et al., 2026; Li et al., 2026a). For agents, SGCD (Ding et al., 2026) and GRSD (Zheng et al., 2026a) use guidance derived from rollout groups to modify credit assignment in reinforcement learning. REVUE instead extracts reflections from students’ visual interactions to reinforce effective behaviors and locate the first failure in Acquire, Read, or Ground. We condition the teacher on these reflections and use their effects on its predictions to group and reweight tokens for direct distillation. 3 Preliminaries Agentic visual reasoning. Given an image–query pairx = (I,u), a student policy π θ with parametersθinteracts with a tool environmentP env .Before stepk, history h k = (x,a 0 ,o 1 ,...,a k−1 ,o k )contains the input, previous actions, and returned observa- tions, withh 0 = x. An actiona k contains reasoning, executable code, or a final answer. An observation o k+1 contains tool-returned text and images. The interaction follows a k ∼ π θ (·| h k ), o k+1 ∼ P env (·| h k ,a k ), h k+1 = (h k ,a k ,o k+1 ).(1) Actions without tool calls yieldo k+1 =∅. A final answerˆyends the interaction, and the complete sequence of actions and observations forms a trajectory τ . Visual evidenceE k includesIand all tool-returned images inh k . In Figure 1 (bottom), the student crops and zooms in on the blue car ’s plate. It readsS QT 911and matches it to optionC. The correct visual factf x needed to answeruprovides a reference for checking the student’s reading. The rule γ x maps this fact to the answer. The evidence chain is x Acquire −→ E k Read −→ f x Ground −→ γ x ˆy.(2) Success requires evidence sufficient to establishf x , an accurate reading, and an answer satis- fyingˆy = γ x (f x ). We attribute evidence-chain failures to the first unmet requirement in the dependency order Acquire, Read, and Ground. These stages can recur throughout an interaction. On-policy distillation.On-policy distillation (OPD) (Agarwal et al., 2024) trains the student on its own sampled trajectories using token-level teacher distributions. For each inputx, the current student samplesMtrajectories per round, formingG x =τ i M i=1 with trajectory indexi. Token positionstare distinct from interaction stepsk. Tokeny i,t is the student’s output at positiontin τ i . Historyh i,t contains the input and preceding student outputs and tool observations. LetT i denote the supervised student-output positions in trajectoryi, with|T i |their number. This set covers reasoning, code, and final answers across interaction rounds. Prompts and tool returns provide context but are not prediction targets. The teacherq φ has fixed parametersφ. Givenh i,t , p i,t and q 0 i,t denote the student and base-teacher next-token distributions: p i,t = π θ (·| h i,t ), q 0 i,t = q φ (·| h i,t ). Superscript0marks the base teacher prediction. WithD KL denoting KL divergence, the reverse- KL objective and its Monte Carlo (MC) estimate fromG x are L OPD (θ) =E τ i ∼π θ (·|x) 1 |T i | X t∈T i D KL (p i,t ∥q 0 i,t ) MC ≈ 1 M M X i=1 1 |T i | X t∈T i D KL (p i,t ∥q 0 i,t ).(3) The objective averages over supervised positions within each trajectory, then over theMtrajec- tories for the same input. 4 On-Policy Visual Evidence Distillation Reflection summarize compare x k x M-k correct agentic rollout wrong agentic rollout Break Pointℬ ! Anchor 풜 ! inputx student policy critic teacher policy inputx w/ reflection inputx w/o reflection Acquire Read Ground Read First broken type Discrepancy: The rollout fails to observe the white spines/ribs on the cactus, .. |Δlog qₜ| Weighted RKL ∇ θ ℒ log qᴿ(yₜ) log q⁰(yₜ) Same sampled trajectory y frozen Token impact |log qᴿ(yₜ) − log q⁰(yₜ)| Group-wise normalization High-impact Low-impact Update student Training Sampling Figure 2: Sampling and training pipeline of REVUE. 4 Method REVUE converts same-query student trajectories into reflection-guided distillation, as shown in Figure 2. It first constructs a trajectory-specific visual-evidence reflection, then measures how the reflection changes a frozen teacher’s token predictions, and finally uses these changes to reweight token-level distillation losses (Algorithm 1). 4.1 Visual-Evidence Reflection Our error analysis identifies Acquire, Read, and Ground as the main failure stages (Figure 1), with examples in Appendix E. Prior work also shows that attending to the correct visual evidence does not guarantee a correct answer (Liu et al., 2026b). Even with the same evidence, students may need different corrections. In the pie-chart example in Appendix E.2, both trajectories answerNo. One misreads the slice ranking, while the other reads it correctly but misapplies the lower-median rule. These failures occur at the Read and Ground stages. Targeted distillation guidance therefore requires examining the student’s actions, visual observations, and reasoning to identify the specific error in evidence use. We construct reflections fromG x , the group of trajectories sampled by the current student for the same input. The visual evidence and correct reasoning in one attempt can provide a reference for others. A critic compares these trajectories and their visual observations to extract correct references and diagnose errors in each incorrect trajectory. For trajectoryi, the complete visual-evidence reflection is R i = (A i ,B i ).(4) The AnchorA i provides a reference for correct visual evidence use. It includes the target visual information and supporting images actually observed withinG x . It also records the correct visual factf x and the ruleγ x that maps this fact to the answer. For an incorrect trajectory, the Break PointB i = (σ i ,δ i )describes its failure relative to this reference.σ i denotes the first failed stage in the dependency order, andδ i describes the specific discrepancy. These images and text serve as training-time teacher context for token-level supervision on the student’s sampled trajectories. The teacher can then reinforce effective evidence use and guide corrections to observed errors. Appendix A.4 shows a concrete visual-evidence reflection, and Appendices A.5–A.6 detail its construction and critic prompts. 5 On-Policy Visual Evidence Distillation Which country is highlighted? Ground Truth: (B) Saint Kitts and Nevis (A) Antigua and Barbuda (B) Saint Kitts and Nevis (C) Grenada (D) Saint Lucia Actual tool crop by student Student trajectory Reasoning after the tool call Upon analyzing the cropped image , the highlighted country is a small island located in the eastern Caribbean. The shape and position of the highlighted area suggest it is one of the smaller islands in the region . Compar ing it with the given options, the highlighted country matches the geographical features of Saint Lucia. Saint Lucia is known for its distinctive shape and location in the eastern Caribbean, which aligns with the highlighted area in the image . The cropped image provides a clearer view, confirming that the highlighted country is indeed Saint Lucia. Final answer The country highlighted in the map is (D) Saint Lucia. The green box is located in the eastern Caribbean, and Saint Lucia fits this description. High-impact tokens: Encourage Suppress Figure 3: Token impact on a trajectory with a reading error. The student misreads two islands as one, answeringSaint Luciainstead ofSaint Kitts and Nevis. Reflection identifies two islands and lowers support for theunderlinedisland,one, and firstLucia. Blue/orange denote increased/decreased teacher support; darker shading indicates larger absolute log- probability shifts. Both evaluations score the same student trajectory; the finalLuciachanges little under the fixed erroneous prefix. 4.2 From Reflection to Token Impact Visual-evidence reflection provides correct references and diagnoses failures in the evidence chain. To locate where this guidance changes supervision, we compare teacher scores along the student’s sampled trajectory. On the same student historyh i,t , the frozen teacher predictsq 0 i,t without reflection (Section 3) andq R i,t = q φ (· | h i,t ,R i )with it. Only the training-time teacher receives reflection; the student retains its original interaction history. Using teacher probabilities normalized over the full vocabulary, we define the log-probability shift for candidate token v: d i,t (v) = logq R i,t (v)− logq 0 i,t (v).(5) To connect these shifts to the reverse-KL objective in Equation 3, we compare both teachers on a shared vocabulary support. The training loss uses the adopted teacher’s top-Kcandi- dates; for the reflection-conditioned branch, we use these candidates as Supp for both teachers. Suppressing(i,t), writep = p i,t andp Supp (v) = p(v)/ P u∈Supp p(u) forv ∈ Supp. Defineq Supp analogously and let ℓ Supp (p,q) =D KL (p Supp ∥q Supp ). Lemma 1 (Reflection and the distillation gradient)For fixed history, teacher predictions, and nonempty shared support, with positive student and teacher probabilities on this support, we have ∇ θ ℓ Supp (p,q R )− ℓ Supp (p,q 0 ) =−E v∼p Supp d(v)∇ θ logp Supp (v) .(6) Equation 6 expresses the gradient change as a sum over candidate tokens, with each term weighted by its teacher-score shiftd(v)(proof in Appendix D.1). We focus on the student’s sampled tokeny i,t to locate where reflection changes teacher support for its actual choices. We define the magnitude of this shift as token impact: s i,t =|d i,t (y i,t )| = logq R i,t (y i,t )− logq 0 i,t (y i,t ) .(7) Taking the absolute value captures both increased and decreased teacher support, covering reinforcement and correction. This score ranks positions by changes in teacher scores, not by full gradient magnitude. Each position’s loss still supervises the distribution over its support. Token impact distinguishes positions within a trajectory by how much their teacher scores change with reflection. In Figure 3, reflection reduces support for the mistaken judgment ofone island. The tokenonehas higher impact than the finalLucia, whose teacher score changes little under the existing erroneous prefix. Section 4.3 groups positions by these scores and assigns greater distillation weight to high-impact positions. 6 On-Policy Visual Evidence Distillation 4.3 Reflection-Aware Grouped Distillation 00.511.52≥2.5 Token impact s t 0 40 80 Tokens (%) Top 20% carry 97.6% of impact LowHigh 080100 Percentile 0 10 20 Impact 99th: 2.32 100th: 19.70 Figure 4: Token-impact sparsity Token impact measures teacher-score changes induced by visual-evidence reflection (Section 4.2). Figure 4 pools early-training scores: approximately 20% of scored positions account for 97.6% of total impact (Ap- pendix B.4). Uniform token averaging allocates total loss weight in proportion to group size, leaving sparse high-impact positions a small share. To emphasize their supervision, we group positions by impact and introduce weightsw i,t into the reverse-KL objective in Equation 3: L ReVuE (θ) = 1 |G| X τ i ∈G 1 |T i | X t∈T i w i,t D KL p Supp i,t i,t (q ∗ i,t ) Supp i,t .(8) G contains retained student trajectories in the current batch, with nonempty supervised setsT i . The teacher targetq ∗ i,t isq R i,t when reflection-conditioned scoring is valid and token-aligned, and q 0 i,t otherwise. Both distributions are renormalized overSupp i,t = TopK(q ∗ i,t ) , as indicated by the superscripts. Within each trajectory, we rank supervised positions bys i,t , assigning the top⌈α|T i |⌉toT i,High and the rest toT i,Low =T i i,High , whereα∈ (0, 1). For nonempty groups, we allocate total loss weightsλ∈ (0, 1)and1− λtoHighandLowand distribute each uniformly within its group. Each position’s coefficient in Equation 8 is w i,t /|T i |, giving the weights w i,t = ( λ|T i |/|T i,High |,t∈T i,High , (1− λ)|T i |/|T i,Low |, t∈T i,Low . (9) α controls group size, whileλcontrols total group weight. We setλ = 0.5, assigning half to each group. With teacher targets, supports, and groups fixed,w i,t scales each position’s gradient relative to uniform token averaging. This multiplier exceeds one forHighwhen the group contains fewer than half the supervised positions. Appendix D.2 and Figure 14 give the derivation and weight curves. If impacts are unavailable or either group is empty, we restore uniform averaging with w i,t = 1 and retain the adopted teacher target (Appendix A.1). 5 Experiments 5.1 Experimental Setup For models, we use Thyme-SFT (Zhang et al., 2026b) cold-start checkpoints of Qwen2.5-VL- 7B (Bai et al., 2025b) and InternVL3.5-4B-Instruct (Wang et al., 2025a) as students. Teachers are our reproduced Thyme-RL experts; Qwen3.5-397B-A17B (Qwen Team, 2026) is the critic. Off- the-shelf comparisons include GPT-4o (OpenAI, 2024), Gemini-3.1-Flash-Lite, Qwen2.5-VL-32B, Qwen3-VL-30B-A3B-Thinking (Bai et al., 2025a), and InternVL3.5-38B-Instruct. For baselines, we compare with Vision-OPD (Yuan et al., 2026) for visual privilege and V-Zero (Sun et al., 2026) and VAD (Zhang et al., 2026a) for visual contrast. We also include Vanilla OPD, RFT (Yuan et al., 2023), and GT-Privileged. All OPD baselines use the same RL expert teacher; details are shown in Appendix B.1. For benchmarks, we evaluate on 11 benchmarks covering perception, math, and general multimodal tasks (Appendix F). Appendix B.3 details the training data, model initialization, and training and evaluation settings. 7 On-Policy Visual Evidence Distillation 5.2 Main Results Takeaway 1.Both model families improve across all three task categories. Both REVUE stu- dents exceed the evaluated OPD baselines and RFT in all three weighted category averages. Perception gains over Vanilla OPD are 2.40% for Qwen2.5-VL-7B and 1.70% for InternVL3.5-4B-Instruct. 2. Higher perception accuracy accompanies shorter responses and less frequent tool use. Qwen2.5-VL-7B improves all three measures over Vanilla OPD on all five perception benchmarks, with the extent of change varying by task. 360390420450480 Mean response length (tokens) ↓ 65 67.5 70 72.5 75 Accuracy (%) RL Expert: 71.75% ReVuE (Ours) 458 tokens (a) HRBench 8K Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 1.20× shorter 380420460500540 Mean response length (tokens) ↓ 76 78 80 82 84 RL Expert: 83.77% ReVuE (Ours) 527 tokens (b) V* Bench Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 1.31× shorter Tool-call rate ↓Accuracy with tool ↑ Accuracy without tool ↑ 40 50 60 70 80 % (c) HRBench 8K 73.0 73.1 48.6 74.5 75.7 60.3 74.2 74.6 58.7 67.8 69.9 56.6 78.5 71.8 61.1 61.3 70.4 63.6 54.0 79.4 67.7 Tool-call rate ↓Accuracy with tool ↑ Accuracy without tool ↑ 30 40 50 60 70 80 90 100 (d) V* Bench 95.3 82.4 33.3 96.9 84.3 66.7 95.8 82.0 62.5 75.9 76.6 76.1 79.1 80.1 70.0 73.3 76.4 78.4 74.3 82.7 80.7 Figure 5: Response length, accuracy, and tool use for Qwen2.5-VL-7B. REVUE improves perception across model families, with gains in math and general tasks. In all three weighted category averages, both REVUE students outperform all evaluated OPD baselines and RFT, which trains only on teacher trajectories with correct answers (Table 1). Perception gains over Vanilla OPD are 2.40% for Qwen2.5-VL-7B and 1.70% for InternVL3.5-4B- Instruct. Qwen’s gains on HRBench 8K (+3.50%) and TreeBench (+3.10%) cover high-resolution understanding and object relations. Both models also improve in math and general tasks, with average gains of 2.83% and 1.65% for InternVL, respectively. REVUE also achieves higher category averages than baselines using privileged visual information or visual contrasts, with matched student initializations and teacher sources. The answer-privilege comparison provides further evidence: REVUE exceeds GT-Privileged on all 11 benchmarks for both students. These comparisons support guiding how students acquire, read, and use evidence; giving the teacher the correct answer alone does not yield the same gains. After reflection-assisted distillation, both students also surpass their RL Expert teachers in all three category averages. Both students also exceed GPT-4o in all three averages and the larger off-the-shelf models in their families in perception. 8 On-Policy Visual Evidence Distillation Table 1: Main results. Bold marks the best OPD result per column within each model family, including ties. Wtd. Avg. uses benchmark sample counts as weights. Baseline details and model labels are provided in Appendix B.1–B.2. Evaluation benchmark details are provided in Appendix F. Model / Method PerceptionMathGeneral HR-4K HR-8KV * Tree Bench Visual Probe Wtd. Avg. Math Vista Math Verse Visu Logic Wtd. Avg. Hallu Bench CQA Pro Info VQA Wtd. Avg. Off-the-Shelf Models GPT-4o61.0054.0061.7849.8823.8850.2858.8339.2125.2041.2251.3728.6771.1753.28 Gemini3.1FL46.0043.0064.9251.8527.9643.8980.8077.9232.4062.6359.9237.0383.5063.57 Qwen2.5-32B75.1369.2578.0148.4045.0563.8977.0053.0525.9051.9050.7430.0283.1059.29 Qwen3-30B-T77.1371.3880.1045.4336.8963.2680.2066.1225.8056.7161.8235.4685.6864.45 InternVL-38B71.5062.1365.9741.9820.9754.3470.9048.4827.2048.8950.7426.9877.7055.71 Qwen2.5-VL-7B Base Model69.0063.5075.3937.0441.1757.7769.1044.0425.4046.3440.9120.7975.1050.53 Cold-start74.3866.6280.1037.2841.5660.7268.0044.2926.4046.3841.7321.1279.0552.68 RL Expert75.5071.7583.7740.2544.8563.8970.6045.4326.2047.5643.8321.6979.7453.60 RFT75.3872.0081.2040.0041.7563.1269.5045.8124.6046.7042.3321.1879.5753.07 Vanilla OPD75.4070.5081.2038.0242.9162.6170.6046.0726.0047.6742.8821.6979.4453.28 GT-Privileged 73.6270.5079.5839.7543.1062.2671.2045.6925.2047.4943.4520.8479.7053.23 Vision-OPD73.0065.6276.4439.0141.3659.9867.7042.3925.7045.4839.8920.7978.7552.08 V-Zero75.6269.5078.0137.2843.1062.0869.8043.9126.6046.9942.1621.0879.1252.79 VAD74.7567.7576.9641.9843.8962.0869.0045.5622.3045.6244.2421.7379.6553.65 REVUE(Ours)77.1074.0082.2041.1244.6665.0171.6047.0826.5048.4945.1221.8180.2754.14 InternVL3.5-4B-Instruct Base Model62.0055.0068.5940.4920.58 49.3268.5028.9325.8042.0040.0426.0166.0447.78 Cold-start69.5063.2566.4940.9929.9055.6669.3047.2125.8047.4548.5026.2472.9852.79 RL Expert72.6264.6271.7340.7434.5658.2069.3046.3226.9047.6046.9230.5973.9354.38 RFT71.2566.0071.7340.2535.0058.2270.0045.9424.3046.8147.7829.3974.1754.26 Vanilla OPD71.1365.8773.3040.2533.7958.0268.4043.4026.6046.3446.8530.4672.1653.48 GT-Privileged 71.2564.8870.1639.7533.2057.3671.3039.3426.2046.0949.2929.9072.4853.91 Vision-OPD66.2560.0065.9737.7828.9353.0466.6044.5426.6046.0244.5730.4674.4754.14 V-Zero68.1361.7567.0240.2528.3554.4568.8046.9526.8047.5649.0328.6573.6153.99 VAD67.5061.0067.0239.0127.9653.7868.3046.0727.7047.4548.0531.4473.3654.61 REVUE(Ours)73.2567.3773.3041.4836.1259.7271.6047.4628.1049.1749.3832.2773.3555.13 REVUE improves perception accuracy with shorter responses and less frequent tool use. Across all five perception benchmarks, the Qwen2.5-VL-7B student is more accurate than Vanilla OPD while producing shorter responses and using tools less often (Figure 5; Appendix C.4). On HRBench 8K, REVUE reduces mean response length from the RL Expert’s 458 tokens to 382, while raising accuracy from 71.75% to 74.00% (+2.25%). On HRBench 8K and V* Bench, its answer accuracy exceeds the other plotted distillation baselines on samples both with and without tool use. Tool-use frequency alone does not explain the performance differences. On V* Bench, VAD uses tools slightly less often (73.3% vs. 74.3%), yet REVUE achieves 5.24% higher overall accuracy. The extent of these behavioral changes varies by task. On TreeBench, REVUE reduces response length by 46.09% and tool-use rate from 64.7% to 15.8%, while improving accuracy by 3.10% over Vanilla OPD. On VisualProbe, it still uses tools on 83.7% of samples and also achieves higher accuracy. These results support more effective visual evidence use: higher accuracy with less generation and less frequent tool use, while tool reliance still varies by task. 9 On-Policy Visual Evidence Distillation 6 Analysis and Discussion Takeaway 1.Reflection improves all three category averages; reweighting adds gains in per- ception and math. On Qwen2.5-VL-7B, reflection raises all three averages, and adding impact reweighting raises perception and math by another 1.31% and 0.71%, respectively. The general-task average drops by 0.28% relative to reflection alone. 2.Impact ranking identifies supervision worth prioritizing. On Qwen2.5-VL-7B, impact-based selection outperforms random selection in perception and math, and masking high-impact positions hurts more than masking equally many low-impact positions. The top 20% of pooled early-training positions carry 97.6% of total impact. 3.Prioritizing 20% of positions gives the best tested perception and math averages. General tasks on Qwen2.5-VL-7B favor the 100% setting, whose average exceeds the 20% setting by 0.28%. 4. Critic stage diagnoses agree closely with human annotation and other judges. Critic labels match one annotator on 93 of 96 rollouts (96.88%), with 95.31% agree- ment on the 64 incorrect rollouts. Valid pairs agree at 99.2% for a critic rerun and 92.8–93.2% across models. Table 2: Component ablation on Qwen2.5-VL-7B. MethodPerception↑ Math↑ General↑ Vanilla OPD62.6147.6753.28 +reflection63.7047.7854.42 +impact reweight65.0148.4954.14 Reflection brings gains across tasks; impact reweighting further improves perception and math. In the cumula- tive ablation on Qwen2.5-VL-7B, adding visual-evidence reflection improves all three category averages and nine of 11 benchmark scores over Vanilla OPD (Table 2). Adding impact reweighting then raises the perception and math averages by another 1.31% and 0.71%, respectively. It also improves tasks that do not benefit from reflection alone. On TreeBench, for example, accuracy falls from 38.02% to 37.78% with reflection alone, then rises to 41.12% with reweighting. The general-task average drops by 0.28% relative to reflection alone, but REVUE still outperforms Vanilla OPD on all 11 benchmarks (Appendix C.1). Table 3: Token-impact interventions. MethodPerception↑ Math↑ General↑ Vanilla OPD62.6147.6753.28 Random select 20%63.6447.4254.10 Mask random 20%63.8547.3553.78 Mask low 20%65.1848.4253.97 Mask top 20%62.2647.1752.51 REVUE(Ours)65.0148.4954.14 Impact ranking identifies supervision worth pri- oritizing.We compare impact-based and random selection on Qwen2.5-VL-7B with fixed group frac- tions and weights (Table 3). Compared with ran- dom selection, REVUE gains 1.37% in perception and 1.07% in math, but only 0.04% on general tasks. Masking yields the same ordering in all three cate- gory averages:Mask low > Mask random > Mask top.Mask lowalso exceedsMask top on all 11 benchmarks (Appendix C.2). The cases show reflection lowering teacher support for erroneous choices and raising support for correct judgments within failed trajectories. In the age case, reflection favors the correct relationaboveand discourages the erroneous compar- isonexceed(Appendix E.4). Scores can change at target selection and intermediate visual judgments while final-answer token scores remain nearly unchanged (Figure 3; Appendix E.4). The top 20% of pooled early-training positions carry 97.6% of total impact, supporting greater weight on these positions (Figure 4).Mask lowgains 0.17% in perception over the full method but slightly reduces math and general averages, showing task-dependent effects. 10 On-Policy Visual Evidence Distillation Table 4: Sensitivity to α. FractionPerception↑ Math↑ General↑ Top 5%64.7047.4954.12 Top 10%64.5546.9253.85 Top 20% (REVUE)65.0148.4954.14 Top 50%64.3347.4254.18 Top 100%63.7047.7854.42 Prioritizing 20% of positions gives the highest perception and math averages.This fraction also leads on all three math benchmarks (Table 4; Ap- pendix C.3). Withλ = 0.5, reducingαshrinks the high-weight group and increases each position’s weight within it (Appendix D.2). Neither 5% nor 10% exceeds the perception and math averages at 20%, supporting moderate concentration in this setting. The general-task average peaks at 100%, 0.28% above the 20% setting, indicating different preferences across tasks. Table 5: Critic stage consistency. Comparatorn Agreement (%)κ Human annotator9696.880.957 Qwen (rerun)74499.2 0.987 Gemini 3.8 Flash73693.2 0.892 GPT-5.6 Luna73692.8 0.886 n: valid pairs. Critic diagnoses agree closely with human judg- ments, including on incorrect trajectories.One anno- tator independently labels 96 rollouts balanced across all-correct,mixed, andall-wronggroups before viewing critic labels. Final stage labels match 93 of 96 human labels (96.88%, Cohen’sκ = 0.957). Agreement remains 95.31% on the 64 incorrect rollouts. On 768 rollouts, we compare archived final training labels with fresh labels from the same critic and two external judges. Valid pairs agree at 99.2% for the same critic and 92.8–93.2% across models (Table 5). These results support human agreement and repeatability of the stage diagnoses used for reflection. Appendix B.6 details sampling and label disagreements. 7 Conclusion To provide targeted supervision for visual agents, we introduced REVUE, which improves perception, mathematical reasoning, and general-task performance through visual-evidence reflection and token reweighting in on-policy distillation. The reflections provide the teacher with references for correct evidence use and diagnoses of errors in the student’s own trajectories. Token reweighting then emphasizes supervision at positions where these reflections most change the teacher’s assessment. Behavioral analysis on Qwen2.5-VL-7B also shows fewer tool calls and shorter responses on perception tasks. Ablations and token-impact interventions support the effectiveness of both components. Our study focuses on two student model families and image-based question answering; broader model coverage and more diverse visual interactions remain to be explored. 11 On-Policy Visual Evidence Distillation References Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos Garea, Matthieu Geist, and Olivier Bachem. On-Policy Distillation of Language Models: Learn- ing from Self-Generated Mistakes. In International Conference on Learning Representations, p. 21246–21263, 2024. URLhttps://proceedings.iclr.c/paper_files/paper/ 2024/file/5be69a584901a26c521c2b51e40a4c20-Paper-Conference.pdf. Aniri, Jinhe Bi, Peng Liao, Zengjie Jin, Volker Tresp, Fei Shen, Yunpu Ma, and Tat-Seng Chua. OPD-V: Visual On-Policy Self-Distillation with Modality Balance, 2026. URLhttps:// arxiv.org/abs/2608.05131. Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-VL Technical Report, 2025a. URL https://arxiv.org/abs/2511.21631. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-VL Technical Report, 2025b. URL https://arxiv.org/abs/2502.13923. Yishuo Cai, Jiahui Liu, Yuanxin Liu, Haobo Deng, Linli Yao, Yuhao Zheng, Kun Ouyang, Zhimo Li, Ziyue Wang, Xu Sun, Haoli Bai, and Xiaohui Li. Thinking Without Images: Internalizing Visual Manipulation with On-Policy Self-Distillation, 2026. URLhttps://arxiv.org/ abs/2606.08719. Tianyu Ding, Jianhong Xin, and Juan Pablo De la Cruz Weinstein. Keep policy gradient in charge: Sibling-guided credit distillation for long-horizon tool-use agents, 2026. URLhttps: //arxiv.org/abs/2606.12634. Chaoyou Fu, Peixian Chen, Yunhang Shen, Yulei Qin, Mengdan Zhang, Xu Lin, Jinrui Yang, Xiawu Zheng, Ke Li, Xing Sun, Yunsheng Wu, Rongrong Ji, Caifeng Shan, and Ran He. MME: A comprehensive evaluation benchmark for multimodal large language models. In D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (eds.), Advances in Neural Information Processing Systems, volume 38. Curran Associates, Inc., 2025. doi: 10.52202/085713-4899. URLhttps://proceedings.neurips.c/paper_files/ paper/2025/file/d79a27cf2772fe00be7f341efc0eb517-Paper-Datasets_ and_Benchmarks_Track.pdf. Yuxian Gu, Li Dong, Furu Wei, and Minlie Huang.MiniLLM: Knowledge Distilla- tion of Large Language Models. In International Conference on Learning Representations, p. 32694–32717, 2024. URLhttps://proceedings.iclr.c/paper_files/paper/ 2024/file/8ac015d409635f196f9e3e9dcfb9a94e-Paper-Conference.pdf. Tianrui Guan, Fuxiao Liu, Xiyang Wu, Ruiqi Xian, Zongxia Li, Xiaoyu Liu, Xijun Wang, Lichang Chen, Furong Huang, Yaser Yacoob, Dinesh Manocha, and Tianyi Zhou. HallusionBench: An advanced diagnostic suite for entangled language hallucination 12 On-Policy Visual Evidence Distillation and visual illusion in large vision-language models.In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 14375–14385, June 2024.URLhttps://openaccess.thecvf.com/content/CVPR2024/html/Guan_ HallusionBench_An_Advanced_Diagnostic_Suite_for_Entangled_Language_ Hallucination_and_CVPR_2024_paper.html. Jack Hong, Chenxiao Zhao, ChengLin Zhu, Weiheng Lu, Guohai Xu, and Xing Yu. DeepEyesV2: Toward Agentic Multimodal Model. In International Conference on Learning Representations, p. 114851–114872, 2026. URLhttps://proceedings.iclr.c/paper_files/paper/ 2026/file/badbef7d421640fb038399e9d7271b1c-Paper-Conference.pdf. Xinhai Hou, Shaoyuan Xu, Manan Biyani, Moyan Li, Jia Liu, Todd C Hollon, and Bryan Wang. CodeV: Code with Images for Faithful Visual Reasoning via Tool-Aware Policy Optimization. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 21500–21510, 2026. URLhttps://openaccess.thecvf.com/content/CVPR2026/ html/Hou_CodeV_Code_with_Images_for_Faithful_Visual_Reasoning_via_ Tool-Aware_CVPR_2026_paper.html. Yushi Hu, Weijia Shi, Xingyu Fu, Dan Roth, Mari Ostendorf, Luke Zettlemoyer, Noah A. Smith, and Ranjay Krishna.Visual Sketchpad:Sketching as a Visual Chain of Thought for Multimodal Language Models.In Advances in Neural In- formation Processing Systems, volume 37, p. 139348–139379, 2024.doi:10.52202/ 079017-4423. URLhttps://proceedings.neurips.c/paper_files/paper/2024/ file/fb82011040977c7712409fbdb5456647-Paper-Conference.pdf. Kimi Team, Tongtong Bai, Yifan Bai, Yiping Bao, S. H. Cai, Yuan Cao, Ziwei Chai, Y. Charles, H. S. Che, Cheng Chen, Guanduo Chen, Huarong Chen, Jia Chen, Jianlong Chen, Jun Chen, Kefan Chen, Liang Chen, Ruijue Chen, Xinhao Chen, Yanru Chen, Yanxu Chen, Yicun Chen, Yimin Chen, Yingjiang Chen, Yuankun Chen, Yujie Chen, Yutian Chen, Zhirong Chen, Ziwei Chen, Dazhi Cheng, Yean Cheng, Minghan Chu, Jialei Cui, Jiaqi Deng, Muxi Diao, Hao Ding, Mengfan Dong, Mengnan Dong, Yuxin Dong, Yuhao Dong, Angang Du, Chenzhuang Du, Dikang Du, Lingxiao Du, Yulun Du, Yu Fan, Shengjun Fang, Qiulin Feng, Yichen Feng, Garimugai Fu, Kelin Fu, Hongcheng Gao, Tong Gao, Yuyao Ge, Shangyi Geng, Chengyang Gong, Xiaochen Gong, Zhuoma Gongque, Qizheng Gu, Xinran Gu, Yicheng Gu, Longyu Guan, Shuhao Guan, Yuanying Guo, Xiaoru Hao, Dailan He, Tianhong He, Weiran He, Wenyang He, Yibo He, Yunjia He, Chao Hong, Hao Hu, Jiaxi Hu, Yangyang Hu, Zhenxing Hu, Ke Huang, Ruiyuan Huang, Weixiao Huang, Zhiqi Huang, Chaobo Jia, Tao Jiang, Zhejun Jiang, Xinyi Jin, Yu Jing, Guokun Lai, Aidi Li, C. Li, Cheng Li, Fang Li, Guanghe Li, Guanyu Li, Haitao Li, Haoyang Li, Jia Li, Jingwei Li, Junxiong Li, Lincan Li, Mo Li, Weihong Li, Wentao Li, Xinhang Li, Xinhao Li, Yang Li, Yanhao Li, Yiwei Li, Yuxiao Li, Zhaowei Li, Zhaoxi Li, Zheming Li, Weilong Liao, Jiawei Lin, Xiaohan Lin, Yibo Lin, Zhishan Lin, Zichao Lin, Cheng Liu, Chenyu Liu, Hongzhang Liu, Liang Liu, Shaowei Liu, Shudong Liu, Shuran Liu, Tianwei Liu, Tianyu Liu, Weizhou Liu, Xiangyan Liu, Yangyang Liu, Yanming Liu, Yibo Liu, Yuanxin Liu, Zhengying Liu, Zhongnuo Liu, Enzhe Lu, Haoyu Lu, Zhiyuan Lu, G. Luo, Junyu Luo, Tongxu Luo, Yashuo Luo, Long Ma, Shaoguang Mao, Yuan Mei, Xin Men, Fanqing Meng, Zhiyong Meng, Yibo Miao, Minqing Ni, Kun Ouyang, Siyuan Pan, Bo Pang, Yuchao Qian, Ruoyu Qin, Zeyu Qin, Jiezhong Qiu, Bowen Qu, Zeyu Shang, Youbo Shao, Tianxiao Shen, Zhennan Shen, Juanfeng Shi, Lidong Shi, Shengyuan Shi, Feifan Song, Pengwei Song, Tianhui Song, Xiaoxi Song, Hongjin Su, Jianlin Su, Zhaochen Su, Lin Sui, Jinsong Sun, Junyao Sun, Tongyu Sun, Flood Sung, Yunpeng Tai, Chuning Tang, Heyi Tang, Xiaojuan Tang, Zhengyang Tang, Jiawen Tao, Shiyuan Teng, Chaoran Tian, Pengfei Tian, Bowen Wang, Chensi Wang, Chuang Wang, Congcong Wang, Dingkun Wang, Dinglu Wang, Dongliang Wang, Feng Wang, Hailong Wang, Haiming Wang, Hao Wang, Hengzhi Wang, Huaqing Wang, Hui Wang, Jiahao Wang, Jinhong 13 On-Policy Visual Evidence Distillation Wang, Jiuzheng Wang, Kaixin Wang, Linian Wang, Qibin Wang, Shengjie Wang, Shuyi Wang, Si Wang, Wei Wang, Xiaochen Wang, Xinyuan Wang, Yao Wang, Yejie Wang, Yipu Wang, Yiqin Wang, Yucheng Wang, Yuzhi Wang, Zhaoji Wang, Zhaowei Wang, Zhengtao Wang, Zhexu Wang, Zifan Wang, Zihan Wang, Zizhe Wang, Chu Wei, Ming Wei, Chuan Wen, Zichen Wen, Chengjie Wu, Haoning Wu, Junyan Wu, Rucong Wu, Wenhao Wu, Yuefeng Wu, Yuhao Wu, Yuxin Wu, Zijian Wu, Chenjun Xiao, Jin Xie, Xiaotong Xie, Yuchong Xie, Bowei Xing, Boyu Xu, Jianfan Xu, Jing Xu, Jinjing Xu, L. H. Xu, Lin Xu, Suting Xu, Weixin Xu, Xinbo Xu, Xinran Xu, Yangchuan Xu, Yichang Xu, Yuemeng Xu, Zelai Xu, Ziyao Xu, Junjie Yan, Yuzi Yan, Guangyao Yang, Hao Yang, Junwei Yang, Kai Yang, Ningyuan Yang, Xiaofei Yang, Xinlong Yang, Xinyu Yang, Ying Yang, Yi Yang, Yi Yang, Zhen Yang, Zhilin Yang, Zonghan Yang, Haotian Yao, Dan Ye, Haoran Ye, Wenjie Ye, Zhuorui Ye, Peng Yebo, Bohong Yin, Chengzhen Yu, Longhui Yu, Tao Yu, Tianxiang Yu, Enming Yuan, Mengjie Yuan, Xiaokun Yuan, Yang Yue, Weihao Zeng, Dunyuan Zha, Haobing Zhan, Dehao Zhang, Hao Zhang, Jin Zhang, Puqi Zhang, Qiao Zhang, Rui Zhang, Xiaobin Zhang, Xiaoyun Zhang, Y. Zhang, Yadong Zhang, Yangkun Zhang, Yichi Zhang, Yizhi Zhang, Yongting Zhang, Yu Zhang, Yushun Zhang, Yutao Zhang, Yutong Zhang, Zheng Zhang, Chenguang Zhao, Feifan Zhao, Jinxiang Zhao, Shuai Zhao, Xiangyu Zhao, Xuanle Zhao, Yikai Zhao, Zijia Zhao, Huabin Zheng, Ruihan Zheng, Shaojie Zheng, Tengyang Zheng, Junfeng Zhong, Longguang Zhong, Weiming Zhong, M. Zhou, Runjie Zhou, Xinyu Zhou, Zaida Zhou, Jinguo Zhu, Liya Zhu, Xinhao Zhu, Yuxuan Zhu, Zhen Zhu, Jingze Zhuang, Weiyu Zhuang, Ying Zou, and Xinxing Zu. Kimi K2.5: Visual Agentic Intelligence, 2026. URL https://arxiv.org/abs/2602.02276. Xin Lai, Junyi Li, Wei Li, Tao Liu, Tianjian Li, and Hengshuang Zhao.Mini- o3:Scaling Up Reasoning Patterns and Interaction Turns for Visual Search. InInternationalConferenceonLearningRepresentations,p.76722–76746,2026. URLhttps://proceedings.iclr.c/paper_files/paper/2026/file/ 7c40c5050bd029a3ea7f8b01412f735-Paper-Conference.pdf. Pengyu Li, Zhitao Gao, Lingling Zhang, Muye Huang, Yuanming Li, Zesheng Yang, Fangzhi Xu, and Jun Liu. Visual-OPSD: Cross-Modal On-Policy Self-Distillation for Efficient Unified Multimodal Reasoning, 2026a. URL https://arxiv.org/abs/2606.18974. Yijiang Li, Yijun Liang, Yunjie Tian, Bingyang Wang, Ke Zhang, Zhenfei Yin, Di Fu, Philip Torr, and Nuno Vasconcelos. Self-Supervised Visual On-Policy Distillation, 2026b. URL https://arxiv.org/abs/2608.14144. Ruiqi Liu, Xiaolei Lv, Gengsheng Li, Ximo Zhu, Zhiheng Wang, Zhengbo Zhang, Junkai Chen, Zhiheng Li, Bo Li, Jun Gao, and Shu Wu. Visual-Advantage On-Policy Distillation for Vision- Language Models, 2026a. URL https://arxiv.org/abs/2605.21924. Zhining Liu, Ziyi Chen, Hui Liu, Chen Luo, Xianfeng Tang, Suhang Wang, Jingying Zeng, Zhenwei Dai, Zhan Shi, Tianxin Wei, Hanqing Lu, Benoit Dumoulin, and Hanghang Tong. Seeing but not believing: Probing the disconnect between visual attention and an- swer correctness in VLMs. In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (eds.), International Conference on Learning Representations, volume 2026, p. 72849–72876, 2026b.URLhttps://proceedings.iclr.c/paper_files/paper/ 2026/file/76818d8d85e05e45ce3a16a8468619d1-Paper-Conference.pdf. Pan Lu, Hritik Bansal, Tony Xia, Jiacheng Liu, Chunyuan Li, Hannaneh Hajishirzi, Hao Cheng, Kai-Wei Chang, Michel Galley, and Jianfeng Gao. MathVista: Evaluating mathematical reason- ing of foundation models in visual contexts. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (eds.), International Conference on Learning Representations, volume 2024, p. 23439–23554, 2024. URLhttps://proceedings.iclr.c/paper_files/paper/ 2024/file/663bce02a0050c4a11f1eb8a7f1429d3-Paper-Conference.pdf. 14 On-Policy Visual Evidence Distillation Zixian Ma, Jianguo Zhang, Zhiwei Liu, Jieyu Zhang, Juntao Tan, Manli Shu, Juan Carlos Niebles, Shelby Heinecke, Huan Wang, Caiming Xiong, Ranjay Krishna, and Silvio Savarese. LATTE: Learning to Think with Vision Specialists. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 11192–11229, 2025. doi: 10.18653/v1/ 2025.emnlp-main.564. URL https://aclanthology.org/2025.emnlp-main.564/. Ahmed Masry, Mohammed Saidul Islam, Mahir Ahmed, Aayush Bajaj, Firoz Kabir, Aaryaman Kartha, Md Tahmid Rahman Laskar, Mizanur Rahman, Shadikur Rahman, Mehrad Shahmo- hammadi, Megh Thakkar, Md Rizwan Parvez, Enamul Hoque, and Shafiq Joty. ChartQAPro: A more diverse and challenging benchmark for chart question answering. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (eds.), Findings of the Association for Computational Linguistics: ACL 2025, p. 19123–19151, Vienna, Austria, July 2025. Association for Computational Linguistics. ISBN 979-8-89176-256-5. doi: 10.18653/v1/2025. findings-acl.978. URL https://aclanthology.org/2025.findings-acl.978/. Minesh Mathew, Viraj Bagal, Rubèn Tito, Dimosthenis Karatzas, Ernest Valveny, and C.V. Jawahar.InfographicVQA.In Proceedings of the IEEE/CVF Win- ter Conference on Applications of Computer Vision (WACV), p. 1697–1706, January 2022. URLhttps://openaccess.thecvf.com/content/WACV2022/html/Mathew_ InfographicVQA_WACV_2022_paper.html. OpenAI.GPT-4o System Card, August 2024.URLhttps://cdn.openai.com/ gpt-4o-system-card.pdf. OpenAI.Thinking with images, April 2025.URLhttps://openai.com/index/ thinking-with-images/. Ji Qi, Ming Ding, Weihan Wang, Yushi Bai, Qingsong Lv, Wenyi Hong, Bin Xu, Lei Hou, Juanzi Li, Yuxiao Dong, and Jie Tang. CogCoM: A Visual Language Model with Chain-of-Manipulations Reasoning. In International Conference on Learning Representations, p. 11090–11110, 2025. URLhttps://proceedings.iclr.c/paper_files/paper/ 2025/file/1dcee1cd6890ab7fcdf173ec10526da9-Paper-Conference.pdf. Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026. URLhttps: //qwen.ai/blog?id=qwen3.5. Alex Su, Haozhe Wang, Weiming Ren, Fangzhen Lin, and Wenhu Chen. Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning. In Advances in Neural Information Processing Systems, volume 38, p. 8222–8251, 2025. doi: 10. 52202/085713-0280. URLhttps://proceedings.neurips.c/paper_files/paper/ 2025/file/0c38f54740062529a4117a04b583f3c-Paper-Conference.pdf. Haoxiang Sun, Zhihang Yi, Langxuan Deng, Yuhao Zhou, Peiqi Jia, Jian Zhao, Li Yuan, Jiancheng Lv, and Tao Wang. V-Zero: Answer-Label-Free On-Policy Distillation with Contrastive Evidence Gating for Fine-Grained Visual Reasoning, 2026. URLhttps://arxiv.org/abs/ 2606.25319. Kanghui Tian, Siyuan Liu, Ziang Yan, Sheng Xia, Shuai Dong, and Yi Wang. ViCuR: Visual cues as recoverable privilege for multimodal on-policy distillation, 2026. URLhttps://arxiv. org/abs/2606.05718. Haochen Wang, Xiangtai Li, Zilong Huang, Anran Wang, Jiacong Wang, Tao Zhang, Jiani Zheng, Sule Bai, Zijian Kang, Jiashi Feng, Zhuochen Wang, and Zhaoxiang Zhang.Traceable evidence enhanced visual grounded reasoning:Evaluation and method.In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust 15 On-Policy Visual Evidence Distillation (eds.), International Conference on Learning Representations, volume 2026, p. 148769– 148794, 2026.URLhttps://proceedings.iclr.c/paper_files/paper/2026/ file/f09ab24a37ae2664b3d923cdb0319b5-Paper-Conference.pdf. Weiyun Wang, Zhangwei Gao, Lixin Gu, Hengjun Pu, Long Cui, Xingguang Wei, Zhaoyang Liu, Linglin Jing, Shenglong Ye, Jie Shao, Zhaokai Wang, Zhe Chen, Hongjie Zhang, Ganlin Yang, Haomin Wang, Qi Wei, Jinhui Yin, Wenhao Li, Erfei Cui, Guanzhou Chen, Zichen Ding, Changyao Tian, Zhenyu Wu, Jingjing Xie, Zehao Li, Bowen Yang, Yuchen Duan, Xuehui Wang, Zhi Hou, Haoran Hao, Tianyi Zhang, Songze Li, Xiangyu Zhao, Haodong Duan, Nianchen Deng, Bin Fu, Yinan He, Yi Wang, Conghui He, Botian Shi, Junjun He, Yingtong Xiong, Han Lv, Lijun Wu, Wenqi Shao, Kaipeng Zhang, Huipeng Deng, Biqing Qi, Jiaye Ge, Qipeng Guo, Wenwei Zhang, Songyang Zhang, Maosong Cao, Junyao Lin, Kexian Tang, Jianfei Gao, Haian Huang, Yuzhe Gu, Chengqi Lyu, Huanze Tang, Rui Wang, Haijun Lv, Wanli Ouyang, Limin Wang, Min Dou, Xizhou Zhu, Tong Lu, Dahua Lin, Jifeng Dai, Weijie Su, Bowen Zhou, Kai Chen, Yu Qiao, Wenhai Wang, and Gen Luo. InternVL3.5: Advancing Open-Source Multimodal Models in Versatility, Reasoning, and Efficiency, 2025a. URL https://arxiv.org/abs/2508.18265. Wenbin Wang, Liang Ding, Minyan Zeng, Xiabin Zhou, Li Shen, Yong Luo, Wei Yu, and Dacheng Tao. Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models. Proceedings of the AAAI Conference on Artificial Intelligence, 39(8):7907–7915, 2025b. doi: 10.1609/aaai.v39i8.32852. URLhttps: //ojs.aaai.org/index.php/AAAI/article/view/32852. Shaohang Wei, Zikun Su, Feifan Song, Wen Luo, Wei Li, Guangyue Peng, and Houfeng Wang. Verifier-Induced Support Reshaping in On-Policy Optimization, 2026. URLhttps://arxiv. org/abs/2608.00220. Mingyuan Wu, Jingcheng Yang, Jize Jiang, Meitang Li, Kaizhuo Yan, Hanchao Yu, Minjia Zhang, ChengXiang Zhai, and Klara Nahrstedt.VTool-R1:VLMs Learn to Think with Images via Reinforcement Learning on Multimodal Tool Use.In International Conference on Learning Representations,p. 78298–78319, 2026.URLhttps://proceedings.iclr.c/paper_files/paper/2026/file/ 7e6f445a74cdb71931aac64f1e3f49c9-Paper-Conference.pdf. Penghao Wu and Saining Xie. V*: Guided visual search as a core mechanism in multimodal LLMs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 13084–13094, 2024. URLhttps://openaccess.thecvf.com/content/CVPR2024/ html/Wu_V_Guided_Visual_Search_as_a_Core_Mechanism_in_Multimodal_ CVPR_2024_paper.html. Weiye Xu, Jiahao Wang, Weiyun Wang, Zhe Chen, Wengang Zhou, Aijun Yang, Lewei Lu, Houqiang Li, Xiaohua Wang, Xizhou Zhu, Wenhai Wang, Jifeng Dai, and Jin- guo Zhu.VisuLogic: A benchmark for evaluating visual reasoning in multi-modal large language models.In C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (eds.), International Conference on Learning Representations, volume 2026, p. 25966–26003, 2026. URLhttps://proceedings.iclr.c/paper_files/paper/ 2026/file/2bbc73b3d3c2de43743ce2d82c8f3d7d-Paper-Conference.pdf. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jing Zhou, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, 16 On-Policy Visual Evidence Distillation Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. Qwen3 Technical Report, 2025. URL https://arxiv.org/abs/2505.09388. Tianzhu Ye, Li Dong, Xun Wu, Shaohan Huang, and Furu Wei. On-policy context distillation for language models, 2026. URL https://arxiv.org/abs/2602.12275. Weichen Yu, Xiaomin Li, Yizhou Zhao, Xiaoze Liu, Ruowang Zhang, Haixin Wang, Yinyi Luo, Chen Henry Wu, Gaurav Mittal, Matt Fredrikson, and Yu Hu. Multi-rollout on-policy distillation via peer successes and failures, 2026. URLhttps://arxiv.org/abs/2605. 12652. Qianhao Yuan, Jie Lou, Xing Yu, Hongyu Lin, Le Sun, Xianpei Han, and Yaojie Lu. Vision-OPD: Learning to see fine details for multimodal LLMs via on-policy self-distillation, 2026. URL https://arxiv.org/abs/2605.18740. Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models, 2023. URL https://arxiv.org/abs/2308.01825. Kangning Zhang, Yixing Li, Shuai Shao, Qingyao Li, Zhengxi Lu, Zhiyuan Yao, Jianghao Lin, Wenxiang Jiao, Yuan Lu, Weiwen Liu, Weinan Zhang, and Yong Yu. VAD: Attributing visual evidence for target reconstruction in multimodal on-policy distillation, 2026a. URL https://arxiv.org/abs/2607.28590. Renrui Zhang, Dongzhi Jiang, Yichi Zhang, Haokun Lin, Ziyu Guo, Pengshuo Qiu, Aojun Zhou, Pan Lu, Kai-Wei Chang, Yu Qiao, Peng Gao, and Hongsheng Li. MATHVERSE: Does your multi-modal LLM truly see the diagrams in visual math problems? In Computer Vision – ECCV 2024, p. 169–186. Springer Nature Switzerland, 2025. doi: 10.1007/978-3-031-73242-3_10. URL https://link.springer.com/chapter/10.1007/978-3-031-73242-3_10. Yi-Fan Zhang, Xingyu Lu, Shukang Yin, Chaoyou Fu, Wei Chen, Xiao Hu, Bin Wen, Kaiyu Jiang, Changyi Liu, Tianke Zhang, Haonan Fan, Kaibing Chen, Jiankang Chen, Hao- jie Ding, Kaiyu Tang, Zhang Zhang, Liang Wang, Fan Yang, Tingting Gao, and Guorui Zhou. Thyme: Think Beyond Images. In International Conference on Learning Representations, p. 79536–79572, 2026b. URLhttps://proceedings.iclr.c/paper_files/paper/ 2026/file/8080912b919ad0344c0d34e68983dfcb-Paper-Conference.pdf. Siyan Zhao, Zhihui Xie, Mengchen Liu, Jing Huang, Guan Pang, Feiyu Chen, and Aditya Grover. Self-Distilled Reasoner: On-Policy Self-Distillation for Large Language Models. In International Conference on Machine Learning, 2026. URLhttps://icml.c/virtual/2026/poster/ 64784. Binbin Zheng, Zijun Xie, Guanqun Zhao, Enlei Gong, Xing Ma, Xiaoliang Fu, and Zeyu Chen. Group-reflective self-distillation for agentic reinforcement learning, 2026a. URLhttps: //arxiv.org/abs/2607.28076. Ziwei Zheng, Michael Yang, Jack Hong, Chenxiao Zhao, Guohai Xu, Le Yang, Chao Shen, and Xing Yu.DeepEyes: Incentivizing “Thinking with Images” via Reinforce- ment Learning. In International Conference on Learning Representations, p. 126775–126798, 2026b.URLhttps://proceedings.iclr.c/paper_files/paper/2026/file/ cdb347d7516d52a9280ea9d6708911f6-Paper-Conference.pdf. 17 On-Policy Visual Evidence Distillation Appendix AExtended Method Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 A.1 Training Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 A.2 Supervision Scope . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 A.3 Teacher Conditioning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 A.4 A Visual-Evidence Reflection Example . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .21 A.5 Critic Feedback . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22 A.6 Critic Prompt and Output Format . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .24 A.7 Training Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .25 BExperimental Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26 B.1Baseline Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .26 B.2Model Labels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27 B.3Training and Evaluation Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27 B.4Token-Impact Sparsity in Figure 4 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .27 B.5Error Attribution in Figure 1 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28 B.6Critic Consistency Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .28 CAdditional Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31 C.1 Per-Benchmark Component Ablation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .31 C.2 Token-Impact Interventions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .32 C.3 Sensitivity to the High-Impact Token Fraction . . . . . . . . . . . . . . . . . . . . . . . . . . . .33 C.4 Response Length and Tool Use on Perception Benchmarks . . . . . . . . . . . . . . . . . .35 C.5 Training-Time Reflection Coverage . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .37 C.6 Offline Comparison with Anchor-Only Context . . . . . . . . . . . . . . . . . . . . . . . . . . .37 D Proofs and Derivations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38 D.1 Reflection-Gradient Proof . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38 D.2 Gradient Weights under Grouped Distillation . . . . . . . . . . . . . . . . . . . . . . . . . . . .39 ECase Study . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40 E.1Errors in Acquiring, Reading, and Using Visual Evidence . . . . . . . . . . . . . . . . . . .40 E.2Different Errors with the Same Evidence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .41 E.3Different Errors on the Same Plot . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42 E.4Token Impact Visualizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .43 FEvaluation Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 F.1Visual Perception . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 F.2Mathematical and Logical Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 F.3General Multimodal Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .44 18 Sampling from student Reflection via critic Teacher evaluation Reweighting Gradient update On-Policy Visual Evidence Distillation A Extended Method Details A.1 Training Procedure Algorithm 1 summarizes training from student trajectories, critic feedback, and reflection construction to student updates. Algorithms 2–4 detail answer verification, evidence-state collection, and reflection assembly, respectively. Algorithm 1 REVUE: Training Procedure Require: DataD, trainable student π θ , frozen teacher q φ , fixed criticC Require: Rollouts per query M , support size K, high-impact fraction α, group weight λ Ensure: Updated student π θ All tokenwise operations use t∈T i . Superscript Supp denotes renormalization over Supp; impacts use full-vocabulary probabilities. 1: for each training minibatchX ⊂D do 2:for each (x,y ⋆ )∈X do 3:G x ← ROLLOUT(π θ ,x,M ) 4:c x ← VERIFYANSWERS(G x ,y ⋆ );E x ← OBSERVEDSTATES(G x ) 5: J x ←C(x,y ⋆ ,G x , c x ,E x ) 6: R i M i=1 ← CHECKANDASSEMBLE(J x ,E x , c x ) 7:end for 8: LetG collect retained trajectories acrossX withT i ̸=∅ 9:for each τ i ∈G do 10:q 0 i,t ← q φ (·| h i,t ); q ∗ i,t ← q 0 i,t ; s i ← unavailable 11:ifR i ̸=∅ then 12:q R i,t ← q φ (·| h i,t ,R i ) 13:if evaluation succeeds and completion tokens align then 14:q ∗ i,t ← q R i,t 15:Attempt s i,t ← logq R i,t (y i,t )− logq 0 i,t (y i,t ) 16:end if 17:end if 18:p i,t ← π θ (·| h i,t ); Supp i,t ← TopK(q ∗ i,t ) 19:ℓ i,t ←D KL p Supp i,t i,t (q ∗ i,t ) Supp i,t 20:Compute ℓ i using Equation 11 21:end for 22: L← mean τ i ∈G ℓ i ▷ 0 ifG =∅ 23: θ ← OPTIMIZERSTEP(θ,∇ θ L) 24: Refresh the rollout policy from θ 25: end for Letu i = 1indicate a valid reflection, a successful teacher evaluation, and aligned completion tokens; otherwise setu i = 0. Letb i = 1indicate that impacts were also computed successfully and at least two supervised positions exist; otherwise setb i = 0. An impact-computation failure retains the adopted reflection-conditioned target and replaces group weighting with uniform averaging. q ∗ i,t = ( q R i,t , u i = 1, q 0 i,t , u i = 0. (10) Letℓ i,t =D KL (p Supp i,t i,t ∥(q ∗ i,t ) Supp i,t )denote the per-position KL term in Equation 8. Each trajectory 19 On-Policy Visual Evidence Distillation with a nonempty supervised set has loss ℓ i = λ |T i,High | X t∈T i,High ℓ i,t + 1− λ |T i,Low | X t∈T i,Low ℓ i,t , b i = 1, 1 |T i | X t∈T i ℓ i,t ,b i = 0. (11) Impacts are sorted in descending order with earlier positions breaking ties, and the high-impact group contains the first⌈α|T i |⌉positions. The final loss averages retained trajectories equally as in Equation 8. A.2 Supervision Scope Supervised positions cover student-generated reasoning, code, and final answers across all interaction rounds, with other labels set to−100: m i,t = 1label i,t ̸=−100, T i =t : m i,t = 1, |T i | = X t m i,t .(12) Prompts, tool-returned text, image placeholders, sandbox boundary markers, and interceptor- appended suffixes are excluded from supervision targets. Tool observations remain in the input history for subsequent student predictions. Impact ranking, group sizes, and loss means use only positions inT i . A.3 Teacher Conditioning The base teacher scores the fixed student completion token by token, conditioned on the original task prompt, image, and preceding student interaction history. The reflection-conditioned teacher appends reflection text to the first user message and inserts supporting images after the original image and before the student’s trajectory images. The re-encoded completion must match the student’s sequence in length and in every token ID. Reflection is generated after the rollout group is complete, so supporting evidence may come from later steps of the target trajectory or from a sibling trajectory. This retrospective information is used only for teacher supervision, while the student prefix retains its original visible history. The reference answer informs critic diagnosis, and the resulting visual fact, grounding rule, or discrepancy may convey answer semantics. Comparing the two teacher evaluations measures the effect of this added context, including both text and supporting images. Impact uses full-vocabulary normalized log probabilities at the sampled token and is detached after computation. RKL renormalizes both distributions over the adopted teacher ’s top-Kcandi- dates, without forcibly adding the sampled token. Impact ranks and weights training positions, while the loss at each position supervises the entire support distribution (Appendix D.1). 20 On-Policy Visual Evidence Distillation A.4 A Visual-Evidence Reflection Example Task and visual evidence What does the central vertical line with the notation e iλn in the figure likely represent? A. A series of discrete energy levels in a quantum system B. A variable resistor in an electrical circuit C. The eigenvalues of a Hamiltonian in a solid-state physics model D. The time evolution operator in a quantum mechanical system Original image Correct attempt Post-tool reasoning The central vertical line with the notation e iλn typically represents a phase factor in quantum mechanics. This phase factor often arises from the time evolution of quantum states under a Hamiltonian, indicating how a state evolves over time according to the principles of quantum mechanics. Answer: D Incorrect attempt Post-tool reasoning The figure shows a ladder diagram with levels labeled by λ, typically representing eigenvalues in physics, specifically in the context of quantum mechanics or solid-state physics where λ could represent different energy states. Answer: C Both attempts received the same tool crop, shown below. Reflection supplied to the teacher for the incorrect attempt Anchor Supporting tool crop Acquire Target: central vertical line notation. Read Observed value: e iλn The central vertical line is labeled with the mathematical expression e iλn , positioned between U 0 −1 and U 0 blocks. Ground In quantum mechanics, the expression e iλn (where λ relates to eigenvalues/energy and n to time/steps) represents the time evolution operator acting on a state, especially when flanked by unitary transformations U 0 and U 0 −1 . Break Point First broken link: Ground Discrepancy The rollout correctly identifies the notation and the ladder diagram context but incorrectly maps the visual evidence to 'eigenvalues of a Hamiltonian' (Option C) instead of recognizing the exponential form as a time evolution operator (Option D). Figure 6: A concrete visual-evidence reflection supplied to the teacher. The mixed group contains seven correct attempts and one incorrect attempt; one of each is shown with its post- tool reasoning and answer. Both attempts receive identical tool crops. The Anchor uses the incorrect attempt’s own crop, and the Break Point records the critic’s Ground diagnosis. 21 On-Policy Visual Evidence Distillation A.5 Critic Feedback VERIFYANSWERS. Algorithm 2 computes binary answer labels, which classify each rollout group as all-correct, mixed, or all-wrong. When deterministic matching does not accept an answer, the answer judge evaluates only the question, extracted student answer, and reference answer as text. Algorithm 2 VERIFYANSWERS: Answer verification Require: Same-query trajectoriesG x , reference answer y ⋆ Ensure: Binary correctness labels c x 1: q ← shared question inG x ; c x ← 0 2: for each trajectory i∈G x do 3: a i ← text inside <answer>...</answer> 4:if a i is missing or empty then continue 5: ̃a i ← strip answer prefixes from a i 6:if strict mathematical matching or symbolic verification accepts ( ̃a i ,y ⋆ ) then 7:c i ← 1 8:else 9:c i ← ANSWERJUDGE(q,a i ,y ⋆ )∈0, 1▷ 0 if unavailable 10:end if 11: end for 12: return c x OBSERVEDSTATES.Algorithm 3 collects the original-image state and recorded post-tool states so the critic can associate visual evidence with a specific interaction step. Each state contains the images available up to that point, with multiple images from one tool call belonging to the same state. The countsn tool (e)andn code (e)record cumulative tool rounds and code tokens, respectively. Algorithm 3 OBSERVEDSTATES: Evidence-state collection Require: Same-query trajectoriesG x , including the original image and interaction records Ensure: Evidence statesE x with provenance, available images, and cumulative costs 1: E x ←original-image state; owner∅, n tool = n code = 0 2: for each recorded post-tool state e of trajectory i∈G x do 3:Add (e,i, images available at e,n tool (e),n code (e)) toE x 4: end for 5: returnE x CHECKANDASSEMBLE. The critic receives the query, reference answer, original image, correctness-labeled trajectories, and evidence states. Up to eight tool images are supplied per group, prioritizing correct trajectories. For each sufficient statee, the critic is asked to identify a smallest supporting image subsetU e that determines the required visual fact using only images available at that state. It also returns shared reading information (query slot, observed value, and visual fact), a grounding rule, and per-trajectory diagnosesB i = (σ i ,δ i ). Algorithm 4 validates required fields, state and image references, and consistency between diagnoses and answer labels. Only mixed groups reconcile inconsistent stage labels before validation using answer correctness and the presence of an own sufficient state, while retaining the discrepancy text. All-correct groups require CORRECT labels with empty discrepancies. Anchor selection prioritizes sufficient evidence already acquired by the trajectory and chooses among states by lexicographic cost: c(e) = n tool (e), n code (e), |U e |, id(e) ,(13) 22 On-Policy Visual Evidence Distillation LetC i contain sufficient post-tool states owned by trajectoryi, and letC + x contain sufficient original-image states and sufficient states from answer-correct trajectories. Complete reflections require a nonempty group reference setC + x , after which each trajectory selects evidence as e ⋆ i = arg min e∈C i c(e), C i ̸=∅, arg min e∈C + x c(e), C i =∅. (14) If the selected state requires no tool use, assembly adds no image because the teacher context already contains the original. Algorithm 4 CHECKANDASSEMBLE: Reflection assembly Require: Critic outputJ x , recorded statesE x , correctness labels c x Ensure: Per-trajectory reflectionsR i , with∅ denoting unavailable reflection 1: R i ←∅ for all i; ifJ x is missing then returnR i 2: if the group is mixed then reconcile stages with c x and own sufficient states 3: ifJ x is inapplicable or fails validation then returnR i 4: B i ← (σ i ,δ i ) from the checked diagnoses, for all i 5: if all answers are wrong then return(∅,B i ) i 6: ResolveJ x ’s sufficient states inE x to formC i andC + x 7: ifC + x =∅ then returnR i 8: for each trajectory i do 9:Select e ⋆ i using Equation 14 10: A i ← (e ⋆ i ,U e ⋆ i ,J x .reading,J x .grounding) 11: R i ← (A i ,B i ) 12: end for 13: returnR i 23 On-Policy Visual Evidence Distillation A.6 Critic Prompt and Output Format The box below reproduces the critic system prompt used for mixed groups. The all-correct prompt requiresCORRECTwith an empty discrepancy for every trajectory. The all-wrong prompt disallowsCORRECTand permits an empty set of sufficient evidence states. It also requires a sufficient state for eachREADorGROUNDdiagnosis, preserves verifier-label conflicts, and returns one diagnosis per trajectory. Critic system prompt: mixed groups You are a training-time multimodal evidence-chain analyst for a think-with-image VQA policy. You receive one task and a group of verifier-labeled CORRECT and INCORRECT student rollouts. A rollout contains chronological policy reasoning, free-form Python code, sandbox text/numeric outputs, zero/ one/multiple images returned by each sandbox interaction, and a final answer. One Python code block may return MULTIPLE images. They belong to the SAME sandbox interaction and may jointly form multi-region evidence. A rollout may have multiple sandbox interactions. Later interactions may correct earlier visual observations. The CORRECT/INCORRECT label is defined only by the supplied VQA correctness label. Trust CORRECT rollouts as correct final solutions. Your task is to identify the visual evidence chain: ACQUIRE: Which actually observed Evidence States already contain sufficient task-relevant visual evidence? READ: What is the minimum atomic visual fact needed by the question? GROUND: How does that visual fact map to the answer semantics? EVIDENCE STATE: ORIGINAL_STATE contains ORIGINAL_IMAGE. Every other state is the state immediately after one real sandbox interaction and lists all images currently available. A state is SUFFICIENT iff some subset of its available images already contains enough visual evidence to determine the task-relevant visual fact without needing a later observation. For each sufficient state, return the SMALLEST supporting image subset. CLOSED-SET RULE: Never invent an image, state, crop, coordinate, code, or synthetic view. Only use supplied state IDs and image IDs. READ: Do not use a predefined taxonomy. Return: - query_slot - observed_value - one atomic visual_fact GROUND: Return one concise evidence-to-answer semantic rule. Do not state only an answer/option letter. ROLLOUT DIAGNOSIS: CORRECT rollout -> stage=CORRECT. For INCORRECT rollouts return the FIRST broken interface: ACQUIRE: No state belonging to this rollout is sufficient. 24 On-Policy Visual Evidence Distillation Critic system prompt: mixed groups (continued) READ: A sufficient state exists, but the relevant visual fact is misread. GROUND: A sufficient state exists and the fact is read correctly, but final answer semantics contradict/mis-map that fact. OTHER: Failure is outside this visual evidence chain. Return JSON only according to schema. The output schema is supplied as a separate structured-output constraint rather than as part of the user message. The table retains its fields, types, and shared constraints. Critic output schema: fields and constraints Field structureType applicableboolean sufficient_statesobject[] state_idstring support_image_idsstring[] readingobject query_slotstring observed_valuestring visual_factstring groundingobject rulestring rollout_diagnosesobject[] rollout_idstring stagestring (enum) discrepancystring Allowed values for rollout_diagnoses[].stage: CORRECT, ACQUIRE, READ, GROUND, OTHER The root is an object; [] denotes an array, with item fields indented below it. All fields are required; additional properties are disallowed for every object; strict = true. A.7 Training Configuration The full method described here usesM = 8trajectories per query,K = 32teacher candidates, a high-impact fractionα = 0.2, and group weightλ = 0.5. Teacher parameters, teacher supports, impacts, and group assignments remain fixed during student updates. Training uses the pure OPD loss, without a GRPO reward term or an additional reference-policy KL. Answer correctness is used for group classification and critic diagnosis, not as an advantage or a trajectory-level loss weight. For the Thyme-based Qwen2.5-VL-7B setting, the student starts from an SFT checkpoint, and the frozen teacher uses step 2150 of the corresponding RL expert. The separate critic uses Qwen3.5-397B-A17B-FP8 with temperature zero and a 2048-output-token limit per call to generate JSON-schema-constrained feedback. Critic runtime and auxiliary resources.Training uses eight NVIDIA H20 GPUs (96 GB each), and the critic runs through a shared vLLM service on a separate machine with another eight H20 GPUs (96 GB each). This deployment leaves the training GPUs available for rollouts and allows critic requests to overlap with independent training-side computations, such as reward and answer-correctness evaluation. The critic is used only during training and is not required at inference. 25 On-Policy Visual Evidence Distillation Each critic request processes one group of eight trajectories, giving 20 requests per optimizer step. A test request with one image and eight trajectories took 5.8 s and produced 664 output tokens. Assuming this latency is maintained with four concurrent requests, five waves give a nominal critic service time of approximately 29 s per step, excluding additional queueing and retries. This estimate is 10.8% of the mean step time of 269 s measured across six training segments. The actual increase in training wall-clock time depends on overlap with independent computations and service scheduling. Extrapolating to 1,075 steps gives approximately 8.7 hours of nominal critic service time. At- tributing all eight service GPUs to this run during these intervals gives a resource estimate of approximately 70 GPU-hours. This estimate accounts for nominal service time; total server allocation depends on service uptime and sharing. B Experimental Details B.1 Baseline Details All baselines start from Thyme-SFT cold-start checkpoints of Qwen2.5-VL-7B or InternVL3.5-4B- Instruct. OPD variants use the corresponding Thyme-RL Expert as teacher; RFT uses demon- strations generated by the same expert. This keeps student initializations and teacher sources consistent across comparisons. Vanilla OPD (Agarwal et al., 2024) provides the standard on-policy distillation reference for evaluating reflection-based supervision. The RL Expert supervises student-generated prefixes token by token, without ground-truth answers, privileged crops, or evidence reflections. Rejection sampling fine-tuning (RFT) (Yuan et al., 2023) compares offline demonstration learning with distillation on student-generated trajectories. For each prompt in the Thyme-RL training set, the corresponding RL Expert samples eight trajectories. We retain answer-correct trajectories for supervised fine-tuning of the same cold-start student. Training uses these fixed demonstrations rather than teacher distributions queried on current student prefixes. GT-Privileged compares ground-truth answers with visual-evidence reflections as teacher privilege. Following the privileged-context design of OPSD (Zhao et al., 2026), we provide the teacher with the Thyme-RL ground-truth answer. We replace its self-teacher with the corresponding RL Expert. The student starts from the same cold-start checkpoint and generates trajectories without access to the privileged answer. Vision-OPD (Yuan et al., 2026) provides a local visual privilege baseline for comparison with reflections that diagnose evidence use. The teacher receives an evidence-centered crop and supervises the student’s own trajectories. We replace its self-teacher with the corresponding RL Expert and use the same cold-start student. With V-Zero (Sun et al., 2026), we compare visual-contrast trajectory weighting with reflection- based supervision. Teacher support for the same student trajectory under relevant and irrelevant visual views determines its distillation weight. The target remains the positive-view teacher distribution. We use the corresponding RL Expert for both views and the same cold-start student. Our comparison with VAD (Zhang et al., 2026a) examines visual target reconstruction as an alternative to direct distillation from a reflection-conditioned teacher. It contrasts teacher predictions with and without visual evidence to estimate visual corrections and reconstruct the distillation target. We replace its initial-model teacher with the corresponding RL Expert and retain the same cold-start student. 26 On-Policy Visual Evidence Distillation B.2 Model Labels In Table 1, Base Model denotes the original model before training in this work. Gemini3.1FL abbreviates Gemini-3.1-Flash-Lite. Qwen2.5-32B, Qwen3-30B-T, and InternVL-38B denote Qwen2.5-VL-32B, Qwen3-VL-30B-A3B-Thinking, and InternVL3.5-38B-Instruct, respectively. B.3 Training and Evaluation Settings We initialize the Qwen2.5-VL-7B student directly from the released Thyme-SFT (Zhang et al., 2026b) checkpoint. 1 We obtain the InternVL3.5-4B cold-start checkpoint by supervised fine- tuning of InternVL3.5-4B-Instruct on Thyme’s released SFT dataset. 2 We train both RL experts from their corresponding cold-start checkpoints by reproducing Thyme-RL on its released RL dataset. 3 Table 6: Training and evaluation settings. ComponentParameterValue OPD objectiveLossReverse KL Vocabulary supportTeacher’s top-32 candidates OPD rolloutsTemperature for code tokens0.0 Temperature for other tokens1.0 Top-p0.9 Top-k50 EvaluationTemperature0.01 Top-p0.001 Top-k1 All experimentsRandom seed42 Training jobHardware8× NVIDIA H20 (96 GB each) Critic serviceSeparate hardware8× NVIDIA H20 (96 GB each) We compute reverse KL after renormalizing teacher and student distributions over the teacher’s top-32 vocabulary candidates at each supervised position. B.4 Token-Impact Sparsity in Figure 4 Figure 4 pools offline rescoring results from 632 trajectories in a five-step early-training archive, comprising 226,726 supervised tokens with valid impact scores. We use the impact definition in Equation 7 and exclude nonsupervised positions and missing scores. The dashed line marks the pooled 80th percentile, approximately s = 0.05524. Tokens at or above this threshold comprise approximately 20% of scored tokens and account for 97.6% of total impact. High and Low in the figure describe the pooled distribution; training selects high-impact positions separately within each trajectory. Histogram heights give the percentage of all scored tokens in each interval. Bins in[0, 2.5)have width 0.05, with the bin containing the 80th percentile split at that threshold. The final bar retains all tokens withs≥ 2.5. The inset plots empirical percentile against impact over the full pooled score range, without smoothing or fitting. 1 Thyme-SFT model: https://huggingface.co/Kwai-Keye/Thyme-SFT 2 Thyme-SFT dataset: https://huggingface.co/datasets/Kwai-Keye/Thyme-SFT 3 Thyme-RL dataset: https://huggingface.co/datasets/Kwai-Keye/Thyme-RL 27 On-Policy Visual Evidence Distillation B.5 Error Attribution in Figure 1 The error-attribution analysis in Figure 1 uses 3,710 questions from V ∗ Bench (Wu & Xie, 2024), HRBench-4K (Wang et al., 2025b), HRBench-8K (Wang et al., 2025b), and MME (Fu et al., 2025). 4 We use GPT-5.6-Luna as the judge for evaluation and error attribution. Four error labels map to the three visual-evidence stages. Acquire combinesfaulty_tool (29.8%) andmissing_tool(7.9%), totaling 37.7%. Read corresponds toperception_error (33.8%). Ground corresponds to hallucination (6.3%). B.6 Critic Consistency Evaluation We evaluate the final training labels of Qwen3.5-397B-A17B-FP8 on 96 groups of eight rollouts. Each group type (all-correct, mixed, and all-wrong) contributes 32 groups. One annotator independently labels one sampled rollout per group before revealing the critic label; all 96 samples receive a label. Agreement measures exact stage matches; unweighted Cohen’sκ adjusts for chance agreement. Table 7: Final critic stage agreement. (a) Human–critic label counts Human\ Critic Correct Acquire Read Ground Other Correct320000 Acquire028000 Read03 2300 Ground00090 Other00001 (b) Pairwise final-label agreement Pairn Agree. (%)κ Qwen (arch.) / rerun74499.2 0.987 Qwen (arch.) / Gemini 73693.2 0.892 Qwen (arch.) / Luna73692.8 0.886 Qwen (rerun) / Gemini 71293.5 0.898 Qwen (rerun) / Luna71293.4 0.896 Gemini / Luna71295.4 0.926 Arch. denotes archived training labels, rerun denotes one fresh evaluation, and n counts valid paired rollouts. Gemini and Luna denote Gemini 3.8 Flash and GPT-5.6 Luna, respectively. Human agreement is 96.88% (κ = 0.957) overall and 95.31% (κ = 0.925) on incorrect rollouts. The three human–critic disagreements occur at the Acquire–Read boundary. All three are from mixed groups: the annotator assigns Read, whereas the critic assigns Acquire. This points to dis- agreement over whether the available evidence is sufficient to support a judgment. Agreement with archived training labels uses 744 valid rollout pairs for a rerun by the same critic and 736 for each of the two external judges (Table 7). These comparisons use valid final stage labels. 4 MME: official dataset repository. 28 On-Policy Visual Evidence Distillation Figure 7: Human annotation interface demo: the upper view presents the sample identifiers, original image, question, and reference answer. The page also provides progress tracking, sample filtering, navigation, and JSONL import and export controls. 29 On-Policy Visual Evidence Distillation Figure 8: Human annotation interface demo: the lower view of the same sample presents the rollout, the initially collapsed judge label, six annotation options, and a notes field. The options comprise five diagnostic labels and a SKIP option for cases that cannot be judged reliably. 30 On-Policy Visual Evidence Distillation C Additional Results This appendix reports per-benchmark component ablations, token-impact interventions, fraction sensitivity, and additional analyses of response length and tool use. C.1 Per-Benchmark Component Ablation Table 8 reports scores on all 11 benchmarks for the three variants in Table 2. Reflection improves most benchmarks, with uneven gains. Adding reflection to Vanilla OPD improves nine of 11 benchmarks and all three category averages. Perception and general-task averages increase by 1.09% and 1.14%, respectively, while math improves by 0.11%. Gains on HRBench 8K (+2.00%) and HallusionBench (+2.54%) cover high-resolution understanding and hallucination detection. TreeBench and MathVerse are the two exceptions, with accuracy decreases of 0.24% and 0.51%, respectively. Impact reweighting improves tasks that do not benefit from reflection alone. Relative to the reflection-only model, reweighting improves seven of eight perception and math benchmarks, while V* Bench remains at 82.20%. TreeBench rises from 37.78% to 41.12% and MathVerse from 45.56% to 47.08%, reversing their declines relative to Vanilla OPD. HRBench 8K and VisualProbe also improve by 1.50% and 0.58%, extending the reweighting gains to tasks that already benefit from reflection. Reweighting introduces a small trade-off on general tasks. The full model scores below the reflection-only model on all three general-task benchmarks, lowering the category average by 0.28%. The decreases are 0.53% on ChartQA-Pro, 0.30% on HallusionBench, and 0.08% on InfographicVQA. All three remain above Vanilla OPD, and the full method improves over Vanilla OPD on all 11 benchmarks. Table 8: Complete component ablation on Qwen2.5-VL-7B. Rows cumulatively add visual- evidence reflection and impact reweighting, and Wtd. Avg. uses the benchmark-sample weighting of Table 1. MathVista and MathVerse denote the Mini and vision-only splits, respec- tively. Higher is better for every metric; gray denotes Vanilla OPD, blue marks REVUE, and bold indicates column-best results, including ties. (a) Perception MethodHR-4KHR-8KV * Tree Bench Visual Probe Wtd. Avg. Vanilla OPD75.4070.5081.2038.0242.9162.61 + Visual-evidence reflection76.2572.5082.2037.7844.0863.70 + Impact reweighting (REVUE)77.1074.0082.2041.1244.6665.01 (b) Math MethodMathVistaMathVerseVisuLogicWtd. Avg. Vanilla OPD70.6046.0726.0047.67 + Visual-evidence reflection71.1045.5626.2047.78 + Impact reweighting (REVUE)71.6047.0826.5048.49 (c) General Method Hallusion BenchChartQA-Pro Infographic VQAWtd. Avg. Vanilla OPD42.8821.6979.4453.28 + Visual-evidence reflection45.4222.3480.3554.42 + Impact reweighting (REVUE)45.1221.8180.2754.14 31 On-Policy Visual Evidence Distillation C.2 Token-Impact Interventions Table 9 expands Table 3 with scores on all 11 benchmarks. Table 10 summarizes how the four variants and REVUE differ in token selection and masking. Masking low-impact positions does not benefit every task. Compared with REVUE, Mask low 20% raises the perception average by 0.17%. Math and general-task averages decrease by 0.07% and 0.17%, respectively. Within perception, HRBench 8K and VisualProbe improve by 1.00% and 0.78%, respectively, while TreeBench drops by 1.61%. The small gain in average perception does not show that low-impact supervision is generally harmful. Across all 11 benchmarks, Mask low 20% exceeds Mask top 20%, supporting impact ranking as an indicator of supervision value. Table 9: Complete token-impact intervention results on Qwen2.5-VL-7B. Wtd. Avg. uses the benchmark-sample weighting of Table 1. MathVista and MathVerse denote the Mini and vision-only splits, respectively. All scores are percentages and higher is better; gray denotes Vanilla OPD, blue marks REVUE, and bold indicates column-best results, including ties. (a) Perception MethodHR-4KHR-8KV * Tree Bench Visual Probe Wtd. Avg. Vanilla OPD75.4070.5081.2038.0242.9162.61 Random select 20%75.4072.6280.1038.0245.4463.64 Mask random 20%76.2573.7581.2037.2843.6963.85 Mask low 20%77.0075.0082.2039.5145.4465.18 Mask top 20%75.0071.6278.0137.7841.3662.26 REVUE(Ours)77.1074.0082.2041.1244.6665.01 (b) Math MethodMathVistaMathVerseVisuLogicWtd. Avg. Vanilla OPD70.6046.0726.0047.67 Random select 20%71.3045.5625.0047.42 Mask random 20%70.9044.4226.1047.35 Mask low 20%71.8047.0826.1048.42 Mask top 20%71.3045.9424.0047.17 REVUE(Ours)71.6047.0826.5048.49 (c) General Method Hallusion BenchChartQA-Pro Infographic VQAWtd. Avg. Vanilla OPD42.8821.6979.4453.28 Random select 20%44.9421.8780.2154.10 Mask random 20%43.5821.8780.0853.78 Mask low 20%44.7721.9679.9353.97 Mask top 20%42.1620.7978.7552.51 REVUE(Ours)45.1221.8180.2754.14 32 On-Policy Visual Evidence Distillation Table 10: Variant definitions for token-impact interventions. Percentages refer to supervised positions, and masking removes the selected positions from the distillation loss. Random select replaces the impact-selected high-weight group with a random group while retaining supervision at every position. MethodMasked positionsRetained supervision Random select 20%NoneAll positions; random 20% and remaining 80% receive group loss weights 0.5 : 0.5 Mask random 20%Random 20%Remaining 80% receive total group loss weight 0.5, shared uniformly Mask low 20%Lowest-impact 20%Highest-impact 80%, including the entire High group Mask top 20%Highest-impact 20%Lowest-impact 80%; the entire High group is masked REVUE(Ours)NoneAll positions; highest-impact 20% and remaining 80% receive group loss weights 0.5 : 0.5 C.3 Sensitivity to the High-Impact Token Fraction Table 11 reports all 11 benchmark scores for the five fractions in Table 4. The fractionαcontrols the share of supervised positions assigned to the high-impact group, as defined in Section 4.3. The blue row marks the 20% setting used by REVUE. The best fraction varies across benchmarks. Among the tested fractions, 20% achieves the highest score on all three math benchmarks. It also leads on HRBench 4K, HRBench 8K, and TreeBench. V* Bench scores highest at 5%, while VisualProbe scores highest at 50%. The general- task average is highest at 100%. In these experiments, 20% yields the highest perception and math averages, supporting its use as a common setting for both categories. 33 On-Policy Visual Evidence Distillation Table 11: Sensitivity to the high-impact token fraction on Qwen2.5-VL-7B. Wtd. Avg. is weighted by benchmark sample counts. Sample counts are (800, 800, 191, 405, 515), (1000, 788, 1000), and (1129, 1948, 2801) for perception, math, and general benchmarks, respectively. MathVista and MathVerse denote the Mini and vision-only splits, respectively. All scores are percentages and higher is better; blue marks the 20% setting of REVUE, and bold indicates column-best results, including ties. (a) Perception High-impact fractionHR-4KHR-8KV * Tree Bench Visual Probe Wtd. Avg. Top 5%77.0073.1283.2540.4944.6664.70 Top 10%75.8873.7582.7240.9944.4764.55 Top 20% (REVUE)77.1074.0082.2041.1244.6665.01 Top 50%76.5073.5080.6339.2644.8564.33 Top 100%76.2572.5082.2037.7844.0863.70 (b) Math High-impact fractionMathVistaMathVerseVisuLogicWtd. Avg. Top 5%71.1046.3224.8047.49 Top 10%71.1044.2924.8046.92 Top 20% (REVUE)71.6047.0826.5048.49 Top 50%71.1045.3025.4047.42 Top 100%71.1045.5626.2047.78 (c) General High-impact fraction Hallusion BenchChartQA-Pro Infographic VQAWtd. Avg. Top 5%46.1621.7479.8454.12 Top 10%44.1321.5380.2553.85 Top 20% (REVUE)45.1221.8180.2754.14 Top 50%45.6621.8680.0954.18 Top 100%45.4222.3480.3554.42 34 On-Policy Visual Evidence Distillation C.4 Response Length and Tool Use on Perception Benchmarks The following five figure pairs report Qwen2.5-VL-7B results on all five perception benchmarks. Each pair shows overall accuracy versus mean response length on the left, and tool-call rate and accuracy on samples with and without tool use on the right. Arrows report RL Expert’s response length divided by that of REVUE; all bar-chart metrics are percentages. 350390430470510 Mean response length (tokens) ↓ 73 74 75 76 77 78 Accuracy (%) RL Expert: 75.50% ReVuE (Ours) 472 tokens (a) HRBench 4K Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 1.26× shorter Tool-call rate ↓Accuracy with tool ↑ Accuracy without tool ↑ 40 50 60 70 80 (b) HRBench 4K Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 78.8 77.5 62.4 76.4 78.1 67.2 76.0 77.5 68.8 61.8 79.4 62.8 76.8 76.1 74.2 54.2 78.3 70.5 50.0 84.0 70.3 Figure 9: Response length, accuracy, and tool use on HRBench 4K for Qwen2.5-VL-7B. 360390420450480 Mean response length (tokens) ↓ 65 67.5 70 72.5 75 Accuracy (%) RL Expert: 71.75% ReVuE (Ours) 458 tokens (a) HRBench 8K Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 1.20× shorter Tool-call rate ↓Accuracy with tool ↑ Accuracy without tool ↑ 40 50 60 70 80 % (b) HRBench 8K Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 73.0 73.1 48.6 74.5 75.7 60.3 74.2 74.6 58.7 67.8 69.9 56.6 78.5 71.8 61.1 61.3 70.4 63.6 54.0 79.4 67.7 Figure 10: Response length, accuracy, and tool use on HRBench 8K for Qwen2.5-VL-7B. 35 On-Policy Visual Evidence Distillation 380420460500540 Mean response length (tokens) ↓ 76 78 80 82 84 Accuracy (%) RL Expert: 83.77% ReVuE (Ours) 527 tokens (a) V* Bench Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 1.31× shorter Tool-call rate ↓Accuracy with tool ↑ Accuracy without tool ↑ 30 40 50 60 70 80 90 100 % (b) V* Bench Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 95.3 82.4 33.3 96.9 84.3 66.7 95.8 82.0 62.5 75.9 76.6 76.1 79.1 80.1 70.0 73.3 76.4 78.4 74.3 82.7 80.7 Figure 11: Response length, accuracy, and tool use on V* Bench for Qwen2.5-VL-7B. 220300380460 Mean response length (tokens) ↓ 37 39 41 43 Accuracy (%) RL Expert: 40.25% ReVuE (Ours) 451 tokens (a) TreeBench Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 1.87× shorter Tool-call rate ↓Accuracy with tool ↑ Accuracy without tool ↑ 10 20 30 40 50 60 70 80 (b) TreeBench Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 74.6 36.1 40.8 66.4 40.2 40.4 64.7 37.4 39.2 28.9 36.8 39.9 76.5 34.2 47.4 25.9 41.0 42.3 15.8 43.8 40.6 Figure 12: Response length, accuracy, and tool use on TreeBench for Qwen2.5-VL-7B. 530550570590610 Mean response length (tokens) ↓ 41 42 43 44 45 46 Accuracy (%) RL Expert: 44.85% ReVuE (Ours) 594 tokens (a) VisualProbe Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 1.10× shorter Tool-call rate ↓Accuracy with tool ↑ Accuracy without tool ↑ 20 30 40 50 60 70 80 90 (b) VisualProbe Cold Start V-Zero RL Expert VAD Vanilla OPD ReVuE (Ours) Vision-OPD 87.0 43.3 29.9 89.3 43.7 54.6 90.9 42.5 46.8 87.4 42.0 36.9 76.9 45.0 37.0 90.1 45.0 33.3 83.7 46.2 36.9 Figure 13: Response length, accuracy, and tool use on VisualProbe for Qwen2.5-VL-7B. 36 On-Policy Visual Evidence Distillation C.5 Training-Time Reflection Coverage After deduplication, the main training run contains 1,075 optimization steps with 20 groups per step and eight rollouts per group, totaling 21,500 groups and 172,000 rollout slots. As shown in Table 12, reflections cover 161,632 slots (93.97%), comprising complete reflections (78.36%) and Break-Point-only reflections (15.61%). The Break-Point-only branch supplies valid reflections for all-wrong groups, while the remaining 6.03% of slots use the base teacher. Table 12: Reflection coverage in the main training run. All percentages use 172,000 rollout slots as the denominator; the final row sums the first two rows. Reflection statusRollout slotsShare (%) Complete reflection134,78478.36 Break-Point-only reflection26,84815.61 No reflection (base teacher)10,3686.03 Any reflection (subtotal)161,63293.97 C.6 Offline Comparison with Anchor-Only Context Complete reflections correct more errors and preserve more correct answers. We compare complete reflections (Anchor plus Break Point) with Anchor-only context in an offline evaluation on fixed paired cases from 311 mixed groups (Table 13). Of 139 initially incorrect answers, complete reflections correct 124, compared with 62 for Anchor-only context. Among the 172 initially correct answers, eight become incorrect with complete reflections, compared with 47 with Anchor-only context. Complete reflections outperform Anchor-only context across all six data sources. These offline results support the value of failure diagnoses for both error correction and preserving correct answers. This pattern is consistent with the token-impact cases in Appendix E.4: reflection reduces teacher support for erroneous choices and increases support for correct judgments within failed trajectories. Table 13: Offline reflection comparison on fixed paired cases. Original predictions are the predictions before adding reflection, and accuracy changes are computed from unrounded values. ContextCorrect / 311Accuracy (%)∆Accuracy (%) Original predictions17255.31— Anchor only18760.13+4.82 Anchor + Break Point28892.60+37.30 37 On-Policy Visual Evidence Distillation D Proofs and Derivations D.1 Proof of Lemma 1 Fix a position(i,t)and writep = π θ (· | h i,t ),q 0 = q 0 i,t ,q R = q R i,t , andd(v) = d i,t (v). Fix the history, teacher predictions, and nonempty shared supportSupp, with positive student and teacher probabilities on it. For either teacher,q(Supp) = P v∈Supp q(v)is its probability mass on the support, whileq Supp (v) = q(v)/q(Supp)is its renormalized distribution. Subtracting the two KL losses cancels their shared student term: ℓ Supp (p,q R )− ℓ Supp (p,q 0 ) = X v∈Supp p Supp (v) log (q 0 ) Supp (v) (q R ) Supp (v) =− X v∈Supp p Supp (v)d(v) + log q R (Supp) q 0 (Supp) .(15) The teacher normalization term is constant inθand vanishes under differentiation. Using ∇ θ p Supp (v) = p Supp (v)∇ θ logp Supp (v) gives ∇ θ ℓ Supp (p,q R )− ℓ Supp (p,q 0 ) =− X v∈Supp d(v)∇ θ p Supp (v) =−E v∼p Supp d(v)∇ θ logp Supp (v) .(16) This proves Equation 6. For the reflection-conditioned branch in Section 4.3, takeSupp = Supp i,t and normalize both teachers over this support. From distribution shifts to a sampled-token proxy. Letz v be a student logit and ̄ d Supp = E v∼p Supp [d(v)] the student-weighted mean shift. For v ∈ Supp, ∂ ∂z v ℓ Supp (p,q R )− ℓ Supp (p,q 0 ) =−p Supp (v) d(v)− ̄ d Supp .(17) The softmax derivative gives this expression and zero direct derivatives for logits outsideSupp. The gradient change depends on student probabilities and centered shifts; parameter gradients also involve the logit Jacobian. Equation 7 measures the teacher shift at the student’s sampled choice for ranking positions. It need not match rankings by full logit- or parameter-gradient norm. For example, takeq 0 = (0.10, 0.15, 0.375, 0.375)andq R = (0.30, 0.45, 0.125, 0.125), withSupp containing the latter’s top two candidates. Both teachers renormalize to(0.4, 0.6)onSupp, so the shared-support gradient difference is zero, although either sampled token in this set has impactlog 3. The uniform shift cancels under support normalization. Impact remains defined for sampled tokens outsideSupp; RKL at that position still supervises the distribution within it. Shared support and teacher-specific top-Ksets. LetSupp R = TopK(q R )andSupp 0 = TopK(q 0 ). Comparing losses on each teacher’s own support gives ℓ Supp R (p,q R )− ℓ Supp 0 (p,q 0 ) = ∆ prob + ∆ support , ∆ prob = ℓ Supp R (p,q R )− ℓ Supp R (p,q 0 ), ∆ support = ℓ Supp R (p,q 0 )− ℓ Supp 0 (p,q 0 ).(18) The lemma characterizes∇ θ ∆ prob ;∇ θ ∆ support is generally nonzero when the supports differ. These terms decompose the loss difference; they are not additional training objectives. 38 On-Policy Visual Evidence Distillation D.2 Gradient Weights under Grouped Distillation Equal group weights increase per-position gradient coefficients in the smaller group. We examine how the weights in Equation 9 scale each position’s gradient contribution relative to uniform token averaging. Letℓ i,t denote the per-position KL term in Equation 8. With teacher targets, supports, and group assignments fixed, w i,t is constant during the student update, so ∇ θ w i,t |T i | ℓ i,t = w i,t ∇ θ ℓ i,t |T i | .(19) The weightw i,t directly gives the per-position gradient multiplier relative to uniform averaging under the same teacher target. Withλ = 0.5and both groups nonempty, Equation 9 assigns half of the total loss coefficient to each group. Ignoring the rounding in|T i,High | =⌈α|T i |⌉gives the continuous approximations in Figure 14: w High ≈ 0.5 α , w Low ≈ 0.5 1− α .(20) In this continuous form, High has a multiplier above one and Low below one whenα < 0.5; both equal one at α = 0.5. For a trajectory with 100 supervised positions,α = 0.2assigns 20 positions to High and the remaining 80 to Low. Uniform averaging assigns each position a coefficient of0.01, while grouping gives High and Low positions coefficients of0.025and0.00625. The corresponding gradient contributions scale by 2.5 and 0.625, with each group’s coefficients summing to 0.5. 0.10.20.30.40.50.60.70.80.9 High-impact fraction α 0 1 2 3 4 5 Per-position gradient multiplier α = 0.2 High: 2.5 Low: 0.625 λ = 0.5 High: 0.5 / αLow: 0.5 / (1 − α)Uniform averaging: 1 Figure 14: Per-position gradient weights atλ = 0.5. Blue and gray curves show the High and Low gradient multipliers relative to uniform token averaging; the black dotted line marks the unit baseline. The continuous curves ignore group-size rounding and cover0.1≤ α≤ 0.9. Markers at α = 0.2 give the exact multipliers 2.5 and 0.625 for the 100-position example. 39 On-Policy Visual Evidence Distillation E Case Study E.1 Errors in Acquiring, Reading, and Using Visual Evidence Figure 15 illustrates three concrete errors in acquiring, reading, and using visual evidence in Thyme-SFT (Zhang et al., 2026b) responses. (a)AcquireCrop misses the target HRBench8K · #87 Original image Question: How many dogs are being walked in the image? A. Five B. Two C. Three D. Four Thyme-SFT crop × Thyme-SFT · Thinking after the cropHuman evidence check Upon examining the cropped image, it is evident thatthere are three dogs being walked.The dogs are positioned close to each other, andtheir distinct shapes and sizes can be distinguished.The person walking the dogs isholding leashes that correspond to the three dogs, confirming the count.Therefore, the correct answer is thatthere are three dogs being walked in the image. The crop contains foliage. No dogs or leashes are visible in it, although the reasoning claims to see both. Final answer: C. Three Reference answer: D. Four (b)ReadMisreading the curve MathVista_MINI · #65 Original graph Question: Is f(3) > 0? A. Yes B. No Thyme-SFT · Thinking (full) The image is clear and provides a well-defined graph with a visible x-axis and y-axis. The function f(x) is plotted, and the value of f(3) can be determined by locating the point on the graph where x = 3.The graph shows that f(3) is below the x-axis, indicating that f(3) is less than 0.Therefore, the image conditions are sufficient to answer the question without further processing. The correct answer is (B) No, as f(3) is not greater than 0. Final answer: B Human evidence check:At x = 3, the curve is above the x-axis. f(3) > 0. Answer: A. Yes (c)GroundIncorrect aggregation MathVista_MINI · #331 Question: Several people compared how many Web pages they had visited. What is the mean of the numbers? Original table Human evidence check All eight values are read correctly. Sum = 512 Mean = 512 / 8 = 64 Thyme-SFT · Reasoning in final answer (full) To find the mean of the numbers, we need to follow these steps: 1. Add up all the numbers in the list. 2. Count how many numbers there are. 3. Divide the total sum by the count of numbers. Let's start with step 1: Add up all the numbers. Bonnie: 84 Winston: 56 Percy: 70 Derek: 68 Jake: 60 Finn: 47 Carter: 75 Kurt: 52 Total sum = 84 + 56 + 70 + 68 + 60 + 47 + 75 + 52= 520 Next, step 2: Count how many numbers there are. There are 8 numbers. Finally, step 3: Divide the total sum by the count of numbers. Mean = 520 / 8 = 65 So, the mean of the numbers is 65. Figure 15: In Acquire, the crop misses the queried dogs; in Read, the model describes a curve above the x-axis as lying below it; in Ground, it reads the table correctly but miscalculates the sum. The panels retain post-crop thinking in (a), full thinking in (b), and the complete reasoning-bearing final answer in (c). Red text marks errors and their consequences. 40 On-Policy Visual Evidence Distillation E.2 Different Errors with the Same Evidence Figure 16 compares two student trajectories for the same six-slice pie chart. Both use the original image without tool calls and answer “No”. Same image, different errors Question: Is Medium Blue the low median? Shared original image · no tool calls Human evidence check There are six slices. Medium Blue is fourth largest. The lower median is the third smallest value, equivalently the fourth largest. Reference answer: Yes Read · Misread rank Ground · Misapplied rule Thinking (full) The image is clear and provides a pie chart with distinct segments and a corresponding legend. The segments are labeled with colors and their corresponding names, making it easy to identify the size of each segment visually. The question asks whether "Medium Blue" is the low median. By visually comparing the sizes of the segments, we can determine the relative proportions.The "Medium Blue" segment appears smaller than "Dark Salmon" and "Orchid," but larger than "Dark Gold" and "Maroon." Therefore, it is not the low median.The image is sufficiently clear and simple for this analysis, so no further processing is required. Thinking (full) The image is a pie chart with six distinct segments, each labeled with a color name. The segment sizes are visually distinguishable, and the labels are clear. The question asks whether "Medium Blue" is the low median. To answer this, we need to compare the sizes of the segments visually. The image is clear enough to make this comparison without needing further processing. The segments are well-separated, and their relative sizes can be estimated visually. Therefore, the image conditions are sufficient to answer the question. Final answer (full) No, Medium Blue is not the low median.In the pie chart, the segments are arranged in descending order of their size. The colors and their corresponding sizes are as follows: 1. Maroon (largest) 2. Dark Gold 3. Medium Aqua 4. Dark Salmon 5. Medium Blue 6. Orchid (smallest) Since Medium Blue is the fifth segment from the largest, it is not the low median. The low median is Dark Salmon. Final answer (full) No, Medium Blue is not the low median.In the pie chart, the segments are ordered by size as follows: 1. Maroon (largest) 2. Medium Aqua 3. Dark Gold 4. Medium Blue 5. Dark Salmon 6. Orchid (smallest) Medium Blue is the fourth largest segment,which is not the low median. The low median is between Medium Blue and Dark Salmon. Correction needed: read the rank correctlyCorrection needed: apply the lower-median rule Figure 16: Medium Blue is the fourth-largest slice, making it the lower median of six slices. The left trajectory misreads the rank (Read); the right trajectory reads it correctly but misapplies the lower-median rule (Ground). Both complete thinking traces and final answers are shown, with errors in red. The required correction is to fix the visual rank on the left and the lower-median rule on the right. Both rollouts belong to the same all-wrong group, whose recorded training feedback contains diagnoses only. 41 On-Policy Visual Evidence Distillation E.3 Different Errors on the Same Plot Figure 17 contrasts two incorrect trajectories for the same neutron-star plot. Same plot, two different errors Question: Which neutron star group predominantly has a characteristic age (τc) less than 10⁵ years according to the plot? A. AXPs and SGRs B. RRATs C. XDINs D. Radio Pulsars Original plot Human evidence check Age contours are gray dotted diagonals. At a fixed period, positions above the 10⁵-year contour indicate younger stars; positions below indicate older stars. Most AXPs and SGRs (green triangles) lie above this contour. Reference answer: A. AXPs and SGRs Misidentified age contourReversed age comparison Thinking before the tool (full) To determine which neutron star group has a characteristic age (τc) less than 10⁵ years, we need to focus on the region of the plot where τc = 10⁵ years is indicated.This region is located in the lower part of the plot, below the τc = 10⁵ yr line. Cropping and zooming into this specific area will help us identify the relevant data points and their corresponding labels. The task involves analyzing the distribution of data points in the lower section of the plot. Given the scale and resolution of the image, zooming in on the area below the τc = 10⁵ yr line will provide a clearer view of the data points and their labels, making it easier to identify the predominant group with τc < 10⁵ years. Here is the Python code to crop and zoom into the relevant area: Thinking before the tool (full) To identify which group of neutron stars primarily has a characteristic age (τc) less than 10⁵ years, we need to concentrate on the area of the plot where τc = 10⁵ years is indicated. This region is positioned in the middle of the plot, with τc values diminishing toward the left side. Focusing on this specific area by cropping and zooming into it will facilitate the clear identification of data points associated with each neutron star category (AXPs and SGRs, RRATs, XDINs, and Radio Pulsars) and their respective τc values. This approach aids in accurately determining which category predominantly falls below 10⁵ years. Here is the Python code to crop and zoom into the relevant area: Actual tool crop Crop (0, 700) to (1932, 1150); enlarge by 1.5× Actual tool crop Crop (300, 400) to (1200, 800); enlarge by 1.5× Tool code is summarized above; all natural-language thinking is retained. Thinking after the tool (full) The green lines on the plot correspond to τc = 10⁵ years,and most of the RRATs (represented by blue circles) are found below these lines,indicating that they predominantly have a characteristic age less than 10⁵ years. Thinking after the tool (full) The plot features lines depicting characteristic ages (τc) for different magnetic field strengths.AXPs and SGRs are primarily located above the τc = 10⁵ years line,indicating that their characteristic ages exceed 10⁵ years.Conversely,RRATs, marked by blue circles, are predominantly situated below the τc = 10⁵ years line,suggesting that their characteristic ages are generally less than 10⁵ years. × Final answer: B. RRATs× Final answer: B. RRATs Figure 17: Both trajectories answer B (RRATs), whereas the reference answer is A (AXPs and SGRs). The left misidentifies green lines as age contours; the right correctly places AXPs and SGRs above the age contour but interprets that position as older. Their different crops and all natural-language thinking are shown; tool code is summarized. 42 On-Policy Visual Evidence Distillation E.4 Token Impact Visualizations Figures 18 and 19 complement Figure 3 with token impact visualizations for visual target selection and age inference. What is the color of the sweater? Actual tool crop by student Ground Truth: The color of the sweater is red. Student trajectory Error: Acquire Target localization and crop plan The task is to determine the color of the sweater worn by one of the individuals in the image . The sweater is visible on the person in the foreground on the left. Although the image is relatively clear, the sweater's details are not entirely distinct due to the distance and lighting conditions. Cro pping and zoom ing in on the person wearing the sweater will help in accurately identifying the color. To achieve this, I will crop the image around the person on the left, focusing specifically on the upper body where the sweater is located. This will enhance the visibility of the sweater's color. The coordinates for cropping will be approximately (20, 50, 220, 280) based on the visible portion of the person. Here is the Python code to crop and zoom in on the image : Reasoning after the tool call Upon examining the processed image , the person in the foreground on the left is wearing a sweater. The sweater appears to be a light purple or lavender color. The lighting and shadows in the image may slightly affect the perception of the exact shade, but the visible portion of the sweater clearly indicates a light purple or lavender hue. The cropped and zoomed-in view provides a clearer perspective, confirming the sweater's color more accurately. High-impact tokens: Encourage Suppress Figure 18: Token impact on visual target selection. The student crops the foreground person and reportslight purple or lavenderinstead ofred. Reflection identifies the background person as the target and lowers support for theunderlinedforegroundandleftin the crop plan. Blue/orange denote increased/decreased teacher support; darker shading indicates larger absolute log-probability shifts on the same student trajectory. Which neutron star group predominantly has a characteristic age ( τ_c ) less than 10⁵ years according to the plot? Ground Truth: A. AXPs and SGRs A. AXPs and SGRs B. RRATs C. XDINs D. Radio Pulsars Actual tool crop by student Student trajectory Error: Read Reasoning after the tool call The plot features lines depicting characteristic ages (\( _c \)) for different magnetic field strengths. AXPs and SGRs are primarily located above the \( \ tau_c = 10^5 \) years line, indicating that their characteristic ages exceed \( 10 ^5 \) years. Conversely, RRATs, marked by blue circles, are predominantly situated below the \( _c = 10^5 \) years line, suggesting that their characteristic ages are generally less than \( 10^5 \) years. Final answer B. RRATs High-impact tokens: Encourage Suppress Figure 19: Token impact on age inference. The student correctly placesAXPs and SGRs above theτ c = 10 5 yr contour but infers older ages, incorrectly answeringRRATs. Reflection links this position to younger ages, increasing support for theunderlinedcorrect relation aboveand decreasing support for the mistaken comparisonexceed. Blue/orange denote increased/decreased teacher support; darker shading indicates larger absolute log-probability shifts on the same student trajectory. 43 On-Policy Visual Evidence Distillation F Evaluation Benchmarks In Table 1, MathVista and MathVerse denote the Mini and vision-only splits, respectively. HalluBench, CQA-Pro, and InfoVQA abbreviate HallusionBench, ChartQA-Pro, and Infograph- icVQA, respectively. F.1 Visual Perception HRBench-4K/8K (Wang et al., 2025b) evaluate high-resolution perception on 4K and 8K images, covering fine-grained recognition and spatial relations. 5 V ∗ Bench (Wu & Xie, 2024) evaluates visual search in high-resolution images, focusing on fine-grained attributes and spatial relations. 6 VisualProbe (Lai et al., 2026) evaluates visual search for small targets among distractors in high-resolution images. 7 TreeBench (Wang et al., 2026) tests fine-grained object perception and relation understanding in complex scenes, with annotations for traceable visual evidence. 8 F.2 Mathematical and Logical Reasoning MathVista-Mini (Lu et al., 2024) is the testmini split of MathVista, covering mathematical reasoning over diagrams, charts, and natural images. 9 MathVerse (Zhang et al., 2025) evaluates diagram understanding in visual mathematics. We use its vision-only version, which presents the problem information entirely in the image. 10 VisuLogic (Xu et al., 2026) tests nonverbal logical reasoning through visual patterns involving quantity, space, and attributes. 11 F.3 General Multimodal Understanding HallusionBench (Guan et al., 2024) diagnoses language hallucinations and visual illusions through questions that require image context. 12 ChartQA-Pro (Masry et al., 2025) covers diverse chart types and question formats, including conversational, hypothetical, and unanswerable questions. 13 InfographicVQA (Mathew et al., 2022) requires joint reasoning over text, layout, graphics, and data visualizations to answer questions about infographics. 14 5 HRBench-4K/8K: https://huggingface.co/datasets/DreamMr/HR-Bench 6 V ∗ Bench: https://huggingface.co/datasets/craigwu/vstar_bench 7 VisualProbe Easy:https://huggingface.co/datasets/Mini-o3/VisualProbe_Easy; Medium: https://huggingface.co/datasets/Mini-o3/VisualProbe_Medium; Hard:https://huggingface. co/datasets/Mini-o3/VisualProbe_Hard 8 TreeBench: https://huggingface.co/datasets/HaochenWang/TreeBench 9 MathVista-Mini: https://huggingface.co/datasets/AI4Math/MathVista 10 MathVerse: https://huggingface.co/datasets/AI4Math/MathVerse 11 VisuLogic: https://huggingface.co/datasets/VisuLogic/VisuLogic 12 HallusionBench: https://huggingface.co/datasets/rayguan/HallusionBench 13 ChartQA-Pro: https://huggingface.co/datasets/ahmed-masry/ChartQAPro 14 InfographicVQA: https://site.docvqa.org/datasets/infographicvqa 44