Paper deep dive
Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding
Nanxing Hu, Qiwei Yan, Jinchao Zhang, Guoliang Kang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Fine-grained visual understanding requires models to recognize small details within complex images. Multimodal on-policy self-distillation (OPSD) addresses this challenge by using a teacher conditioned on evidence-centered crops to supervise a student conditioned on original images along student-generated trajectories. Ideally, teacher corrections, the distributional changes from the student toward the privileged teacher, should be driven by task-relevant visual evidence. However, the designs that make the teacher effective also introduce other interference. Using a lagged or frozen teacher improves training stability but introduces a model-state gap from the evolving student, while cropping enhances task-relevant evidence but also loses the visual context. These two sources of interference make the teacher corrections not purely rely on the visual evidence. We introduce Evidence-Aligned multimodal on-policy self-Distillation (EAD), which retains the crop-conditioned teacher as the target but constructs a separate evidence reference for weighting the corrections. To exclude the effect of lagged model-state from this reference, EAD measures prediction changes using the current student. To avoid crop-induced context changes, EAD masks the evidence region in the original image while preserving the other visual context. The change from the student's masked-image prediction to its original-image prediction provides a controlled reference for the direction in which the visual evidence shifts the student's prediction. EAD weights each teacher correction by its cosine alignment with the reference, i.e., retaining aligned corrections and downweighting the rest. Retaining only 6\% of the supervision mass of dense OPSD, EAD consistently outperforms previous state-of-the-art methods.
Tags
Links
- Source: https://arxiv.org/abs/2609.34672v1
- Canonical: https://arxiv.org/abs/2609.34672v1
Trouble viewing inline? Open PDF directly ā
Full Text
59,244 characters extracted from source content.
Expand or collapse full text
Evidence-Aligned Multimodal On-Policy Self-Distillation for Fine-Grained Visual Understanding Nanxing Hu Affiliation: Beihang University Qiwei Yan Affiliation: University of the Chinese Academy of Sciences*Corresponding authors Jinchao Zhang* Guoliang Kang Affiliation: Beihang University Abstract Fine-grained visual understanding requires models to recognize small details within complex images. Multimodal on-policy self-distillation (OPSD) addresses this challenge by using a teacher conditioned on evidence-centered crops to supervise a student conditioned on original images along student-generated trajectories. Ideally, teacher corrections, the distributional changes from the student toward the privileged teacher, should be driven by task-relevant visual evidence. However, the designs that make the teacher effective also introduce other interference. Using a lagged or frozen teacher improves training stability but introduces a model-state gap from the evolving student, while cropping enhances task-relevant evidence but also loses the visual context. These two sources of interference make the teacher corrections not purely rely on the visual evidence. We introduce Evidence-Aligned multimodal on-policy self-Distillation (EAD), which retains the crop-conditioned teacher as the target but constructs a separate evidence reference for weighting the corrections. To exclude the effect of lagged model-state from this reference, EAD measures prediction changes using the current student. To avoid crop-induced context changes, EAD masks the evidence region in the original image while preserving the other visual context. The change from the studentās masked-image prediction to its original-image prediction provides a controlled reference for the direction in which the visual evidence shifts the studentās prediction. EAD weights each teacher correction by its cosine alignment with the reference, i.e., retaining aligned corrections and downweighting the rest. Retaining only 6% of the supervision mass of dense OPSD, EAD consistently outperforms previous state-of-the-art methods. We also show that with small-scale backbones, EAD achieves competitive performance compared to substantially larger open-weight and closed-source models. 1 Introduction Fine-grained visual understanding tasks require multimodal large language models (MLLMs) to recognize small, localized evidence within large or cluttered visual contexts. However, reliably extracting and exploiting such evidence remains challenging, even for models with strong general-purpose visual and reasoning capabilities (Khayatkhoei et al., 2025; Liu et al., 2025; Wang et al., 2026). Recent work (Yuan et al., 2026) addresses this limitation with On-Policy Self-Distillation (OPSD), where the student generates trajectories from the original image and a crop-conditioned teacher, with privileged access to the visual evidence, provides next-token targets conditioned on the same prefixes. This encourages the student to make better use of fine-grained visual evidence from the original image, which has shown obvious gains on fine-grained visual understanding tasks. The value of the teacher supervision comes from corrections driven by the task-relevant visual evidence exposed through the crop. However, the designs that make the teacher effective also introduce other interferences into these corrections. First, a lagged or frozen teacher is important for stable self-distillation. Vision-OPD (Yuan et al., 2026) shows that directly using the current student as the teacher can destabilize training. The trained model collapses to near-zero accuracy across all benchmarks. However, using a lagged or frozen teacher introduces a model-state gap, so part of the teacher correction can arise from differences in model state. Second, the cropping is essential for exposing fine-grained visual evidence that may be difficult to extract from the original image, but cropping also removes surrounding visual context and changes the visual-token input presented to the model. This will introduce crop-induced variation into the teacher correction. Section 3 quantifies these two effects at the token level, showing that both model-state and crop-induced variations substantially alter the teacher corrections. Therefore, we introduce Evidence-Aligned multimodal on-policy self-Distillation (EAD), as illustrated in Figure 1. EAD retains the crop-conditioned EMA teacher as the supervision target, while constructing a separate evidence reference to determine how much of each teacher correction to retain. To exclude model-state gap from this reference, EAD computes the prediction change of current student. To avoid crop-induced context changes, EAD masks only the evidence region in the original image while preserving the surrounding visual context. Under the same generation prefix, the change from the studentās masked-image prediction to its original-image prediction provides a controlled reference for the direction in which the visual evidence shifts the studentās prediction. EAD then compares each privileged teacher correction with this reference. Their directional agreement is measured by cosine similarity and directly determines the token-level supervision weight: downweighting weakly aligned corrections and excluding opposing ones from supervision. Retaining only 6% of the supervision mass of dense OPSD, EAD consistently outperforms previous state-of-the-art methods. We also show that with small-scale backbones, EAD achieves competitive performance compared to substantially larger open-weight and closed-source models. These results suggest that OPSD is most effective when teacher corrections are selectively retained according to their alignment with privileged evidence. Our contributions are (1) We identify two sources of interference in multimodal OPSD: model-state gap and crop-induced context loss. Our analysis shows that they substantially affect the teacher corrections. (2) We propose EAD, which retains the crop-conditioned EMA teacher as the supervision target and constructs a separate evidence reference from the shift in the current studentās prediction caused by restoring the masked visual evidence. EAD uses the directional alignment between the teacher correction and the evidence reference to weight the distillation supervision. (3) Experiments show that EAD consistently outperforms previous state-of-the-art methods and is competitive with substantially larger models with retaining only about 6% of the supervision mass of dense OPSD. Figure 1: Motivation and overview of EAD. The privileged teacher correction in multimodal OPSD contains both the desired evidence-driven signal and interference arising from model-state gap and crop-induced context changes (top left). EAD constructs a separate student-side evidence reference from the current studentās prediction shift induced by evidence restoration, which keeps the model state fixed and preserves the surrounding visual context (top right). EAD then weights teacher corrections by the alignment with this evidence reference, concentrating supervision on visually supported corrections (bottom). 2 Related Work Fine-Grained Visual Understanding. Fine-grained visual understanding requires MLLMs to recover small, localized evidence from high-resolution or cluttered images. Existing methods often enhance fine-grained perception at inference time through visual search, localization, or image manipulation (Wu and Xie, 2024; Khayatkhoei et al., 2025; Wang et al., 2025; Zheng et al., 2026; Zhang et al., 2025a). Recently, some approaches improve fine-grained perception through training without additional test-time operations. Zooming without Zooming (Wei et al., 2026) distills the benefits of zooming into full-image training, enabling single-pass fine-grained perception. Vision-OPD (Yuan et al., 2026) uses crop-based teacher supervision to improve recognition of fine details in the original image. Our work also aims to strengthen fine-grained perception without additional inference-time cropping or tool use. On-Policy Self-Distillation. Knowledge distillation transfers prediction distributions from a teacher to a student (Hinton et al., 2015), and has been extended to autoregressive sequence generation (Kim and Rush, 2016). Recently, on-policy distillation (Gu et al., 2024; Agarwal et al., 2024) applies teacher supervision to prefixes generated by the studentās own policy, reducing the mismatch between training and inference prefix distributions. On-policy self-distillation further uses the same model as both teacher and student, with the teacher conditioned on additional privileged information (Zhao et al., 2026). In multimodal on-policy self-distillation, Vision-OPD (Yuan et al., 2026) uses an evidence-centered crop-conditioned teacher to supervise an original-image student. Our work follows this setting but focuses on improving the quality of the teacher corrections. Evidence-Aware Selective Distillation. Selective distillation selectively retains or reweights teacher supervision at the sequence or token level, rather than treating all teacher signals equally (Wang et al., 2021; Tavor et al., 2026). Visual interventions provide a way to identify how model predictions depend on visual evidence (Leng et al., 2024; Hu et al., 2025; Gao et al., 2026). Building on this idea, multimodal distillation methods use visual contrasts to select or weight supervision. Recent multimodal OPD methods use visual contrasts to identify supervision that is more closely tied to visual evidence (Liu et al., 2026; Qian et al., 2026; Sun et al., 2026). Concurrent works in multimodal OPSD explore evidence-aware selection of teacher supervision. OPD-V (Bi et al., 2026) uses contrastive privileged views to select visually supported supervision, while VAD (Zhang et al., 2026) uses teacher responses to degraded crop to identify visually relevant corrections. Our work adapts visual-evidence extraction to the particular structure of multimodal OPSD, yielding a more controlled signal for weighting teacher corrections. Figure 2: Diagnosing interference in teacher corrections. (a) Total Variation (TV) distance between the normalized token-wise JS distributions of DtteacherD_t^teacher and DtcropD_t^crop over 512 images, showing the effect of model-state gap across training. (b) TV distance between DtcropD_t^crop and DtmaskD_t^mask under the current student, showing the difference between crop-induced changes and evidence removal. (c) The corresponding token-wise JS distributions for one example. DtmaskD_t^mask places greater emphasis on tokens associated with the task-relevant visual evidence. DtcropD_t^crop shows large divergence on less vision-relevant tokens. Figure 3: Overview of EAD. The crop-conditioned EMA teacher provides the privileged target pĪøĀÆ,tCp_ Īø,t^C, while the current student produces pĪø,tRp_Īø,t^R and pĪø,tRāp_Īø,t^R^- on the original and masked images, respectively. The OriginalāMask prediction change pĪø,tRāpĪø,tRāp_Īø,t^R-p_Īø,t^R^- forms the evidence reference. EAD compares the teacher correction and evidence reference after a shared JS-motivated normalization, and uses their clipped cosine similarity to weight the distillation loss. 3 Diagnosing Teacher Corrections In Vision-OPD (Yuan et al., 2026), the current student parameterized by Īø is conditioned on the original image R, while the EMA teacher parameterized by ĪøĀÆ Īø is conditioned on the cropped image C. Both are evaluated under the same student-generated prefix ht=(x,y<t)h_t=(x,y_<t), where x denotes the input prompt and y<ty_<t the generated response tokens. For visual input I, we denote the next-token distribution as pĪø,tI=pĪø(ā ā£I,ht)p_Īø,t^I=p_Īø(Ā· I,h_t). The crop-conditioned teacher provides pĪøĀÆ,tCp_ Īø,t^C as the target for the original-image student prediction pĪø,tRp_Īø,t^R. Let sgsg denote the stop-gradient operator. The token-level distillation divergence is Dtteacher=JSā”(sgā”(pĪøĀÆ,tC),pĪø,tR).D_t^teacher=JS\! (sg(p_ Īø,t^C),p_Īø,t^R ). (1) So, the teacher distillation divergence involves both a change in model state (Īø versus ĪøĀÆ Īø) and visual input (R versus C), motivating a further diagnosis of how these factors influence the correction. 3.1 Model State difference Alters Teacher Corrections To isolate the effect of model state, we compare the teacher correction DtteacherD_t^teacher with a single-state variation that uses the current student for both visual inputs: Dtcrop=JSā”(pĪø,tR,pĪø,tC).D_t^crop=JS\! (p_Īø,t^R,p_Īø,t^C ). (2) To quantify how the two divergences differ across response tokens, we normalize their token-level JS values within each response and compute the Total Variation (TV) distance between the resulting distributions, with TVā[0,1]TVā[0,1]: Ļtk=Dtkātā²Dtā²k,TVā”(Ļteacher,Ļcrop)=12āāt|ĻtteacherāĻtcrop|, _t^k= D_t^k _t D_t ^k, (Ļ^teacher,Ļ^crop )= 12 _t | _t^teacher- _t^crop |, (3) where kāteacher,cropkā\teacher,crop\. We analyze a Qwen3.5-4B model trained with Vision-OPD (Yuan et al., 2026) using 512 randomly sampled images across training checkpoints. As shown in Figure 2(a), the two token-level distributions coincide at initialization and diverge during training. The median TV distance reaches 25.3%25.3\% at step 25 and remains substantial throughout training, showing that model-state gap alters the allocation of supervision across response tokens. 3.2 Crop-Induced Change Differs from Evidence Removal We next examine whether the prediction change induced by the privileged crop faithfully reflects the visual evidence. We decompose the original image as R=(E,B)R=(E,B), where E denotes the task-relevant evidence and B the remaining visual context. The RawāCrop comparison changes both components: cropping emphasizes E while removing B. To obtain a cleaner evidence reference, we construct a masked image Rā=(Eā,B)R^-=(E^-,B) which masks the evidence region while preserving the surrounding context. Using the same current student and generation prefix, we compute Dtmask=JSā”(pĪø,tR,pĪø,tRā).D_t^mask=JS\! (p_Īø,t^R,p_Īø,t^R^- ). (4) Following Eq. 3, we normalize DtmaskD_t^mask and DtcropD_t^crop within each response and compute the TV distance between them. Figure 2(b) shows consistently large median TV distances of approximately 4242ā44%44\% across training checkpoints. These results reveal a substantial token-level difference between crop-induced change and evidence removal, highlighting the influence of surrounding-context removal in the cropped view. 3.3 DtmaskD_t^mask Emphasizes Visually Relevant Tokens Figure 2(c) compares the token-wise JS distributions of the three supervision signals in a color-recognition example. Replacing the EMA teacher with the current student redistributes JS mass across response tokens. More importantly, DtmaskD_t^mask places greater emphasis on tokens associated with the task-relevant visual evidence, such as blue, whereas DtcropD_t^crop shows large divergence on tokens less directly tied to the visual evidence, such as is. This suggests that the OriginalāMask comparison is more closely tied to task-relevant visual evidence, motivating its use as the evidence reference for weighting teacher corrections. More case studies are provided in Appendix A.8. 4 Evidence-Aligned multimodal on-policy self-Distillation Motivated by the analysis in Section 3, we introduce Evidence-Aligned multimodal on-policy self-Distillation (EAD). As illustrated in Figure 3, EAD retains pĪøĀÆ,tCp_ Īø,t^C as the privileged teacher target, while using the current studentās OriginalāMask prediction change pĪø,tRāpĪø,tRāp_Īø,t^R-p_Īø,t^R^- as a separate evidence reference. EAD compares this reference with the teacher correction vector pĪøĀÆ,tCāpĪø,tRp_ Īø,t^C-p_Īø,t^R and uses their directional agreement to weight the distillation loss. 4.1 Aligning Teacher Corrections with Visual Evidence At response step t, the crop-conditioned EMA teacher provides the privileged target distribution pĪøĀÆ,tCp_ Īø,t^C for the original-image student distribution pĪø,tRp_Īø,t^R. EAD additionally computes the current studentās next-token distribution conditioned on the masked image, denoted by pĪø,tRāp_Īø,t^R^-. Since pĪø,tRp_Īø,t^R and pĪø,tRāp_Īø,t^R^- are produced by the same model under the same generation prefix and differ only in the evidence region, their distributional change provides a controlled reference for the prediction shift induced by the visual evidence. For two distributions p and p+Ī“p+Ī“, a second-order Taylor expansion of the JensenāShannon divergence around p gives JSā”(p,p+Ī“)ā18āājĪ“j2pj.JS(p,p+Ī“)ā 18 _j _j^2p_j. (5) where j indexes vocabulary entries. This motivates representing distributional changes in the locally normalized coordinates Ī“/pĪ“/ p, so that the directional comparison is consistent with the geometry of JensenāShannon divergence. Based on the shared current student distribution pĪø,tRp_Īø,t^R, we define utcorr=pĪøĀÆ,tCāpĪø,tRmaxā”(pĪø,tR,Ļµ),utevid=pĪø,tRāpĪø,tRāmaxā”(pĪø,tR,Ļµ).u_t^corr= p_ Īø,t^C-p_Īø,t^R (p_Īø,t^R,Īµ), u_t^evid= p_Īø,t^R-p_Īø,t^R^- (p_Īø,t^R,Īµ). (6) utcorru_t^corr represents the direction from the student distribution toward the privileged teacher target, while utevidu_t^evid represents the prediction shift induced by the visual evidence. ĻµĪµ is a small numerical constant. We quantify the directional agreement using the clipped cosine similarity: wt=maxā”(āØutcorr,utevidā©āutcorrā2āāutevidā2,0),w_t= ( u_t^corr,u_t^evid \|u_t^corr\|_2\,\|u_t^evid\|_2,0 ), (7) Teacher corrections that agree more strongly with the evidence-induced direction receive larger weights, while opposing directions receive zero weight. 4.2 Learning from Evidence-Aligned Corrections We use wi,tw_i,t to weight the original distillation loss, yielding the final EAD objective: āEAD=1Bāāi=1B1Tiāāt=1Tisgā”(wi,t)āJSā”(sgā”(pĪøĀÆ,i,tC),pĪø,i,tR),L_EAD= 1B _i=1^B 1T_i _t=1^T_isg(w_i,t)JS\! (sg\! (p_ Īø,i,t^C ),p_Īø,i,t^R ), (8) where B is the number of on-policy responses, TiT_i is the number of valid response tokens, and sgsg denotes stop-gradient. The weighted token losses are averaged within each response and then equally across responses. The teacher and masked-image branch are used only during training, and inference only uses the trained student on the original image. Table 1: Fine-grained visual understanding. We report results on six fine-grained visual benchmarks under the single-forward-pass evaluation setting. Results for closed-source and open-weight models are taken from Yuan et al. (2026), while all Qwen3.5-4B/9B-based methods are obtained from our evaluation. All post-training methods use the same backbone and training data, are trained for one epoch, and are evaluated using the final checkpoint. Avg. is the unweighted mean over the six benchmarks. Best results within each Qwen3.5 scale are shown in bold. Model Params V*Bench ZoomBench HR-Bench MME-RealWorld Avg. 4K 8K EN CN Closed-Source Models (Single Forward Pass) GPT-5.2 (OpenAI, 2025) ā 79.06 50.89 81.12 78.38 72.60 68.80 71.81 GPT-5.4 (OpenAI, 2026) ā 76.96 52.66 84.00 77.88 74.20 70.93 72.77 Gemini-3.1-Pro (Google, 2026a) ā 87.96 61.18 89.63 86.88 76.53 73.31 79.25 Gemini-3.5-Flash (Google, 2026b) ā 89.01 61.42 89.12 86.62 75.31 73.97 79.24 Open-Weight Models (Single Forward Pass) DeepEyes (Zheng et al., 2026) 7B 85.86 46.51 75.13 72.63 64.10 64.09 68.05 Thyme (Zhang et al., 2025a) 7B 82.20 45.09 77.00 72.00 64.80 64.59 67.61 DeepEyesV2 (Hong et al., 2026) 7B 81.68 44.97 77.88 73.75 64.90 65.07 68.04 SenseNova-MARS (Chng et al., 2025) 8B 92.15 47.81 83.13 78.38 67.90 68.90 73.05 MiMo-VL-RL (Yue et al., 2025) 7B 83.25 45.68 73.50 69.38 62.73 55.89 65.07 ZwZ (Wei et al., 2026) 8B 87.96 56.69 83.63 81.75 66.57 68.09 74.12 MiniCPM-V-4.5 (Yu et al., 2026) 9B 70.68 42.60 69.63 61.50 62.65 61.64 61.45 GLM-4.6V (Hong et al., 2025) 106B 86.91 50.06 82.13 78.88 65.57 65.62 71.53 Qwen3-VL-Instruct (Bai et al., 2025) 235B 91.10 56.09 86.13 80.38 71.74 69.04 75.75 Qwen3.5 (Qwen Team, 2026) 397B 87.96 57.16 89.38 85.50 74.82 69.82 77.44 Kimi-K2.6 (Team, Kimi, 2026) 1T 88.48 53.14 81.88 78.00 69.22 66.13 72.81 Qwen3.5-4B Post-Training Strategies Vanilla 4B 82.20 48.76 86.12 79.62 58.81 60.44 69.33 GRPO (Shao et al., 2024) 4B 86.91 58.93 84.50 79.38 72.49 70.78 75.50 Vision-OPD (Yuan et al., 2026) 4B 87.43 59.76 80.62 80.25 70.77 69.38 74.70 OPD-V (Bi et al., 2026) 4B 87.43 58.93 83.75 80.12 70.12 68.80 74.86 VAD (Zhang et al., 2026) 4B 91.10 60.71 85.50 81.25 72.75 70.07 76.90 EAD (Ours) 4B 91.62 60.83 86.38 82.88 72.39 71.10 77.53 Qwen3.5-9B Post-Training Strategies Vanilla 9B 89.01 53.49 85.38 81.88 71.83 67.47 74.84 GRPO (Shao et al., 2024) 9B 89.53 57.51 84.75 81.62 66.47 65.59 74.25 Vision-OPD (Yuan et al., 2026) 9B 89.53 66.04 85.62 82.38 71.11 69.61 77.38 OPD-V (Bi et al., 2026) 9B 90.05 62.25 82.75 82.62 71.65 70.63 76.66 VAD (Zhang et al., 2026) 9B 91.10 61.42 87.62 84.12 72.32 70.07 77.78 EAD (Ours) 9B 93.19 62.01 88.62 84.62 72.46 71.07 78.66 Table 2: Effect of model state on the evidence reference. All variants use the same OriginalāMask image pair. Cross-state uses different model states for the two predictions, whereas Teacher-side and Student-side (EAD) use the EMA teacher and current student alone, respectively. Avg. denotes the unweighted mean over the six benchmarks. Best results in each column are shown in bold. Model State V*Bench ZoomBench HR-Bench MME-RealWorld Avg. 4K 8K EN CN Cross-state 89.53 59.05 85.12 81.38 71.05 69.75 75.98 Teacher-side 89.01 59.76 85.25 83.62 73.18 71.35 77.03 Student-side (EAD) 91.62 60.83 86.38 82.88 72.39 71.10 77.53 Table 3: Effect of visual contrast on the evidence reference. The first two variants use the current student and differ only in the visual inputs, while the VAD-style variant uses the EMA teacher on the cropped image and its downsampled counterpart. All variants use the same EAD objective. Avg. denotes the unweighted mean over the benchmarks. Best results in each column are shown in bold. Visual Contrast V*Bench ZoomBench HR-Bench MME-RealWorld Avg. 4K 8K EN CN OriginalāCrop 88.48 60.00 83.00 80.25 70.61 68.62 75.16 OriginalāMask (EAD) 91.62 60.83 86.38 82.88 72.39 71.10 77.53 CropāDownsampled Crop (VAD) 91.62 58.82 86.50 83.00 69.33 68.73 76.33 Table 4: Effect of local JS scaling and clipped cosine similarity. No JS Scaling removes the local normalization before computing directional alignment, while Shifted-Cosine retains negatively aligned teacher corrections. Variant V*Bench ZoomBench HRBench-4K HRBench-8K MME-EN MME-CN Avg. No JS Scaling 89.01 59.29 85.12 82.38 72.37 71.29 76.58 Shifted-Cosine 90.58 61.18 83.50 81.25 72.22 70.05 76.46 EAD 91.62 60.83 86.38 82.88 72.39 71.10 77.53 5 Experiments 5.1 Experimental Setup Implementation. We closely follow the Vision-OPD (Yuan et al., 2026) training configuration. We train Qwen3.5-4B and Qwen3.5-9B on the same 6.2K fine-grained examples for one epoch and evaluate the final checkpoint. Distillation uses JensenāShannon divergence computed from the top-100 logits with partial softmax, and the privileged teacher is updated by EMA with a rate of 0.050.05. To reduce premature collapse toward short responses that directly output the option letter, we fix the first generated token to The, exclude this token from the training loss, and sample all subsequent tokens normally. Further implementation details are provided in Appendix A.1. Benchmarks. We evaluate fine-grained visual perception on six benchmarks. V*Bench (Wu and Xie, 2024) evaluates the ability to locate and recognize fine visual details in high-resolution and visually crowded images. ZoomBench (Wei et al., 2026) focuses on fine-grained perception where decisive evidence is localized and difficult to recover from the original image. HR-Bench (Wang et al., 2025) evaluates fine-grained single- and cross-instance perception at both 4K and 8K resolutions. MME-RealWorld (Zhang et al., 2025b) further evaluates high-resolution perception and reasoning in challenging real-world scenarios. We use both its English and Chinese subsets. We additionally evaluate held-out generalization in Appendix A.2. For evaluation, we follow the released Vision-OPD pipeline with a calibrated multiple-choice answer-extraction procedure, applied consistently to all methods. Unresolved responses are evaluated by Qwen3.5-122B-A10B (Qwen Team, 2026) as a fallback judge. More details are provided in Appendix A.3. Baselines. We compare against two groups of baselines. First, we include representative closed-source and open-weight MLLMs under single-forward-pass inference. Their benchmark scores are taken directly from the evaluation reported by Vision-OPD. Second, we compare alternative post-training strategies using the same Qwen3.5 (Qwen Team, 2026) backbones and training data. At both the 4B and 9B scales, we compare EAD against the Vanilla model, GRPO (Shao et al., 2024), Vision-OPD (Yuan et al., 2026), and the concurrent methods OPD-V (Bi et al., 2026) and VAD (Zhang et al., 2026). We retrain each post-training method on the same backbone and training data and evaluate all resulting checkpoints with the same pipeline. All variants are trained for one epoch, and the final checkpoint is used for evaluation. 5.2 Comparison with State-of-the-Art MLLMs Table 1 reports the comparison on six fine-grained visual understanding benchmarks. For the Qwen3.5 post-training setting, all methods use the same backbone, training data, and evaluation pipeline. EAD achieves the highest average performance at both the 4B and 9B scales. Compared with Vision-OPD, EAD improves the average by 2.832.83 and 1.281.28 points at 4B and 9B, respectively. It also outperforms the strongest concurrent baseline, VAD, by 0.630.63 and 0.880.88 points at the two scales. We further compare EAD with representative open-weight and closed-source MLLMs. Despite using substantially smaller backbones, EAD reaches performance comparable to substantially larger models. These consistent gains demonstrate the effectiveness of selectively retaining teacher corrections according to their alignment with visual evidence. Figure 4: Sparsity and concentration of EAD supervision. (a) Fraction of the original distillation-loss mass retained by EAD throughout training. (b) Concentration of the retained loss across response tokens, compared with the original dense distillation loss on the same responses. (c) On-policy rollout accuracy of EAD and Vision-OPD during training. Results are shown for Qwen3.5-4B and Qwen3.5-9B. 5.3 EAD Retains Sparse and Concentrated Supervision EAD substantially reduces the amount of teacher supervision used during training, retaining only 5.87%5.87\% and 6.15%6.15\% of the original distillation-loss mass for the 4B and 9B models, respectively (Figure 4(a)). Furthermore, this reduction is not uniform: the top 10%10\% highest-loss tokens account for over 95%95\% of the retained loss mass, compared with about 6161ā62%62\% under dense distillation (Figure 4(b)). Thus, EAD concentrates supervision on a much smaller subset of corrections rather than simply reducing the overall distillation strength. Despite retaining only a small fraction of the original supervision, EAD maintains higher on-policy rollout accuracy than Vision-OPD throughout training (Figure 4(c)). These results show that EAD produces a sparser and more concentrated distillation objective while maintaining more effective optimization. 5.4 Ablation Studies We ablate the evidence-reference construction by only varying the prediction difference Īt _t used to form utevid=Īt/maxā”(pĪø,tR,Ļµ)u_t^evid= _t/ (p_Īø,t^R,Īµ), keeping the remaining EAD objective and training procedure fixed. Tables 2 and 3 report the fine-grained results; held-out results are provided in Appendix A.5. All the experiments are conducted on Qwen3.5-4B. Effect of model state. We first examine which model state should be used to construct the utevidu_t^evid. All three variants use the same OriginalāMask image pair and differ only in the model states used to compute the two predictions. Cross-state uses Īt=pĪø,tRāpĪøĀÆ,tRā _t=p_Īø,t^R-p_ Īø,t^R^-, while Teacher-side and Student-side (EAD) use Īt=pĪøĀÆ,tRāpĪøĀÆ,tRā _t=p_ Īø,t^R-p_ Īø,t^R^- and Īt=pĪø,tRāpĪø,tRā _t=p_Īø,t^R-p_Īø,t^R^-, respectively. As shown in Table 2, Student-side (EAD) performs best, followed by Teacher-side, while Cross-state performs worst. Both same-state variants outperform Cross-state, showing the benefit of excluding model-state gap when constructing the evidence reference. The further advantage of Student-side (EAD) over Teacher-side shows that the studentās own OriginalāMask prediction change provides a more effective evidence reference than the teacherās. Effect of the visual contrast. We next examine which image pair should be used to construct the evidence reference. For a controlled comparison, both variants use the current student to make predictions and differ only in the visual inputs. OriginalāCrop uses Īt=pĪø,tCāpĪø,tR _t=p_Īø,t^C-p_Īø,t^R, whereas OriginalāMask (EAD) uses Īt=pĪø,tRāpĪø,tRā _t=p_Īø,t^R-p_Īø,t^R^-. As shown in Table 3, OriginalāMask (EAD) consistently outperforms OriginalāCrop across all six benchmarks. This shows that removing the evidence while preserving the surrounding visual context provides a more effective reference than altering the context through cropping. Comparison with a VAD-style reference. We further consider the image pair used by VAD (Zhang et al., 2026). CropāDownsampled Crop (VAD) uses Īt=pĪøĀÆ,tCāpĪøĀÆ,tCdeg _t=p_ Īø,t^C-p_ Īø,t^C_deg, where CdegC_deg is a downsampled version of the cropped image, while keeping the EAD weighting objective and training procedure unchanged. The CropāDownsampled Crop (VAD) comparison keeps both the model state and cropped region fixed, varying only the visual degradation. However, this distributional change captures the EMA teacherās evidence sensitivity in the cropped view, rather than the current studentās evidence sensitivity in the original image. OriginalāMask (EAD) instead captures the current studentās response to visual evidence within the original-image context. As shown in Table 3, OriginalāMask (EAD) outperforms this VAD-style reference, supporting the use of an evidence reference constructed from the current student in the original-image context. Effect of local JS scaling. We next examine the effect of the local JS scaling used to compare the teacher correction with the evidence reference. No JS Scaling removes the shared 1/maxā”(pĪø,tR,Ļµ)1/ (p_Īø,t^R,Īµ) scaling and directly computes cosine similarity between the probability differences pĪøĀÆ,tCāpĪø,tRp_ Īø,t^C-p_Īø,t^R and pĪø,tRāpĪø,tRāp_Īø,t^R-p_Īø,t^R^-, while keeping the remaining EAD objective and training procedure unchanged. As shown in Table 4, removing the local JS scaling reduces the average performance from 77.53 to 76.58. This shows that comparing the two prediction changes under the shared local JS scaling provides a more effective alignment signal. Effect of clipped cosine similarity. We further examine whether negatively aligned teacher corrections should be retained for supervision. Shifted-Cosine replaces the EAD weight maxā”(cosā”(utcorr,utevid),0) ( (u_t^corr,u_t^evid),0) with (cosā”(utcorr,utevid)+1)/2( (u_t^corr,u_t^evid)+1)/2, so that negatively aligned corrections retain non-zero supervision weights. All other settings remain unchanged. As shown in Table 4, Shifted-Cosine reduces the average performance from 77.53 to 76.46. This supports excluding teacher corrections that are directionally opposed to the evidence reference. 6 Conclusion We introduced Evidence-Aligned multimodal on-policy self-Distillation (EAD) for fine-grained visual understanding. EAD retains the privileged teacher as the supervision target while using the current studentās OriginalāMask prediction change as a separate evidence reference for selectively weighting teacher corrections. By keeping the model state fixed and preserving the surrounding visual context, this reference more cleanly captures the prediction shift induced by task-relevant visual evidence. Across 4B and 9B models, EAD consistently outperforms strong post-training baselines and remains competitive with substantially larger open-weight and closed-source models on fine-grained visual understanding benchmarks. AI use statement Generative AI tools were used to assist with language editing and to improve the clarity, grammar, and readability of the manuscript. They were not used to formulate research hypotheses, design the methodology or experiments, analyze data, interpret experimental results, or generate scientific claims. All AI-assisted edits were carefully reviewed and revised by the authors. The authors take full responsibility for the final content of the manuscript. References Agarwal et al. (2024) R. Agarwal, N. Vieillard, Y. Zhou, P. Stanczyk, S. Ramos Garea, M. Geist, and O. Bachem On-policy distillation of language models: learning from self-generated mistakes. In International Conference on Learning Representations, Vol. 2024, p. 21246ā21263. Cited by: Ā§2. Bai et al. (2025) S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: Table 1. Bi et al. (2026) J. Bi, P. Liao, Z. Jin, V. Tresp, F. Shen, Y. Ma, T. Chua, et al. OPD-v: visual on-policy self-distillation with modality balance. arXiv preprint arXiv:2608.05131. Cited by: Table 5, Table 5, Ā§2, Table 1, Table 1, Ā§5.1. Chen et al. (2024) L. Chen, J. Li, X. Dong, P. Zhang, Y. Zang, Z. Chen, H. Duan, J. Wang, Y. Qiao, D. Lin, et al. Are we on the right way for evaluating large vision-language models?. Advances in Neural Information Processing Systems 37, p. 27056ā27087. Cited by: Ā§A.2. Chng et al. (2025) Y. X. Chng, T. Hu, W. Tong, X. Li, J. Chen, H. Yu, J. Lu, H. Guo, H. Deng, C. Xie, et al. Sensenova-mars: empowering multimodal agentic reasoning and search via reinforcement learning. arXiv preprint arXiv:2512.24330. Cited by: Table 1. Gao et al. (2026) S. Gao, Y. Wang, J. Yan, Z. Wu, and Y. Jiang Thinking with deltas: incentivizing reinforcement learning via differential visual reasoning policy. arXiv preprint arXiv:2601.06801. Cited by: Ā§2. Google (2026a) Google Gemini 3.1 Pro: a smarter model for your most complex tasks. External Links: Link Cited by: Table 1. Google (2026b) Google Gemini 3.5: frontier intelligence with action. External Links: Link Cited by: Table 1. Gu et al. (2024) Y. Gu, L. Dong, F. Wei, and M. Huang Minillm: knowledge distillation of large language models. In International Conference on Learning Representations, Vol. 2024, p. 32694ā32717. Cited by: Ā§2. Hinton et al. (2015) G. Hinton, O. Vinyals, and J. Dean Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. Cited by: Ā§2. Hong et al. (2026) J. Hong, C. Zhao, C. Zhu, W. Lu, and G. Xu Deepeyesv2: toward agentic multimodal model. In International Conference on Learning Representations, Vol. 2026, p. 114851ā114872. Cited by: Table 1. Hong et al. (2025) W. Hong, W. Yu, X. Gu, G. Wang, G. Gan, H. Tang, J. Cheng, J. Qi, J. Ji, L. Pan, et al. Glm-4.5 v and glm-4.1 v-thinking: towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006. Cited by: Table 1. Hu et al. (2025) N. Hu, X. Duan, J. Zhang, and G. Kang Enhancing visual reliance in text generation: a bayesian perspective on mitigating hallucination in large vision-language models. In Proceedings of the 33rd ACM International Conference on Multimedia, p. 4778ā4787. Cited by: Ā§2. Khayatkhoei et al. (2025) M. Khayatkhoei, P. Chhikara, F. Ilievski, et al. Mllms know where to look: training-free perception of small visual details with multimodal llms. In International Conference on Learning Representations, Vol. 2025, p. 68194ā68213. Cited by: Ā§1, Ā§2. Kim and Rush (2016) Y. Kim and A. M. Rush Sequence-level knowledge distillation. In Proceedings of the 2016 conference on empirical methods in natural language processing, p. 1317ā1327. Cited by: Ā§2. Leng et al. (2024) S. Leng, H. Zhang, G. Chen, X. Li, S. Lu, C. Miao, and L. Bing Mitigating object hallucinations in large vision-language models through visual contrastive decoding. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 13872ā13882. Cited by: Ā§2. Li et al. (2023) Y. Li, Y. Du, K. Zhou, J. Wang, X. Zhao, and J. Wen Evaluating object hallucination in large vision-language models. In Proceedings of the 2023 conference on empirical methods in natural language processing, p. 292ā305. Cited by: Ā§A.2. Liu et al. (2025) P. Liu, H. Shen, C. Fang, Z. Sun, J. Liao, and T. Zhao Vlm-fo1: bridging the gap between high-level reasoning and fine-grained perception in vlms. arXiv preprint arXiv:2509.25916. Cited by: Ā§1. Liu et al. (2026) R. Liu, X. Lv, G. Li, X. Zhu, Z. Wang, Z. Zhang, J. Chen, Z. Li, B. Li, J. Gao, et al. Visual-advantage on-policy distillation for vision-language models. arXiv preprint arXiv:2605.21924. Cited by: Ā§2. OpenAI (2025) OpenAI Introducing GPT-5.2. External Links: Link Cited by: Table 1. OpenAI (2026) OpenAI Introducing GPT-5.4. External Links: Link Cited by: Table 1. Qian et al. (2026) Y. Qian, J. Yu, J. Liu, M. Wang, H. B. Li, and X. Hu Med-opd: improving medical vision-language models via evidence-aware on-policy distillation. arXiv preprint arXiv:2607.16303. Cited by: Ā§2. Qwen Team (2026) Qwen Team Qwen3.5: towards native multimodal agents. External Links: Link Cited by: Ā§A.3, Table 1, Ā§5.1, Ā§5.1. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: Table 5, Table 5, Table 1, Table 1, Ā§5.1. Sun et al. (2026) H. Sun, Z. Yi, L. Deng, Y. Zhou, P. Jia, J. Zhao, L. Yuan, J. Lv, and T. Wang V-zero: answer-label-free on-policy distillation with contrastive evidence gating for fine-grained visual reasoning. arXiv preprint arXiv:2606.25319. Cited by: Ā§2. Tavor et al. (2026) A. Tavor, I. Ebenspanger, N. Cnaan, and M. Geva Rethinking selective knowledge distillation. arXiv preprint arXiv:2602.01395. Cited by: Ā§2. Team, Kimi (2026) Team, Kimi Kimi k2.6: from code to creation, from one to many. External Links: Link Cited by: Table 1. Tong et al. (2024a) S. Tong, E. Brown, P. Wu, S. Woo, M. Middepogu, S. C. Akula, J. Yang, S. Yang, A. Iyer, X. Pan, et al. Cambrian-1: a fully open, vision-centric exploration of multimodal llms. Advances in Neural Information Processing Systems 37, p. 87310ā87356. Cited by: Ā§A.2. Tong et al. (2024b) S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. LeCun, and S. Xie Eyes wide shut? exploring the visual shortcomings of multimodal llms. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 9568ā9578. Cited by: Ā§A.2. Wang et al. (2021) F. Wang, J. Yan, F. Meng, and J. Zhou Selective knowledge distillation for neural machine translation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), p. 6456ā6466. Cited by: Ā§2. Wang et al. (2026) H. Wang, Y. Wang, T. Zhang, Y. Zhou, Y. Li, J. Wang, Y. Tian, J. Meng, Z. Huang, G. Mai, et al. Grasp any region: towards precise, contextual pixel understanding for multimodal llms. In International Conference on Learning Representations, Vol. 2026, p. 2222ā2250. Cited by: Ā§1. Wang et al. (2025) W. Wang, L. Ding, M. Zeng, X. Zhou, L. Shen, Y. Luo, W. Yu, and D. Tao Divide, conquer and combine: a training-free framework for high-resolution image perception in multimodal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 7907ā7915. Cited by: Ā§2, Ā§5.1. Wei et al. (2026) L. Wei, L. He, J. Lan, L. Dong, Y. Cai, S. Li, H. Zhu, W. Wang, L. Kong, Y. Wang, et al. Zooming without zooming: region-to-image distillation for fine-grained multimodal perception. arXiv preprint arXiv:2602.11858. Cited by: Ā§2, Table 1, Ā§5.1. Wu and Xie (2024) P. Wu and S. Xie V*: guided visual search as a core mechanism in multimodal llms. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 13084ā13094. Cited by: Ā§2, Ā§5.1. Yu et al. (2026) T. Yu, Z. Wang, C. Wang, F. Huang, W. Ma, Z. He, T. Cai, W. Chen, Y. Huang, R. Zhao, et al. Minicpm-v 4.5: cooking efficient mllms via architecture, data, and training recipe. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 11704ā11715. Cited by: Table 1. Yuan et al. (2026) Q. Yuan, J. Lou, X. Yu, H. Lin, L. Sun, X. Han, and Y. Lu Vision-opd: learning to see fine details for multimodal llms via on-policy self-distillation. arXiv preprint arXiv:2605.18740. Cited by: Table 5, Table 5, Ā§1, Ā§1, Ā§2, Ā§2, Ā§3.1, Ā§3, Table 1, Table 1, Table 1, Ā§5.1, Ā§5.1. Yue et al. (2025) Z. Yue, Z. Lin, Y. Song, W. Wang, S. Ren, S. Gu, S. Li, P. Li, L. Zhao, L. Li, et al. Mimo-vl technical report. arXiv preprint arXiv:2506.03569. Cited by: Table 1. Zhang et al. (2026) K. Zhang, Y. Li, S. Shao, Q. Li, Z. Lu, Z. Yao, J. Lin, W. Jiao, Y. Lu, W. Liu, et al. VAD: attributing visual evidence for target reconstruction in multimodal on-policy distillation. arXiv preprint arXiv:2607.28590. Cited by: Table 5, Table 5, Ā§2, Table 1, Table 1, Ā§5.1, Ā§5.4. Zhang et al. (2025a) Y. Zhang, X. Lu, S. Yin, C. Fu, W. Chen, X. Hu, B. Wen, K. Jiang, C. Liu, T. Zhang, et al. Thyme: think beyond images. arXiv preprint arXiv:2508.11630. Cited by: Ā§2, Table 1. Zhang et al. (2025b) Y. Zhang, H. Zhang, H. Tian, C. Fu, S. Zhang, J. Wu, F. Li, K. Wang, Q. Wen, Z. Zhang, et al. Mme-realworld: could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans?. In International Conference on Learning Representations, Vol. 2025, p. 89655ā89701. Cited by: Ā§5.1. Zhao et al. (2026) S. Zhao, Z. Xie, M. Liu, J. Huang, G. Pang, F. Chen, and A. Grover Self-distilled reasoner: on-policy self-distillation for large language models. arXiv preprint arXiv:2601.18734. Cited by: Ā§2. Zheng et al. (2026) Z. Zheng, M. Yang, J. Hong, C. Zhao, G. Xu, L. Yang, and C. Shen Deepeyes: incentivizingā thinking with imagesā via reinforcement learning. In International Conference on Learning Representations, Vol. 2026, p. 126775ā126798. Cited by: Ā§2, Table 1. Appendix A Appendix A.1 Additional Implementation Details Training configuration. We build EAD on the Vision-OPD training pipeline and keep its training configuration unchanged unless otherwise specified. Both Qwen3.5-4B and Qwen3.5-9B are trained for one epoch on the same 6.2K fine-grained examples used by Vision-OPD. We use a training batch size of 96, sample n=8n=8 on-policy responses per example, and optimize the student with a learning rate of 2Ć10ā62Ć 10^-6 and 10 warmup steps. The maximum prompt and response lengths are 8192 and 1024 tokens, respectively. The privileged teacher is updated by exponential moving average (EMA) with an update rate of 0.05. On-policy rollout generation. Training trajectories are generated by the current original-image student. For each sampled response y=(y1,ā¦,yT)y=(y_1,ā¦,y_T), all subsequent teacher and evidence reference predictions are evaluated under the same student-generated textual prefix ht=(x,y<t)h_t=(x,y_<t). We fix the first generated token to The and exclude this token from the training loss; all subsequent response tokens are sampled normally. This construction ensures that the privileged teacher correction and the evidence reference are compared at exactly the same autoregressive states. Privileged crop and evidence-masked view. We directly use the original-image/crop pairs released by Vision-OPD. Thus, the crop-conditioned EMA teacher receives the same privileged visual input as Vision-OPD. EAD introduces only an additional evidence-masked view of the original image. We construct RāR^- by masking the GT bounding-box interior with the mean RGB color of its surrounding region, while preserving the red-box annotation and the remaining image context unchanged. Shared top-K vocabulary support. Following Vision-OPD, all token-level distribution computations use K=100K=100 vocabulary entries. At response position t, we first select the top-100 vocabulary entries according to the current full-image student distribution pĪø,tRp^R_Īø,t. The same vocabulary indices are then used to gather the corresponding logits from the crop-conditioned EMA teacher pĪøĀÆ,tCp^C_ Īø,t and the masked-image student pĪø,tRāp^R^-_Īø,t. Therefore, the three distributions used by EAD are represented in the same vocabulary coordinates, which allows their distributional changes to be compared directly. More explicitly, if t=TopKā”(pĪø,tR,K),K=100,S_t=TopK\! (p^R_Īø,t,K ), K=100, then all quantities in Eqs. 6ā7 are computed after restricting pĪø,tRp^R_Īø,t, pĪøĀÆ,tCp^C_ Īø,t, and pĪø,tRāp^R^-_Īø,t to the common support tS_t. A.2 Preservation of broader visual capabilities. We additionally evaluate the held-out generalization on MMVP (Tong et al., 2024b), CV-Bench (Tong et al., 2024a), MMStar (Chen et al., 2024), and POPE (Li et al., 2023). We use the same 4B and 9B model checkpoints as those evaluated in Table 1. Table 5 shows that EAD largely preserves the base modelās average performance across the four held-out benchmarks, reducing the performance degradation relative to dense Vision-OPD by about 83% at both model scales. These results show that EAD improves fine-grained perception while largely preserving broader visual capabilities. Table 5: Preservation of broader visual capabilities. We evaluate the same 4B and 9B post-training checkpoints from Table 1 on four held-out benchmarks without additional training. Avg. denotes the unweighted mean over the four benchmarks. Best results within each model scale are shown in bold. Method MMVP CV-Bench MMStar POPE Avg. Qwen3.5-4B Post-Training Strategies Vanilla 79.00 86.97 74.00 89.33 82.33 GRPO (Shao et al., 2024) 78.00 85.78 69.33 87.87 80.25 Vision-OPD (Yuan et al., 2026) 76.67 85.48 70.80 88.99 80.49 OPD-V (Bi et al., 2026) 77.00 84.38 71.40 89.43 80.55 VAD (Zhang et al., 2026) 79.67 87.30 71.80 88.79 81.89 EAD (Ours) 80.33 86.95 73.07 87.70 82.01 Qwen3.5-9B Post-Training Strategies Vanilla 82.67 88.41 77.87 89.86 84.70 GRPO (Shao et al., 2024) 80.33 87.87 72.80 89.97 82.74 Vision-OPD (Yuan et al., 2026) 81.00 86.52 74.33 89.80 82.91 OPD-V (Bi et al., 2026) 80.33 85.36 73.13 89.63 82.11 VAD (Zhang et al., 2026) 85.00 88.82 76.60 89.54 84.99 EAD (Ours) 84.33 88.11 76.40 88.73 84.39 A.3 Answer Extraction and Evaluation Evaluation protocol. All Qwen3.5 post-training variants are evaluated under the same inference and evaluation pipeline. To make multiple-choice grading more robust to free-form model responses, we revise the answer-extraction procedure in the released Vision-OPD evaluator and apply the revised evaluator uniformly to all variants. Answer extraction. For multiple-choice responses, the released evaluator extracts the predicted option using a sequence of regular-expression patterns. Its final fallback, ([A-Z]), may extract an uppercase letter from ordinary response text rather than the selected option. For example, for the response Answer: **D**, this fallback may match the A in Answer rather than the stated option D. Therefore, we remove the unrestricted uppercase-letter fallback and only extract option labels from explicit answer expressions or standard option formats. If no label can be resolved, the evaluator matches the response against the option texts and returns a prediction only when a single option is identified. When multiple explicit answer expressions appear, the last one is taken as the final choice. Only unresolved responses are passed to the fallback judge. Fallback judging. For unresolved responses, we use Qwen3.5-122B-A10B (Qwen Team, 2026) as a text-only fallback judge. The judge receives the question, reference answer, and generated response, but no image, and determines whether the response is equivalent to the reference. We use temperature zero with thinking disabled and constrain the output to Yes or No. Evaluation of the released Vision-OPD checkpoint. We evaluate the publicly released Vision-OPD 4B checkpoint with our revised evaluator. This result serves as an additional reference for comparison under a consistent evaluation protocol. Table 6: Released Vision-OPD 4B checkpoint under our evaluation pipeline. All results are obtained using the same revised evaluator applied to the Qwen3.5-based models in this work. Benchmark Vision-OPD 4B V*Bench 90.05 ZoomBench 59.29 HRBench-4K 82.25 HRBench-8K 79.25 MME-RealWorld-EN 70.71 MME-RealWorld-CN 69.43 MMVP 79.00 CV-Bench 86.82 MMStar 70.87 POPE 89.12 Fine Avg. 75.16 Holdout Avg. 81.45 All-10 77.68 A.4 Details of the Supervision Retention Analysis Retained supervision mass. Figure 4(a) shows the fraction of distillation loss retained by EAD over the 65 training steps. Let LsdenseL_s^dense and LsEADL_s^EAD denote the unweighted and EAD-weighted distillation losses at step s, respectively. The per-step and cumulative retained fractions are Rs=LsEADLsdense,Rā¤S=ās=1SLsEADās=1SLsdense.R_s= L_s^EADL_s^dense, R_ā¤ S= _s=1^SL_s^EAD _s=1^SL_s^dense. (9) Solid curves show RsR_s and dashed curves show Rā¤sR_ā¤ s. The final cumulative fractions are 5.87%5.87\% for 4B and 6.15%6.15\% for 9B. The dense reference is the unweighted loss on the same EAD training trajectories. Within-response supervision concentration. For Figure 4(b), we analyze 4B and 9B responses to the same 128 randomly sampled questions from training set. Token-level losses and EAD weights are computed using the step-65 Student and EMA Teacher checkpoints. For each response i, we sort the retained token losses ai,t=wi,tāDi,tāeāaācāhāeāra_i,t=w_i,tD^teacher_i,t in descending order. With nin_i valid tokens, the fraction of total loss carried by the top k tokens is Ci(kni)=āj=1kai,(j)āj=1niai,(j),k=0,ā¦,ni,C_i ( kn_i )= _j=1^ka_i,(j) _j=1^n_ia_i,(j), k=0,ā¦,n_i, (10) where ai,(j)a_i,(j) is the j-th largest retained token loss. The dense reference follows the same procedure, independently sorting the unweighted losses Di,tāeāaācāhāeārD^teacher_i,t. On-policy rollout accuracy. Figure 4(c) compares 4B/9B EAD and Vision-OPD over 65 training steps. Each step uses 96 questions with eight responses per question. All responses are scored using the same answer extractor; unresolved answers count as incorrect. Faint curves show raw per-step accuracy. Bold curves show the mean accuracy over the current step and the preceding four steps. Table 7: Summary of supervision retention, concentration, and rollout accuracy. All values are percentages. Top-10% loss mass measures how much of each responseās total loss comes from its 10% highest-loss tokens, averaged across responses. Rollout accuracy is computed over all responses from training steps 1ā65. Statistic 4B 9B Cumulative retained loss mass 5.87 6.15 Top-10% loss mass: EAD 95.98 96.81 Top-10% loss mass: dense 61.10 62.27 Training rollout accuracy: EAD 59.53 64.31 Training rollout accuracy: Vision-OPD 55.06 57.49 A.5 Held-Out Generalization of Evidence References We further evaluate the evidence-reference variants from Section 5.4 on four held-out benchmarks using the same trained Qwen3.5-4B checkpoints. Table 8 reports the held-out results together with the Fine Avg. from the six fine-grained benchmarks and the average over all ten benchmarks. Table 8: Held-out generalization of evidence-reference variants. The same checkpoints evaluated in Tables 2 and 3 are evaluated on MMVP, CV-Bench, MMStar, and POPE. Holdout Avg. is the unweighted mean over the four held-out benchmarks, Fine Avg. is the mean over the six fine-grained benchmarks, and All-10 averages all ten benchmarks. Evidence Reference MMVP CV-Bench MMStar POPE Holdout Avg. Fine Avg. All-10 Cross-state masking 77.00 85.49 73.53 86.66 80.67 75.98 77.86 Crop-induced change 78.33 86.61 70.87 89.11 81.23 75.16 77.59 Teacher-side masking 78.67 87.59 72.67 89.24 82.04 77.03 79.03 Teacher crop degradation 76.33 84.15 71.87 83.30 78.91 76.33 77.37 Current student (EAD) 80.33 86.95 73.07 87.70 82.01 77.53 79.33 The held-out results are consistent with the main fine-grained comparison. Both same-state OriginalāMask variants outperform the cross-state construction, supporting the benefit of removing teacherāstudent state differences from the evidence reference. Among the visual comparisons, OriginalāMask also generalizes better than the teacher-side crop-degradation reference. Although teacher-side masking achieves a marginally higher Holdout Avg. than EAD, EAD attains the strongest Fine Avg. and All-10 score while maintaining comparable held-out performance. A.6 Effect of Local JS Scaling EAD represents both the privileged teacher correction and the learner evidence response in the locally normalized coordinates motivated by the second-order approximation of JensenāShannon divergence. We ablate this design by directly computing cosine similarity between the corresponding probability differences without the 1/pĪø,tR1/ p_Īø,t^R scaling. Specifically, the unscaled variant uses u~tcorr=pĪøĀÆ,tCāpĪø,tR,u~tevid=pĪø,tRāpĪø,tRā, u^corr_t=p^C_ Īø,t-p^R_Īø,t, u^evid_t=p^R_Īø,t-p^R^-_Īø,t, (11) and computes w~t=maxā”(cosā”(u~tcorr,u~tevid),0). w_t= ( ( u^corr_t, u^evid_t ),0 ). (12) All other training settings are kept identical to EAD. Table 9: Effect of local JS scaling. Vanilla cosine directly compares the unscaled probability differences, whereas EAD applies the shared local normalization 1/maxā”(pĪø,tR,Ļµ)1/ (p_Īø,t^R,Īµ) before measuring directional alignment. Both variants otherwise use the same evidence reference and training procedure. Benchmark Vanilla Cosine EAD V*Bench 89.01 91.62 ZoomBench 59.29 60.83 HRBench-4K 85.12 86.38 HRBench-8K 82.38 82.88 MME-RealWorld-EN 72.37 72.39 MME-RealWorld-CN 71.29 71.10 MMVP 79.67 80.33 CV-Bench 87.03 86.95 MMStar 73.47 73.07 POPE 87.57 87.70 Fine Avg. 76.58 77.53 Holdout Avg. 81.94 82.01 All-10 78.72 79.33 As shown in Table 9, directly comparing the original probability differences already provides a meaningful alignment signal, while local normalization yields stronger performance. These results support the use of the shared local JS geometry. A.7 Effect of Excluding Negatively Aligned Corrections EAD uses positive cosine weighting, so teacher corrections with negative alignment to the evidence reference are excluded from supervision. We test this choice with a shifted cosine that retains negatively aligned corrections. Specifically, the shifted-cosine variant uses w~t=cosā”(utcorr,utevid)+12, w_t= (u_t^corr,u_t^evid )+12, (13) while keeping all other training settings unchanged. Table 10: Effect of excluding negatively aligned corrections. Shifted-Cosine retains negatively aligned corrections by mapping cosine similarity from [ā1,1][-1,1] to [0,1][0,1]. All other settings are unchanged. Benchmark Shifted-Cosine EAD V*Bench 90.58 91.62 ZoomBench 61.18 60.83 HRBench-4K 83.50 86.38 HRBench-8K 81.25 82.88 MME-RealWorld-EN 72.22 72.39 MME-RealWorld-CN 70.05 71.10 MMVP 77.33 80.33 CV-Bench 87.21 86.95 MMStar 73.93 73.07 POPE 88.88 87.70 Fine Avg. 76.46 77.53 Holdout Avg. 81.84 82.01 All-10 78.61 79.33 As shown in Table 10, Shifted-Cosine yields lower overall performance than EAD. These results support excluding opposing teacher corrections from supervision. Figure 5: Additional token-level case studies. Each example shows the original, cropped, and masked images together with the normalized token-wise JS divergence of DtteacherD_t^teacher, DtcropD_t^crop, and DtmaskD_t^mask. Across different fine-grained visual tasks, DtmaskD_t^mask tends to emphasize tokens more closely associated with the task-relevant visual evidence, while the other comparisons can assign large divergence to less directly related tokens. A.8 Additional Token-Level Case Studies Figure 5 provides additional examples across different fine-grained visual tasks, including attribute recognition, text recognition, shape recognition, and object identification. For each example, we visualize the normalized token-wise JS divergence of DtteacherD_t^teacher, DtcropD_t^crop, and DtmaskD_t^mask on the same generated response. Across these cases, DtmaskD_t^mask places greater emphasis on tokens associated with the task-relevant visual evidence, whereas DtcropD_t^crop and DtteacherD_t^teacher exhibit large divergence on tokens less directly related to that evidence. These examples are consistent with the observation in Figure 2 and further motivate using the OriginalāMask comparison as the evidence reference.