Paper deep dive
VAttnPO: Visual Attention-Guided Policy Optimization
Lihao Zheng, Zhenwei Shao, Zhou Yu, Yan Yang, Haoyi Sun, Qian Wang, Ning Mao, Pan Zhou
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal reinforcement learning with verifiable rewards commonly assigns one sequence advantage to an entire response. This outcome-level signal distinguishes overall response quality but leaves changes in visual-evidence use across generated tokens implicit. We introduce VAttnPO (Visual Attention-Guided Policy Optimization), which uses answer-conditioned visual attention to refine token-level credit while preserving the verifier-derived trajectory-mean advantage. A frozen old policy constructs a visual reference by teacher forcing the known correct answer and produces a visual distribution for each rollout token. VAttnPO measures their agreement in visual-attention space using Jensen--Shannon divergence, rather than reading guidance from teacher output-token distributions or introducing an attention-matching objective. Trajectory centering, relative-position weighting, and credit balancing turn this agreement into a zero-mean, position-aware correction to the GRPO sequence advantage. Token intervention, visual-index permutation, and trajectory analysis provide diagnostic evidence linking this signal to model behavior, visual correspondence, and generation position. On Qwen3-VL-8B-Instruct, VAttnPO raises six-task image AVE from 70.97 with GRPO to 73.50 and scores higher on every reported structured, video, and MME metric. Gains over GRPO also hold on Qwen2.5-VL-7B-Instruct and InternVL3-8B-Instruct.
Tags
Links
- Source: https://arxiv.org/abs/2610.16735v1
- Canonical: https://arxiv.org/abs/2610.16735v1
Trouble viewing inline? Open PDF directly →
Full Text
80,163 characters extracted from source content.
Expand or collapse full text
VAttnPO: Visual Attention-Guided Policy Optimization Lihao Zheng Zhenwei Shao School of Computer Science and Technology, School of Computer Science and Technology, Hangzhou Dianzi University Hangzhou Dianzi University Zhou Yu Yan Yang School of Computer Science and Technology, School of Computer Science and Technology, Hangzhou Dianzi University Hangzhou Dianzi University Haoyi Sun Qian Wang Base Model, Li Auto Base Model, Li Auto Ning Mao Pan Zhou Base Model, Li Auto Base Model, Li Auto Abstract Multimodal reinforcement learning with verifiable rewards commonly assigns one sequence advantage to an entire response. This outcome-level signal distinguishes overall response quality but leaves changes in visual-evidence use across generated tokens implicit. We introduce VAttnPO (Visual Attention-Guided Policy Optimization), which uses answer-conditioned visual attention to refine token-level credit while preserving the verifier-derived trajectory-mean advantage. A frozen old policy constructs a visual reference by teacher forcing the known correct answer and produces a visual distribution for each rollout token. VAttnPO measures their agreement in visual-attention space using Jensen–Shannon divergence, rather than reading guidance from teacher output-token distributions or introducing an attention-matching objective. Trajectory centering, relative-position weighting, and credit balancing turn this agreement into a zero-mean, position-aware correction to the GRPO sequence advantage. Token intervention, visual-index permutation, and trajectory analysis provide diagnostic evidence linking this signal to model behavior, visual correspondence, and generation position. On Qwen3-VL-8B-Instruct, VAttnPO raises six-task image AVE from 70.97 with GRPO to 73.50 and scores higher on every reported structured, video, and MME metric. Gains over GRPO also hold on Qwen2.5-VL-7B-Instruct and InternVL3-8B-Instruct. 1 Introduction Multimodal large language models are advancing rapidly, spanning tool-integrated reasoning, streaming perception, multi-image grounding, policy optimization, and large-scale evaluation (Chen et al., 2026b; Shen et al., 2026; Chen et al., 2026a; Zheng et al., 2026; Han et al., 2025; Xu et al., 2025). Reinforcement learning with verifiable rewards (RLVR) uses automatic feedback, such as answer correctness and format compliance, to improve reasoning. Methods such as GRPO estimate policy advantages from relative rewards among multiple responses to the same prompt, avoid training a separate value network, and have expanded from text reasoning to vision-language models (Shao et al., 2024; Guo et al., 2025; Huang et al., 2025; Shen et al., 2025; Meng et al., 2025; Chen et al., 2025). Yet the environment usually returns only one scalar reward per response, and the resulting sequence advantage is applied to the entire generated trajectory. Every token in a response therefore receives the same advantage coefficient, leaving differences in credit within the trajectory implicit, even when individual tokens rely on different visual evidence. This granularity gap is especially important in multimodal reasoning: a correct answer does not imply consistent use of relevant visual regions, and equally rewarded responses may follow different visual reasoning paths. Existing fine-grained approaches rely on preference or process supervision (Yu et al., 2024; Wang et al., 2025a), or read dense guidance from teacher- and privileged-context output-token distributions (Agarwal et al., 2024; Yang et al., 2026b; Pan et al., 2026). A vision-language model also produces attention over visual inputs at every generated token, providing an internal readout distinct from output behavior. We treat visual attention as a proxy to be evaluated through behavioral diagnostics and controlled training comparisons, rather than as a causal attribution or certificate of correctness (Liu et al., 2026; Zhao et al., 2026a; Lu et al., 2026; Yang et al., 2026a). This leads to our central question: Can token-varying internal visual statistics refine credit within a trajectory while preserving the verifier-derived trajectory-mean advantage? Attention magnitude itself has no intrinsic task direction, so directly rewarding larger attention does not address this question. We instead use the visual distribution formed by the frozen old policy πθold _ _old under the known correct answer as a reference and measure each generated token’s agreement with it. Token intervention tests whether attention rank tracks behavioral sensitivity; visual-index permutation tests whether correct–incorrect separation depends on patch identity rather than the reference’s marginal shape; and trajectory analysis tests whether this separation varies with generation position. These diagnostics provide evidence for a behaviorally relevant token proxy, reference-patch correspondence, and position-dependent variation. The position-dependent alignment profile further motivates us to adopt a smooth, bounded relative-position sigmoid and evaluate this design through controlled ablations. Motivated by this evidence, we introduce VAttnPO. Within each update, the frozen old policy both samples responses and produces the visual reference by teacher forcing the correct answer, placing reference and rollout attention in the same parameter state; the answer supplies conditioning context without process labels or region annotations. VAttnPO converts token-wise visual similarity into trajectory-centered credit, modulates it with a smooth bounded function of relative position, and balances it into a zero-sum correction. Adding this correction to the GRPO sequence advantage allows token coefficients to vary while preserving the verifier-derived trajectory-mean advantage. To summarize, our contributions are as follows: • Diagnostic Evidence. We identify behavioral, spatial, and positional structure in internal visual attention that motivates token-level credit differentiation. • Visual Attention-Guided Credit. VAttnPO derives a token-level correction from answer-conditioned visual attention through similarity centering, relative-position weighting, and sign-preserving balancing, varying credit within a response while preserving the trajectory-mean advantage. • Empirical Results. On Qwen3-VL-8B-Instruct, VAttnPO improves AVE over six image tasks by 2.53 points over GRPO and records higher scores on every reported metric for structured understanding, video understanding, and MME. 2 Related Work 2.1 Multimodal RLVR and Credit Assignment GRPO and DeepSeek-R1 establish group-relative RLVR (Shao et al., 2024; Guo et al., 2025); Vision-R1, VLM-R1, M-Eureka, R1-V, and Video-R1 extend it to image and video reasoning (Huang et al., 2025; Shen et al., 2025; Meng et al., 2025; Chen et al., 2025; Feng et al., 2025), while SAPO further modifies policy optimization (Gao et al., 2025). These methods retain response-level verifiable outcomes as task supervision, and standard GRPO shares one sequence advantage across a completion. Preference learning and process supervision offer finer feedback but generally require preference data, step labels, a process verifier, or an additional reward model (Yu et al., 2024; Sun et al., 2024; Li et al., 2023a; Lightman et al., 2024; Wang et al., 2024b; Wang et al., 2025a). VAttnPO adds none of these sources and leaves verifier rewards unchanged; it only redistributes the resulting sequence advantage within each response. 2.2 On-policy Distillation and Self-Distilled RL GKD and OPD provide token-level targets from teacher output distributions on student trajectories (Agarwal et al., 2024; Lu and Lab, 2025), while OPSD uses the same model as teacher and student under different contexts (Zhao et al., 2026b). RLSD reweights verifier-derived advantages using token-likelihood relations between original and correct-answer contexts (Yang et al., 2026b), whereas RLCSD contrasts policy distributions under correct and incorrect hints (Pan et al., 2026). All read reference behavior from output-token distributions. VAttnPO instead reads an answer-conditioned visual-attention distribution and constructs a zero-sum correction to verifier-derived advantages, without an output-distribution or attention-matching objective; verifiable rewards remain the source of task supervision. 2.3 Attention-Derived Supervision and Visual Credit Attention transfer and distillation match teacher attention maps for knowledge transfer (Zagoruyko and Komodakis, 2016; Zhou et al., 2025), whereas diagnostic and intervention work studies links between visual attention, grounding, hallucination, and behavior (Liu et al., 2026; Zhao et al., 2026a; Lu et al., 2026; Yang et al., 2026a). These studies show that attention is neither a causal explanation nor a certificate of correctness, yet it is not behaviorally arbitrary. VAttnPO neither increases attention at prescribed locations nor matches attention maps: the same frozen old policy produces the answer-conditioned reference and rollout attention, and the derived statistics are used only to construct the token-level GRPO correction. 3 Pilot Study We examine whether visual attention tracks behavioral sensitivity, whether correctness-related separation depends on visual-patch correspondence, and how this separation varies over completion progress. The answer-conditioned reference PgtP_gt pools the frozen model’s attention over correct-answer queries and normalizes it over visual patches (Equation (2)); lower JSD indicates that a rollout token’s visual distribution is closer to this reference. We conduct three targeted pilot studies to examine these questions. All studies use layer 20 of Qwen3-VL-8B-Instruct, the same fixed 200 examples, and G=8G=8 responses per input. The analysis includes 1,429 valid trajectories, comprising 771 correct and 658 incorrect responses. Correct–incorrect statistics are aggregated within input before inputs are weighted equally, with input-cluster bootstrap intervals; Appendix E provides additional protocol details. (a) Token intervention (b) GT vs. shuffled reference (c) Alignment profiles Figure 1: Diagnostic evidence from visual attention. (a) Paired Cohen’s d for High tokens against Low, Random, and Position-matched controls. (b) Wrong-minus-correct JSD gaps under the correct-answer and patch-shuffled references. (c) Mean JSD for correct and incorrect responses and their wrong-minus-correct gap over completion progress. The final point denotes the last valid marker. Error bars in (a) and (b), and shaded bands around the correct and incorrect curves in (c), show confidence intervals obtained by bootstrap resampling at the example level. Token Intervention. We first test whether attention rank tracks behavioral sensitivity. Using the top 20% of visual patches under PgtP_gt as the reference region, we rank tokens by the preceding query’s normalized attention to that region and form equal-size high, low, random, and position-matched sets. We then mask the selected tokens’ complete self-attention key–value columns. Over all responses, high-attention tokens cause larger drops in the original answer log-probability than all controls, with paired Cohen’s d=0.85d=0.85–1.011.01 and intervals excluding zero; high and position-matched sets have similar mean positions. Figure 1(a) additionally reports the correct and incorrect response subsets, and its legend abbreviates Position-matched as Matched. Attention rank therefore distinguishes tokens with different intervention effects even after matching average position. GT Versus Shuffled Reference. We compare rollout-token JSD to the correct-answer reference and to a visual-index-permuted version that preserves its attention values and entropy but breaks visual correspondence. Incorrect trajectories are farther from the correct-answer reference on average, whereas the shuffled-reference gap is near zero; the interval for the difference between these gaps excludes zero. In Figure 1(b), True denotes the wrong-minus-correct gap under the correct-answer reference, Shuffled denotes the gap under the permuted reference, and Specificity is the True gap minus the Shuffled gap. Correct–incorrect separation therefore depends on reference-patch correspondence rather than only the marginal distribution of reference values. Alignment Over Completion Progress. Correct responses have lower mean JSD than incorrect responses at every displayed position. The wrong-minus-correct gap varies non-monotonically and is largest at the final valid token, showing position-dependent alignment without implying a general late-reasoning mechanism (Figure 1(c)). These diagnostics motivate using answer-conditioned visual attention to differentiate tokens and examining its variation over generation progress. The position-dependent alignment profile motivates us to adopt a smooth, bounded relative-position sigmoid and evaluate this design through controlled ablations in Section 5.3. Centering and credit balancing preserve the trajectory mean of the verifier-derived advantage. 4 Method VAttnPO adds a token-level visual correction to the GRPO sequence advantage while preserving its trajectory mean. The verifiable reward and its group-relative advantage evaluate the complete response, while the internal visual signal redistributes that advantage within the response. Constructing this correction requires explicit constraints: the answer-conditioned reference supplies task direction, trajectory centering removes response-wide offset, bounded position weights modulate local credit, and credit balancing restores zero mean while retaining nonzero credit signs. The three stages construct reference and rollout attention, compute centered credit, and form position-aware token advantages. Concretely, PgtP_gt and Qi,tQ_i,t produce visual similarity si,ts_i,t; trajectory centering yields ri,tr_i,t; position modulation forms ui,tu_i,t; and credit balancing produces di,td_i,t. The final coefficient A~i,t=A^i+αdi,t A_i,t= A_i+α d_i,t retains A^i A_i as the verifier-derived response-level evaluation while di,td_i,t controls its local allocation within the response. Figure 2: VAttnPO overview. Within each update, the frozen old policy extracts visual distributions by teacher forcing the correct answer and by processing sampled responses. Their similarity is centered within each trajectory, weighted by relative position, and balanced into a token correction of the sequence advantage. 4.1 Preliminaries Given multimodal input x, the frozen old policy πθold _ _old samples G responses yi=(yi,1,…,yi,Li)y_i=(y_i,1,…,y_i,L_i), where LiL_i is the valid completion length. The old policy remains fixed within an update, placing sampling probabilities, response attention, and the visual reference obtained under the correct answer in the same parameter state. The policy πref _ref used for KL regularization is a separate frozen model with a different role. The verifiable reward RiR_i gives the group-relative advantage A^i=Ri−mean(R1:G)std(R1:G)+ϵ. A_i= R_i-mean(R_1:G)std(R_1:G)+ε. (1) Standard GRPO broadcasts A^i A_i to every valid token, so token-level differences arise only from likelihood ratios, clipping states, and score functions rather than explicit credit. VAttnPO leaves RiR_i and the group-normalized sequence advantage A^i A_i unchanged. It replaces the broadcast A^i A_i with token-varying A~i,t A_i,t having the same trajectory mean. Figure 2 summarizes the path from visual attention to these coefficients. 4.2 Stage 1: Visual Reference and Rollout Attention For each input, we run two forward paths through the same πθold _ _old. The reference branch teacher-forces the known correct answer ygty^gt, while the response branch reads attention from responses sampled by that old policy. Both paths share model parameters, input visual tokenization, and the target attention layer. Therefore, their comparison focuses on the distribution of visual attention generated under the same strategy state across two text conditions, rather than on representation drift across models or updates. At the fixed layer, we retain only post-softmax attention from textual queries to the N visual-patch keys. Let gtm,h,:∈ℝ+Na^gt_m,h,: _+^N and (i)t,h,:∈ℝ+Na^(i)_t,h,: _+^N denote visual attention at correct-answer token m and rollout token t for head h. The reference branch pools over LgtL_gt valid answer queries and H heads, whereas the rollout branch retains one distribution for every valid generation position. Normalizing the pooled weights over visual patches gives Pgt=Norm(1LgtH∑m=1Lgt∑h=1Hm,h,:gt),Qi,t=Norm(1H∑h=1Ht,h,:(i)).P_gt=Norm\! ( 1L_gtH _m=1^L_gt _h=1^Ha^gt_m,h,: ), Q_i,t=Norm\! ( 1H _h=1^Ha^(i)_t,h,: ). (2) All responses to an input share PgtP_gt and are therefore compared with the same visual reference. The reference is obtained from the old policy under the known correct answer; the answer supplies conditioning context but no region supervision. After generation, VAttnPO assigns Qi,tQ_i,t to token t at the same index, enabling relative position normalization by the final valid length without altering sampling. Appendix C outlines attention aggregation and the query-index distinction from the intervention in Section 3. Appendix I.1 examines the layer-wise profile of correctness-related visual-attention separation across three backbones, while Appendix I.2 illustrates spatial attention patterns across layers. 4.3 Stage 2: Token-Level Visual Credit Both Qi,tQ_i,t and PgtP_gt are probability distributions over the same visual-patch support. We use Jensen–Shannon divergence (JSD) because it is symmetric, always finite, and bounded above by the sample-independent constant ln2 2. These properties provide a stable mapping from distance to [0,1][0,1] without requiring identical nonzero support. The resulting bounded similarity is si,t=1−JSD(Qi,t∥Pgt)ln2∈[0,1].s_i,t=1- JSD(Q_i,t\|P_gt) 2∈[0,1]. (3) Absolute similarity can vary with task content, the size of the visual support, and the concentration of the reference distribution. Treating si,ts_i,t directly as an added reward would carry the similarity offset shared by a response into local credit and could duplicate sequence-level differences. We therefore subtract the mean over valid generated tokens: ri,t=si,t−1Li∑u=1Lisi,u,∑t=1Liri,t=0.r_i,t=s_i,t- 1L_i _u=1^L_is_i,u, _t=1^L_ir_i,t=0. (4) The sign of ri,tr_i,t indicates whether token t is more or less aligned than its trajectory average. Centering removes the response-wide offset and yields zero-sum local credit. It preserves every within-trajectory similarity difference because ri,t−ri,u=si,t−si,ur_i,t-r_i,u=s_i,t-s_i,u, but does not rescale by variance, so different responses retain their own credit dispersion. 4.4 Stage 3: Position-Aware Advantage Construction Position Modulation. We use a smooth, bounded position weight to modulate visual credit over completion progress without directly scaling A^i A_i. For Li≥2L_i≥ 2, let pi,t=(t−1)/(Li−1)p_i,t=(t-1)/(L_i-1) denote relative completion position. With κ>0κ>0, location parameter μ, and 0<gmin≤10<g_ ≤ 1, the endpoint-normalized sigmoid, position weight, and weighted credit are Sκ(p) S_κ(p) =σ(κ(p−μ))−σ(−κμ)σ(κ(1−μ))−σ(−κμ), = σ(κ(p-μ))-σ(-κμ)σ(κ(1-μ))-σ(-κμ), (5) gi,tpos g^pos_i,t =gmin+2(1−gmin)Sκ(pi,t), =g_ +2(1-g_ )S_κ(p_i,t), ui,t u_i,t =gi,tposri,t. =g^pos_i,tr_i,t. Here, μ locates the sigmoid transition, κ controls its steepness, and gming_ sets the bounded weight range from gming_ to 2−gmin2-g_ . When μ=0.5μ=0.5, the midpoint weight is one. The construction uses only valid length and relative position, placing different-length responses on the same smooth progress coordinate; for Li=1L_i=1, we use unit weight. Position alone creates no independent advantage and changes only the magnitude of existing positive and negative visual credits. Credit Balancing. Although ∑tri,t=0 _tr_i,t=0, position modulation generally gives ∑tui,t≠0 _tu_i,t≠ 0. Directly re-centering iu_i would restore zero sum but could change a local credit’s sign. Let [z]+=max(z,0)[z]_+= (z,0). We compute positive and negative weighted-credit totals separately and scale both sides to the smaller total: Pi+=∑t[ui,t]+,Ni−=∑t[−ui,t]+,Ci=min(Pi+,Ni−).P_i^+= _t[u_i,t]_+, N_i^-= _t[-u_i,t]_+, C_i= (P_i^+,N_i^-). (6) di,t=Ci([ui,t]+Pi+−[−ui,t]+Ni−),∑tdi,t=0.d_i,t=C_i ( [u_i,t]_+P_i^+- [-u_i,t]_+N_i^- ), _td_i,t=0. (7) CiC_i is the largest credit mass that both signs can retain, so the smaller side remains unchanged and only the larger side is contracted. In a nondegenerate trajectory, both scaling factors lie in (0,1](0,1]; hence each di,td_i,t retains the sign of its corresponding ui,tu_i,t without increasing its magnitude, while the positive and negative totals cancel exactly. If either total does not exceed the numerical threshold τ, the entire id_i is set to zero. Policy Update. The final token advantage uses correction strength α: A~i,t=A^i+αdi,t,∑t=1LiA~i,t=LiA^i. A_i,t= A_i+α d_i,t, _t=1^L_i A_i,t=L_i A_i. (8) Thus, αdi,tα d_i,t changes token coefficients within a response, while ∑tdi,t=0 _td_i,t=0 preserves the trajectory mean of the verifier-derived sequence advantage. All attention, similarity, position-weight, and balance statistics are stop-gradient. During training, A~i,t A_i,t directly replaces the broadcast A^i A_i in the clipped GRPO objective, while likelihood ratios, clipping, and KL regularization against the separate frozen policy πref _ref remain unchanged. Appendix C gives the full objective, degenerate branches, and Algorithm 1; Appendix D provides the corresponding properties and derivations. 5 Experiments 5.1 Experimental Setup All primary experiments start from Qwen3-VL-8B-Instruct (Bai et al., 2025a) and train directly, without SFT warm-up, on a fixed set of 168,460 multiple-choice image–video examples from Video-R1-260k (Feng et al., 2025). Except in the training-step ablation, the training budget is 400 optimization steps. We post-train separate models with GRPO (Shao et al., 2024), SAPO (Gao et al., 2025), On-policy GKD (Agarwal et al., 2024), OPD (Lu and Lab, 2025), OPSD (Zhao et al., 2026b), RLSD (Yang et al., 2026b), RLCSD (Pan et al., 2026), and VAttnPO. Each method retains its original objective while sharing the training data, optimization budget, and evaluation protocol. GRPO and VAttnPO additionally share the verifiable reward, sampling, clipping, and KL regularization; their sole method difference is that VAttnPO replaces the broadcast sequence advantage with Equation (8). Appendices A and B provide the full data composition, reward, sampling, and method parameters. Table 1 reports VStar (Wu and Xie, 2024), HRBench (Wang et al., 2025b), MMVP (Tong et al., 2024), MMMU-Pro (Yue et al., 2024), MuirBench (Wang et al., 2024a), and BLINK (Fu et al., 2024), whose mean is AVE. These tasks span general visual reasoning, multidisciplinary understanding, multi-image relations, fine-grained perception, and visual pattern discrimination, so AVE summarizes complementary image capabilities rather than mixing every metric into one scale. Table 2 separately reports ChartQA (Masry et al., 2022), OCRBench (Liu et al., 2024b), DocVQA (Mathew et al., 2021), InfoVQA (Mathew et al., 2022), POPE (Li et al., 2023b), HallusionBench (Guan et al., 2024), and CountBenchQA (Beyer et al., 2024). Table 3 reports MVBench (Li et al., 2024), LongVideoBench (Wu et al., 2024), TempCompass (Liu et al., 2024a), Video-MME (Fu et al., 2025b), and MME (Fu et al., 2025a). ChartQA, all video metrics, MME, and the heterogeneously scaled metrics in Table 2 do not enter AVE. 5.2 Main Results Table 1: Image reasoning and fine-grained perception. AVE is the mean of the six tasks shown. Best results are shown in bold. Model Type Method Fine-grained Perception Reasoning AVE VStar HRBench 4K MMVP MMMU-Pro MuirBench BLINK Base Base model 81.15 77.38 80.00 52.95 60.35 67.18 69.84 RLVR GRPO 83.25 77.50 81.00 54.28 61.54 68.23 70.97 SAPO 84.29 77.75 81.33 53.82 60.27 68.33 70.97 On-policy Distillation GKD 79.06 79.00 76.33 53.87 62.62 63.97 69.14 OPD 85.34 78.75 79.33 54.57 63.65 65.97 71.27 OPSD 82.72 80.13 80.67 54.74 60.46 66.65 70.90 Self-distilled RL RLSD 82.20 77.62 82.00 53.18 60.58 65.65 70.21 RLCSD 83.77 77.50 79.67 54.10 61.85 69.02 70.99 Ours VAttnPO 87.43 80.62 82.33 55.95 65.62 69.07 73.50 Image Reasoning and Fine-Frained Perception. VAttnPO raises AVE from 70.97 with GRPO to 73.50 (+2.53+2.53) and exceeds the strongest compared baseline, OPD at 71.27, by 2.23 points. All six image tasks improve over GRPO: the largest gains are 4.18 on VStar and 4.08 on MuirBench, while the other four tasks also remain positive. This distribution shows that the average gain spans both fine-grained perception and visual reasoning rather than being driven by one benchmark. Table 2: Structured, document, and reliability results. HallusionBench reports aAcc only. Best results are shown in bold. Model Type Method Structured & Document Hallucination CountBenchQA ChartQA OCRBench DocVQA InfoVQA POPE HallusionBench Base Base model 82.72 88.10 95.27 82.38 87.77 70.87 88.71 RLVR GRPO 82.92 89.50 95.40 82.97 88.21 71.61 89.53 SAPO 82.92 89.50 95.41 82.44 88.19 70.56 89.94 On-policy Distillation GKD 81.36 88.70 94.86 82.11 88.44 71.50 85.42 OPD 81.44 88.70 95.04 82.18 87.58 73.29 88.71 OPSD 82.60 90.00 95.26 82.38 88.44 71.19 89.73 Self-distilled RL RLSD 81.44 89.50 94.87 81.89 88.39 70.14 89.12 RLCSD 83.20 89.40 95.24 82.85 87.92 72.03 89.12 Ours VAttnPO 83.84 90.10 95.59 83.64 88.67 73.82 91.17 Structured, Document, and Reliability. VAttnPO obtains the highest score among compared methods on all seven structured, document, and reliability metrics in Table 2. Its largest margins over GRPO are 2.21 points on HallusionBench aAcc and 1.64 on CountBenchQA (Beyer et al., 2024), whereas gains on document tasks are smaller, including 0.19 on DocVQA. The magnitudes are therefore heterogeneous, but their direction extends the improvement beyond the six image tasks to visual reliability, counting, and document understanding. Table 3: Video understanding and MME. MME-per and MME-rea denote MME-Perception and MME-Reasoning. Best results are shown in bold. Model Type Method Video Understanding MME MVBench LongVideoBench TempCompass Video-MME MME-per MME-rea Base Base model 70.72 62.2 69.10 67.7 1724.7 671.4 RLVR GRPO 71.25 62.8 70.04 68.4 1736.1 673.9 SAPO 71.58 62.9 69.97 68.4 1733.2 652.1 On-policy Distillation GKD 68.53 57.3 70.04 64.4 1722.7 656.8 OPD 70.97 62.1 71.38 66.6 1727.5 681.8 OPSD 70.83 62.7 70.89 67.6 1732.4 657.9 Self-distilled RL RLSD 70.08 61.0 68.77 66.4 1724.6 658.9 RLCSD 70.60 62.6 70.46 68.0 1732.3 661.1 Ours VAttnPO 72.15 63.6 71.58 68.6 1755.0 683.9 Video Understanding and MME. VAttnPO’s gains extend beyond the six image tasks. It improves all four video benchmarks over GRPO, by 0.90 on MVBench, 0.8 on LongVideoBench, and 1.54 on TempCompass, with a smaller margin of 0.2 on Video-MME. It also improves MME-per and MME-rea by 18.9 and 10.0 points, respectively (Table 3). Consequently, these performance improvements span short-form video, long-form video, temporal aspects, and other multimodal evaluations, with the magnitude of improvement varying across different benchmarks. 5.3 Ablation and Controlled Analysis We examine VAttnPO through controls on its components and similarity metric, position design, reference and token correspondence, and credit balancing. Figure 3 additionally compares data compositions and training checkpoints. Every AVE uses the six tasks in Table 1. Table 4: Four controlled ablation groups. AVE denotes the mean over the six image tasks in Table 1. (a) Core components and metric Variant Metric Position Balance AVE GRPO None No No 70.97 Position-only None Yes Yes 70.84 KL variant KL Yes Yes 71.37 No position JSD No Yes 72.35 VAttnPO JSD Yes Yes 73.50 (b) Position design Variant Setting AVE GRPO No visual credit 70.97 Position-only Pure position 70.84 No position Uniform 72.35 Position-shuffled Shuffled sigmoid 72.19 VAttnPO Relative sigmoid 73.50 (c) Reference and correspondence Variant Reference Token credit AVE GRPO None N/A 70.97 Wrong ref. Wrong GT Aligned 68.92 Shuffled ref. Shuffled GT Aligned 70.46 Token-permuted GT Permuted 70.86 VAttnPO GT Aligned 73.50 (d) Credit balance Variant Guarantees zero sum Preserves credit sign AVE GRPO N/A N/A 70.97 No balance No Yes 70.84 Mean recenter Yes No 71.35 VAttnPO Yes Yes 73.50 Core components and metric. Position-only replaces visual credit with a centered, RMS-calibrated position curve; No position retains visual credit with uniform position weights; and the KL variant replaces JSD similarity with reference smoothing, forward KL, and an exponential map while retaining position modulation and credit balancing. Position-only reaches 70.84 AVE, close to GRPO at 70.97; No position reaches 72.35, and the full VAttnPO reaches 73.50. Visual credit therefore contributes beyond a positional prior, with position modulation providing a further gain. The KL variant reaches 71.37; this comparison concerns only the construction defined in Appendix F and leaves policy KL regularization unchanged. Position Design. Position-shuffled permutes the same position weights within each response, preserving their values while breaking their correspondence with tokens. Relative-position weighting raises AVE from 72.35 for No position to 73.50, whereas Position-shuffled reaches 72.19. Together with Position-only at 70.84, this pattern favors coupling visual credit to its original relative positions. Reference And Token Correspondence. Wrong-GT reference replaces the teacher-forced correct option with a deterministic incorrect option; Shuffled reference permutes only the reference’s visual-patch indices; and Token-permuted shuffles centered credits ri,tr_i,t within each response. They reach 68.92, 70.46, and 70.86 AVE, respectively, compared with 73.50 for the complete VAttnPO, showing that reference content, visual correspondence, and token correspondence each affect the construction. Credit Balancing. No balance directly uses the position-weighted credit ui,tu_i,t; Mean recenter restores zero sum by subtracting the trajectory mean; and the full rule rescales positive and negative credits separately to retain local signs. They reach 70.84, 71.35, and 73.50 AVE, respectively, supporting the combination of zero-sum restoration and local sign preservation. (a) Data composition (b) Checkpoint comparison Figure 3: (a) AVE under image-only and image–video training. (b) AVE at steps 200 and 400. Data Composition. VAttnPO improves over GRPO with both image-only and image–video training, with a larger gain on the mixed data: +0.88+0.88 and +2.53+2.53 AVE, respectively (Figure 3(a)). The variant without position weighting lies between GRPO and the full method in both settings. Checkpoint Comparison. VAttnPO leads GRPO and the variant without position weighting at both evaluated checkpoints, reaching 72.52 and 73.50 AVE at steps 200 and 400, respectively (Figure 3(b)). This comparison concerns two discrete training checkpoints rather than a continuous convergence trend. 5.4 Cross-Backbone Generalization Table 5: Cross-backbone AVE. Δ denotes the VAttnPO gain over GRPO. Backbone Base GRPO VAttnPO Δ Qwen2.5-VL-7B-Instruct 59.44 61.64 62.55 +0.91 InternVL3-8B-Instruct 59.54 60.46 61.03 +0.56 VAttnPO continues to demonstrate performance gains over GRPO across two additional backbone models. Specifically, VAttnPO improves the AVE by 0.91 on Qwen2.5-VL-7B (Bai et al., 2025b) and by 0.56 on InternVL3-8B (Zhu et al., 2025). The detail results for each benchmark are listed in Appendix H. 6 Conclusion We introduced VAttnPO, which converts answer-conditioned visual attention from a frozen old policy into a zero-mean, position-aware correction to the GRPO sequence advantage. On Qwen3-VL-8B-Instruct, VAttnPO raises six-task image AVE from 70.97 to 73.50, scores higher than GRPO on every reported structured, video, and MME metric, and retains positive gains on two additional backbones. Together, the diagnostics and controlled comparisons show that internal visual statistics can refine within-response credit while preserving the trajectory-mean advantage. AI use statement We used generative AI tools to polish the writing, check mathematical derivations and formulas, and assist with writing and running parts of the research code, including submitting training jobs. The authors reviewed the edited text, verified the mathematical arguments, and inspected the AI-assisted code and resulting runs. We take full responsibility for the final manuscript and results. Reproducibility Statement We document the training-data filtering and composition in Appendix A, and report optimization settings, sampling parameters, random seeds, and the software and hardware environment in Appendix B. Section 4 and Appendix C specify the visual-reference construction, attention aggregation, numerical handling, and policy objective, with the update procedure summarized in Algorithm 1. Derivations of the credit-balancing and trajectory-mean preservation properties appear in Appendix D. Diagnostic protocols, statistical procedures, controlled variants, and benchmark evaluation are described in Appendices E–G. Appendices H and I provide additional cross-backbone results and layer-wise analyses. References Agarwal et al. (2024) R. Agarwal, N. Vieillard, Y. Zhou, P. Stanczyk, S. R. Garea, M. Geist, and O. Bachem On-policy distillation of language models: learning from self-generated mistakes. In International Conference on Learning Representations, Cited by: §1, §2.2, §5.1. Bai et al. (2025a) S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: §5.1. Bai et al. (2025b) S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923. Cited by: §5.4. Beyer et al. (2024) L. Beyer, A. Steiner, A. S. Pinto, A. Kolesnikov, X. Wang, D. Salz, M. Neumann, I. Alabdulmohsin, M. Tschannen, E. Bugliarello, et al. PaliGemma: a versatile 3B VLM for transfer. arXiv preprint arXiv:2407.07726. Cited by: §5.1, §5.2. Chen et al. (2026a) J. Chen, Z. Chen, C. Du, M. He, W. He, H. Li, Q. Li, Z. Liu, H. Ma, X. Pan, C. Ren, X. Rao, X. Shen, C. Wang, T. Wei, C. Yu, P. Yu, S. Yao, C. Zhou, K. Zhan, L. Zheng, P. Zhou, X. Zhu, and Y. Zheng StreamingClaw technical report. External Links: 2603.22120, Link Cited by: §1. Chen et al. (2026b) J. Chen, X. Shen, L. Zheng, Z. Shao, H. Cui, C. Du, L. Gong, F. Gu, X. Hao, W. He, J. He, Y. Hu, B. Huang, S. Li, Q. Li, J. Luo, Z. Liu, X. Liu, N. Mao, L. Mu, X. Pan, Z. Qu, C. Ren, X. Rao, H. Sun, Q. Wang, S. Wang, Z. Wang, W. Wang, L. Wen, J. Zhan, H. Yang, S. Yang, J. Yang, P. Yu, H. Zhang, B. Zhang, C. Zhou, Z. Zhou, S. Zhou, S. Xie, Y. Zhu, H. Ma, T. Wei, P. Zhou, and W. Chen MindWatcher: toward smarter multimodal tool-integrated reasoning. External Links: 2512.23412, Link Cited by: §1. Chen et al. (2025) L. Chen, L. Li, H. Zhao, Y. Song, and Vinci R1-v: reinforcing super generalization ability in vision-language models with less than $3. Note: GitHub repository Cited by: §1, §2.1. Chen et al. (2024) L. Chen, J. Li, X. Dong, P. Zhang, Y. Zang, Z. Chen, H. Duan, J. Wang, Y. Qiao, D. Lin, et al. Are we on the right way for evaluating large vision-language models?. Advances in Neural Information Processing Systems. Cited by: Figure I.1. Duan et al. (2026) H. Duan, X. Fang, J. Yang, X. Zhao, Z. Ma, Y. Qiao, M. Li, T. Liang, L. Zhu, A. Agarwal, X. Li, S. Ding, J. Bu, Z. Liu, Z. Qi, Y. Li, Y. Zang, Z. Chen, L. Chen, Y. Liu, Y. Ma, H. Sun, Y. Zhang, S. Lu, T. H. Wong, W. Wang, P. Zhou, C. Fu, J. Cui, J. Chen, E. Song, S. Mao, J. Lin, X. Wei, J. Li, Z. Sun, Z. Wang, Z. Zhang, X. Dong, J. He, P. Zhang, J. Wang, D. Lin, and K. Chen VLMEvalKit: an open-source toolkit for evaluating large multi-modality models. External Links: 2407.11691, Link Cited by: Appendix G. Feng et al. (2025) K. Feng, K. Gong, B. Li, Z. Guo, Y. Wang, T. Peng, J. Wu, X. Zhang, B. Wang, and X. Yue Video-r1: reinforcing video reasoning in mllms. In Advances in Neural Information Processing Systems, Vol. 38, p. 99114–99137. Cited by: §A.1, §2.1, §5.1. Fu et al. (2025a) C. Fu, P. Chen, Y. Shen, Y. Qin, M. Zhang, X. Lin, J. Yang, X. Zheng, K. Li, X. Sun, et al. MME: a comprehensive evaluation benchmark for multimodal large language models. In NeurIPS Datasets and Benchmarks Track, Cited by: §5.1. Fu et al. (2025b) C. Fu, Y. Dai, Y. Luo, L. Li, S. Ren, R. Zhang, Z. Wang, C. Zhou, Y. Shen, M. Zhang, et al. Video-MME: the first-ever comprehensive evaluation benchmark of multi-modal llms in video analysis. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §5.1. Fu et al. (2024) X. Fu, Y. Hu, B. Li, Y. Feng, H. Wang, X. Lin, D. Roth, N. A. Smith, W. Ma, and R. Krishna BLINK: multimodal large language models can see but not perceive. arXiv preprint arXiv:2404.12390. Cited by: §5.1. Gao et al. (2025) C. Gao, C. Zheng, X. Chen, K. Dang, S. Liu, B. Yu, A. Yang, S. Bai, J. Zhou, and J. Lin Soft adaptive policy optimization. External Links: 2511.20347 Cited by: §2.1, §5.1. Guan et al. (2024) T. Guan, F. Liu, X. Wu, R. Xian, Z. Li, X. Liu, X. Wang, L. Chen, F. Huang, Y. Yacoob, et al. HallusionBench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Cited by: §5.1. Guo et al. (2025) D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, et al. DeepSeek-r1 incentivizes reasoning in llms through reinforcement learning. Nature 645 (8081), p. 633–638. Cited by: §1, §2.1. Han et al. (2025) J. Han, J. Chen, H. Shao, H. Ma, M. Li, X. Shen, L. Zheng, W. Chen, T. Wei, and L. Zhang COPO: consistency-aware policy optimization. External Links: 2508.04138, Link Cited by: §1. Huang et al. (2025) W. Huang, B. Jia, Z. Zhai, S. Cao, Z. Ye, F. Zhao, Z. Xu, Y. Hu, and S. Lin Vision-r1: incentivizing reasoning capability in multimodal large language models. External Links: 2503.06749 Cited by: §1, §2.1. Li et al. (2024) K. Li, Y. Wang, Y. He, Y. Li, Y. Wang, Y. Liu, Z. Wang, J. Xu, G. Chen, P. Luo, et al. MVBench: a comprehensive multi-modal video understanding benchmark. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 22195–22206. Cited by: §5.1. Li et al. (2023a) L. Li, Z. Xie, M. Li, S. Chen, P. Wang, L. Chen, Y. Yang, B. Wang, and L. Kong Silkie: preference distillation for large visual language models. arXiv preprint arXiv:2312.10665. Cited by: §2.1. Li et al. (2023b) Y. Li, Y. Du, K. Zhou, J. Wang, W. X. Zhao, and J. Wen Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355. Cited by: §5.1. Lightman et al. (2024) H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe Let’s verify step by step. In International Conference on Learning Representations, Cited by: §2.1. Liu et al. (2024a) Y. Liu, S. Li, Y. Liu, Y. Wang, S. Ren, L. Li, S. Chen, X. Sun, and L. Hou TempCompass: do video llms really understand videos?. arXiv preprint arXiv:2403.00476. Cited by: §5.1. Liu et al. (2024b) Y. Liu, Z. Li, M. Huang, B. Yang, W. Yu, C. Li, X. Yin, C. Liu, L. Jin, and X. Bai OCRBench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences 67 (12). External Links: ISSN 1869-1919, Link, Document Cited by: §5.1. Liu et al. (2026) Z. Liu, Z. Chen, H. Liu, C. Luo, X. Tang, S. Wang, J. Zeng, Z. Dai, Z. Shi, T. Wei, et al. Seeing but not believing: probing the disconnect between visual attention and answer correctness in vlms. In International Conference on Learning Representations, Cited by: §1, §2.3. Lu et al. (2026) H. Lu, B. Chu, W. Fu, G. Nan, J. Liu, M. Pan, Q. Li, Y. Yu, H. Wang, and K. Wang Reallocating attention across layers to reduce multimodal hallucination. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 4157–4167. Cited by: §1, §2.3. Lu and Lab (2025) K. Lu and T. M. Lab On-policy distillation. Thinking Machines Lab: Connectionism. External Links: Document Cited by: §2.2, §5.1. Lu et al. (2023) P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K. Chang, M. Galley, and J. Gao MathVista: evaluating mathematical reasoning of foundation models in visual contexts. arXiv preprint arXiv:2310.02255. Cited by: Figure I.1. Masry et al. (2022) A. Masry, D. Long, J. Q. Tan, S. Joty, and E. Hoque ChartQA: a benchmark for question answering about charts with visual and logical reasoning. In Findings of the Association for Computational Linguistics: ACL 2022, Dublin, Ireland, p. 2263–2279. External Links: Link, Document Cited by: §5.1. Mathew et al. (2022) M. Mathew, V. Bagal, R. Tito, D. Karatzas, E. Valveny, and C. V. Jawahar InfographicVQA. In 2022 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), p. 2582–2591. Cited by: §5.1. Mathew et al. (2021) M. Mathew, D. Karatzas, and C. V. Jawahar DocVQA: a dataset for vqa on document images. In 2021 IEEE Winter Conference on Applications of Computer Vision (WACV), p. 2199–2208. Cited by: §5.1. Meng et al. (2025) F. Meng, L. Du, Z. Liu, Z. Zhou, Q. Lu, D. Fu, T. Han, B. Shi, W. Wang, J. He, et al. M-eureka: exploring the frontiers of multimodal reasoning with rule-based reinforcement learning. arXiv preprint arXiv:2503.07365. Cited by: §1, §2.1. Pan et al. (2026) L. Pan, S. Tao, Y. Zhai, L. Zhang, Z. Liu, B. Ding, A. Liu, and L. Wen RLCSD: reinforcement learning with contrastive on-policy self-distillation. External Links: 2606.11709 Cited by: §1, §2.2, §5.1. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, et al. DeepSeekMath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: §1, §2.1, §5.1. Shen et al. (2025) H. Shen, P. Liu, J. Li, C. Fang, Y. Ma, J. Liao, Q. Shen, Z. Zhang, K. Zhao, Q. Zhang, et al. VLM-r1: a stable and generalizable r1-style large vision-language model. arXiv preprint arXiv:2504.07615. Cited by: §1, §2.1. Shen et al. (2026) X. Shen, J. Chen, L. Zheng, H. Ma, T. Wei, and K. Zhan Evolving from tool user to creator via training-free experience reuse in multimodal reasoning. External Links: 2602.01983, Link Cited by: §1. Sun et al. (2024) Z. Sun, S. Shen, S. Cao, H. Liu, C. Li, Y. Shen, C. Gan, L. Gui, Y. Wang, Y. Yang, et al. Aligning large multimodal models with factually augmented rlhf. In Findings of the Association for Computational Linguistics: ACL 2024, p. 13088–13110. Cited by: §2.1. Tong et al. (2024) S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. LeCun, and S. Xie Eyes wide shut? exploring the visual shortcomings of multimodal llms. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 9568–9578. Cited by: §5.1. Wang et al. (2024a) F. Wang, X. Fu, J. Y. Huang, Z. Li, Q. Liu, X. Liu, M. D. Ma, N. Xu, W. Zhou, K. Zhang, et al. MuirBench: a comprehensive benchmark for robust multi-image understanding. arXiv preprint arXiv:2406.09411. Cited by: §5.1. Wang et al. (2024b) P. Wang, L. Li, Z. Shao, R. Xu, D. Dai, Y. Li, D. Chen, Y. Wu, and Z. Sui Math-shepherd: verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, p. 9426–9439. Cited by: §2.1. Wang et al. (2025a) W. Wang, Z. Gao, L. Chen, Z. Chen, J. Zhu, X. Zhao, Y. Liu, Y. Cao, S. Ye, X. Zhu, et al. VisualPRM: an effective process reward model for multimodal reasoning. arXiv preprint arXiv:2503.10291. Cited by: §1, §2.1. Wang et al. (2025b) W. Wang, L. Ding, M. Zeng, X. Zhou, L. Shen, Y. Luo, W. Yu, and D. Tao Divide, conquer and combine: a training-free framework for high-resolution image perception in multimodal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 7907–7915. Cited by: §5.1. Wu et al. (2024) H. Wu, D. Li, B. Chen, and J. Li LongVideoBench: a benchmark for long-context interleaved video-language understanding. External Links: 2407.15754, Link Cited by: §5.1. Wu and Xie (2024) P. Wu and S. Xie V*: guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 13084–13094. Cited by: §5.1. Xu et al. (2025) P. Xu, S. Xiong, J. Zhang, Y. Chen, B. Zhou, C. C. Loy, D. Clifton, K. M. Lee, L. Van Gool, R. He, R. Yao, X. Long, J. Huang, K. Tian, S. Yang, Y. Shao, J. Feng, Y. Zhong, J. Zhou, C. Tang, T. Zou, Y. Zhang, J. Liang, G. Li, Z. Wang, Q. Zhou, Y. Zhao, S. Xiong, H. Nam, J. Lee, J. Chung, J. Park, J. Oh, K. Lee, W. Lee, J. Ro, T. Osman, C. Hu, C. Liao, C. Chen, C. Han, C. Qiu, C. Peng, C. Xu, D. Li, F. Wang, F. Gao, G. Zhu, G. Tang, H. Lu, H. Fang, H. Qi, H. Wu, H. Cheng, H. Sun, H. Chen, H. Hu, H. Li, J. Ma, J. Yu, J. Wang, J. Yang, J. He, J. Zhou, J. Li, J. Kittler, L. Zheng, L. Zhao, M. Jia, M. Yan, T. T. Nguyen, P. Luo, Q. Li, S. Song, S. Dong, S. Shao, S. Li, T. Xue, T. Xu, T. Gao, T. Li, W. Zhang, W. Su, X. Dong, X. Wu, X. Zhou, X. Chen, X. Wei, X. You, X. Kang, X. Zhou, X. Liu, Y. Wang, Y. Huang, Y. Liu, Y. Yang, Y. Deng, Y. Kang, Y. Yuan, Y. Wen, Y. Tian, Y. Tao, Y. Tang, Y. Lin, Y. Wang, Y. Xi, Y. Yu, Y. Li, Y. Qin, Y. Chen, Y. Cen, Z. Zou, Z. Liu, Z. Shen, Z. Du, Z. Li, Z. Huang, Z. Shao, Z. Song, Z. Feng, Z. Wang, Z. Yu, Z. Li, Z. Zhai, Z. Zhang, Z. Peng, Z. Xiao, and Z. Li MARS2 2025 challenge on multimodal reasoning: datasets, methods, results, discussion, and outlook. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) Workshops, p. 6576–6605. Cited by: §1. Yang et al. (2026a) C. Yang, S. Cai, J. Yuan, and C. Hu Mitigating hallucination in multimodal large language models via cross-layer visual anchors. Pattern Recognition, p. 114380. Cited by: §1, §2.3. Yang et al. (2026b) C. Yang, C. Qin, Q. Si, M. Chen, N. Gu, D. Yao, Z. Lin, W. Wang, J. Wang, and N. Duan Self-distilled rlvr. External Links: 2604.03128 Cited by: §1, §2.2, §5.1. Yu et al. (2024) T. Yu, Y. Yao, H. Zhang, T. He, Y. Han, G. Cui, J. Hu, Z. Liu, H. Zheng, M. Sun, et al. RLHF-v: towards trustworthy mllms via behavior alignment from fine-grained correctional human feedback. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 13807–13816. Cited by: §1, §2.1. Yue et al. (2024) X. Yue, T. Zheng, Y. Ni, Y. Wang, K. Zhang, S. Tong, Y. Sun, B. Yu, G. Zhang, H. Sun, Y. Su, W. Chen, and G. Neubig MMMU-Pro: a more robust multi-discipline multimodal understanding benchmark. arXiv preprint arXiv:2409.02813. Cited by: §5.1. Zagoruyko and Komodakis (2016) S. Zagoruyko and N. Komodakis Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. arXiv preprint arXiv:1612.03928. Cited by: §2.3. Zhao et al. (2026a) J. Zhao, F. Zhang, X. Sun, C. Feng, and Z. Tan Tell model where to look: mitigating hallucinations in mllms by vision-guided attention. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 32582–32591. Cited by: §1, §2.3. Zhao et al. (2026b) S. Zhao, Z. Xie, M. Liu, J. Huang, G. Pang, F. Chen, and A. Grover Self-distilled reasoner: on-policy self-distillation for large language models. External Links: 2601.18734 Cited by: §2.2, §5.1. Zheng et al. (2026) L. Zheng, J. Chen, X. Shen, H. Ma, and T. Wei MIRG-rl: multi-image reasoning and grounding with reinforcement learning. In ICASSP 2026 - 2026 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), Vol. , p. 12292–12296. External Links: Document Cited by: §1. Zhou et al. (2025) Y. Zhou, X. Gao, Z. Chen, and H. Huang Attention distillation: a unified approach to visual characteristics transfer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 18270–18280. Cited by: §2.3. Zhu et al. (2025) J. Zhu, W. Wang, Z. Chen, Z. Liu, S. Ye, L. Gu, H. Tian, Y. Duan, W. Su, J. Shao, Z. Gao, E. Cui, X. Wang, Y. Cao, Y. Liu, X. Wei, H. Zhang, H. Wang, W. Xu, H. Li, J. Wang, N. Deng, S. Li, Y. He, T. Jiang, J. Luo, Y. Wang, C. He, B. Shi, X. Zhang, W. Shao, J. He, Y. Xiong, W. Qu, P. Sun, P. Jiao, H. Lv, L. Wu, K. Zhang, H. Deng, J. Ge, K. Chen, L. Wang, M. Dou, L. Lu, X. Zhu, T. Lu, D. Lin, Y. Qiao, J. Dai, and W. Wang InternVL3: exploring advanced training and test-time recipes for open-source multimodal models. External Links: 2504.10479, Link Cited by: §5.4. Appendix A Training Data and Preprocessing A.1 Data Sources and Composition The main training data are cleaned multiple-choice samples selected from Video-R1-260k (Feng et al., 2025). All primary methods use the same frozen training manifest. The data-composition ablation changes only this manifest and leaves the other training settings unchanged. Table A.1: Training-set composition. Data type Number of examples Image 64,473 Video 103,987 Total 168,460 A.2 Filtering and Preprocessing Starting from 263,071 examples in Video-R1-260k, we retain the 168,769 examples whose problem_type is multiple choice and whose data_type is image or video. The answer letter is extracted from the <answer>...</answer> field in the solution, and 10 examples without an extractable answer letter are removed. We then reserve the final 32 examples in the original data order for validation, leaving 168,727 candidate training examples. An independent video-availability scan reads the first and last frames of 26,896 unique video paths and identifies 15 invalid paths. After removing the 250 examples associated with these paths, we exclude 17 examples for which no valid incorrect answer can be constructed from the parsed options. The resulting fixed training manifest contains 168,460 examples: 64,473 image examples and 103,987 video examples. The 32 validation examples are not included in this training set. Appendix B Reproducibility and Training Configuration B.1 Optimization and Sampling GRPO and VAttnPO share the training data, learning rate, random seed, 400-step budget, sampling, and optimization settings. VAttnPO replaces the broadcast sequence advantage with the token advantage constructed in Equation (8). Each input has G=8G=8 sampled responses. Table B.1: Shared training configuration for GRPO and VAttnPO. Item Setting Backbone Qwen3-VL-8B-Instruct SFT warm-up None Training steps 400 Loss type GRPO Learning rate 2×10−62× 10^-6 Optimizer / schedule AdamW Torch Fused / cosine Per-device batch / processes / accumulation 1 / 8 / 32 Nominal global batch size 256 Response group size G 8 Temperature / top-p / top-k 1.0 / 0.85 / unrestricted Maximum sequence / completion length 8192 / 4096 Dynamic sampling / resampling attempts Enabled / 3 Overlong filtering Enabled Clipping ε / ratio range 0.20 / [0.80,1.20][0.80,1.20] Policy KL coefficient βKL _KL 0.04 Training / data seed 42 / 42 Parameter updates Full fine-tuning Warm-up ratio 0.05 Precision / distributed strategy bfloat16 / ZeRO-2 VAttnPO constructs visual attention from all 32 query heads at zero-based layer 20 and uses the dataset answer as the reference. We use JSD similarity with α=1α=1, μ=0.5μ=0.5, κ=8κ=8, and gmin=0.05g_ =0.05, followed by position weighting and credit balancing for both image and video responses. This layer is the 21st of 36 decoder blocks; grouped-query attention expands its eight key–value heads to 32 query heads. B.2 Verifiable Reward The sequence reward for multiple-choice training combines accuracy, format, and length terms: R=0.9Racc+0.1Rfmt+0.1Rover.R=0.9R_acc+0.1R_fmt+0.1R_over. The accuracy reward extracts an answer from <answer>...</answer>, the final ... expression, or the full response, in that order. If both prediction and ground truth resolve to option letters A–J, they are compared case-insensitively; otherwise, the scorer uses mathematical equivalence. A match gives Racc=1R_acc=1 and a mismatch gives 0. The format reward checks whether the response ends with the case-sensitive form <answer> ...</answer>; a match gives Rfmt=1R_fmt=1 and otherwise 0. Let L denote the completion length in tokens. The length penalty starts at Lstart=3072L_start=3072 and reaches −1-1 at the maximum completion length of 4096: Rover(L)=min(−L−30721024,0)=0,L≤3072,−L−30721024,3072<L≤4096.R_over(L)= \! (- L-30721024,0 )= cases0,&L≤ 3072,\\[2.84526pt] - L-30721024,&3072<L≤ 4096. cases B.3 Software and Hardware Training uses ms-swift 4.4.1 with its default visual preprocessing and video-sampling settings, running on eight NVIDIA L20X GPUs with bfloat16 precision and DeepSpeed ZeRO-2. Except for the explicit checkpoint comparison, the main results use the prespecified 400-step endpoint rather than selecting checkpoints by test performance. Appendix C VAttnPO Implementation Details C.1 Visual Reference from the Correct Answer For each input, the frozen old policy enters an assistant turn after the original system and multimodal user prompt and teacher-forces the dataset answer. All 168,460 training targets strictly use <answer>X</answer>, where X∈A,…,JX∈\A,…,J\ is the correct option letter. They contain no explanation, chain of thought, or , and the reference branch appends neither EOS nor <|im_end|>. Valid ground-truth queries are defined by labels != -100 within the assistant response. The tokens pooled into PgtP_gt therefore include the opening tag, correct option letter, and closing tag, but exclude the assistant header and padding. Their attention to visual keys is averaged over the LgtL_gt queries and all H heads and then normalized over the N visual positions. This branch is stop-gradient; the correct answer supplies conditioning context but no region annotation. C.2 Sampled Response Attention The main method uses the attention query at the same index as yi,ty_i,t and constructs retrospective credit after the complete response has been generated. The behavioral intervention in Section 3 instead uses the query immediately before the predicted target token. These indexing choices correspond to different analyses and should not be interchanged. At the target layer, attention softmax is first computed over all causal keys, after which only key columns whose input token IDs denote image or video patches are selected. Text, system/user/assistant, visual-boundary, and padding tokens are excluded. Selected patches preserve their original order in the model input sequence. Video-frame patches are flattened in the processor’s spatiotemporal order. We do not normalize individual frames separately: all frames of a video, and all visual patches in multi-image or multi-video inputs, form one shared support V. The selected post-softmax weights are renormalized in L1: Qi,t,p=ai,t,p∑p′∈ai,t,p′+10−8,p∈.Q_i,t,p= a_i,t,p _p a_i,t,p +10^-8, p . The same visual mask, patch order, and normalization are used for PgtP_gt. Both distributions therefore describe spatial attention allocation over one common visual support rather than the total attention weight assigned to visual inputs. Main training uses zero-based layer 20 and averages all 32 query heads after grouped-query expansion, without single-head, top-k-head, or other head selection. C.3 Masks, Length, and Numerical Boundaries Let LiL_i denote the number of valid completion tokens in a response. The terminal token in the response contributes to LiL_i and follows the same-index attention-credit rule as the other valid tokens. Relative position is defined only over these valid tokens. For Li≥2L_i≥ 2, we use pi,t=(t−1)/(Li−1)p_i,t=(t-1)/(L_i-1); when Li=1L_i=1, the position weight is one. Model computation and the query/key projections use BF16, and the attention-logit matrix multiplication follows the model precision. Logits are explicitly cast to FP32 before softmax; head averaging, patch L1 normalization, JSD, position weighting, balancing, and final credits are then computed in FP32. Both the L1 denominator and logarithms use an epsilon of 10−810^-8. With natural logarithms, let M=(P+Q)/2M=(P+Q)/2 and JSD(Q∥P)=12KL(Q∥M)+12KL(P∥M).JSD(Q\|P)= 12KL(Q\|M)+ 12KL(P\|M). The raw JSD lies in [0,ln2][0, 2], and the implementation clips 1−JSD/ln21-JSD/ 2 to [0,1][0,1]. The KL metric variant’s ϵKL=10−4 _KL=10^-4 is not used by the main JSD construction. All visual statistics are stop-gradient. Credit balancing uses τ=10−8τ=10^-8; if Pi+≤τP_i^+≤τ or Ni−≤τN_i^-≤τ, the entire id_i is set to zero. The fail-closed zero-sum check uses tolerance 10−6max(∑t|di,t|,1)10^-6 ( _t|d_i,t|,1). C.4 Full Clipped Optimization Objective Define the likelihood ratio ρi,t(θ)=πθ(yi,t∣x,yi,<t)πθold(yi,t∣x,yi,<t). _i,t(θ)= _θ(y_i,t x,y_i,<t) _ _old(y_i,t x,y_i,<t). The frozen old policy πθold _ _old is used both to sample responses and to extract visual signals; a separate frozen πref _ref is used only for KL regularization. VAttnPO inserts A~i,t A_i,t into the clipped GRPO objective, with symmetric clipping ε=0.20 =0.20, corresponding to the ratio range [0.80,1.20][0.80,1.20]: J(θ)=[1Li∑t J(θ)=E [ 1L_i _t \ min(ρi,t(θ)A~i,t,clip(ρi,t(θ),1−ε,1+ε)A~i,t) \! ( _i,t(θ) A_i,t,clip( _i,t(θ),1- ,1+ ) A_i,t ) −βKLDKL,i,t]. - _KLD_KL,i,t \ ]. The position weight acts only on visual credit and never directly scales A^i A_i; gradients propagate only through this policy objective. Algorithm 1 summarizes the VAttnPO update. Algorithm 1 VAttnPO update 1: Input x, ground-truth answer ygty_gt, rollout policy πθold _ _old, group size G, and credit coefficient α 2: Updated policy πθ _θ 3: Sample yii=1G∼πθold\y_i\_i=1^G _ _old and compute Ri,A^ii=1G\R_i, A_i\_i=1^G ⊳ sequence-level signal 4: Teacher-force ygty_gt and construct PgtP_gt with Equation (2) ⊳ shared reference 5: for i=1i=1 to G do 6: for t=1t=1 to LiL_i do 7: Extract Qi,tQ_i,t and compute si,ts_i,t with Equation (3) 8: end for 9: Compute ri,tr_i,t and ui,tu_i,t with Equations (4)–(5) 10: Compute di,td_i,t with Equations (6)–(7) ⊳ zero-mean correction 11: Set A~i,t=A^i+αdi,t A_i,t= A_i+α d_i,t for all valid t 12: end for 13: Update πθ _θ with the clipped objective above 14: return πθ _θ Appendix D Properties and Derivations D.1 Endpoint-Normalized Position Weight For κ>0κ>0 and finite μ, the sigmoid is strictly increasing and the denominator in Equation (5) is positive. Thus SκS_κ increases strictly from Sκ(0)=0S_κ(0)=0 to Sκ(1)=1S_κ(1)=1. With 0<gmin≤10<g_ ≤ 1, the position weight is positive and nondecreasing on [0,1][0,1], with range [gmin,2−gmin][g_ ,2-g_ ]; it is strictly increasing when gmin<1g_ <1. For μ=0.5μ=0.5, sigmoid symmetry gives Sκ(0.5)=0.5S_κ(0.5)=0.5 and a midpoint weight of one. For Li=1L_i=1, we use unit weight directly. D.2 Centering and Credit Balancing Centering removes the shared similarity offset and gives ∑tri,t=0 _tr_i,t=0, but leaves within-trajectory variance unchanged and does not standardize credit scales across responses. After position weighting, the residual satisfies ∑tui,t=LiCovt(gipos,ri) _tu_i,t=L_iCov_t(g_i^pos,r_i), where the covariance is over valid token positions. In a nondegenerate trajectory, Equation (7) rescales positive and negative credits by positive factors, preserving their signs. The total positive credit after balancing is ∑t:ui,t>0di,t=CiPi+Pi+=Ci, _t:u_i,t>0d_i,t= C_iP_i^+P_i^+=C_i, and the total negative credit is ∑t:ui,t<0di,t=−CiNi−Ni−=−Ci. _t:u_i,t<0d_i,t=- C_iN_i^-N_i^-=-C_i. Their sum is zero, and entries with ui,t=0u_i,t=0 remain zero. In either degenerate branch, i=d_i=0, which also has zero sum. This sign property concerns di,td_i,t relative to ui,tu_i,t. Without position weighting, ui,t=ri,tu_i,t=r_i,t; in exact arithmetic and the nondegenerate case, the two totals are equal, so balancing reduces to the identity di,t=ri,td_i,t=r_i,t. D.3 Preservation of the Trajectory-Mean Advantage Because A~i,t=A^i+αdi,t A_i,t= A_i+α d_i,t and ∑tdi,t=0 _td_i,t=0, 1Li∑t=1LiA~i,t=A^i. 1L_i _t=1^L_i A_i,t= A_i. This identity preserves the trajectory mean of advantage coefficients, not token gradients, their vector sum, or their norm. The final advantage can change sign when αdi,tα d_i,t opposes a nonzero A^i A_i and exceeds it in magnitude; when A^i=0 A_i=0, a nonzero correction can still produce positive and negative token advantages with zero mean. D.4 Computational Complexity Equation (3) computes JSD over an N-dimensional visual distribution for each valid rollout token. For T valid rollout tokens in a group, this credit computation requires O(NT)O(NT) operations. Appendix E Diagnostic Protocols and Supplementary Analyses E.1 Shared Sampling and Statistics All three diagnostics use layer 20 of Qwen3-VL-8B-Instruct and the same image-only candidate pool. The candidate record contains 10,199 responses to 1,485 prompts from the image distribution of the training set; 732 prompts contain both correct and incorrect responses. We sample 200 of these prompts without replacement using seed 42 and obtain 1,429 valid trajectories, comprising 771 correct and 658 incorrect responses. For correct–incorrect comparisons, responses are aggregated within each prompt before prompts receive equal weight. The 95% intervals use 2,000 bootstrap resamples with the prompt as the cluster unit. E.2 Analysis 1: Token Intervention For each example, select the top 20% of visual patches under PgtP_gt as SgtS_gt. Let a¯i,t−1,pprobe a^probe_i,t-1,p be head-aggregated attention from the query before yi,ty_i,t to visual patch p. Define Qi,t,pprobe=a¯i,t−1,pprobe∑p′∈visuala¯i,t−1,p′probe+ϵ,wi,t=∑p∈SgtQi,t,pprobe.Q^probe_i,t,p= a^probe_i,t-1,p _p a^probe_i,t-1,p +ε, w_i,t= _p∈ S_gtQ^probe_i,t,p. The reasoning mask covers reasoning tokens before the committed answer span. For reasoning length LrL_r, every intervention set has size m=max(3,round(0.15Lr))m= (3,round(0.15L_r)). High and low select the m tokens with the largest and smallest wi,tw_i,t, respectively. Random samples without replacement from the middle pool after removing high and low tokens. Position-matched greedily assigns each high token the nearest unused non-high token in relative position. The selected tokens’ complete self-attention key–value columns are then masked for subsequent queries. The intervention effect is the log-probability drop of the model’s original prediction: Δℓ(S)=logpfull(yipred)−logpmask(S)(yipred). (S)= p_full(y_i^pred)- p_mask(S)(y_i^pred). The paired effects are Cohen’s d=0.957d=0.957 (95% CI [0.867,1.052][0.867,1.052]) for High versus Low, 1.0051.005 ([0.925,1.083][0.925,1.083]) for High versus Random, and 0.8520.852 ([0.764,0.942][0.764,0.942]) for High versus Position-matched. The mean relative positions of the High and Position-matched sets are 0.666 and 0.654, respectively; the effect direction is positive in both the correct and incorrect response subsets. All effect sizes use paired differences within the same prompt and the prompt-cluster bootstrap above; position matching is deterministic and greedy. E.3 Analysis 2: GT versus Shuffled Visual Reference We compute each sampled token distribution’s JSD from the GT reference PgtP_gt and from a patch-index-shuffled version. The permutation preserves the multiset of reference values but breaks visual-patch identity. For an example e containing both correct and incorrect responses, the full-completion paired gap is δe=meani∈emeantDi,t−meani∈emeantDi,t, _e=mean_i _emean_tD_i,t-mean_i _emean_tD_i,t, where eW_e and eC_e are its incorrect and correct responses. The comparison statistic is the difference between the GT-reference and shuffled-reference gaps. One patch permutation is generated and fixed for each (source_row_idx,rollout_index)( source\_row\_idx, rollout\_index) pair. All token subsets of the same response share it, and it is not resampled per token or bootstrap replicate. This pilot uses the image-only candidate pool, so no video-frame boundary is involved. E.4 Analysis 3: Correct–Incorrect JSD over Completion Progress For Li>1L_i>1, valid tokens in each response are assigned to 20 bins by relative generation position: bi,t=⌊19t−1Li−1⌋.b_i,t= 19 t-1L_i-1 . The final bin contains only the terminal token of each response rather than an equal-width interval over its final segment. We first aggregate within each response and bin, then across correct or incorrect responses within each example, and finally weight examples equally. Appendix F Ablation and Control Construction F.1 Position and Balance Variants Table F.1: Controlled variants and their sole change from VAttnPO. Variant Sole change Position-only Center and unit-RMS normalize each trajectory’s position curve, then multiply by the fixed r-scale proxy 0.14 No position Set gi,tpos=1g^pos_i,t=1, so ui,t=ri,tu_i,t=r_i,t Position-shuffled Deterministically permute gi,tposg^pos_i,t over valid tokens per response with seed 42 No balance Set di,t=ui,td_i,t=u_i,t Mean recenter Set di,t=ui,t−Li−1∑vui,vd_i,t=u_i,t-L_i^-1 _vu_i,v Table F.2: Per-benchmark results for position and credit-balance variants. Variant VStar MMMU-Pro MuirBench BLINK HRBench 4K MMVP AVE GRPO 83.25 54.28 61.54 68.23 77.50 81.00 70.97 Position-only 83.34 54.18 62.23 67.85 78.12 79.33 70.84 No balance 83.77 54.41 62.04 67.75 77.38 79.67 70.84 Mean recenter 84.82 54.47 62.04 68.23 77.88 80.67 71.35 No position 85.34 55.55 63.35 68.70 79.50 81.67 72.35 Position-shuffled 84.95 55.48 63.25 68.39 79.38 81.67 72.19 VAttnPO 87.43 55.95 65.62 69.07 80.62 82.33 73.50 Similarity metric control. The KL variant uses the forward divergence DKL(Qi,t∥Pgtϵ)D_KL(Q_i,t\|P_gt^ε). We smooth the reference with the uniform distribution Uj=1/NU_j=1/N, using ϵKL=10−4 _KL=10^-4, and map the divergence to similarity with a fixed temperature τKL=1 _KL=1: Pgtϵ P_gt^ε =(1−ϵKL)Pgt+ϵKLU, =(1- _KL)P_gt+ _KLU, Di,tQP D^QP_i,t =DKL(Qi,t∥Pgtϵ), =D_KL(Q_i,t\|P_gt^ε), si,tKL s^KL_i,t =exp(−DQPi,t/τKL), = \! (-D^QP_i,t/ _KL ), ri,tKL r^KL_i,t =si,tKL−1Li∑v=1Lisi,vKL. =s^KL_i,t- 1L_i _v=1^L_is^KL_i,v. Centering is performed over valid completion tokens. The same position weights then form ui,tKL=gi,tposri,tKLu^KL_i,t=g^pos_i,tr^KL_i,t, followed by the same credit-balancing rule to obtain di,tKLd^KL_i,t. This replaces the similarity construction in Stage 2; policy KL regularization remains unchanged. The variant scores 84.82, 54.16, 60.92, 68.12, 77.88, and 82.33 on VStar, MMMU-Pro, MuirBench, BLINK, HRBench 4K, and MMVP, respectively, for an AVE of 71.37. The exponential map changes the scale of within-trajectory similarity differences, so this comparison concerns the specified KL construction. F.2 Reference and Token-Correspondence Controls Shuffled reference applies a fixed permutation to the visual-patch indices, Pgtπ[j]=Pgt[π(j)]P_gt^π[j]=P_gt[π(j)], preserving the multiset of probabilities, entropy, and concentration while disrupting spatial correspondence. Image and video patches are permuted separately, without crossing modalities; patches within each modality share one support, allowing permutations across images or video frames. Each example in the current training manifest contains either one image or one video. The permutation is determined per example with seed 42, shared by all responses to that input, and fixed throughout training. Token-permuted directly permutes the normalized centered similarity credit ri,t=si,t−s¯ir_i,t=s_i,t- s_i from Equation (4) among valid completion positions, where si,t=1−JSD(Qi,t∥Pgt)/ln2s_i,t=1-JSD(Q_i,t\|P_gt)/ 2. Let πi _i denote the permutation for response i; the position weights remain at their original positions: ui,tperm=gi,tposri,πi(t),iperm=Balance(iperm),u^perm_i,t=g^pos_i,tr_i, _i(t), _i^perm=Balance(u_i^perm), where BalanceBalance follows Equations (6)–(7) and the same degenerate-case rule. With seed 42, the permutation is determined by the input and complete response-token sequence, so different responses generally receive different mappings while identical responses receive the same mapping. It is not resampled by training step, and padding positions are excluded. The operation preserves the multiset, mean, and RMS of ir_i, but need not preserve the multiset or RMS of iu_i or id_i after position weighting and balancing; the balanced credit still has zero sum. The Wrong-GT reference parses valid options, orders them alphabetically, and selects the first option different from the ground truth. It teacher-forces <answer>X</answer> without , where X is the selected incorrect letter; its training manifest and all other settings match VAttnPO. Appendix G Evaluation Protocol All benchmark evaluations use VLMEvalKit (Duan et al., 2026) and follow its default protocol for each benchmark, including prompting, decoding, answer extraction, and scoring. We apply no benchmark-specific protocol overrides. AVE is the equally weighted mean of VStar, HRBench 4K, MMVP, MMMU-Pro, MuirBench, and BLINK. ChartQA, all video metrics, MME, and the heterogeneously scaled metrics in Table 2 are excluded. Appendix H Cross-Backbone Results The VAttnPO runs for Qwen2.5-VL-7B-Instruct and InternVL3-8B-Instruct both use zero-based layer 14. Apart from the backbone and attention layer, their settings follow Table B.1. Table H.1: Cross-backbone results on the six tasks used to compute AVE. Backbone Method VStar MMMU-Pro MuirBench BLINK HRBench 4K MMVP AVE Qwen2.5-VL-7B-Instruct Base 73.82 38.44 53.42 51.76 68.88 70.33 59.44 GRPO 75.92 39.08 53.69 54.55 72.25 74.33 61.64 VAttnPO 76.39 40.02 55.58 54.71 73.62 75.00 62.55 InternVL3-8B-Instruct Base 68.59 26.36 55.65 56.60 70.37 79.67 59.54 GRPO 71.20 27.59 56.42 56.81 70.75 80.00 60.46 VAttnPO 71.68 28.36 56.98 57.08 71.38 80.67 61.03 Appendix I Layer-Wise Visual Attention Analysis I.1 Layer-Wise Separation During Inference We compare correctness-related visual-attention separation across network layers during inference. All three frozen models use the same fixed set of 200 diagnostic prompts as Section 3, with G=8G=8 sampled responses per prompt, temperature 1.0, top-p 1.0, and generation seed 271828. Each backbone contributes only prompts containing both valid correct and incorrect responses, yielding 198, 44, and 173 paired prompts, respectively; the eligible subsets can therefore differ across backbones. We use the 0-based completion-progress bins defined in Appendix E.4. At each layer, visual attention is first aggregated across heads as in Equation (2), after which JSD is computed. For layer ℓ , prompt p, and progress bin b, the within-prompt wrong-minus-correct gap is δp,b(ℓ)=meani∈pmeant∈b(i)Di,t(ℓ)−meani∈pmeant∈b(i)Di,t(ℓ),δ^( )_p,b=mean_i _pmean_t∈ b(i)D^( )_i,t-mean_i _pmean_t∈ b(i)D^( )_i,t, where pW_p and pC_p denote the incorrect and correct sampled responses. We summarize late-completion separation by Δℓlate=15∑b=1519meanpδp,b(ℓ). ^late_ = 15 _b=15^19mean_pδ^( )_p,b. This statistic gives equal weight to the final five bins, including the terminal-token bin, and covers approximately 78.9–100% completion progress; it is not a pooled average over all tokens in that span. A positive value indicates that incorrect responses are farther from the answer-conditioned visual-attention reference. Confidence intervals use 2,000 prompt-cluster bootstrap resamples. For zero-based layer index ℓ in a model with LdecL_dec decoder layers, normalized depth is ℓ/(Ldec−1) /(L_dec-1). Table I.1: Layer-wise wrong-minus-correct JSD gaps, averaged equally over the final five completion-progress bins. Positive gaps indicate lower reference alignment for incorrect responses; intervals are 95% prompt-cluster bootstrap. Bold marks Qwen3-VL layer 20, used in the main experiments, and does not denote statistical superiority. (a) Qwen3-VL-8B-Instruct Layer Depth Δℓlate ^late_ 95% CI L8 23% 4.4×10−34.4× 10^-3 [1.8×10−3, 7.1×10−3][1.8× 10^-3,\,7.1× 10^-3] L12 34% 3.6×10−33.6× 10^-3 [7.0×10−4, 6.6×10−3][7.0× 10^-4,\,6.6× 10^-3] L16 46% 1.0×10−21.0× 10^-2 [6.3×10−3, 1.4×10−2][6.3× 10^-3,\,1.4× 10^-2] L20 57% 1.1×−1.1× 10^-2 [6.5×−, 1.5×−][6.5× 10^-3,\,1.5× 10^-2] L24 69% 6.7×10−36.7× 10^-3 [3.2×10−3, 1.0×10−2][3.2× 10^-3,\,1.0× 10^-2] L28 80% 6.9×10−36.9× 10^-3 [3.2×10−3, 1.1×10−2][3.2× 10^-3,\,1.1× 10^-2] L32 91% 5.3×10−35.3× 10^-3 [2.4×10−3, 8.2×10−3][2.4× 10^-3,\,8.2× 10^-3] (b) Qwen2.5-VL-7B-Instruct Layer Depth Δℓlate ^late_ 95% CI L0 0% 8.2×10−48.2× 10^-4 [9.5×10−5, 1.6×10−3][9.5× 10^-5,\,1.6× 10^-3] L4 15% 3.5×10−43.5× 10^-4 [−4.1×10−3, 4.5×10−3][-4.1× 10^-3,\,4.5× 10^-3] L9 33% 1.1×10−31.1× 10^-3 [−4.1×10−3, 5.6×10−3][-4.1× 10^-3,\,5.6× 10^-3] L14 52% 4.8×10−34.8× 10^-3 [−1.5×10−3, 1.1×10−2][-1.5× 10^-3,\,1.1× 10^-2] L18 67% 3.5×10−33.5× 10^-3 [−8.2×10−4, 7.3×10−3][-8.2× 10^-4,\,7.3× 10^-3] L22 81% 2.8×10−32.8× 10^-3 [−3.3×10−3, 8.9×10−3][-3.3× 10^-3,\,8.9× 10^-3] L27 100% 2.8×10−32.8× 10^-3 [−2.3×10−3, 8.1×10−3][-2.3× 10^-3,\,8.1× 10^-3] (c) InternVL3-8B-Instruct Layer Depth Δℓlate ^late_ 95% CI L0 0% 6.9×10−46.9× 10^-4 [1.1×10−4, 1.3×10−3][1.1× 10^-4,\,1.3× 10^-3] L4 15% 1.3×10−31.3× 10^-3 [−1.1×10−3, 3.3×10−3][-1.1× 10^-3,\,3.3× 10^-3] L9 33% 2.8×10−32.8× 10^-3 [−3.2×10−4, 5.5×10−3][-3.2× 10^-4,\,5.5× 10^-3] L14 52% 4.3×10−34.3× 10^-3 [−2.0×10−3, 1.1×10−2][-2.0× 10^-3,\,1.1× 10^-2] L18 67% 4.3×10−34.3× 10^-3 [1.6×10−4, 8.7×10−3][1.6× 10^-4,\,8.7× 10^-3] L22 81% 3.9×10−33.9× 10^-3 [−7.7×10−4, 8.6×10−3][-7.7× 10^-4,\,8.6× 10^-3] L27 100% 2.3×10−32.3× 10^-3 [−1.6×10−4, 5.0×10−3][-1.6× 10^-4,\,5.0× 10^-3] Among the scanned layers, the largest point estimates occur at intermediate normalized depths: 0.57 for Qwen3-VL-8B and 0.52 for both Qwen2.5-VL-7B-Instruct and InternVL3-8B-Instruct. In Qwen3-VL-8B, L20 is close to L16: their paired difference is 6.6×10−46.6× 10^-4, with a 95% bootstrap interval of [−2.5,3.6]×10−3[-2.5,3.6]× 10^-3. The Qwen2.5-VL estimates are less precise with 44 paired prompts, while the displayed InternVL3 estimates for L14 and L18 both round to 4.3×10−34.3× 10^-3. The observed profile therefore provides supporting context for using Qwen3-VL layer 20 rather than identifying a uniquely preferred layer. I.2 Qualitative Layer-Wise Attention Examples The quantitative analysis summarizes correctness-related separation across layers; the following examples illustrate spatial attention patterns at individual layers. Figure I.1 presents nine benchmark examples as one continuous visualization. Each PDF shows the original input and head-mean attention heatmaps from layers 8, 12, 16, 20, 24, 28, and 32, with the question and reference answer printed above. Layers within an example use a shared q95 color scale, fixed opacity, and identical smoothing, allowing qualitative inspection of spatial migration across layers within that example; absolute color intensity should not be compared across examples. The figure illustrates qualitative changes in attention patterns across layers rather than establishing causal attribution or correctness. Figure I.1: Layer-wise visual attention across nine examples, ordered as BLINK, DocVQA, InfoVQA, two MathVista (Lu et al., 2023) examples, MMStar (Chen et al., 2024), two OCRBench examples, and OCRBench v2. Each example uses one shared scale across its displayed layers.