Paper deep dive
CVIF: A Criticality-Driven Visual Intervention Framework for Geometric Diagram Understanding in MLLMs
Jiahui Kang, Bifan Wei, Lingling Zhang, Tianwen Jiang, Qiuyong Xiao, Jihong Zhang, Jun Liu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Despite significant progress in visual tasks by Multimodal Large Language Models (MLLMs), geometric diagram understanding remains challenging due to the presence of sparse visual cues and ambiguous symbol-primitive associations. MLLMs may therefore rely on textual priors, producing interpretations that conflict with visual evidence. We introduce the training-free Criticality-Driven Visual Intervention Framework (CVIF), an inference-time method that localizes critical layers and executes visual interventions during the transition from evidence aggregation to semantic decoding. At these layers, a Geometry-Constrained Local Relation Reconstruction (GCLR) module selects and weights vertex-centered visual evidence, while an Adaptive Visual Steering Operator (AVSO) redistributes attention mass toward the selected tokens. Experiments on PGPS9K and PGDP5K show that CVIF raises Overall F1 from 77.85 to 85.58 and from 75.23 to 82.84, respectively, establishing a novel inference-time visual intervention paradigm.
Tags
Links
- Source: https://arxiv.org/abs/2610.06399v1
- Canonical: https://arxiv.org/abs/2610.06399v1
Trouble viewing inline? Open PDF directly →
Full Text
46,045 characters extracted from source content.
Expand or collapse full text
CVIF: A Criticality-Driven Visual Intervention Framework for Geometric Diagram Understanding in MLLMs Jiahui Kang1, Bifan Wei1,†, Lingling Zhang1, Tianwen Jiang2 Qiuyong Xiao2, Jihong Zhang2,†, Jun Liu1 1School of Computer Science and Technology, Xi’an Jiaotong University 2Tencent Hy AI Data jhkang@stu.xjtu.edu.cn †Corresponding authors Abstract Despite significant progress in visual tasks by Multimodal Large Language Models (MLLMs), geometric diagram understanding remains challenging due to the presence of sparse visual cues and ambiguous symbol-primitive associations. MLLMs may therefore rely on textual priors, producing interpretations that conflict with visual evidence. We introduce the training-free Criticality-Driven Visual Intervention Framework (CVIF), an inference-time method that localizes critical layers and executes visual interventions during the transition from evidence aggregation to semantic decoding. At these layers, a Geometry-Constrained Local Relation Reconstruction (GCLR) module selects and weights vertex-centered visual evidence, while an Adaptive Visual Steering Operator (AVSO) redistributes attention mass toward the selected tokens. Experiments on PGPS9K and PGDP5K show that CVIF raises Overall F1 from 77.85 to 85.58 and from 75.23 to 82.84, respectively, establishing a novel inference-time visual intervention paradigm. 1 Introduction In recent years, Multimodal Large Language Models (MLLMs) have achieved substantial progress in the field of multimodal understanding (Lu et al., 2024; Wang et al., 2024; Zhang et al., 2025; Qiao et al., 2025; Chen et al., 2026). Geometric Problem Solving (GPS), which integrates geometric diagrams with textual questions, has emerged as a crucial benchmark for evaluating their cognitive capabilities (Ma et al., 2026; Chen et al., 2021). However, existing MLLMs tend to deduce visual information for geometric reasoning from textual contexts rather than genuinely perceiving the diagrams. Extensive empirical studies reveal that the current end-to-end GPS paradigm (Zhang et al., 2024a; Zhang et al., 2024b) conflates perception errors with reasoning failures, making it difficult to isolate the true source of model limitations (Sun et al., 2026; Zhang et al., 2024b; Guo et al., 2025; Wang et al., 2026; Cho et al., 2025). Multiple diagnostic studies consistently confirm that perception errors remain the primary cause of failures in geometric reasoning tasks (Wang et al., 2025a; Wang et al., 2025b) (Figure 1(a)). Therefore, it is imperative to explicitly decouple visual perception from logical reasoning by requiring models to independently parse geometric diagrams and generate formal language descriptions, thereby enabling an objective and unambiguous evaluation. Figure 1: Attention failure modes of MLLMs in geometric diagram perception. (a) Perceptual errors during the process of geometric reasoning lead to prediction failures. (b) When predicting which line segment intersects BDBD at point E, shallow layers attend to the visually correct point C. As depth increases, attention shifts toward evidence that confirms a prior-compatible candidate (e.g., AFAF); by a later layer, the visually correct point C receives little attention. (c) When determining which angle is annotated by 43∘43 , late-layer attention may concentrate on the label “B” and nearby symbols while failing to recover the local topology of the three edges incident to B. In particular, the model does not sufficiently use edge BDBD as structural evidence for resolving the angle assignment. Why do MLLMs readily make errors in perceiving geometric diagrams? Our study finds that these errors can be traced to: (1) visual cues captured at early layers being overlooked in deeper layers, and (2) attention mechanisms in deeper layers becoming excessively focused on letter symbols while overlooking geometric structures. This is precisely consistent with recent observations from extensive research on MLLM vision that such failures are primarily not caused by an inability to perceive visual information (Khayatkhoei et al., 2025; Liu et al., 2026; Shao et al., 2026). First, attending to relevant visual tokens does not ensure their continued use in later layers. As illustrated in Figure 1(b), textual priors can favor an incorrect interpretation of sparse visual inputs (Favero et al., 2024; Liu et al., 2024). Subsequent attention updates may reinforce features consistent with that interpretation while reducing the contribution of conflicting visual evidence. We refer to this reinforcement of an existing interpretation as confirmation bias in the following discussion. Second, in geometric tasks, attending to the correct symbols does not by itself establish the geometric relations between them. As shown in Figure 1(c), recognizing a number or a vertex label is insufficient to determine which incident edges define the annotated angle. Human perception combines prior knowledge with incoming sensory evidence. When the two conflict, visual evidence can revise an initial expectation before a final judgment is formed. By contrast, an MLLM may increasingly favor a prior-consistent interpretation during autoregressive generation, even when the diagram contains conflicting visual cues. Once these cues lose influence in the model’s internal representations, output-level correction alone cannot reliably restore them. This observation motivates intervention inside the model, rather than only through external prompting or post-hoc correction. Building on the above analysis, we aim to design an active intervention mechanism that can identify the critical locations for visual intervention and provide relevant geometric cues. First, conceptualizing Transformer inference as an energy-descent process, we can view the failure illustrated in Figure 1 as a consequence of hidden states failing to sufficiently leverage visual cues to establish the direction of energy descent. As a result, hidden states are overly misled by textual priors and gradually become trapped in a local-minimum state. We introduce the concept of representational criticality to describe a state in which a hidden representation has integrated diverse cues but has not yet formed a stable preference for a specific interpretation. Representations in such a state are particularly sensitive to changes in the relative contribution weights of visual evidence and textual priors, making them ideal locations for visual intervention. Second, in geometric diagrams, geometric vertices not only serve as anchors for linguistic descriptions but also act as key nodes for constructing higher-level geometric topological structures. We can therefore use geometric vertices as entry points and combine them with the model’s attention history to identify relevant visual evidence that encodes local topological information. Building on this idea, we propose CVIF, a training-free framework that uses layer-wise prediction diagnostics to select intervention layers and steers attention toward vertex-guided visual features. Our main contributions are: (i) We propose a Criticality-Driven Visual Intervention Framework, which mitigates confirmation bias and visual hallucinations in MLLMs by applying mechanism-level perturbations at the criticality. (i) We design a Geometry-Constrained Local Relation Reconstruction (GCLR) mechanism, which reconstructs topological relations within the high-dimensional semantic space. (i) We introduce an Adaptive Visual Steering Operator (AVSO), which safely reshapes multi-head attention distributions, enabling the model to resist deep-layer textual inertia without disrupting native logical reasoning. 2 Related Work Geometric Problem Solving and Geometry Diagram Parsing. Geometric Problem Solving has long served as a core benchmark for evaluating the complex cognitive and spatial reasoning capabilities of artificial intelligence (Lu et al., 2021; Seo et al., 2014). Recently, MLLMs have demonstrated strong capabilities on various mathematical reasoning benchmarks (Lu et al., 2024; Qiao et al., 2025). However, their performance on GPS tasks remains unsatisfactory (Xu et al., 2025). Most existing GPS works rely on end-to-end benchmarks, which tend to conflate perception errors with reasoning failures (Zhang et al., 2024a; Zhang et al., 2024b). In fact, recent studies have identified that MLLMs often suffer from severe perceptual impairments during long-sequence generation (Peng et al., 2026). For geometric diagrams, this challenge is further exacerbated by their extremely sparse and abstract topological features. Models may rely on textual priors when interpreting geometric diagrams, producing descriptions that are plausible from the text but inconsistent with the visual evidence. To address this, Geometry Diagram Parsing approaches attempt to translate diagrams into formal languages (Zhang et al., 2022; Wang et al., 2026) to decouple perception from reasoning. Mechanistic Interpretability and Representation Dynamics. Mechanistic interpretability studies seek to characterize the computations implemented by neural networks(Elhage et al., 2021). Layer-wise analyses identify depth-dependent linguistic representations(Tenney et al., 2019) and show that feed-forward networks (FFNs) can function as key-value memories whose outputs contribute to vocabulary-space predictions(Geva et al., 2021; Geva et al., 2022). Lad et al. (2026) propose stages of inference involving contextual feature construction, prediction ensembling, and final distribution sharpening. These studies motivate a functional interpretation of changes across layers, rather than a uniform view of the forward pass. Connections between attention and modern Hopfield networks also provide an energy-based perspective on associative retrieval(Ramsauer et al., 2021). In multimodal settings, studies report reliance on textual priors and limitations in visual grounding(Deng et al., 2025; Lee et al., 2025; Tong et al., 2024). Together, these findings motivate investigating where visual evidence can be reinforced during prediction formation. 3 Analysis of Intervention Locations and Geometric Cues 3.1 Critical Locations for Visual Intervention Figure 2: Prior-induced local minima and vision-guided trajectory correction. (a) When examining the angle at vertex B, textual priors tend to focus on the conventional ∠ ABC, whereas visual perception is naturally drawn to the salient feature "43∘43 ". As depicted in the energy landscape, this causes the prediction state to be pulled by prior forces (black arrow), making it highly susceptible to being trapped in a neighboring local minimum that represents an erroneous prediction (i.e., ∠ABC=43∘ ABC=43 ). During the critical state, introducing the local topological structure at vertex B provides a vision-driven steering force (blue arrow), thereby reducing the confidence of local minima induced by prior and steering the prediction toward the global minimum. (b) Layer-wise normalized predictive entropy for Qwen3-VL-32B at the question-end token. To conceptualize the dynamics of forward inference, we adopt an energy-landscape perspective inspired by continuous Hopfield networks (Ramsauer et al., 2021). From this viewpoint, we treat the forward pass as a process in which hidden states continually evolve toward stable states in a high-dimensional semantic space. For geometric diagram understanding, interpretations supported by textual priors and those supported by local visual evidence may compete with each other (Figure 2(a)). At representational criticality, small changes in the relative contributions of visual evidence and textual priors can alter the subsequent evolution of hidden states. Representational criticality therefore identifies a critical location for visual intervention: if intervention occurs too early, the model has not yet formed a semantic judgment relevant to the current prediction; if it occurs too late, the prior-favored interpretation has already become stable, making it difficult for local visual evidence to redirect the prediction trajectory. To identify these critical locations, we quantitatively track changes in hidden-state predictions across network depth. Let the hidden state at the l-th Transformer layer be (l)∈ℝdh^(l) ^d. Mapping it to the vocabulary V through the language model head lmW_lm yields the predictive distribution P(l)=Softmax(lm(l))P^(l)=Softmax(W_lmh^(l)). We use normalized predictive entropy to characterize the uncertainty of the prediction state at layer l: ℋnorm(l)=−1log|V|∑v∈VP(l)(v)logP(l)(v).H_norm^(l)=- 1 |V| _v∈ VP^(l)(v) P^(l)(v). (1) The entropy trajectory reflects three stages (Figure 2(b)). First, the model aggregates multimodal evidence in a high-entropy state. It then enters a stage of semantic decoding and prediction formation, during which the entropy curve deviates from its steady trend and exhibits pronounced nonlinear changes. Finally, the model consolidates and refines candidate predictions, causing entropy to decrease to a low level and gradually converge. Based on the geometric inflection points of the entropy curve, we use linear fitting to identify a set of critical layers ℒL at which visual intervention is applied. 3.2 Vertex-Centered Geometric Cues Unlike natural images, angles, intersections, and collinearity relations in geometric diagrams are typically expressed jointly by discrete vertices, thin line segments, and nearby annotations. Such local topological information is visually subtle, yet crucial for correctly parsing geometric relations. During feature aggregation, attention tends to focus more readily on letters and numbers with explicit linguistic semantics, while weak edges, intersections, and their relative positional relations are easily overlooked. Geometric vertices provide a natural organizing center for addressing this issue. Visually, edges, angles, intersections, and higher-order local configurations are organized around their incident vertices. Linguistically, geometric expressions such as BDBD and ∠ABC ABC also refer to diagram entities through vertex labels. A vertex relevant to the current prediction can therefore be regarded as the center of a local semantic cluster: the visual representations surrounding it jointly encode the vertex label, incident edges, nearby annotations, and their topological relations (Figure 3). Because a vertex token covers a local image region and has been contextualized by the vision encoder, it can encode information about nearby incident edges and local structures. Preserving the salience of these vertex-centered representations enables subsequent FFN computations to organize nearby geometric evidence around the relevant vertices, thereby helping the model recover geometric relations such as edge connectivity, angle structures, and multi-edge intersections. Figure 3: The Geometry-Constrained Local Relation Reconstruction (GCLR) mechanism. Using vertex B as a spatial anchor, GCLR identifies its incident edges (l1l_1, l2l_2, l3l_3) and nearby geometric annotations. The resulting target token set and weight matrix explicitly compensate for the relative topological relations of the local structures. 4 Method Figure 4: Overview of CVIF. Layer-wise normalized predictive entropy provides a diagnostic for selecting candidate critical layers during prediction formation. At these layers, GCLR utilizes geometric vertices as anchors to explicitly decouple and reconstruct spatial-semantic mappings for local structures. Subsequently, the AVSO performs distribution shaping and boost control over the attention, achieving active visual recall and prediction rectification. 4.1 Criticality-Driven Visual Intervention Framework As illustrated in Figure 4, the CVIF workflow proceeds as follows. Given an image-text prompt, we identify a set of critical layers ℒL based on the prediction entropy of the question-end token. For each layer l, let curr(l)∈ℝH×LseqA_curr^(l) ^H× L_seq denote the multi-head attention matrix, where H is the number of heads and LseqL_seq is the sequence length. We define ¯∈ℝLseq A ^L_seq as the historical smoothed attention from the preceding n layers (averaged across head dimensions), and verticesP_vertices as the geometric vertex coordinates obtained from a vertex detector (Jocher et al., 2023). If l∉ℒl , the model retains its native feed-forward path. Conversely, if l∈ℒl , the GCLR module leverages ¯ A and verticesP_vertices to compute a target token set Ωtgt _tgt and a topological weight matrix targetW_target: Ωtgt,target=GCLR(¯,vertices). _tgt,W_target=GCLR( A,P_vertices). (2) AVSO then safely reshapes the original attention curr(l)A_curr^(l) within the multi-head attention module: steered(l)=AVSO(curr(l),¯,Ωtgt,target),A_steered^(l)=AVSO(A_curr^(l), A, _tgt,W_target), (3) The steered attention matrix steeredA_steered is multiplied by the Value matrix, fusing key visual features to push the hidden state across the potential barrier in subsequent FFNs, steering subsequent representations toward visually supported predictions. Finally, to ensure rigorous quantitative evaluation, the generated geometric descriptions are translated into formal language using Qwen3-VL-32B. 4.2 Geometry-Constrained Local Relation Reconstruction GCLR uses detected geometric vertices as anchors to select local visual evidence relevant to the current prediction. It first evaluates vertex relevance from historical attention and vertex-centered spatial priors, then identifies candidate tokens around selected vertices, and finally produces a target token set and topology-aware weights for subsequent attention steering. Let vertices=kk=1NvP_vertices=\p_k\_k=1^N_v denote the detected vertex coordinates, where NvN_v is the number of vertices. For each visual token t∈Ωvist∈ _vis, let tp_t denote the center of its corresponding image region. The distance between vertex k and token t is defined as dk(t)=‖t−k‖2d_k(t)= \|p_t-p_k \|_2. For each vertex k, we construct a topology prior and an anchor prior: Mtopo(k)(t)=ρexp(−dk(t)22σg2)+(1−ρ)exp(−dk(t)σl)M_topo^(k)(t)=ρ (- d_k(t)^22 _g^2 )+(1-ρ) (- d_k(t) _l ), Manch(k)(t)=exp(−dk(t)22σg2)M_anch^(k)(t)= (- d_k(t)^22 _g^2 ), where ρ=0.2ρ=0.2 controls the mixture of the two distance-decay components, and σg _g and σl _l control their spatial scales. The base relevance of vertex k is obtained through the weighted score of its historical attention under topological priors: sbase(k)=∑t∈Ωvis¯(t)mtopo(k)(t).s_base^(k)= _t∈ _vis A(t)\,m_topo^(k)(t). (4) To incorporate distance-decay compensation, let dkrefd_k^ref denote the distance between vertex k and the prediction-relevant reference location. The final vertex score is sv(k)=sbase(k)+ηmaxu∈1,…,Nvsbase(u)exp(−dkrefτ),s_v^(k)=s_base^(k)+η _u∈\1,…,N_v\s_base^(u) (- d_k^refτ ), (5) where η controls the compensation strength and τ controls its decay rate. We retain vertices whose scores exceed a dynamic threshold: act=k|sv(k)≥μv+αvσv,V_act= \k\, |\,s_v^(k)≥ _v+ _v _v \, where μv _v and σv _v are the mean and standard deviation of the candidate vertex scores, respectively. The normalized vertex weight is ωv(k)=sv(k)∑u∈actsv(u),k∈act. _v^(k)= s_v^(k) _u _acts_v^(u),k _act. For each activated vertex k, the token-level response is stok(k)(t)=Mtopo(k)(t)¯(t),t∈Ωvis.s_tok^(k)(t)=M_topo^(k)(t) A(t), t∈ _vis. (6) After applying the dynamic threshold μtok(k)+αtokσtok(k) _tok^(k)+ _tok _tok^(k), we retain the representative tokens as feature(k)C_feature^(k). To preserve the vertex-centered anchor, we additionally select ck=argmaxt∈ΩvisManch(k)(t)¯(t).c_k= _t∈ _visM_anch^(k)(t) A(t). The final candidate set for vertex k is k=feature(k)∪ck.U_k=C_feature^(k)∪\c_k\. Within kU_k, we combine anchor localization and historical attention to compute token-level weights: ωtok(k,j)=(1−γ)manch(k)(j)∑u∈kmanch(k)(u)+γ¯(j)∑u∈k¯(u),j∈k, _tok^(k,j)=(1-γ) m_anch^(k)(j) _u _km_anch^(k)(u)+γ A(j) _u _k A(u), j _k, (7) where γ balances spatial anchoring and historical attention. The final target token set is Ωtgt=⋃k∈actk. _tgt= _k _actU_k. For each target token t∈Ωtgtt∈ _tgt, we aggregate its contributions from all activated vertices: target(t)=∑k∈actωv(k)ωtok(k,t)[t∈k].W_target(t)= _k _act _v^(k) _tok^(k,t)I[t _k]. (8) 4.3 Adaptive Visual Steering Operator Figure 5: The principle of the Adaptive Visual Steering Operator (AVSO). As illustrated in Figure 5, AVSO reshapes attention at critical layers without modifying model parameters. It uses the target-token set and topology-aware weights produced by GCLR, together with historical and current attention, to increase the influence of key geometric evidence while compensating for the added attention mass through text tokens. For each target token t∈Ωtgtt∈ _tgt, curr(l)(t)A_curr^(l)(t) denotes its current attention values across heads, ¯(t) A(t) denotes its head-aggregated historical attention prior, and target(t)W_target(t) denotes its normalized topology-aware weight. All vector operations below are applied element-wise along the head dimension. AVSO first compares current attention with the historical attention prior to compute a shaping factor for each target token: shape(l)(t)=(1−α)+α¯(t)curr(l)(t)+ϵ,t∈Ωtgt,M_shape^(l)(t)=(1-α)1+α A(t)A_curr^(l)(t)+ε, t∈ _tgt, (9) where α∈[0,1]α∈[0,1] controls the contribution of historical attention, ϵε is a numerical stability constant, and 1 is an all-one vector over attention heads. The shaped target attention is then defined as shaped(l)(t)=curr(l)(t)⊙shape(l)(t),t∈Ωtgt,A_shaped^(l)(t)=A_curr^(l)(t) _shape^(l)(t), t∈ _tgt, (10) where ⊙ denotes element-wise multiplication. AVSO selectively triggers intervention only for heads exhibiting high attention toward visual tokens yet insufficient alignment with the target tokens. gate(l)=(∑t∈Ωviscurr(l)(t)≥τdown∧∑t∈Ωtgtcurr(l)(t)≤τup), _gate^(l)=I ( _t∈ _visA_curr^(l)(t)≥ _down\; \; _t∈ _tgtA_curr^(l)(t)≤ _up ), (11) where τdown _down and τup _up are the lower visual-attention threshold and upper target-attention threshold, respectively. To ensure that the shaped target attention reaches a competitive visual-attention scale, we compute a guarantee coefficient: guarantee(l)=Clamp(maxt∈Ωviscurr(l)(t)mint∈Ωtgtshaped(l)(t)+ϵ, 1.0,+∞). γ_guarantee^(l)=Clamp ( _t∈ _visA_curr^(l)(t) _t∈ _tgtA_shaped^(l)(t)+ε,\,1.0,\,+∞ ). For each target token, AVSO updates its attention as boosted(l)(t)=curr(l)(t)⊙[gate(l)⊙shape(l)(t)⊙guarantee(l)⋅λ⋅target(t)+(−gate(l))],t∈Ωtgt,A_boosted^(l)(t)=A_curr^(l)(t) [ _gate^(l) _shape^(l)(t) γ_guarantee^(l)·λ·W_target(t)+ (1- _gate^(l) ) ],t∈ _tgt, (12) where λ>0λ>0 controls the intervention strength. The attention mass added to target tokens is Δ(l)=∑t∈Ωtgt(boosted(l)(t)−curr(l)(t)). ^(l)= _t∈ _tgt (A_boosted^(l)(t)-A_curr^(l)(t) ). Let the current attention mass assigned to text tokens be text(l)=∑t∈Ωtextcurr(l)(t).m_text^(l)= _t∈ _textA_curr^(l)(t). The text-token scaling factor is defined as text(l)=Clamp(−Δ(l)text(l)+ϵ, 0, 1). β_text^(l)=Clamp (1- ^(l)m_text^(l)+ε,\,0,\,1 ). The compensated text attention is text(l)(t)=curr(l)(t)⊙text(l),t∈Ωtext.A_text^(l)(t)=A_curr^(l)(t) β_text^(l), t∈ _text. (13) Combining the boosted attention of target tokens, the compensated attention of text tokens, and the unchanged attention of all remaining tokens, followed by normalization over the token dimension, yields the final steered attention steered(l)A_steered^(l). 5 Experiment 5.1 Experimental Setup We evaluated the performance of geometric diagram parsing on 2,650 image–formal language pairs from PGPS9K (Zhang and others, 2023) and 2,650 pairs from PGDP5K (Hao et al., 2022). To enable unified evaluation, we convert PGDP5K annotations into the PGPS representation. By comparing the predicted set with the ground-truth set, we report precision (P), recall (R), and F1 for structural, semantic, and overall predicates. CVIF uses Qwen3-VL-32B-Instruct (Bai et al., 2025a) as the intervention backbone. We compare it with open-source models from the Qwen, InternVL, and GLM families (Bai et al., 2025b; Bai et al., 2025a; Wang et al., 2025c; Team, 2026; Team et al., 2025), as well as proprietary models (Anthropic, 2025; OpenAI, 2025; Google, 2025). Table 1: Performance comparison on PGPS9K. Methods Structure Semantic Overall P R F1 P R F1 P R F1 Open-source MLLMs Qwen2.5-VL-7B-Instruct 73.58 38.00 50.12 38.32 43.82 40.89 47.19 40.70 43.70 InternVL3.5-8B-Instruct 66.14 43.70 52.63 43.42 51.55 47.14 51.44 47.38 49.33 InternVL3.5-30B-Instruct 70.03 58.89 63.98 57.39 56.12 56.75 62.32 57.61 59.87 InternVL3.5-8B 68.90 54.26 60.71 61.52 59.06 60.26 64.00 56.49 60.01 Qwen2.5-VL-32B-Instruct 70.15 53.01 60.39 63.33 58.9 61.03 65.49 55.74 60.22 Qwen3-VL-8B-Instruct 72.75 52.12 60.73 73.56 66.82 70.03 71.18 58.92 64.47 GLM-4.6V-Flash 77.25 52.08 62.22 77.39 62.68 69.26 76.42 56.98 65.29 Qwen3.5-9B 80.45 66.94 73.08 78.38 75.87 77.10 77.60 71.08 74.19 Qwen3-VL-32B-Instruct 82.91 76.16 79.39 79.63 74.67 77.07 80.37 75.48 77.85 Closed-source MLLMs Claude-4.5-Sonnet 81.47 69.25 74.87 78.17 68.93 73.26 79.04 69.10 73.74 GPT-5.2-1211 95.52 90.66 93.03 88.93 84.68 86.75 91.98 87.90 89.89 Gemini-3-Flash 98.84 91.14 94.84 86.73 84.14 85.41 92.91 87.90 90.33 CVIF (Ours) 90.73 88.41 89.56 83.28 81.95 82.61 85.72 85.43 85.58 5.2 Diagram Parsing Results CVIF improves Overall F1 over Qwen3-VL-32B-Instruct from 77.85 to 85.58 on PGPS9K and from 75.23 to 82.84 on PGDP5K (Tables 1 and 2). On PGPS9K, structural and semantic F1 improve by 10.17 and 5.54 points, respectively; the corresponding gains on PGDP5K are 9.77 and 5.95 points. CVIF also consistently improves overall precision and recall: by 5.35 and 9.95 points on PGPS9K, and by 4.84 and 10.15 points on PGDP5K. The larger recall gains indicate more complete recovery of geometric predicates, while the simultaneous precision gains show that this improved coverage does not introduce additional spurious relations. CVIF achieves the best Overall F1 among the evaluated open-source models on both datasets. Table 2: Performance comparison on PGDP5K. Methods Structure Semantic Overall P R F1 P R F1 P R F1 Qwen2.5-VL-32B-Instruct 70.15 53.01 60.39 63.33 58.9 61.03 65.49 55.74 60.22 Qwen3-VL-8B-Instruct 73.16 51.04 60.13 71.15 60.15 65.19 69.89 55.15 61.65 GLM-4.6V-Flash 76.95 53.18 62.89 71.27 56.73 63.18 72.04 54.78 62.24 Qwen3.5-9B 79.72 65.21 71.74 77.98 72.45 75.11 76.70 68.48 72.36 Qwen3-VL-32B-Instruct 81.28 74.58 77.79 77.43 69.81 73.43 78.26 72.43 75.23 GPT-5.2-1211 94.03 89.47 91.69 84.50 79.67 82.01 89.20 85.04 87.07 Gemini-3-Flash 97.15 93.54 95.31 77.62 75.67 76.63 88.30 85.47 86.86 CVIF (Ours) 88.76 86.38 87.56 80.84 77.97 79.38 83.10 82.58 82.84 Table 3: Sample Accuracy and Perfect Parsing Rate (PPR) on PGPS9K and PGDP5K. Methods PGPS9K PGDP5K Lines Circles Structures Semantics PPR Lines Circles Structures Semantics PPR Qwen2.5-VL-32B-Instruct 36.57 77.51 34.75 38.42 21.43 34.64 78.64 32.60 38.42 19.36 Qwen3-VL-8B-Instruct 42.79 77.51 38.08 50.75 25.06 42.45 77.85 37.40 50.00 24.00 GLM-4.6V-Flash 43.70 80.60 39.09 49.81 25.77 46.34 80.00 39.70 51.02 25.40 Qwen3.5-9B 56.00 85.43 50.64 61.89 37.36 55.55 82.60 48.30 63.47 36.34 Qwen3-VL-32B-Instruct 64.23 92.57 62.38 58.79 43.47 64.38 87.62 58.08 58.79 40.11 GPT-5.2-1211 84.42 91.36 79.13 73.55 62.04 85.81 86.42 75.66 71.40 58.26 Gemini-3-Flash 89.32 91.77 83.92 77.21 67.77 94.23 89.85 85.02 68.15 60.68 CVIF (Ours) 79.09 95.62 77.25 69.96 59.47 80.00 88.72 72.23 69.77 55.77 5.3 Diagram-level Exact Match Evaluation Category-level F1 does not guarantee that an entire diagram is parsed correctly. We therefore report Sample Accuracy (SA), which requires all predicates within a category to match the ground truth, and Perfect Parsing Rate (PPR), which requires all predicates across the diagram to be correct. As shown in Table 3, CVIF achieves the best results among the evaluated open-source configurations on all metrics. Compared with Qwen3-VL-32B-Instruct, CVIF raises PPR from 43.47 to 59.47 on PGPS9K and from 40.11 to 55.77 on PGDP5K. These results show that CVIF improves not only individual predicate recovery, but also the global completeness and correctness of diagram parsing. Table 4: Ablation results on PGPS9K. Methods Structure Semantic Overall P R F1 P R F1 P R F1 Qwen3-VL-32B (Baseline) 82.91 76.16 79.39 79.63 74.67 77.07 80.37 75.48 77.85 Qwen3-VL-32B 2-stage 84.29 78.92 81.52 81.11 77.85 79.45 81.11 77.85 79.45 Uniform Intv. 86.00 80.13 82.96 79.66 77.03 78.32 81.07 78.70 79.86 w/o GCLR 86.46 81.21 83.75 81.49 78.79 80.12 82.51 80.10 81.28 w/o AVSO 86.12 81.11 83.54 80.34 78.54 79.43 81.12 79.92 80.52 CVIF (Ours) 90.73 88.41 89.56 83.28 81.95 82.61 85.72 85.43 85.58 5.4 Ablation Study Table 4 shows that the two-stage pipeline, which generates a natural-language description before formalizing it, reaches 79.45 Overall F1, substantially below CVIF (85.58). Thus, CVIF’s gains cannot be explained by changing the output pipeline alone. “Uniform Intv.” replaces criticality-guided layer selection with fixed interventions at layers 16, 32, and 48 of the 64-layer backbone, yielding 79.86 Overall F1 versus 85.58 for CVIF and supporting criticality-guided placement. Replacing GCLR with Top-K visual-token selection reduces Overall F1 to 81.28, while replacing AVSO with direct attention amplification reduces it to 80.52. These results verify the importance of both criticality-guided layer selection and the two proposed modules. Figure 6: Qualitative case analysis on representative geometric diagrams. 5.5 Case Analysis Figure 6 compares the parsing results of the baseline, the two-stage pipeline, and CVIF on representative diagrams. Although the baseline and two-stage pipeline recover many individual predicates, they often assign attributes to incorrect geometric entities or miss local structural relations required for correct parsing. In contrast, CVIF consistently recovers the relevant vertex-centered topology, including angle assignments, line connectivity, collinearity, and perpendicular relations. These examples illustrate that the proposed visual intervention helps the model retain and utilize local geometric evidence, thereby producing structurally consistent diagram parses. 5.6 Downstream Geometry Reasoning We evaluate the downstream utility of diagram representations on the Vision Dominant split of the Plane Geometry subset of MathVerse (Zhang et al., 2024b). This representative and challenging geometry-reasoning setting requires models to rely primarily on visual evidence, making it well suited to evaluate whether improved diagram perception transfers to downstream reasoning. We compare five reasoning models under three settings: direct answering, augmentation with a formal-language representation, and augmentation with a natural-language diagram description (Table 5). Natural-language descriptions improve accuracy for all five models, with gains from 1.76 to 5.88 points. Formal-language representations improve four models, while the preferred representation varies by backbone. These results show that explicit diagram representations can improve downstream geometry reasoning in visually demanding settings. Table 5: Downstream reasoning accuracy on MathVerse. Methods Qwen3-VL-32B Qwen3-VL-8B InternVL3.5-8B Qwen2.5-VL-7B Qwen3-VL-4B Direct Answer 53.73 45.10 41.18 42.16 41.18 + Formal Language 53.73 49.61 45.29 46.08 41.37 + Text Description 58.63 48.24 47.06 46.47 42.94 6 Conclusion In this paper, we proposed a training-free Criticality-Driven Visual Intervention Framework to mitigate textual-prior bias in MLLMs’ geometric diagram understanding. An energy-landscape perspective motivates reinforcing local visual evidence at representational critical states, where competing interpretations have not yet become stable. The framework uses layer-wise prediction diagnostics to select candidate critical layers and directs attention toward vertex-guided visual evidence. We designed the GCLR, which uses geometric vertices as anchors to reconstruct "spatial-semantic" mappings, overcoming the failure of global attention to capture complex topologies. Furthermore, by introducing the AVSO, our framework achieved robust distribution reshaping. The reported experiments show improved diagram-parsing performance with the intervention, and evaluate the use of generated descriptions in downstream geometry reasoning, demonstrating a novel inference-time intervention paradigm for mitigating visual hallucinations. References Anthropic (2025) Anthropic The Claude 3 and 4.5 model family. Note: https://w.anthropic.com Cited by: §5.1. Bai et al. (2025a) S. Bai, Y. Cai, R. Chen, et al. Qwen3-VL technical report. arXiv preprint arXiv:2511.21631. Cited by: §5.1. Bai et al. (2025b) S. Bai, K. Chen, X. Liu, et al. Qwen2.5-VL technical report. arXiv preprint arXiv:2502.13923. Cited by: §5.1. Chen et al. (2021) J. Chen, J. Tang, J. Qin, X. Liang, L. Liu, E. Xing, and L. Lin Geoqa: a geometric question answering benchmark towards multimodal numerical reasoning. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, p. 513–523. Cited by: §1. Chen et al. (2026) S. Chen, H. Yuan, Y. Xu, P. Liu, T. Feng, J. Cen, Z. Huang, and Y. Yang Mathflow: enhancing the perceptual flow of mllms for visual mathematical problems. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 967–992. Cited by: §1. Cho et al. (2025) S. Cho, Z. Qin, Y. Liu, Y. Choi, S. Lee, and D. Kim Geodano: geometric vlm with domain agnostic vision encoder. arXiv preprint arXiv:2502.11360. Cited by: §1. Deng et al. (2025) A. Deng, T. Cao, Z. Chen, and B. Hooi Words or vision: do vision-language models have blind faith in text?. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 3867–3876. Cited by: §2. Elhage et al. (2021) N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y. Bai, A. Chen, A. Conerly, et al. A mathematical framework for transformer circuits. Transformer Circuits Thread. Cited by: §2. Favero et al. (2024) A. Favero, L. Zancato, M. Trager, S. Choudhary, P. Perera, A. Achille, A. Swaminathan, and S. Soatto Multi-modal hallucination control by visual information grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 14303–14312. Cited by: §1. Geva et al. (2022) M. Geva, A. Caciularu, K. Wang, G. Dar, and J. Berant Transformer feed-forward layers build predictions by promoting concepts in the vocabulary space. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 30–45. Cited by: §2. Geva et al. (2021) M. Geva, R. Schuster, J. Berant, and O. Levy Transformer feed-forward layers are key-value memories. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), Cited by: §2. Google (2025) Google Gemini 3 Pro. Note: https://deepmind.google/models/gemini/pro/ Cited by: §5.1. Guo et al. (2025) S. Guo, S. Jiang, Q. He, Y. Xiao, J. Liang, B. Yude, M. He, S. Tao, and L. Zhang Do large language models truly understand cross-cultural differences?. arXiv preprint arXiv:2512.07075. Cited by: §1. Hao et al. (2022) Y. Hao, M. Zhang, F. Yin, and L. Huang PGDP5K: a diagram parsing dataset for plane geometry problems. In Proceedings of the 26th International Conference on Pattern Recognition (ICPR), p. 1763–1769. Cited by: §5.1. Jocher et al. (2023) Ultralytics yolov8 Note: https://github.com/ultralytics/ultralytics Cited by: §4.1. Khayatkhoei et al. (2025) M. Khayatkhoei, P. Chhikara, F. Ilievski, et al. Mllms know where to look: training-free perception of small visual details with multimodal llms. In International Conference on Learning Representations (ICLR), Vol. 2025, p. 68194–68213. Cited by: §1. Lad et al. (2026) V. Lad, J. H. Lee, W. Gurnee, and M. Tegmark Remarkable robustness of llms: stages of inference?. Advances in Neural Information Processing Systems 38, p. 130050–130083. Cited by: §2. Lee et al. (2025) K. Lee, M. Kim, S. Yoon, M. Kim, D. Lee, H. Koh, and K. Jung Vlind-bench: measuring language priors in large vision-language models. In Findings of the Association for Computational Linguistics: NAACL 2025, p. 4129–4144. Cited by: §2. Liu et al. (2024) S. Liu, K. Zheng, and W. Chen Paying more attention to image: a training-free method for alleviating hallucination in lvlms. In European Conference on Computer Vision (ECCV), p. 125–140. Cited by: §1. Liu et al. (2026) Z. Liu, Z. Chen, H. Liu, C. Luo, X. Tang, S. Wang, J. Zeng, Z. Dai, Z. Shi, T. Wei, et al. Seeing but not believing: probing the disconnect between visual attention and answer correctness in vlms. In International Conference on Learning Representations (ICLR), Vol. 2026, p. 72849–72876. Cited by: §1. Lu et al. (2024) P. Lu, H. Bansal, T. Xia, J. Liu, C. Li, H. Hajishirzi, H. Cheng, K. Chang, M. Galley, and J. Gao MathVista: evaluating mathematical reasoning of foundation models in visual contexts. In International Conference on Learning Representations (ICLR), Cited by: §1, §2. Lu et al. (2021) P. Lu, R. Gong, S. Jiang, L. Qiu, S. Huang, L. Xia, and S. Zhu Inter-gps: interpretable geometry problem solving with formal language and natural language. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics (ACL), Cited by: §2. Ma et al. (2026) J. Ma, W. Wang, and Q. Jin A survey of deep learning for geometry problem solving. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 39416–39448. Cited by: §1. OpenAI (2025) OpenAI GPT-5 system card. Technical report OpenAI. Note: https://openai.com/index/gpt-5-system-card/ Cited by: §5.1. Peng et al. (2026) R. Peng, X. Wu, J. Lei, L. Hou, Y. Ma, and X. Li Deeper thought, weaker aim: understanding and mitigating perceptual impairment during reasoning in multimodal large language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §2. Qiao et al. (2025) R. Qiao, Q. Tan, G. Dong, et al. We-math: does your large multimodal model achieve human-like mathematical reasoning?. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL), p. 20023–20070. Cited by: §1, §2. Ramsauer et al. (2021) H. Ramsauer, B. Schäfl, J. Lehner, P. Seidl, M. Widrich, T. Adler, L. Gruber, M. Holzleitner, M. Pavlović, G. K. Sandner, et al. Hopfield networks is all you need. In International Conference on Learning Representations (ICLR), Cited by: §2, §3.1. Seo et al. (2014) M. J. Seo, H. Hajishirzi, A. Farhadi, and O. Etzioni Diagram understanding in geometry questions. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, p. 2831–2838. Cited by: §2. Shao et al. (2026) J. Shao, Y. Yang, Y. Liao, Y. Shen, C. Yang, and Y. Xu Thinking with tools, not with pixels: tool calls as text scaffolds for visual reasoning. arXiv preprint arXiv:2608.09682. Cited by: §1. Sun et al. (2026) Y. Sun, S. Zhang, W. Tang, A. Chen, P. Koniusz, K. Zou, Y. Xue, and A. Hengel Math blind: failures in diagram understanding undermine reasoning in mllms. In International Conference on Learning Representations (ICLR), Vol. 2026, p. 124385–124422. Cited by: §1. Team (2026) Q. Team Qwen3.5-Omni technical report. arXiv preprint arXiv:2604.15804. Cited by: §5.1. Team et al. (2025) V. Team, W. Hong, W. Yu, X. Gu, G. Wang, G. Gan, H. Tang, J. Cheng, J. Qi, J. Ji, et al. GLM-4.5v and GLM-4.1v-thinking: towards versatile multimodal reasoning with scalable reinforcement learning. arXiv preprint arXiv:2507.01006. Cited by: §5.1. Tenney et al. (2019) I. Tenney, D. Das, and E. Pavlick BERT rediscovers the classical nlp pipeline. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, p. 4593–4601. Cited by: §2. Tong et al. (2024) S. Tong, Z. Liu, Y. Zhai, Y. Ma, Y. LeCun, and S. Xie Eyes wide shut? exploring the visual shortcomings of multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 9568–9578. Cited by: §2. Wang et al. (2024) K. Wang, J. Pan, W. Shi, Z. Lu, H. Ren, A. Zhou, M. Zhan, and H. Li Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural Information Processing Systems 37, p. 95095–95169. Cited by: §1. Wang et al. (2025a) P. Wang, Z. Li, F. Yin, D. Ran, and C. Liu MV-math: evaluating multimodal math reasoning in multi-visual contexts. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 19541–19551. Cited by: §1. Wang et al. (2025b) P. Wang, C. Yang, Z. Li, et al. SolidGeo: measuring multimodal spatial math reasoning in solid geometry. arXiv preprint arXiv:2505.21177. Cited by: §1. Wang et al. (2026) P. Wang, M. Zhang, J. Cao, et al. Geoparsing: diagram parsing for plane and solid geometry with a unified formal language. In Findings of the Association for Computational Linguistics: ACL 2026, p. 29876–29903. Cited by: §1, §2. Wang et al. (2025c) W. Wang, Z. Gao, L. Gu, et al. InternVL 3.5: advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265. Cited by: §5.1. Xu et al. (2025) L. Xu, Y. Zhao, J. Wang, et al. GeoSense: evaluating identification and application of geometric principles in multimodal reasoning. arXiv preprint arXiv:2504.12597. Cited by: §2. Zhang et al. (2024a) J. Zhang, Z. Li, M. Zhang, F. Yin, C. Liu, and Y. Moshfeghi GeoEval: benchmark for evaluating llms and multimodal models on geometry problem-solving. In Findings of the Association for Computational Linguistics: ACL 2024, p. 1258–1276. Cited by: §1, §2. Zhang et al. (2023) M. Zhang et al. PGPS9K: a large-scale dataset for geometric problem solving and theorem proving. In Proceedings of the 31st ACM International Conference on Multimedia, Cited by: §5.1. Zhang et al. (2022) M. Zhang, F. Yin, Y. Hao, and C. Liu Plane geometry diagram parsing. In Proceedings of the Thirty-First International Joint Conference on Artificial Intelligence (IJCAI), p. 1636–1643. Cited by: §2. Zhang et al. (2024b) R. Zhang, D. Jiang, Y. Zhang, et al. MathVerse: does your multi-modal LLM truly see the diagrams in visual math problems?. In European Conference on Computer Vision (ECCV), p. 169–186. Cited by: §1, §2, §5.6. Zhang et al. (2025) S. Zhang, A. Chen, Y. Sun, J. Gu, Y. Zheng, P. Koniusz, K. Zou, A. v. d. Hengel, and Y. Xue Open eyes, then reason: fine-grained visual mathematical understanding in mllms. arXiv preprint arXiv:2501.06430. Cited by: §1.