Paper deep dive

AdaVSkip: Adaptive Visual Token Skipping Across Layers For Efficient MLLMs Inference

Yuyao Sun, Tao Deng, Shuang Li, Deqing Wang

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 54

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Multimodal large language models (MLLMs) require substantial computation to process numerous visual tokens across all transformer layers. Most methods for efficient MLLM inference exploit horizontal redundancy by compressing visual tokens. Beyond token reduction, recent studies exploit vertical redundancy through early exit or fixed-layer skipping. However, we find that the extent and distribution of this redundancy vary across inputs and differ between self-attention and MLP modules. Motivated by these observations, we propose AdaVSkip, which equips each layer with two lightweight routers that independently determine whether visual tokens pass through by or skip the self-attention and MLP modules. These decisions collectively define an input-specific visual-computation path, but their discrete and non-differentiable nature makes learning effective paths challenging. To address this challenge, we develop a progressive two-stage training framework that updates only the routers while keeping the backbone frozen. Stage I establishes an initial routing policy through supervised training with input-specific targets derived from module-wise necessity scores. To further align the routing policy with task performance, Stage II uses reinforcement learning to optimize routing decisions with direct feedback from generated answers. It combines an answer correctness reward with a skip-consistency reward that discourages excessive retention of visual-token computation. Across three MLLM backbones, AdaVSkip maintains strong task performance with substantially less computation. On LLaVA-NeXT-7B, AdaVSkip reduces FLOPs by 53.2\% while preserving the original model's average performance. Combining it with visual token compression increases this reduction to 91.2\%, while retaining 97.2\% of the original performance on average.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

50,323 characters extracted from source content.

Expand or collapse full text

AdaVSkip: Adaptive Visual Token Skipping Across Layers For Efficient MLLMs Inference Yuyao Sun Tao Deng11footnotemark: 1 Shuang Li† Deqing Wang† †thanks: Equal contribution. †Corresponding authors. Affiliation: Beihang University Abstract Multimodal large language models (MLLMs) require substantial computation to process numerous visual tokens across all transformer layers. Most methods for efficient MLLM inference exploit horizontal redundancy by compressing visual tokens. Beyond token reduction, recent studies exploit vertical redundancy through early exit or fixed-layer skipping. However, we find that the extent and distribution of this redundancy vary across inputs and differ between self-attention and MLP modules. Motivated by these observations, we propose AdaVSkip, which equips each layer with two lightweight routers that independently determine whether visual tokens pass through by or skip the self-attention and MLP modules. These decisions collectively define an input-specific visual-computation path, but their discrete and non-differentiable nature makes learning effective paths challenging. To address this challenge, we develop a progressive two-stage training framework that updates only the routers while keeping the backbone frozen. Stage I establishes an initial routing policy through supervised training with input-specific targets derived from module-wise necessity scores. To further align the routing policy with task performance, Stage I uses reinforcement learning to optimize routing decisions with direct feedback from generated answers. It combines an answer correctness reward with a skip-consistency reward that discourages excessive retention of visual-token computation. Across three MLLM backbones, AdaVSkip maintains strong task performance with substantially less computation. On LLaVA-NeXT-7B, AdaVSkip reduces FLOPs by 53.2% while preserving the original model’s average performance. Combining it with visual token compression increases this reduction to 91.2%, while retaining 97.2% of the original performance on average. 1 Introduction Multimodal large language models (MLLMs) have achieved remarkable progress across diverse vision-language tasks Alayrac et al. (2022); Li et al. (2023b); Dai et al. (2023); Liu et al. (2023); Liu et al. (2024a). Recent models increasingly adopt high-resolution inputs or fine-grained visual representations, often producing large numbers of visual tokens Liu et al. (2024b); Li et al. (2024a); Chen et al. (2024c); Zhu et al. (2025); Bai et al. (2025a). These visual tokens are processed alongside text tokens by both self-attention and MLP modules at every transformer layer, resulting in substantial inference costs. To improve efficiency, existing methods reduce visual-token counts by learning compact representations before feeding them into the language model Zhang et al. (2025b); Li et al. (2024b); Lu et al. (2025) or by pruning less informative tokens during inference Chen et al. (2024a); Vasu et al. (2025); Yang et al. (2025); Ye et al. (2025); Zhang et al. (2025a). These methods primarily exploit horizontal redundancy within the visual-token sequence, but the retained tokens may still undergo computation in layers where their contribution is limited. To exploit this vertical redundancy across model depth, recent studies allow visual tokens to exit early Lin et al. (2025); Wu et al. (2024) or freeze visual-token updates in input-invariant layer subsets Yuan et al. (2025). However, it remains unclear whether fixed or prefix-based skipping patterns can adequately accommodate the varying visual-computation needs of different questions. To investigate this question, we systematically analyze how skipping visual-token computation in individual self-attention and MLP modules affects the model’s output distribution across different inputs. Our analysis yields two key observations. First, vertical redundancy is input-dependent: both the extent of skippable visual-token computation and its distribution across model depth vary substantially across inputs, limiting the effectiveness of a fixed skipping pattern. Second, even within the same layer, self-attention and MLP often differ in their need for visual-token computation, suggesting that a shared layer-wise skipping decision may skip useful computation in one module or retain redundant computation in the other. Together, these observations motivate an input-adaptive, module-wise visual-token skipping mechanism. Figure 1: Fixed layer-wise skipping vs. adaptive module-wise visual token skipping. To this end, we propose AdaVSkip, an input-adaptive framework for module-wise visual-token skipping. At each transformer layer, AdaVSkip introduces two lightweight routers that independently determine whether to perform visual-token computation in the self-attention and MLP modules, respectively. When a router chooses to skip, visual tokens bypass the module with their hidden states unchanged. They remain in the sequence and can resume computation in subsequent modules. Text tokens continue to be processed in every module regardless of the routing decisions. Collectively, these decisions define an input-specific visual-computation path, allowing AdaVSkip to adapt both the extent and distribution of visual-token skipping across layers and modules. However, the non-differentiability of discrete routing decisions poses a key challenge to training these routers. We address this challenge through a progressive two-stage training strategy that updates only the routers while keeping the backbone frozen. In Stage I, we construct input-specific routing targets from module-wise necessity scores. For each module type, we assign keep targets to the smallest subset of highest-scoring modules whose cumulative necessity reaches a predefined fraction of the total necessity mass. We then train the routers using both necessity-guided supervision and the downstream language modeling loss to establish an initial routing policy. However, this training relies on estimated routing targets and approximate gradients through discrete routing decisions, which may limit the quality of the learned policy. Stage I therefore refines this policy through reinforcement learning, using the correctness of answers generated under sampled routing paths as direct feedback. To discourage overly conservative retention of visual-token computation, we additionally introduce a skip-consistency reward that regularizes sampled keep ratios toward their sample-specific Stage-I references, encouraging the policy to explore better-performing routing paths at comparable computational cost. Extensive experiments across three MLLM backbones and multiple benchmarks demonstrate that AdaVSkip improves performance–efficiency trade-offs over fixed-layer and prefix-based skipping methods, particularly under aggressive skipping. At moderate skipping levels, AdaVSkip reduces FLOPs by approximately 50% while largely preserving average performance. Moreover, AdaVSkip is complementary to visual-token compression. On LLaVA-NeXT-7B, combining it with CDPruner reduces FLOPs by 91.2% while retaining 97.2% of the original performance on average. Our main contributions are summarized as follows: • We characterize vertical redundancy in visual-token computation, revealing that its extent and distribution vary across inputs and that self-attention and MLP exhibit distinct patterns. • We propose AdaVSkip, an input-adaptive, module-wise visual-token skipping framework with two lightweight routers per layer that independently determine whether visual tokens participate in the computation of the self-attention and MLP modules. • We develop a progressive two-stage training strategy that first learns an initial routing policy through necessity-guided supervision and then refines it through reinforcement learning with correctness and skip-consistency rewards, while keeping the backbone frozen. • Across three MLLMs and multiple benchmarks, AdaVSkip offers better performance–efficiency trade-offs than fixed-layer and early-exit methods. With visual token compression, it saves more computation with less performance loss than token compression alone. 2 Related Work 2.1 Multimodal Large Language Models Recent multimodal large language models (MLLMs) extend the reasoning and instruction-following capabilities of LLMs to visual inputs Alayrac et al. (2022); Li et al. (2023b); Dai et al. (2023); Chen et al. (2024c); Bai et al. (2025b). A widely adopted design is the projector-based paradigm, which connects a pretrained vision encoder to an LLM through a lightweight projector, mapping images into visual tokens that are processed together with text tokens in the LLM Liu et al. (2023); Liu et al. (2024a). Building on this paradigm, recent MLLMs improve visual perception by using higher-resolution inputs or introducing more visual tokens for finer-grained visual representation Liu et al. (2024b); Zhu et al. (2025); Guo et al. (2025); Bai et al. (2025a). However, this trend substantially increases computational costs, making efficient visual-token computation increasingly important for MLLMs. 2.2 Horizontal Redundancy in Visual Tokens Existing methods for efficient MLLM inference mainly exploit horizontal redundancy by reducing the number of visual tokens processed by the language model. These methods fall into two categories. The first learns compact visual representations before the LLM stage, often through redesigned visual projectors Zhang et al. (2025b); Li et al. (2024b); Lu et al. (2025), but usually requires architectural modifications and training. The second performs training-free token reduction during inference by estimating token importance or redundancy and pruning or merging less informative tokens Chen et al. (2024a); Zhang et al. (2024); Yang et al. (2025); Ye et al. (2025); Zhang et al. (2025a). Although effective, both approaches reduce the visual-token count while processing the remaining tokens through all LLM layers, leaving vertical redundancy in visual-token computation largely unexploited. 2.3 Vertical Redundancy Across Layers Vertical redundancy has been explored in LLMs through early-exit and adaptive-depth methods, including SkipDecode Del Corro et al. (2023), LayerSkip Elhoushi et al. (2024), Mixture-of-Depths Raposo et al. (2024), and SkipGPT Zhao et al. (2025). In MLLMs, visual-token computation may be necessary in some layers but contribute little in others. This vertical redundancy remains less explored than horizontal redundancy. VTW Lin et al. (2025) and DyVTE Wu et al. (2024) reduce visual-token computation through early exit, with DyVTE supporting input-dependent exit depths. Both methods restrict visual-token computation to a prefix of layers. Challenging this prefix assumption, ShortV Yuan et al. (2025) show that important visual-token computation is not confined to early layers and preserve it in selected layers. Rather than using a fixed layer subset or a single exit point, AdaVSkip determines how much visual-token computation to skip and where for each input, with independent decisions for self-attention and MLP at each layer. 3 Method 3.1 Patterns of Vertical Redundancy To characterize vertical redundancy, we quantify the necessity of visual-token computation in the self-attention and MLP modules of each transformer layer, yielding 2L2L scores for an L-layer MLLM. For each input x with ground-truth answer y=(y1,…,yT)y=(y_1,…,y_T), we first run the original model on the input and answer in a single forward pass with teacher forcing, recording its output distributions at all answer-token positions. We then repeat the forward pass, skipping visual-token computation in one module at a time. For layer l and module type c∈=attn,mlpc =\attn,mlp\, we define the necessity score by comparing the resulting distributions with those of the original model: slc(x)=1T∑t=1TDKL(Pfull(⋅∣x,y<t)∥P−(l,c)(⋅∣x,y<t)),s_l^c(x)= 1T _t=1^TD_KL (P_full(· x,y_<t)\, \|\,P_-(l,c)(· x,y_<t) ), (1) where PfullP_full denotes the original model, and P−(l,c)P_-(l,c) denotes the intervened model where visual-token computation in the module of type c at layer l is skipped while other computations remain unchanged. A larger slc(x)s_l^c(x) indicates that skipping visual-token computation in this module causes a larger deviation from the original model, suggesting higher visual-token necessity. (a) Self-attn necessity across layers. (b) MLP necessity across layers. (c) Training-set t-SNE. Figure 2: Layer-wise visual-token necessity varies across benchmarks and samples. (a) and (b) report necessity scores for self-attention and MLP, respectively, on representative benchmarks. (c) shows a t-SNE visualization of LLaVA-1.5 SFT samples, each represented by a 64-dimensional vector formed by concatenating its self-attention and MLP necessity scores across all layers. We examine the resulting necessity scores at both the benchmark and sample levels. Figures 2(a) and 2(b) show the necessity distributions across layers for self-attention and MLP on representative benchmarks, revealing two patterns. First, both the locations and spans of high-necessity regions vary across benchmarks. For example, POPE exhibits concentrated necessity peaks with low scores in most layers, whereas TextVQA shows elevated necessity across a broader range of depths. Second, self-attention and MLP can exhibit markedly different necessity scores within the same layer. Across benchmarks, self-attention generally shows high necessity at layer 0, while MLP necessity at that layer remains relatively low. On POPE, self-attention exhibits a clear necessity peak at layer 11, while the corresponding MLP necessity is much weaker. Together, these results reveal distinct patterns of vertical redundancy across benchmarks and between modules within the same layer. We further examine sample-level variation on the LLaVA-1.5 SFT training set, as shown in Figure 2(c). For each sample, we concatenate the necessity scores of the 32 self-attention and 32 MLP modules into a 64-dimensional vector and project it into two dimensions using t-SNE. Samples from the same data source tend to cluster in the visualization, suggesting that visual-token redundancy is associated with task characteristics. Together, the benchmark-level and sample-level analyses motivate input-adaptive visual-token skipping with independent decisions for self-attention and MLP at each layer. 3.2 Adaptive Visual-Token Skipping We therefore propose AdaVSkip, which adaptively determines whether visual tokens participate in each layer’s self-attention and MLP modules, while text tokens continue to be processed in both modules. For layer l, let the input hidden states consist of visual tokens Vl−1∈ℝNv×dV^l-1 ^N_v× d and text tokens Tl−1∈ℝNt×dT^l-1 ^N_t× d, where NvN_v and NtN_t denote the numbers of visual and text tokens, respectively, and d is the hidden dimension. At the layer entrance, two lightweight routers, ℛattnlR_attn^l and ℛmlplR_mlp^l, independently determine visual-token participation in the subsequent self-attention and MLP modules. Both routers use the same text-side representation as input. Text hidden states encode the user query and can incorporate visual evidence from preceding layers, providing a compact signal for estimating the necessity of visual-token computation. Specifically, we construct the router input as zl=[Tlastl−1;1Nt∑i=1NtTil−1]∈ℝ2d,z^l= [\,T^l-1_last\;;\; 1N_t _i=1^N_tT^l-1_i\, ] ^2d, (2) where [⋅;⋅][·\,;\,·] denotes concatenation, and the two terms represent the last text-token hidden state and the average-pooled text representation, respectively. Both are computed only over prompt text tokens in training and inference, excluding answer tokens and padding. Given zlz^l, the two routers produce keep probabilities p(l,attn)p_(l,attn) and p(l,mlp)p_(l,mlp). At inference, the corresponding binary gates g(l,attn)g_(l,attn) and g(l,mlp)g_(l,mlp) are set to 1 when their keep probabilities exceed 0.5 and to 0 otherwise. Each gate controls all visual tokens in the corresponding module, with 1 preserving their computation and 0 skipping it. Routing is applied only during prefill, with no routing decisions during autoregressive decoding. For self-attention, g(l,attn)=1g_(l,attn)=1 preserves the original attention computation over visual and text tokens. When g(l,attn)=0g_(l,attn)=0, visual hidden states skip the module unchanged, while text tokens are updated through text-only attention. This differs from ShortV, which freezes visual-token updates but retains visual keys and values for text attention Yuan et al. (2025). For MLP, text tokens are always updated, whereas visual tokens receive the MLP update only when g(l,mlp)=1g_(l,mlp)=1. Otherwise, their hidden states at the MLP input are passed unchanged to the next layer. As shown in Figure 3, we train the routers in two stages while keeping the backbone frozen. Stage I establishes an initial routing policy through necessity-guided supervised training, and Stage I further refines this policy through reinforcement learning with task-level feedback. Figure 3: Overview of AdaVSkip’s progressive router training framework. 3.3 Stage I: Necessity-Guided Supervised Training In Stage I, we construct input-specific routing targets from module-wise visual-token necessity scores. Let m=(l,c)m=(l,c) index the module of type c∈=attn,mlpc =\attn,mlp\ at layer l, and let ℳcM_c denote the set of modules of type c across all layers. For each input, we rank the modules in ℳcM_c by their necessity scores in descending order and select the smallest top-ranked subset whose cumulative necessity accounts for at least a fraction τc _c of the total necessity mass within ℳcM_c. Selected modules are assigned qm=1q_m=1 to preserve visual-token computation, while the remaining modules are assigned qm=0q_m=0 to skip it. For a fixed threshold τc _c, the number and locations of preserved modules depend on each input’s necessity distribution. The threshold also controls the extent of skipping: larger values favor retaining more modules, while smaller values encourage more skipping. In addition to necessity-guided routing supervision, we incorporate the downstream language modeling loss to train the routers. During training, the forward pass uses hard gates obtained by thresholding the router probabilities, while a straight-through estimator (STE) approximates gradients through these gates during backpropagation. Let ℳ=⋃c∈ℳcM= _c M_c, and let pmp_m and gmg_m denote the keep probability and binary gate for module m, respectively. For an input x with ground-truth answer y=(y1,…,yT)y=(y_1,…,y_T), the Stage-I training objective is defined as ℒSFT=λroute∑m∈ℳBCE⁡(pm,qm)−λlm∑t=1Tlog⁡P⁡(yt∣y<t,x;gmm∈ℳ).L_SFT= _route _m BCE(p_m,q_m)- _lm _t=1^T P\! (y_t y_<t,x;\g_m\_m ). (3) Here, λroute _route and λlm _lm control the relative contributions of the two loss terms. The routing loss encourages keep probabilities to match the necessity-derived targets, while the language modeling loss encourages accurate prediction of ground-truth answer tokens under the selected routing path. To reduce the offline cost of full module-wise profiling, we also provide an accelerated variant for estimating visual-token necessity scores. Appendix describes this variant in detail and compares it with full profiling, showing that the two approaches yield comparable downstream performance after router training. All results in the main text use routing targets constructed through full profiling. 3.4 Stage I: Routing Policy Refinement Stage-I routing targets are derived from necessity scores, which may not fully capture the importance of visual-token computation for final-answer correctness. Moreover, router updates from the downstream language modeling loss rely on approximate gradients through hard routing decisions. We therefore refine the routing policy through reinforcement learning, using the correctness of answers generated under sampled routing paths as direct feedback. For an input x, a routing trajectory =gmm∈ℳg=\g_m\_m specifies the binary routing decisions across all skippable modules. The policy acts on visual-token computation rather than output tokens: each action determines whether visual tokens participate in the computation of a self-attention or MLP module. For each input, we sample a group of G routing trajectories by drawing actions from the Bernoulli distributions predicted by the routers. The frozen backbone then generates an answer under each sampled trajectory for subsequent correctness evaluation. The correctness reward is 1 if the generated answer exactly matches the ground-truth answer after normalization and 0 otherwise. However, optimizing correctness alone may encourage overly conservative routing that preserves excessive visual-token computation. To discourage this tendency, SCR penalizes deviations from sample-specific reference keep ratios. Before refinement, we run the Stage-I routers on the Stage-I training set to record these references separately for self-attention and MLP. For each sampled trajectory, SCR is defined as RSCR(i)=−∑c∈|ρc(i,x)−ρ¯c(x)|,R_SCR(g^i)=- _c | _c(g^i,x)- ρ_c(x) |, (4) where ρc(i,x) _c(g^i,x) denotes the fraction of modules of type c that preserve visual-token computation under trajectory ig^i, and ρ¯c(x) ρ_c(x) is the corresponding Stage-I reference. The coefficient λSCR _SCR controls the strength of this regularization. SCR thus encourages keep ratios to remain close to their Stage-I references while allowing the policy to adjust which modules process visual tokens. Finally, we adapt GRPO to optimize the router policy. For each input, we normalize rewards within the sampled group to obtain a trajectory-level advantage A^i A^i, which is shared by all routing actions in trajectory ig^i. This yields a relative learning signal for the following RL objective: ℒRL=−⁡[1G|ℳ|∑i=1G∑m∈ℳmin⁡(ηmiA^i,clipϵ(ηmi)A^i)],ηmi=πθ(gmi∣x,m)πθold(gmi∣x,m).L_RL=-E\! [ 1G|M| _i=1^G _m \! ( _m^i A^i,\,clip_ε( _m^i) A^i ) ], _m^i= _θ(g_m^i x,m) _ _old(g_m^i x,m). (5) Here, gmig_m^i denotes the routing action at module m in trajectory ig^i. The policies πθ _θ and πθold _ _old denote the current router policy and the policy used to sample the trajectories, respectively. 4 Experiments Table 1: Performance comparison of layer skipping and visual token compression on LLaVA-1.5-7B. AdaVSkip-S1 denotes the Stage-I-only variant, while AdaVSkip denotes the full method. Method FLOPs↓ VQAv2 GQA TextVQA POPE MME MMB SEED MMStar Avg.↑ Vanilla 100.0% 78.5 61.9 58.2 85.9 1510.7 64.3 66.1 33.6 100.0% Part I: Layer Skipping Methods Approximately 16 Skipped Layers VTW (AAAI’25)(N=16)(N=16) 56.5% 66.3 55.1 51.8 86.0 1497.0 64.0 66.2 32.8 94.9% DyVTE (NeurIPS’25) 56.4% 76.6 60.0 56.6 81.6 1491.4 64.7 66.1 – 98.0% ShortV (ICCV’25)(N=19)(N=19) 56.9% 75.7 60.8 54.4 85.5 1502.2 64.2 65.5 33.0 98.0% AdaVSkip-S1(K=16)(K=16) 51.8% 77.6 61.1 56.8 86.3 1516.3 64.0 66.2 33.3 99.3% AdaVSkip(K=16)(K=16) 52.0% 78.3 61.8 57.9 86.4 1512.8 64.9 65.9 33.4 100.0% Approximately 20 Skipped Layers VTW (AAAI’25)(N=20)(N=20) 45.6% 59.6 51.3 50.6 82.1 1475.4 63.4 64.9 32.7 91.6% ShortV (ICCV’25)(N=23)(N=23) 47.8% 75.3 59.4 50.4 85.6 1420.0 61.9 65.1 33.1 95.7% AdaVSkip-S1(K=20)(K=20) 44.7% 77.0 60.7 55.6 86.3 1465.7 63.9 65.7 32.8 98.2% AdaVSkip(K=20)(K=20) 45.2% 77.7 61.2 56.7 86.1 1449.9 64.0 66.4 33.8 99.0% Approximately 24 Skipped Layers VTW (AAAI’25)(N=24)(N=24) 34.8% 46.1 43.3 44.2 51.5 815.3 41.4 45.4 26.8 66.4% ShortV (ICCV’25)(N=27)(N=27) 38.7% 68.9 54.7 45.6 72.9 1169.0 58.6 56.8 31.1 85.8% AdaVSkip-S1(K=24)(K=24) 32.3% 72.2 57.0 52.2 83.1 1362.6 60.6 62.6 33.0 93.5% AdaVSkip(K=24)(K=24) 32.5% 74.0 57.6 52.6 83.3 1407.9 61.3 63.1 34.4 95.1% Part I: Visual Token Compression Methods 128 Retained Visual Tokens (22.2%) CDPruner (NeurIPS’25) 32.0% 76.6 59.9 56.2 87.4 1431.4 63.1 63.2 33.4 97.6% MMTok (ICLR’26) 32.0% 76.3 59.2 57.0 86.3 1442.2 62.3 63.1 33.7 97.4% ZOO-Prune (CVPR’26) 32.0% 76.5 59.4 57.8 87.1 1418.0 61.8 62.8 33.2 97.2% CDPruner + AdaVSkip 30.5% 77.6 60.8 57.1 87.3 1447.9 63.8 65.1 35.1 99.4% 64 Retained Visual Tokens (11.1%) CDPruner (NeurIPS’25) 22.5% 75.4 58.6 55.3 87.1 1415.1 61.1 62.1 32.4 95.8% MMTok (ICLR’26) 22.5% 75.2 58.2 56.0 85.7 1394.9 61.2 61.4 33.4 95.7% ZOO-Prune (CVPR’26) 22.5% 75.0 58.5 55.3 85.8 1369.9 60.2 60.9 31.7 94.5% CDPruner + AdaVSkip 21.8% 76.4 60.1 56.4 87.5 1424.0 63.0 64.1 34.9 98.3% 32 Retained Visual Tokens (5.6%) CDPruner (NeurIPS’25) 17.7% 73.6 57.0 53.2 87.4 1373.0 59.6 60.8 30.4 93.1% MMTok (ICLR’26) 17.7% 73.4 56.6 53.4 85.0 1343.6 59.4 59.7 33.6 93.4% ZOO-Prune (CVPR’26) 17.7% 72.2 55.8 54.1 84.7 1336.9 58.9 58.2 31.8 92.1% CDPruner + AdaVSkip 17.4% 75.3 58.5 55.0 87.6 1405.9 60.6 62.6 32.9 95.9% Datasets and Models. To empirically validate the effectiveness of AdaVSkip, we conduct experiments on three MLLMs: LLaVA-1.5-7B Liu et al. (2024a), LLaVA-NeXT-7B Liu et al. (2024b), and Qwen3-VL-8B-Instruct Bai et al. (2025a). For LLaVA-1.5-7B and LLaVA-NeXT-7B, we evaluate on VQAv2 Goyal et al. (2017), GQA Hudson and Manning (2019), TextVQA Singh et al. (2019), POPE Li et al. (2023c), MME Fu et al. (2023), MMBench Liu et al. (2024c), SEED-Bench Li et al. (2023a), and MMStar Chen et al. (2024b). For Qwen3-VL-8B-Instruct, we report results on ChartQA Masry et al. (2022), HallusionBench Guan et al. (2024), AI2D Kembhavi et al. (2016), DocVQA Mathew et al. (2021), InfoVQA Mathew et al. (2022), MMBench, MME, and SEED-Bench. Compared Methods. To evaluate AdaVSkip, we consider two dimensions of visual-token redundancy: vertical redundancy across layers and horizontal redundancy among tokens. For vertical redundancy, we compare AdaVSkip with VTW Lin et al. (2025), DyVTE Wu et al. (2024), and ShortV Yuan et al. (2025). VTW and DyVTE reduce layer-wise computation through early exiting, while ShortV freezes visual-token updates at predefined layers. For horizontal redundancy, we compare with visual token pruning methods, including VisionZip Yang et al. (2025), CDPruner Zhang et al. (2025a), MMTok Dong et al. (2026) and ZOO-Prune Kim et al. (2026). 4.1 Main Results Tables 1, 2, and 3 report the main experimental results obtained across three MLLM backbones. We compare AdaVSkip with fixed-layer and early-exit methods to evaluate its effectiveness in identifying and exploiting vertical redundancy. Visual token compression targets horizontal redundancy among tokens, whereas AdaVSkip skips visual-token computation in modules where its contribution is limited. Since these approaches target different sources of redundancy, a standalone comparison does not isolate the benefits of adaptive skipping. We therefore compare token compression alone with its combination with AdaVSkip at comparable FLOPs. Equivalent visual token budgets refer to the standalone compression method’s retention ratios. On LLaVA-1.5-7B, AdaVSkip achieves better performance–efficiency trade-offs than existing skipping methods across all evaluated settings. At K=16K=16, it achieves 100.0%100.0\% average relative performance with only 52.0%52.0\% of the original FLOPs, maintaining the original model’s average performance while outperforming ShortV by 2.02.0 percentage points at lower computational cost. The performance gap widens as skipping becomes more aggressive. At K=20K=20, AdaVSkip achieves 99.0%99.0\% average relative performance with 45.2%45.2\% of the original FLOPs, exceeding ShortV and VTW by 3.33.3 and 7.47.4 percentage points, respectively. At K=24K=24, it still retains 95.1%95.1\% average relative performance with only 32.5%32.5\% of the original FLOPs, whereas ShortV and VTW drop to 85.8%85.8\% and 66.4%66.4\%, respectively. The corresponding margins increase to 9.39.3 and 28.728.7 percentage points, indicating that AdaVSkip better preserves performance as the computational budget tightens. Beyond standalone skipping, combining AdaVSkip with CDPruner achieves 99.4%99.4\%, 98.3%98.3\%, and 95.9%95.9\% average relative performance at equivalent visual token budgets of 22.2%22.2\%, 11.1%11.1\%, and 5.6%5.6\%, respectively. Compared with CDPruner alone, these configurations improve average relative performance by 1.81.8, 2.52.5, and 2.82.8 percentage points while requiring fewer FLOPs in each setting. These benefits also extend to stronger backbones. On LLaVA-NeXT-7B, AdaVSkip achieves 100.4%100.4\% average relative performance at K=16K=16 with 46.8%46.8\% of the original FLOPs, exceeding ShortV and VTW by 1.21.2 and 5.15.1 percentage points, respectively, at lower computational cost. At the 5.6%5.6\% equivalent visual token budget, the combination of AdaVSkip and CDPruner achieves 97.2%97.2\% average relative performance with only 8.8%8.8\% of the original FLOPs, compared with 94.5%94.5\% at 9.7%9.7\% FLOPs for CDPruner alone. On Qwen3-VL-8B-Instruct, AdaVSkip achieves 98.6%98.6\% average relative performance at K=18K=18 with 51.4%51.4\% of the original FLOPs, outperforming ShortV and VTW by 1.81.8 and 16.816.8 percentage points, respectively. At the 10%10\% equivalent visual token budget, its combination with VisionZip achieves 85.5%85.5\% average relative performance with 13.5%13.5\% FLOPs, compared with 71.7%71.7\% at 13.9%13.9\% FLOPs for VisionZip alone. Here, aggressive token compression substantially degrades performance on tasks requiring fine-grained visual perception. Jointly exploiting vertical and horizontal redundancy mitigates these losses: the combination outperforms MMTok, the strongest standalone compression baseline at this budget, by 18.418.4, 13.813.8, and 7.77.7 points on ChartQA, DocVQA, and InfoVQA, respectively. We provide results under additional experimental settings in the appendix. Table 2: Performance analysis of layer skipping and visual token compression on LLaVA-NeXT-7B. Method FLOPs↓ VQAv2 GQA TextVQA POPE MME MMB SEED MMStar Avg.↑ Vanilla 100.0% 81.8 64.2 61.3 86.5 1519.0 67.4 70.2 35.8 100.0% (a) Layer Skipping Methods (∼ 50% of layers skipped) VTW (AAAI’25)(N=16)(N=16) 51.8% 75.6 55.8 47.3 87.5 1518.2 67.1 70.2 37.6 95.3% ShortV (ICCV’25)(N=19)(N=19) 51.6% 78.8 63.4 56.8 87.3 1526.0 67.2 70.4 37.4 99.2% AdaVSkip-S1(K=16)(K=16) 46.3% 81.0 63.6 57.3 87.3 1509.2 67.0 69.4 37.6 99.4% AdaVSkip(K=16)(K=16) 46.8% 81.3 64.1 59.4 87.3 1535.4 66.4 70.0 38.3 100.4% (b) Visual Token Compression Methods (5.6% visual tokens retained) VisionZip (CVPR’25) 9.7% 71.4 55.2 55.0 75.8 1327.8 58.6 58.3 31.8 87.1% CDPruner (NeurIPS’25) 9.7% 76.7 60.8 55.4 86.8 1425.3 64.2 65.5 33.9 94.5% MMTok (ICLR’26) 9.7% 75.6 60.0 54.2 83.8 1425.1 62.9 64.5 34.0 93.1% ZOO-Prune (CVPR’26) 9.7% 76.1 59.9 55.4 83.1 1397.9 64.2 64.1 35.0 93.6% CDPruner + AdaVSkip 8.8% 78.6 62.4 56.2 88.8 1473.9 65.9 67.6 35.3 97.2% Table 3: Performance analysis of layer skipping and visual token pruning on Qwen3-VL-8B-Instruct. Method FLOPs↓ ChartQA HallB AI2D DocVQA InfoVQA MMB MME SEED Avg.↑ Vanilla 100.0% 85.9 62.4 84.1 96.1 83.1 84.5 1746.2 78.7 100.0% (a) Layer Skipping Methods (∼ 50% of layers skipped) VTW (AAAI’25) (N=18)(N=18) 52.2% 27.7 58.5 81.7 94.3 37.6 79.6 1709.9 75.2 81.8% ShortV (ICCV’25) (N=19)(N=19) 51.5% 80.0 59.8 82.8 94.4 78.7 82.6 1702.7 78.0 96.8% AdaVSkip-S1 (K=18)(K=18) 51.5% 80.2 63.3 82.4 94.4 79.0 83.7 1720.1 77.5 97.8% AdaVSkip (K=18)(K=18) 51.4% 81.2 63.8 83.0 95.2 80.8 84.0 1709.2 78.2 98.6% (b) Visual Token Compression Methods (10.0% visual tokens retained) VisionZip (CVPR’25) 13.9% 44.8 45.0 72.4 53.8 35.5 75.5 1528.0 69.3 71.7% CDPruner (NeurIPS’25) 13.9% 33.3 45.6 69.4 43.4 31.9 74.9 1534.7 68.9 67.7% MMTok (ICLR’26) 13.9% 48.6 54.5 77.0 62.4 41.9 77.9 1636.3 73.3 78.7% ZOO-Prune (CVPR’26) 13.9% 50.5 48.2 70.7 51.6 33.4 76.1 1516.6 72.7 72.9% VisionZip + AdaVSkip 13.5% 67.0 56.0 79.2 76.2 49.6 81.4 1628.2 73.9 85.5% 4.2 Ablation Study of AdaVSkip Training Components Table 4: Ablation study of AdaVSkip training components on LLaVA-1.5-7B under K=16K=16. Method FLOPs↓ VQAv2 GQA TextVQA POPE MME MMB SEED MMStar Avg.↑ Layer-wise Routing (Full Stage-I) 52.4% 77.7 61.1 56.1 86.0 1500.7 64.0 65.9 33.1 98.9% Module-wise + Necessity Targets 54.4% 76.8 60.6 54.9 85.4 1448.4 64.8 66.2 33.3 98.2% + Downstream CE Loss 53.9% 77.2 59.2 55.6 86.5 1470.8 64.5 65.8 33.9 98.5% + Input-specific Routing Targets 51.8% 77.6 61.1 56.8 86.3 1516.3 64.0 66.2 33.3 99.3% + RL Refinement w/o SCR 53.5% 78.2 61.6 57.6 86.4 1512.9 64.6 66.0 33.9 100.0% + Skip-Consistency Reward 52.0% 78.3 61.8 57.9 86.4 1512.8 64.9 65.9 33.4 100.0% We conduct an ablation study of AdaVSkip on LLaVA-1.5-7B under K=16K=16 to examine the effects of routing granularity, Stage I supervision, and Stage I refinement, as shown in Table 4. We first compare layer-wise routing with module-wise routing under the full Stage I setting. The module-wise design improves average performance from 98.9%98.9\% to 99.3%99.3\% while reducing retained FLOPs from 52.4%52.4\% to 51.8%51.8\%, demonstrating the benefit of making separate routing decisions for self-attention and MLP modules. We then progressively add the downstream CE loss and input-specific routing targets to module-wise routing with necessity targets. The downstream CE loss improves average performance from 98.2%98.2\% to 98.5%98.5\%, while input-specific routing targets further increase it to 99.3%99.3\% and reduce retained FLOPs to 51.8%51.8\%. For Stage I, RL refinement without SCR achieves 100.0%100.0\% average performance but increases retained FLOPs to 53.5%53.5\%. Adding SCR reduces retained FLOPs to 52.0%52.0\% while maintaining a comparable average performance of 100.0%100.0\%, confirming its effectiveness in controlling the computation budget during policy refinement. 4.3 Analysis of Adaptive Routing Method GQA TextVQA POPE MME MMBench Avg. Fixed-Budget AdaVSkip 60.6 55.4 86.8 1484.5 64.6 98.6% Count-Matched Prior 61.5 57.7 85.6 1455.9 63.7 98.7% AdaVSkip 61.8 57.9 86.4 1512.8 64.9 100.2% Skipped Modules (%) 45.0 36.1 62.2 57.6 60.2 52.2 Table 5: Ablation study of adaptive routing on LLaVA-1.5-7B under K=16K=16. Method FLOPs (%) Prefill (ms) E2E (ms) KV Cache (MB) MME Vanilla 100.0 161.16 194.32 1087.74 1519.0 AdaVSkip 44.5 77.10 (2.09×2.09×) 109.36 (1.78×1.78×) 517.58 (↓52.4% 52.4\%) 1535.4 VisionZip 9.0 25.01 (6.44×6.44×) 63.39 (3.07×3.07×) 106.20 (↓90.2% 90.2\%) 1327.8 MMTok 9.0 25.01 (6.44×6.44×) 73.30 (2.65×2.65×) 112.71 (↓89.6% 89.6\%) 1425.1 CDPruner + AdaVSkip 8.3 26.40 (6.10×6.10×) 64.69 (3.00×3.00×) 103.74 (↓90.5% 90.5\%) 1473.9 Table 6: Comparison of inference efficiency and performance on MME using LLaVA-NeXT-7B. Table 5 examines two aspects of AdaVSkip’s input adaptivity: how many modules to skip and which modules to skip. First, Fixed-Budget AdaVSkip retains router-based module selection but fixes the skip ratio at 50%50\% for every input. At comparable FLOPs, AdaVSkip achieves 100.2%100.2\% average relative performance versus 98.6%98.6\% for this variant, supporting the benefit of input-dependent computation budgets. For example, its average skip ratio is lower on TextVQA (36.1%36.1\%) than on POPE (62.2%62.2\%). Second, Count-Matched Prior uses AdaVSkip’s skip counts for each input, separately for self-attention and MLP. Within each type, it selects the prescribed number of modules to skip using a fixed necessity ranking shared across all input samples. With both skip counts matched, AdaVSkip achieves 100.2%100.2\% average relative performance versus 98.7%98.7\% for Count-Matched Prior, demonstrating the benefit of input-dependent module selection. 4.4 Inference Efficiency Analysis Table 6 reports inference efficiency and MME performance for LLaVA-NeXT-7B. When used alone, AdaVSkip achieves 2.09×2.09× prefill and 1.78×1.78× end-to-end speedups without sacrificing task performance, improving the MME score from 1519.01519.0 to 1535.41535.4. It retains only 44.5%44.5\% of the original FLOPs and reduces the KV cache by 52.4%52.4\%. Combining AdaVSkip with CDPruner further increases the prefill and end-to-end speedups to 6.10×6.10× and 3.00×3.00×, respectively, while reducing the KV cache by 90.5%90.5\%. The combined method achieves an MME score of 1473.91473.9, with only a modest drop from vanilla inference. It offers inference speeds comparable to VisionZip and MMTok despite additional routing overhead, while outperforming both on MME. Figure 4: AdaVSkip routing visualization. Red boxes denote preserved visual-token computation. 4.5 Visualization and Case Study To further illustrate AdaVSkip’s adaptive behavior, we visualize the module-wise routing decisions for three representative samples under the K=24K=24 budget. As shown in Figure 4, different inputs lead to different amounts and locations of preserved visual-token computation across model depth. For the scene-understanding sample in the middle, the question can be answered with relatively coarse visual information, and AdaVSkip skips a large number of modules while still producing the correct answer. In contrast, the OCR sample requires reading fine-grained text from the image, and the spatial-relation sample requires recognizing the object and its relation to the surrounding scene; AdaVSkip preserves more modules for these two cases. Moreover, although AdaVSkip preserves more modules for both samples, their selected subsets differ, showing that the routing pattern adapts to the specific visual reasoning requirements of each input. These examples show that AdaVSkip adapts not only the amount of visual computation but also how it is allocated across modules. 5 Conclusion This work studies vertical redundancy in visual-token computation for efficient MLLM inference. Our analysis shows that its extent and distribution vary across inputs and between self-attention and MLP. Motivated by these findings, we propose AdaVSkip, which uses lightweight routers to independently control visual-token computation in self-attention and MLP at each layer. A progressive two-stage training scheme combines necessity-guided supervision with policy refinement, updating only the routers while keeping the backbone frozen. Experiments across three backbones and multiple benchmarks demonstrate better performance–efficiency trade-offs than fixed-layer and early-exit methods. Combining AdaVSkip with visual token compression jointly exploits vertical and horizontal redundancy, achieving better performance at lower computational cost than compression alone. References Alayrac et al. (2022) J. Alayrac, J. Donahue, P. Luc, A. Miech, I. Barr, Y. Hasson, K. Lenc, A. Mensch, K. Millican, M. Reynolds, et al. Flamingo: a visual language model for few-shot learning. Advances in Neural Information Processing Systems 35, p. 23716–23736. Cited by: §1, §2.1. Bai et al. (2025a) S. Bai, Y. Cai, R. Chen, K. Chen, X. Chen, Z. Cheng, L. Deng, W. Ding, C. Gao, C. Ge, et al. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Cited by: §1, §2.1, §4. Bai et al. (2025b) S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, et al. Qwen2.5-vl technical report. arXiv preprint arXiv:2502.13923. Cited by: §2.1. Chen et al. (2024a) L. Chen, H. Zhao, T. Liu, S. Bai, J. Lin, C. Zhou, and B. Chang An image is worth 1/2 tokens after layer 2: plug-and-play inference acceleration for large vision-language models. In European Conference on Computer Vision, p. 19–35. Cited by: §1, §2.2. Chen et al. (2024b) L. Chen, J. Li, X. Dong, P. Zhang, Y. Zang, Z. Chen, H. Duan, J. Wang, Y. Qiao, D. Lin, et al. Are we on the right way for evaluating large vision-language models?. Advances in Neural Information Processing Systems 37, p. 27056–27087. Cited by: §4. Chen et al. (2024c) Z. Chen, W. Wang, H. Tian, S. Ye, Z. Gao, E. Cui, W. Tong, K. Hu, J. Luo, Z. Ma, et al. How far are we to gpt-4v? closing the gap to commercial multimodal models with open-source suites. arXiv preprint arXiv:2404.16821. Cited by: §1, §2.1. Dai et al. (2023) W. Dai, J. Li, D. Li, A. Tiong, J. Zhao, W. Wang, B. Li, P. N. Fung, and S. Hoi Instructblip: towards general-purpose vision-language models with instruction tuning. Advances in Neural Information Processing Systems 36, p. 49250–49267. Cited by: §1, §2.1. Del Corro et al. (2023) L. Del Corro, A. Del Giorno, S. Agarwal, B. Yu, A. Awadallah, and S. Mukherjee Skipdecode: autoregressive skip decoding with batching and caching for efficient llm inference. arXiv preprint arXiv:2307.02628. Cited by: §2.3. Dong et al. (2026) S. Dong, J. Hu, M. Zhang, M. Yin, Y. Fu, and Q. Qian Mmtok: multimodal coverage maximization for efficient inference of vlms. In International Conference on Learning Representations, Vol. 2026, p. 48075–48099. Cited by: §4. Elhoushi et al. (2024) M. Elhoushi, A. Shrivastava, D. Liskovich, B. Hosmer, B. Wasti, L. Lai, A. Mahmoud, B. Acun, S. Agarwal, A. Roman, et al. Layerskip: enabling early exit inference and self-speculative decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 12622–12642. Cited by: §2.3. Fu et al. (2023) C. Fu, P. Chen, Y. Shen, Y. Qin, M. Zhang, X. Lin, J. Yang, X. Zheng, K. Li, X. Sun, et al. Mme: a comprehensive evaluation benchmark for multimodal large language models. arXiv preprint arXiv:2306.13394. Cited by: §4. Goyal et al. (2017) Y. Goyal, T. Khot, D. Summers-Stay, D. Batra, and D. Parikh Making the v in vqa matter: elevating the role of image understanding in visual question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6904–6913. Cited by: §4. Guan et al. (2024) T. Guan, F. Liu, X. Wu, R. Xian, Z. Li, X. Liu, X. Wang, L. Chen, F. Huang, Y. Yacoob, et al. Hallusionbench: an advanced diagnostic suite for entangled language hallucination and visual illusion in large vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 14375–14385. Cited by: §4. Guo et al. (2025) D. Guo, F. Wu, F. Zhu, F. Leng, G. Shi, H. Chen, H. Fan, J. Wang, J. Jiang, J. Wang, et al. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07062. Cited by: §2.1. Hudson and Manning (2019) D. A. Hudson and C. D. Manning Gqa: a new dataset for real-world visual reasoning and compositional question answering. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 6700–6709. Cited by: §4. Kembhavi et al. (2016) A. Kembhavi, M. Salvato, E. Kolve, M. Seo, H. Hajishirzi, and A. Farhadi A diagram is worth a dozen images. In European Conference on Computer Vision, p. 235–251. Cited by: §4. Kim et al. (2026) Y. Kim, Y. Zhang, H. Liu, A. Jung, S. Lee, and S. Hong Zoo-prune: training-free token pruning via zeroth-order gradient estimation in vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 39572–39582. Cited by: §4. Li et al. (2024a) B. Li, Y. Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, P. Zhang, Y. Li, Z. Liu, et al. Llava-onevision: easy visual task transfer. arXiv preprint arXiv:2408.03326. Cited by: §1. Li et al. (2023a) B. Li, R. Wang, G. Wang, Y. Ge, Y. Ge, and Y. Shan Seed-bench: benchmarking multimodal llms with generative comprehension. arXiv preprint arXiv:2307.16125. Cited by: §4. Li et al. (2023b) J. Li, D. Li, S. Savarese, and S. Hoi Blip-2: bootstrapping language-image pre-training with frozen image encoders and large language models. In International Conference on Machine Learning, p. 19730–19742. Cited by: §1, §2.1. Li et al. (2024b) W. Li, Y. Yuan, J. Liu, D. Tang, S. Wang, J. Qin, J. Zhu, and L. Zhang Tokenpacker: efficient visual projector for multimodal llm. arXiv preprint arXiv:2407.02392. Cited by: §1, §2.2. Li et al. (2023c) Y. Li, Y. Du, K. Zhou, J. Wang, W. X. Zhao, and J. Wen Evaluating object hallucination in large vision-language models. arXiv preprint arXiv:2305.10355. Cited by: §4. Lin et al. (2025) Z. Lin, M. Lin, L. Lin, and R. Ji Boosting multimodal large language models with visual tokens withdrawal for rapid inference. In Proceedings of the AAAI Conference on Artificial Intelligence, p. 5334–5342. Cited by: §1, §2.3, §4. Liu et al. (2024a) H. Liu, C. Li, Y. Li, and Y. J. Lee Improved baselines with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 26296–26306. Cited by: §1, §2.1, §4. Liu et al. (2024b) H. Liu, C. Li, Y. Li, B. Li, Y. Zhang, S. Shen, and Y. J. Lee Llava-next: improved reasoning, ocr, and world knowledge. Cited by: §1, §2.1, §4. Liu et al. (2023) H. Liu, C. Li, Q. Wu, and Y. J. Lee Visual instruction tuning. Advances in Neural Information Processing Systems 36, p. 34892–34916. Cited by: §1, §2.1. Liu et al. (2024c) Y. Liu, H. Duan, Y. Zhang, B. Li, S. Zhang, W. Zhao, Y. Yuan, J. Wang, C. He, Z. Liu, et al. Mmbench: is your multi-modal model an all-around player?. In European Conference on Computer Vision, p. 216–233. Cited by: §4. Lu et al. (2025) D. Lu, Y. Sun, Z. Zhang, L. Huang, J. Zeng, M. Shu, and H. Cao Internvl-x: advancing and accelerating internvl series with efficient visual token compression. arXiv preprint arXiv:2503.21307. Cited by: §1, §2.2. Masry et al. (2022) A. Masry, D. X. Long, J. Q. Tan, S. Joty, and E. Hoque Chartqa: a benchmark for question answering about charts with visual and logical reasoning. arXiv preprint arXiv:2203.10244. Cited by: §4. Mathew et al. (2022) M. Mathew, V. Bagal, R. Tito, D. Karatzas, E. Valveny, and C. Jawahar Infographicvqa. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 1697–1706. Cited by: §4. Mathew et al. (2021) M. Mathew, D. Karatzas, and C. Jawahar Docvqa: a dataset for vqa on document images. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, p. 2200–2209. Cited by: §4. Raposo et al. (2024) D. Raposo, S. Ritter, B. Richards, T. Lillicrap, P. C. Humphreys, and A. Santoro Mixture-of-depths: dynamically allocating compute in transformer-based language models. arXiv preprint arXiv:2404.02258. Cited by: §2.3. Singh et al. (2019) A. Singh, V. Natarajan, M. Shah, Y. Jiang, X. Chen, D. Batra, D. Parikh, and M. Rohrbach Towards vqa models that can read. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 8317–8326. Cited by: §4. Vasu et al. (2025) P. K. A. Vasu, F. Faghri, C. Li, C. Koc, N. True, A. Antony, G. Santhanam, J. Gabriel, P. Grasch, O. Tuzel, et al. Fastvlm: efficient vision encoding for vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 19769–19780. Cited by: §1. Wu et al. (2024) Q. Wu, W. Lin, Y. Zhou, W. Ye, Z. Zen, X. Sun, and R. Ji Accelerating multimodal large language models via dynamic visual-token exit and the empirical findings. arXiv preprint arXiv:2411.19628. Cited by: §1, §2.3, §4. Yang et al. (2025) S. Yang, Y. Chen, Z. Tian, C. Wang, J. Li, B. Yu, and J. Jia Visionzip: longer is better but not necessary in vision language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 19792–19802. Cited by: §1, §2.2, §4. Ye et al. (2025) W. Ye, Q. Wu, W. Lin, and Y. Zhou Fit and prune: fast and training-free visual token pruning for multi-modal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, p. 22128–22136. Cited by: §1, §2.2. Yuan et al. (2025) Q. Yuan, Q. Zhang, Y. Liu, J. Chen, Y. Lu, H. Lin, J. Zheng, X. Han, and L. Sun Shortv: efficient multimodal large language models by freezing visual tokens in ineffective layers. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 329–339. Cited by: §1, §2.3, §3.2, §4. Zhang et al. (2025a) Q. Zhang, M. Liu, L. Li, M. Lu, Y. Zhang, J. Pan, Q. She, and S. Zhang Beyond attention or similarity: maximizing conditional diversity for token pruning in mllms. arXiv preprint arXiv:2506.10967. Cited by: §1, §2.2, §4. Zhang et al. (2025b) S. Zhang, Q. Fang, Z. Yang, and Y. Feng LLaVA-mini: efficient image and video large multimodal models with one vision token. arXiv preprint arXiv:2501.03895. Cited by: §1, §2.2. Zhang et al. (2024) Y. Zhang, C. Fan, J. Ma, W. Zheng, T. Huang, K. Cheng, D. Gudovskiy, T. Okuno, Y. Nakata, K. Keutzer, et al. Sparsevlm: visual token sparsification for efficient vision-language model inference. arXiv preprint arXiv:2410.04417. Cited by: §2.2. Zhao et al. (2025) A. Zhao, F. Ye, Y. Fan, J. Tong, Z. Fei, H. Su, and X. Shen Skipgpt: dynamic layer pruning reinvented with token awareness and module decoupling. arXiv preprint arXiv:2506.04179. Cited by: §2.3. Zhu et al. (2025) J. Zhu, W. Wang, Z. Chen, Z. Liu, S. Ye, L. Gu, H. Tian, Y. Duan, W. Su, J. Shao, et al. Internvl3: exploring advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479. Cited by: §1, §2.1.

Instant research discovery