Paper deep dive

LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models

Yuxiang Shen, Hailong Huang, Zhenkun Gao, Xueheng Li, Man Zhou, Chengjun Xie, Haoxuan Che, Xuanhua He, Jie Zhang

Year: 2026Venue: arXiv preprintArea: cs.CVType: PreprintEmbeddings: 70

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%

Last extracted: 7/20/2026, 9:19:48 AM

Summary

The paper introduces LookWise, a training-free framework for Multimodal Large Language Models (MLLMs) that enhances fine-grained visual reasoning by adaptively deciding when and where to look. It addresses the computational inefficiency and noise of indiscriminate cropping and the semantic drift of raw attention maps. LookWise uses a confidence-based module to determine if additional visual evidence is needed and a semantic-guided localization module to precisely crop relevant regions, achieving higher accuracy and a 4.0x speedup over search-based methods like ZoomEye.

Entities (10)

LookWise · framework · 98%Multimodal Large Language Models · model-class · 95%confidence-based module · component · 93%semantic-guided localization module · component · 93%ZoomEye · method · 92%AOKVQA · dataset · 90%POPE · dataset · 90%InternVL · model · 88%LLaVA-NeXT · model · 88%DeepEyes · method · 85%

Relation Signals (8)

LookWise → improves → Accuracy

confidence 95% · LookWise consistently improves accuracy over strong baselines

LookWise → outperforms → ZoomEye

confidence 92% · achieving an approximately 4.0× inference speedup over the search-based method ZoomEye

LookWise → uses → confidence-based module

confidence 90% · LookWise follows a two-stage pipeline: a confidence-based module decides when to look more carefully

LookWise → uses → semantic-guided localization module

confidence 90% · and a semantic-guided localization module determines where to look.

ZoomEye → has → high inference latency

confidence 85% · resulting in high inference latency

Multimodal Large Language Models → suffersfrom → Semantic Drift

confidence 85% · and a drift between semantic intent and spatial attention

Multimodal Large Language Models → suffersfrom → perceptual redundancy

confidence 85% · existing training-free methods suffer from two flaws: perceptual redundancy from indiscriminate cropping

DeepEyes → requires → Computational Resources

confidence 80% · these methods require extensive computational resources and task-specific training data

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Multimodal Large Language Models (MLLMs) are shifting towards "Thinking with Images" by actively exploring image details. While effective, large-scale training is computationally expensive, which has spurred growing interest in lightweight, training-free solutions. However, existing training-free methods suffer from two flaws: perceptual redundancy from indiscriminate cropping, which increases computational cost and introduces noise; and a drift between semantic intent and spatial attention, which prevents accurate localization of user-focused regions. To address these challenges, we propose LookWise, a framework for adaptive visual reasoning. LookWise follows a two-stage pipeline: a confidence-based module decides when to look more carefully, and a semantic-guided localization module determines where to look. This design enables MLLMs to adaptively acquire fine-grained visual evidence without additional training. Experiments on fine-grained and high-resolution visual reasoning benchmarks show that LookWise consistently improves accuracy over strong baselines while achieving an approximately $4.0\times$ inference speedup over the search-based method ZoomEye, demonstrating robust cross-model generalization.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

68,800 characters extracted from source content.

Expand or collapse full text

IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY1 LookWise: Knowing When and Where to Look for Fine-Grained Visual Reasoning in Multimodal Large Language Models Yuxiang Shen, Hailong Huang, Zhenkun Gao, Xueheng Li, Man Zhou, Chengjun Xie, Haoxuan Che, Xuanhua He, and Jie Zhang Abstract—Multimodal Large Language Models (MLLMs) are shifting towards "Thinking with Images" by actively exploring image details. While effective, large-scale training is compu- tationally expensive, which has spurred growing interest in lightweight, training-free solutions. However, existing training- free methods suffer from two flaws: perceptual redundancy from indiscriminate cropping, which increases computational cost and introduces noise; and a drift between semantic intent and spatial attention, which prevents accurate localization of user-focused regions. To address these challenges, we propose LookWise, a framework for adaptive visual reasoning. LookWise follows a two-stage pipeline: a confidence-based module decides when to look more carefully, and a semantic-guided localization module determines where to look. This design enables MLLMs to adaptively acquire fine-grained visual evidence without additional training. Experiments on fine-grained and high-resolution visual reasoning benchmarks show that LookWise consistently improves accuracy over strong baselines while achieving an approximately 4.0× inference speedup over the search-based method ZoomEye, demonstrating robust cross-model generalization. Code is avail- able at: https://github.com/Xiaoxiang100/LookWise. Index Terms—Training-free inference, adaptive visual reason- ing, semantic-guided localization, fine-grained perception, multi- modal large language models. I. INTRODUCTION In recent years, Multimodal Large Language Models (MLLMs) [1], [2], [3], [4] have made significant progress in visual-language understanding [5], [6], [7], [8]. The prevailing paradigm, “Thinking about Images,” encodes images into This work was supported by the Anhui Provincial Natural Science Foun- dation under Grant 2408085MD090. Yuxiang Shen, Hailong Huang, and Zhenkun Gao contributed equally to this work. Xuanhua He and Jie Zhang are the corresponding authors. Yuxiang Shen and Xueheng Li are with the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China, and also with the University of Science and Technology of China, Hefei 230026, China (e-mail: shenyuxiang_xmu@163.com; lixue- heng@mail.ustc.edu.cn). Man Zhou is with the University of Science and Technology of China, Hefei 230026, China (e-mail: manman@mail.ustc.edu.cn). Chengjun Xie and Jie Zhang are with the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei 230031, China (e-mail: cjxie@iim.ac.cn; zhangjie@iim.ac.cn). Hailong Huang is with Zhejiang University, Hangzhou 310027, China (e- mail: huanghailong73@gmail.com). Zhenkun Gao is with East China Normal University, Shanghai 200241, China (e-mail: 51275901149@stu.ecnu.edu.cn). Haoxuan Che and Xuanhua He are with The Hong Kong University of Science and Technology, Hong Kong, China (e-mail: hche@ust.hk; xhecd@connect.ust.hk). static visual features and then applies language-based reason- ing mechanisms such as Chain-of-Thought [9], [10]. However, this passive paradigm limits adaptive visual acquisition, as inputs are uniformly resized, cropped, or tiled into fixed visual tokens, producing static representations that cannot recover fine-grained details or adjust focus according to reasoning demands. As a result, performance often degrades on tasks involving tiny targets, subtle attributes, or high-resolution scenes [11], [12]. To address this limitation, a new paradigm termed “Think- ing with Images” [13] has emerged, empowering models with active perception capabilities. Instead of relying only on a single static encoding, models can dynamically decide when and where to zoom, crop, or scan images in response to the current query [14], [15]. This capability has been demonstrated by training-based approaches such as DeepEyes [16] and V- Star [17]. However, these methods require extensive computa- tional resources and task-specific training data, motivating the exploration of training-free alternatives [18], [19] that enable adaptive visual reasoning at inference time. Recent training-free methods attempt to acquire fine-grained visual details through inference-time strategies. Search-based methods, such as DC 2 [20] and ZoomEye [21], iteratively evaluate image regions but incur 5–10× inference overhead. Single-pass methods, such as MLLMs-Know [22], rely on attention maps for efficient localization but often struggle with fine-grained and multi-object cases. These methods reveal two fundamental challenges: (1) determining when visual en- hancement is necessary, since indiscriminate cropping wastes computation and may introduce noise; and (2) identifying where to precisely localize targets, especially when several objects compete for attention. Together, these questions of when to look and where to look constitute the core challenge of training-free adaptive visual reasoning. To validate these hypotheses and understand the underly- ing mechanisms, we conducted controlled experiments across multiple benchmarks, revealing two critical patterns (as shown in Sec I-A). Observation 1: Models know when to look, but current methods do not ask. Existing methods usually crop all inputs under the assumption that finer visual details are always beneficial. However, our analysis shows that cropping can degrade performance in some cases, as it may fail to add useful information for simple queries and instead introduce interfering noise, as shown in Figure 3. We further observe arXiv:2603.00171v3 [cs.CV] 30 May 2026 IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY2 systematic differences in the model’s confidence distributions across samples of varying difficulty, indicating that MLLMs inherently provide a signal for when more visual information is needed. This motivates our confidence-based strategy for deciding when to look more carefully. Observation 2: Attention-based methods struggle with where to look. While attention maps provide an efficient way to locate visual regions, they often fail in multi-object scenarios. The model may attend to only one target, drift toward a salient distractor, or merge nearby instances into a single response. Figure 5 reveals that raw attention captures “what is relevant” rather than reliably resolving “where each queried object is.” This motivates our semantic-guided local- ization module, which decouples language intent from raw visual attention to determine where to look. Building on these insights, we propose LookWise. Un- like prior methods that uniformly crop all inputs, LookWise dynamically determines whether cropping is beneficial and precisely localizes the queried targets. Specifically, LookWise first decides when to look more carefully by using token-level confidence from the initial global prediction as an efficient indicator of information sufficiency. Once additional visual evidence is needed, it determines where to look through semantic-guided localization, which uses question-derived tar- get semantics to guide visual attention toward the queried regions. In this way, LookWise achieves decision-aware zoom- ing without iterative search or additional training. Tissue box is hard to identify, I need look more carefully Answer : Blue The target is highly salient,I could provide a more accurate response to the question. What is the color of the truck? What is the color of the tissue box? Answer : White Confidence = 0.87 Answer : Black Confidence = 0.34 When When should I look more carefully? Where Where should I look more carefully? Fig. 1. Illustration of LookWise: enabling MLLMs to know when and where to look for adaptive visual reasoning. Our contributions can be summarized as follows: (1) We identify two key deficiencies of existing training-free visual reasoning methods: indiscriminate cropping that causes per- ceptual redundancy, and raw attention localization that suf- fers from semantic drift in multi-object scenes. (2) We pro- pose LookWise, a plug-and-play framework that combines confidence-based decision making with semantic-guided local- ization to answer when and where to look. (3) Extensive exper- iments demonstrate that LookWise achieves superior accuracy across multiple benchmarks while delivering a 4.0× speedup over the search-based method ZoomEye and generalizing across different MLLM backbones. I. RELATED WORK A. Multimodal Large Language Models Multimodal Large Language Models (MLLMs) have made significant progress by aligning visual perception with textual reasoning [23]. In modern architectures, a pre-trained vision encoder is connected to a frozen large language model via learnable fusion modules. These modules range from simple linear projections [24], [25] to query-based networks like Q- Formers [26], which map visual features into the semantic space of the LLM. However, traditional MLLMs typically encode images at a fixed, low resolution, leading to the loss of fine-grained details. This limits their performance on tasks involving small objects or complex scenes [27], [28]. To preserve more visual informa- tion, recent models such as LLaVA-NeXT [29] and InternVL [30] adopt dynamic high-resolution strategies. They divide a high-resolution image into smaller patches, encode each patch independently, and directly concatenate the resulting visual tokens. While these high-resolution strategies improve image clar- ity, they still process image patches passively and uniformly. The language model must consume all generated visual tokens, regardless of whether a patch contains task-relevant evidence or background noise. This wastes computation and may still miss the critical details required by the query. Therefore, fine- grained MLLM inference requires an adaptive mechanism that knows when and where to look. B. Vision-Language Reasoning To overcome the limitations of fixed-resolution processing, recent research enables MLLMs to actively interact with visual content, similar to how human eyes focus on specific regions [14], [15]. Current approaches to this active perception generally fall into two categories: training-based and training- free methods. Training-based approaches, such as DeepEyes [16] and Pixel Reasoner [31], teach models how to crop and zoom through reinforcement learning or supervised step-by-step visual trajectories. While effective, these methods are highly expensive. They require massive amounts of specialized train- ing data and computational resources. Moreover, the learned perception strategies are tightly bound to the specific model during training, making it difficult to apply them to other off- the-shelf MLLMs. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY3 To avoid these high training costs, training-free methods attempt to simulate human-like zooming directly during the inference stage. Early training-free methods rely on iterative search. For example, ZoomEye [21] crops images into smaller patches and explores them through a hierarchical tree. Al- though this search-based approach finds fine-grained details accurately, the multi-round evaluation process is extremely slow, resulting in high inference latency. More efficient ap- proaches, such as ViCrop [22], use internal cross-attention maps to directly locate targets. However, they typically adopt an “always-crop” policy, focusing on where to look while ne- glecting whether zooming is needed. For visually simple tasks, this blind cropping wastes computation and often introduces background noise that misleads the model. Furthermore, they struggle to separate and locate targets accurately in complex, multi-object scenarios. In contrast, LookWise unifies token confidence and atten- tion maps for autonomous, decision-aware zooming. Token confidence determines whether more visual information is necessary before any extra visual operation is performed, while semantic-guided attention determines where to crop once visual enhancement is triggered. By deciding whether to crop before performing localization, LookWise retains the efficiency of attention-based methods and achieves competitive localization quality without the high latency of iterative search. C. Confidence Estimation and Adaptive Inference Confidence estimation allows models to evaluate their own uncertainty. In Large Language Models (LLMs), token-level probabilities serve as simple and effective indicators of how confident the model is about its predictions [32], [33]. When a model shows low confidence, it is more likely to hallucinate or make reasoning errors. Recent studies have introduced various methods to quantify this uncertainty, such as aggregating token probabilities [34] or calculating entropy [35], essentially helping models "know what they don’t know." Although confidence signals are widely used in text gen- eration, they are rarely used to control visual computation in MLLMs. Many training-free methods still crop every input, wasting computation on visually simple samples. We use token confidence from the initial global prediction as a routing signal. If the confidence is high, the model answers directly; otherwise, LookWise activates semantic-guided localization and performs a second prediction with local visual details. This adaptive strategy reduces redundant visual processing and improves inference efficiency. I. METHODS A. Analysis: When and Where to Look Before introducing LookWise, we analyze the fundamental limitations of existing training-free methods in determining when and where to crop. Our analysis aims to answer two critical questions: (1) Can MLLMs know when additional visual information is necessary? (2) What causes attention- based localization to fail on fine-grained tasks? 1) Can MLLMs know when to look?: In the Thinking with Images paradigm, some training-based methods learn when to look through additional supervision or reinforcement learning. Most training-free methods, however, adopt indiscriminate cropping and re-encoding for all samples, neglecting the critical question of when to crop or zoom. For visually simple samples, forced cropping incurs unnecessary computational overhead and may introduce redundant or noisy local informa- tion that misleads the model into incorrect answers. Figure 3 presents two representative examples. First, a crop containing a salient “SLOW DOWN” sign misleads the model to answer with the sign text, overriding the correct global prediction. Second, when the target “cocoa” is already clear in the global image, adding a crop provides little new information but still increases inference cost. Additional visual evidence is often expected to improve the prediction confidence of MLLMs. However, this assumption has not been systematically validated across different sam- ples. To investigate this, we conduct a token-level confidence analysis before and after cropping. Based on the confidence change after cropping, we categorize samples into two groups: (i) Need Processing, where confidence increases (Score crop > Score org ), and (i) No Need Processing, where confidence remains unchanged or decreases (Score crop ≤ Score org ). The confidence distributions of the two groups are shown in Figure 2. Fig. 2. Probability density distributions of initial average confidence on the AOKVQA and POPE datasets. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY4 Slowing down is a recommended action, not a prohibited one Example of Cropping-Induced Misjudgment 1.Original Image Input • Question : What is prohibited when driving on the street? • Correct Answer : " speeding" 2.Original Image + Cropped Sub-Image Input When the original image and a cropped sub-image (which highlights the "20 ZONE" and "SLOW DOWN!") are input together, the model is confused by the sign information • Question: What is prohibited when driving on the street? • Incorrect Answer: " slow down" Slowing down is a recommended action, not a prohibited one Example of Crop with No Effect 1.Original Image Input • Question: " What food is the brown powder sprinkled on the drink? " • Correct Answer: " cocoa" 2.Original Image + Cropped Sub-Image Input When a precisely cropped sub-image is added, the model’s response remains unchanged • Question: " What food is the brown powder sprinkled on the drink?” • Correct Answer: " cocoa" Cropping can’t provide new information and it’s a redundant operation that wastes resources Fig. 3. Limitations of indiscriminate cropping. Left: Cropping a region containing the “SLOW DOWN” sign misleads the model to answer “slow down,” even though the question asks what action is prohibited and the correct answer is “speeding.” Right: Cropping the drink area adds no new visual information, but the model’s prediction remains unchanged while additional computation is introduced. As shown in Figure 2, the Need Processing and No Need Processing samples exhibit a clear separation in their original average confidence distributions. On both AOKVQA and POPE, the No Need Processing samples show higher initial confidence, indicating that the original image provides suffi- cient visual evidence for prediction. In contrast, the Need Pro- cessing samples are concentrated in lower-confidence regions, suggesting that additional visual details are more beneficial when the model’s initial prediction is uncertain. This analysis reveals that cropping has a heterogeneous effect on model confidence: while some samples benefit from additional details, many show unchanged or even reduced confidence. Importantly, the initial confidence reflects the model’s need for additional visual information, enabling a simple and reliable criterion for deciding when to look. 2) Why does attention fail to decide where to look?: MLLMs possess inherent visual grounding capabilities [22], and their spatial attention enables more efficient localization than search-based methods such as DC 2 [20] and Zoom- Eye [21]. However, directly using raw attention maps intro- duces two challenges for precise target localization. The first problem occurs when a question involves multiple objects, e.g., “Is the bicycle on the left or right side of the motorcycle?” The attention response may spread across several entities and fail to isolate the queried object. We address this by first extracting the main target from the question using few-shot in-context learning (ICL). The text tokens of this target are then used as queries over image tokens, producing a semantic-guided attention map that focuses on the object of interest more precisely. The second problem appears when many objects of the same category are close together, e.g., “How many people?” The attention map often merges nearby instances into one region, making instance-level reasoning difficult. We mitigate this issue by applying attention thresholding to generate candidate boxes, followed by an NMS-inspired overlap filtering step to remove redundant regions. The model then produces the final answer using the refined visual evidence. By addressing these two issues, LookWise obtains more reliable target regions and provides a practical answer to where to look in complex scenes. B. LookWise: Overview To address the limitations of existing training-free methods, we propose LookWise, a two-stage framework for adaptive IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY5 Calculate Confidence Low High Is the steel ladder on the left or right side of the wagon? Question Step 1: Target Extraction <Reason> To answer this question...</Reason> <Target> "steel ladder“ , "wagon“ </Target> Target : “steel ladder ” Target : “wagon ” Image embedding Target 1 embedding Target 2 embedding Step 2 ： localization Crop Image Original Image Answering directly If with high confidence If with low confidence The steel ladder is on the left side of the wagon Final Answer Step 3: Crop the image Fig. 4.Overview of the LookWise framework. The model evaluates token confidence to decide when to look more carefully. If uncertain, it extracts key textual targets and uses them as queries in cross-attention over visual tokens to determine where to crop. The selected local regions are then provided together with the global image to produce the final answer. visual reasoning. The first stage is a confidence-based module that decides when to crop, avoiding redundant processing for simple instances. The second stage is a semantic-guided local- ization module that determines where to crop by integrating the target semantics extracted from the question with the model’s spatial attention. Together, these two stages enable MLLMs to look more carefully only when needed and at the regions most relevant to the query. The overall workflow of LookWise is illustrated in Figure 4. C. Confidence-Based Module: Determining When to Look For a given input image I and query Q, the model first performs an initial inference to obtain a preliminary answer sequence, denoted as Answer pre . Relying solely on the global view may be insufficient for fine-grained reasoning, but cropping every sample is inefficient. We therefore design a confidence-based module to determine whether a second visual enhancement stage should be triggered. Unlike explicit prompting methods that require an additional generation step to make a routing decision, our strategy directly uses the token probabilities from the initial forward pass. This allows the model to estimate its own uncertainty with virtually zero extra computational cost. 1) Model Confidence Quantification.: We quantify the con- fidence of the model by utilizing the average confidence of the tokens in the generated answer sequence. Specifically, the sequence is defined as Y =y 1 ,y 2 ,...,y T ,(1) where y t represents the t-th token, and P (y t |·) denotes the probability of the model generating this token at step t. We define the confidence score C as the arithmetic mean of the Softmax probabilities of all tokens in the sequence. This confidence score C intuitively reflects the model’s epistemic uncertainty regarding the preliminary answer Answer pre . C = 1 T T X t=1 P (y t |·).(2) 2) Optimal Threshold Selection.: A simple yet effective ap- proach is to use a fixed decision threshold τ . As demonstrated in Section IV-F, cropping only samples with confidence scores below this threshold improves performance while reducing computational overhead. When representative validation data are available and closely aligned with the test distribution, we further consider a principled threshold optimization strategy. Specifically, thresh- old selection is formulated as a statistical decision problem, where the optimal τ is obtained by maximizing the following utility function: J λ (τ ) = TPR(τ )− λ· FPR(τ ), λ > 0.(3) Here, TPR and FPR denote the true positive rate and false positive rate, respectively, and λ serves as a cost-weighting hy- perparameter that balances potential performance gains against unnecessary local inspection. This optimization strategy assumes access to representa- tive validation data. In open-domain settings where such data cannot be reliably obtained, employing a high fixed threshold remains a stable and effective alternative, as evi- denced by the results in Table VI. These observations further demonstrate the robustness and generalization capability of our approach. 3) Final Decision Process.: Based on the optimal decision boundary, we perform the following binary decision: (1) C ≥ τ : the model exhibits high confidence on the global view. In this case, we consider the available visual information IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY6 sufficient for reasoning and directly output Answer pre as the final answer. (2) C < τ : the model shows high uncertainty, indicating that the current visual information may be insuffi- cient to support fine-grained reasoning. We therefore trigger the subsequent semantic-guided localization module to acquire high-resolution local evidence. D. Semantic-GuidedLocalizationModule:Determining Where to Look Once cropping is triggered, the model must precisely lo- calize the target region. However, relying exclusively on raw visual attention often leads to attention misalignment. In com- plex scenes or multi-object queries, unguided attention may activate irrelevant backgrounds or conflate distinct instances. To address this limitation, we introduce a semantic-guided localization module that uses explicit language intent to guide spatial localization. The process has two steps. First, we decouple the core physical target from the user query. Second, the text tokens of the extracted target act as queries over image tokens, producing a semantic-guided attention map. This text- to-vision guidance translates the abstract query intent into a concrete bounding box for local evidence extraction. 1) Semantic Decoupling (What to Localize).: To accurately extract key visual entities, We design a prompt template based on Chain-of-Thought. We guide the model to first generate a <Reason> tag before outputting the final <target>. Formally, given a question Q, the model outputs a semantic target set E = e 1 ,e 2 ,.... For example, for "What kind of animal is on the red sign?", the model generates: <Reason>The question asks about something on a sign. The core subject to locate is the sign.</Reason><Target>red sign</Target>. Then it extracts E ="red sign". To illustrate this extraction capability across complex sce- narios, we integrate a standardized prompt template and pro- vide comprehensive few-shot examples, as shown below. Target Extraction Prompt Question Template: Based on the question, identify ONLY the primary, physical objects or subjects mentioned. Do not include adjectives, locations, or states. Please respond using <Reason> and <target> tags. Guidelines: • A ’target’ must be a simple, concrete noun (e.g., ’car’, ’per- son’, ’table’). • For multiple distinct subjects, separate them with a comma (e.g., ’cat, dog’). Question: What is the brand name on the laptop? Response: <Reason>The question asks about a brand name found on a laptop. The core physical subject is the lap- top.</Reason><target>laptop</target> Question: How many people are wearing hats in the image? Response: <Reason>Thequestionaskstocountpeople who are wearing hats. The primary subjects are people and hats.</Reason><target>people, hats</target> Question: Is the cat on the left or right side of the wooden chair? Response: <Reason>The question asks about the position of a cat relative to a wooden chair. The core physical subjects are the cat and the chair.</Reason><target>cat, wooden chair</target> Question: What kind of food is on the white plate? Response: <Reason>The question asks about the type of food on a plate. The primary physical subjects are the food and the plate.</Reason><target>food, white plate</target> Question: What is the man in the blue shirt holding in his hand? Response: <Reason>The question asks about an object held by a man. The core physical subjects are the man and the object he is holding.</Reason><target>man, object</target> Question: What time is displayed on the clock on the wall? Response: <Reason>Thequestionasksforthetime displayed on a clock. The core physical subject is the clock.</Reason><target>clock</target> 2) Spatial Attention Mapping (Where to Localize).: After identifying the semantic target E, we instantiate a target- specific localization prompt as “where is the <target>”. We then feed this prompt to the MLLM and extract the cross- attention responses between the target tokens and image tokens to obtain a target-conditioned attention map. Compared with attention derived from the original question, this target-specific formulation suppresses irrelevant objects and reduces attention drift in multi-object scenes. 2D Attention Map Generation. Let Z img denote the visual token sequence obtained from the image patches, and let Z E text denote the text tokens corresponding to the extracted target. We extract the cross-attention responses from the MLLM and average them across all attention heads: A 1D = Agg Attn(Z E text → Z img ) (4) The resulting vector A 1D corresponds to the flattened visual token grid. We reshape it according to the patch layout of the vision encoder to obtain a 2D attention map A map ∈R h×w , where each value indicates the attention intensity of the corresponding image region. Adaptive Multi-Scale Sliding Window. Given the spatial attention map A map , our goal is to translate this activation grid into a precise bounding box. To accommodate the variance in input resolutions, we dynamically set an adaptive base dimension S base (typically 224× 224, scaling to 448× 448 for ultra-high-resolution images) based on the original image size. Building upon this adaptive base, we apply a predefined set of scaling ratios R =1.0, 1.2, 1.4, 1.6, 1.8, 2.0, 4.0, 6.0. For each ratio r ∈ R, the candidate extraction size is calculated as S r = S base ×r. This physical dimension is then projected onto the A map grid to determine a corresponding window covering w r × h r attention blocks. We slide this window across the 2D grid to find the position (x ∗ ,y ∗ ) that yields the maximum attention sum V max . To select the most fitting scale, we evaluate the localization sharpness (or contrast) ∆ r for the peak window of each ratio. The sharpness is calculated by taking the difference between the peak attention V max and the average attention of its four adjacent surrounding windows V adj , normalized by the window area: ∆ r = V max − Mean(V adj ) w r × h r (5) Intuitively, a higher ∆ r means the window tightly encapsulates the highly activated target while excluding the unactivated background. We select the optimal ratio r ∗ that maximizes this sharpness metric. Coordinate Mapping. Once the optimal grid window is found, we calculate its geometric center and linearly map it IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY7 back to the original high-resolution image space. Based on the selected box size, we deduce the final pixel coordinates (x 1 ,y 1 ,x 2 ,y 2 ) and clamp them to the image boundaries. The original image I is then cropped accordingly to produce a detail-preserving local view I crop for the final prediction stage. 3) Dealing with Multi-instances.: For queries involving multiple instances of the same category (e.g., "How many people?"), we introduce an attention-based post-processing scheme to separate closely spaced targets. Specifically, an attention score threshold is applied to filter foreground regions, with high-response areas identified as candidate boxes. To remove highly overlapping boxes, we apply an NMS-inspired [36] deduplication step: boxes with an IoU greater than 0.5 are treated as duplicates and pruned. This process effectively reduces spatial redundancy, enabling accurate localization and counting of individual objects. IV. EXPERIMENT A. Implementation Details Experimental Setup & Network Configuration. To validate the effectiveness of our approach, we integrate the Look- Wise framework with two widely-used open-source models: Qwen2.5VL-3B [1] and LLaVA-1.5-7B [24]. In our exper- iments, we set the maximum input resolution for Qwen to 3,211,264 pixels. All experiments are conducted on NVIDIA H200 GPUs. For attention map acquisition, we extract at- tention scores from specific layers to isolate visual regions relevant to the target entity. Specifically, for Qwen2.5-VL, we extract attention scores from Layer 22, utilizing the last token of the target sequence as the query. For LLaVA-1.5, we use attention weights from Layer 14, averaging the attention scores of all tokens comprising the final word of the target phrase to create a stable spatial representation. We conduct a comprehensive comparison against both the original baselines and several state-of-the-art methods, including training-free approaches (e.g., ZoomEye [21], MLLMs-Know [22], DC 2 [20], and ViCrop [37]) and training-based models (e.g., Pix- elReasoner [31]). Datasets and Benchmarks. We evaluate our model across various dimensions using four benchmarks. First, we select AOKVQA [38] and POPE [39] to assess general reasoning capabilities on knowledge-intensive tasks. Furthermore, to validate the effectiveness of our confidence-based decision module on fine-grained features, we introduce two challenging high-resolution datasets: V*-Bench [28] and HR-Bench [20]. These benchmarks emphasize fine-grained attribute recogni- tion and complex spatial reasoning, strictly evaluating the model’s ability to localize tiny visual details. B. Qualitative Analysis of Semantic-Guided Localization To visually demonstrate the effectiveness of the proposed where-to-look mechanism, we compare the attention maps and final bounding regions generated by LookWise with those from a baseline attention-based localization method (MLLMs- Know). B Single-Target Only Identification Question: Is the steel ladder on the left or right side of the wagon? targets: " steel ladder", " wagon" Answer: The steel ladder is on the left side of the wagon. Error examples Incorrect Target Identification Question: Does the kid have curly hair or straight hair? targets: "kid" Answer: The kid has straight hair. Our Results Two-Target Simultaneous Localization Question: Is the steel ladder on the left or right side of the wagon? targets: “steel ladder”, “wagon” Answer: The steel ladder is on the right side of the wagon. Correct Target Identification Question: Does the kid have curly hair or straight hair? targets: "kid" Answer: The kid has curly hair. A C D Fig. 5. Comparison of localization precision. The baseline model often suffers from single-target bias (A) and attention drift (C). In contrast, LookWise explicitly decouples multiple queried targets to ensure complete coverage (B) and accurately identifies the intended subject (D). The attention maps show that semantic-guided localization suppresses noisy activations and maintains precise spatial focus. 1) Accurate multi-object localization:: In Figure 5(A), the baseline suffers from a single-object bias, attending only to the ‘steel ladder’ while missing the ‘wagon’. In contrast, LookWise explicitly decouples multiple targets, activating both the ‘steel ladder’ and ‘wagon’. As shown in Figure 5(B), the resulting union of bounding boxes effectively covers both objects, providing the MLLM with a complete and precise local visual source to accurately reason about their spatial relationship. 2) Robust target identification:: In Figure 5(C), the base- line’s attention drifts to the dominant adult in the scene (in- correct target). Guided by our prompt-driven target extraction, LookWise accurately focuses on the "child", ensuring the extracted region aligns with the intended semantic target, as shown in Figure 5(D). These qualitative results demonstrate that semantic-guided localization significantly mitigates atten- tion diffusion and drift, ensuring that the visual enhancement process relies on accurate local evidence. C. Quantitative Analysis Our method shows strong performance on benchmarks that require fine-grained visual evidence, such as HR-Bench and V*-Bench. Although LLaVA operates at a fixed resolution, IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY8 TABLE I COMPARISON WITH STATE-OF-THE-ART METHODS. HIGHEST RESULTS ARE MARKED IN DARKER COLORS (BLUE FOR LLAVA,RED FOR QWEN). THE GRAY ROWS SHOW THE ABSOLUTE IMPROVEMENT (∆) OF OURS OVER THE BASELINE. ∗ RESULTS ARE REPRODUCED BY US USING OFFICIAL IMPLEMENTATIONS. ModelMethodTraining-freeAOKVQAPOPEV ∗ BenchHR-Bench 4KHR-Bench 8K LLaVA-v1.5-7B Baseline ∗ [24]✓71.0086.9848.6836.1332.13 DC 2 [20]✓--57.60-39.50 VisCrop [37]✓--62.3046.2535.75 MLLMs-Know ∗ [22]✓ 72.3187.2556.0244.3837.25 ZoomEye [21]✓70.5688.9483.2549.8848.63 Ours (LookWise)✓ 72.9087.3762.8047.3842.00 ∆ vs. Baseline+1.90+0.39+14.12+11.25+9.87 Qwen2.5VL-3B Baseline ∗ [1]✓71.4487.2075.9067.5058.88 Pixel Reasoner [31]✗--84.82-66.00 MLLMs-Know ∗ [22]✓71.6289.1275.9066.3664.88 ZoomEye [21]✓71.26 88.9389.0170.1368.38 Ours (LookWise)✓73.1089.1286.3873.2570.00 ∆ vs. Baseline+1.66+1.92+10.48+5.75+11.12 which naturally restricts its attention granularity, LookWise still brings notable improvements: +11.25% on HR-Bench 4K, +9.87% on HR-Bench 8K, and +14.12% on V*-Bench. When integrated with Qwen2.5-VL, LookWise reaches a state-of- the-art 70.00% on HR-Bench 8K, exceeding the baseline by 11.12% and outperforming ZoomEye (65.63%). These gains indicate that adaptive visual reasoning can compensate for res- olution bottlenecks while avoiding unnecessary computation on irrelevant regions. For reasoning-heavy benchmarks such as AOKVQA and POPE, the improvements are more moderate, suggesting that their main bottleneck lies in logical reasoning and common- sense knowledge rather than visual resolution. Nevertheless, the consistent gain, e.g., improving Qwen2.5-VL to 73.10% on AOKVQA, shows the benefit of confidence-based routing over indiscriminate cropping. By looking more carefully only on uncertain samples, LookWise preserves the global context for easy cases and provides additional details for visually ambiguous ones. Overall, the results show that the proposed inference strategy improves fine-grained perception while maintaining robustness on general visual reasoning tasks. D. Efficiency analysis 1) Overall Inference Efficiency: Inference efficiency is an important factor for training-free methods. We evaluate LookWise against two representative baselines: ZoomEye (the current SOTA) and MLLMs-Know (an attention-based method). ZoomEye organizes the image into a hierarchical tree and performs iterative search to simulate zooming. Although effective, this global traversal becomes increasingly expensive. As shown in Figure 6, processing HR-Bench 8K requires more than 10 hours. In contrast, LookWise localizes informative regions directly through semantic-guided attention, avoiding exhaustive search over the full image. On HR-Bench 8K, this yields an approximately 4.0× speedup compared with ZoomEye, while maintaining competitive performance. aokvqapopevstarhr-bench-4khr-bench-8k Datasets 0 100 200 300 400 500 600 700 Time (Minutes) 54min 5h35min 2h7min 5h9min 10h35min (42m) 12min (267m) 1h8min (45m) 1h22min (213m) 1h36min (454m) 3h1min (42m) 11m43s (269m) 1h6m (44m) 1h23m (210m) 1h39m (472m) 2h43m Inference Time Comparison (Lower is Better) Zoomeyes Ours MLLMs Know Fig. 6. Comparison of total GPU inference time (in minutes) across different methods (measured on NVIDIA H200 GPUs). Furthermore, LookWise runs at a similar speed to MLLMs- Know across all evaluated datasets, while achieving higher ac- curacy on challenging high-resolution benchmarks. For exam- ple, LookWise reaches 86.38% on V*-Bench and 70.00% on HR-Bench 8K, compared with 75.90% and 64.88% achieved by MLLMs-Know, respectively. These results suggest that the improvement mainly comes from more effective allocation of local visual computation rather than increased inference cost. 2) Contrast with LLM-Prompting Strategy: An intuitive alternative to our decision strategy is to directly prompt the MLLM to make the choice, e.g., asking "Do you need to look more carefully to answer this question? Answer Yes or No". However, we demonstrate that utilizing token confidence is significantly more efficient than direct prompting. To clearly illustrate the efficiency gap, we compare the inference pipelines of both methods in Algorithm 1. As shown in Algorithm 1, the LLM-Prompting approach requires a strictly sequential pipeline. The model must explic- IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY9 Algorithm 1 Inference Workflow: LLM-Prompting vs. Ours 1: # Method 1: LLM-Prompting 2: Decision ← LLM_Generate(I global , "Need local de- tails?") 3: if Decision == "No" then 4:# Still requires 1 round to answer 5:Return LLM_Generate(I global , Question) 6: else 7:# Decision is "Yes": Extract local details via attention 8: I local ← Target_Guided_Extract(I global ) 9:Return LLM_Generate(I global , I local , Question) 10: end if 11: # Method 2: Ours (Token Confidence) LLM_Generate(I global , Question) 12: if Confidence(Logits) ≥ Threshold then 13:# High confidence: Answer directly 14:Return Answer 15: else 16:# Low confidence: Extract local details via attention 17: I local ← Target_Guided_Extract(I global ) 18:Return LLM_Generate(I global , I local , Question) 19: end if TABLE I TOTAL INFERENCE TIME COMPARISON BETWEEN LLM-PROMPTING AND OUR TOKEN CONFIDENCE DECISION MECHANISMS (MEASURED ON NVIDIA H200 GPUS). Decision MechanismAOKVQAPOPEV*-BenchHR 4KHR 8K LLM-Prompting16m45s1h33m1h29m1h47m3h06m Ours (Confidence)12m33s1h08m1h22m1h36m3h01m itly generate a text-based decision before solving the actual task. Consequently, even for simple queries that do not require careful visual inspection, the model wastes computational resources and time performing an additional LLM_Generate step just to output "No". In contrast, our Token Confidence method avoids this bot- tleneck by attempting to answer the question directly. We calculate the confidence score from the internal probability distribution (Softmax logits) of this first attempt. Since these logits are generated automatically during the forward pass, obtaining the confidence score costs virtually zero extra time. If the confidence is high, we directly output the initial answer, bypassing the need for a second inference round entirely. To empirically validate this workflow advantage, we compare the total inference time of the two decision mechanisms. As shown in Table I, our strategy is consistently faster across all benchmarks under identical hardware conditions. The effi- ciency gains are particularly pronounced on general reasoning datasets (e.g., saving 25 minutes on POPE, a 26.8% reduction), where a large portion of queries can be answered using only the global view. Even on challenging datasets like HR-Bench 8K where careful visual inspection is frequently triggered, our method still reduces overall latency by eliminating the redundant "Yes/No" token generation step. E. Ablation study 1) Ablation on Confidence-based Decision module: As shown in Table I, our confidence-based decision module consistently improves accuracy compared to indiscriminate cropping (w/o confidence-based decision). For Qwen2.5-VL, our strategy yields a +1.14% on AOKVQA and +0.52% on V*- Bench. This confirms that indiscriminate cropping introduces visual noise and misjudgments, which our selective strategy effectively avoids. TABLE I QUANTITATIVE EVALUATION OF CONFIDENCE-BASED DECISION MODULE. ModelAOKVQAV*-Bench Qwen2.5VL-3B w/o confidence-based decision71.96%85.86% w/ confidence-based decision73.10%86.38% ∆+1.14%+0.52% Table IV demonstrates the efficiency of our confidence- based decision module. On AOKVQA, we are able to skip redundant cropping for 67.60% of the samples, effectively halving the inference time without compromising accuracy. Interestingly, this adaptive approach achieves even higher accuracy than applying full cropping (+1.14), indicating that it successfully avoids the interference noise caused by unnec- essary local details. For more challenging benchmarks such as V*-Bench, the model naturally performs cropping more frequently, yet it still manages to reduce redundant processing by 22.52% while sustaining strong performance. These results highlight that LookWise dynamically adjusts its processing behavior according to task complexity, providing a robust and practical trade-off between computational efficiency and predictive accuracy. TABLE IV PERFORMANCE AND EFFICIENCY COMPARISON (MEASURED ON NVIDIA H200 GPUS). BenchmarkAccTimeCrop% AOKVQA w/o confidence-based decision71.9626m24s100% w/ confidence-based decision73.1012m33s32.40% ∆+1.14-13m51s-67.60% V*-Bench w/o confidence-based decision85.861h04m100% w/ confidence-based decision86.381h18m77.48% ∆+0.52+14m-22.52% 2) Ablation on Semantic-Guided Localization: To validate the effectiveness of the semantic-guided localization mod- ule, we compare model performance with and without this component (Table V). The quantitative results reveal a clear distinction depending on the task requirements. On standard visual reasoning datasets (AOKVQA and POPE), the performance improvements are marginal. This is reasonable, as these benchmarks primarily emphasize complex logical reasoning and commonsense knowledge rather than the perception of fine-grained visual details. For such tasks, the IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY10 bottleneck typically lies in the reasoning capabilities of the language model rather than visual clarity. Since the base global encoding already captures the necessary semantic elements to support this reasoning, extracting high-resolution local crops provides limited additional benefit. However, on high-resolution and detail-oriented benchmarks (V*-Bench, HR 4K, and HR 8K), the module provides sub- stantial gains. When processing complex scenes, base models often fail to capture small target objects. By incorporating our module, LLaVA-v1.5 improves by 6.78% on V*-Bench and 4.75% on HR 8K. The impact is even more pronounced for Qwen2.5-VL, which achieves a 9.96% boost on V*-Bench and a 6.14% increase on HR 4K. These consistent improvements across different model ar- chitectures demonstrate the core advantage of the proposed module. By explicitly using the queried target to guide visual attention, the module isolates task-relevant regions more ac- curately. This mechanism prevents background noise from interfering with the reasoning process and is crucial for fine- grained localization in complex, high-resolution scenes. TABLE V ABLATION ON SEMANTIC-GUIDED LOCALIZATION MODULE. Model & StrategyAOKVQAPOPEV*-BenchHR 4KHR 8K LLaVA-v1.5-7B w/o Semantic-guided Localization72.31%87.25%56.02%44.38%37.25% w/ Semantic-guided Localization72.90%87.37%62.80%47.38%42.00% ∆0.59%0.12%6.78%3.00%4.75% Qwen2.5VL-3B w/o Semantic-guided Localization71.62%89.12%75.90%66.36%64.88% w/ Semantic-guided Localization71.96%89.12%85.86%72.50%70.00% ∆0.34%–9.96%6.14%5.12% F. Confidence Sensitivity Analysis To evaluate the robustness of our confidence-based decision module, we conduct a sensitivity analysis aligned with the two scenarios introduced in subsection I-C: optimizing the decision threshold via the utility function, and applying direct fixed thresholds for open-domain settings. 1) Threshold Optimization via Utility Function: In our formulation, the decision of whether to extract local evidence relies on the utility function J λ (τ ) = TPR(τ )− λ· FPR(τ ). The hyperparameter λ serves as a cost-weighting factor that balances the expected benefit of local inspection against the potential risk of unnecessary computational redundancy. Specifically, larger values of λ impose stronger penalties on false-positive decisions, encouraging a more conservative local-inspection policy (i.e., yielding a lower optimal threshold τ ). Conversely, smaller values of λ favor more aggressive local inspection to ensure the capture of fine-grained visual details, thereby expanding the triggering region by adopting a higher threshold. TABLE VI SENSITIVITY ANALYSIS OF THE CONFIDENCE-BASED DECISION MODULE. Model BaselineFull Local InspectionConfidence-based Decision (λ)Fixed Thresh. (Original)(All Inspect)1.01.52.0(τ = 0.92) Threshold (τ )--0.8300.7340.5330.92 Qwen2.5-VL71.4471.9672.5873.1072.4972.05 When representative validation data is available, λ can be tuned to maximize task-specific performance. We analyze this optimization on the AOKVQA benchmark by varying λ. As shown in Table VI, the performance peaks at 73.10% when λ = 1.5, which derives an optimal threshold of τ = 0.734. Notably, this dynamically optimized approach clearly out- performs both the original baseline (71.44%) and the rigid “Full Local Inspection” strategy (71.96%). This performance gap indicates that indiscriminate zooming not only wastes computational resources but can also introduce visual noise that degrades reasoning. Ultimately, this confirms that our algorithm successfully finds a favorable trade-off between extracting necessary fine-grained details and preventing redun- dant local inspection. TABLE VII SENSITIVITY ANALYSIS ACROSS MULTIPLE BENCHMARKS UNDER VARYING CONFIDENCE THRESHOLDS τ . DatasetBase randomcontinuous 0.800.900.940.960.981.00 HR-4K67.50 72.75 72.5072.75 72.50 72.50 72.50 HR-8K58.88 68.25 69.88 70.00 70.00 70.00 70.00 V*-Bench 75.90 81.15 84.8186.38 86.38 85.86 85.86 AOKVQA 71.44 72.31 72.4072.23 72.23 72.14 71.97 2) Threshold Robustness in Open-Domain Settings: In real- world open-domain scenarios, it is often impractical to care- fully tune the hyperparameter λ due to the lack of validation data. To evaluate whether our framework can still perform reliably without such tuning, we sweep across a range of threshold values τ (see Table VII). The results show that our method is not sensitive to the exact choice of the threshold. Even without precise calibration, the model maintains stable performance, and its accuracy consistently remains above the baseline across a wide range of relatively high thresholds (e.g., τ ∈ [0.94, 1.00]). TABLE VIII EVALUATION UNDER A FIXED THRESHOLD (τ = 0.96) BenchmarkBaseline τ = 0.96Extraction Reduced AOKVQA71.4472.2310.39% V*-Bench75.9086.3814.13% HR-Bench 8K58.8870.008.00% HR-Bench 4K67.5072.5010.00% Average ∆–+9.35%10.63% 3) Robustness with a fixed Threshold: Building on the observed stability, we evaluate a single, conservative fixed threshold (τ = 0.96) across all datasets without any per- task adjustment (Table VIII). Under this strictly unified zero- shot setting, LookWise consistently outperforms the baseline, achieving an average accuracy gain of +9.35%. Simultane- ously, it skips redundant local inspection and reduces pro- cessed image regions by an average of 10.63%. This confirms that LookWise can function as a robust plug-and-play module that adapts to diverse tasks using a simple fixed threshold. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY11 What is the color of the bag? Question ： The color of the bag is purple. Answer ： (b) Attention / Cropped Region Target Object ： Bag. (a) Original Image(b) Attention / Cropped Region (a) Original Image What kind of animal is on the blue sail? Question ： The animal on the blue sail is a spider. Answer ： Target Object ： Blue Sail. Baby stroller, Person in orange Is the baby stroller on the left or right side of the person in orange? Question ： The baby stroller is on the right side of the person in orange. Answer ： (b) Attention / Cropped Region Target Object ： (c) Attention / Cropped Region(a) Original Image Fig. 7. Qualitative examples of adaptive visual reasoning. Left: For small object perception, LookWise precisely targets the “Bag” to identify its purple color despite a complex background. Right: For fine-grained attribute recognition, the model isolates the “Blue Sail” to reveal the subtle spider logo. What is the color of the bag? Question ： The color of the bag is purple. Answer ： (b) Attention / Cropped Region Target Object ： Bag. (a) Original Image(b) Attention / Cropped Region (a) Original Image What kind of animal is on the blue sail? Question ： The animal on the blue sail is a spider. Answer ： Target Object ： Blue Sail. Baby stroller, Person in orange Is the baby stroller on the left or right side of the person in orange? Question ： The baby stroller is on the right side of the person in orange. Answer ： (b) Attention / Cropped Region Target Object ： (c) Attention / Cropped Region (a) Original Image Fig. 8. Spatial Reasoning with Multiple Targets. The model attends to both the "Baby stroller" and the "Person in orange," extracting a region that preserves their relative spatial position. G. Discussion & Generalization 1) Detailed Performance Analysis on V*-Bench: While LookWise demonstrates strong performance across most benchmarks, we provide a deeper analysis of its behavior on V*-Bench, focusing on the specific challenges posed by representation constraints and complex spatial reasoning tasks. Resolution and Representation Constraints. LLaVA-v1.5 processes images at a fixed resolution of 336× 336, which limits the level of visual detail the model can capture. For benchmarks such as V*-Bench that contain many extremely small objects, this resolution becomes a clear limitation. Since each visual token corresponds to a relatively large image region, the resulting attention maps are inherently coarse. As a result, the model often lacks the precision needed to localize very small targets and produce tight bounding boxes. This lim- itation is reflected in the much stronger performance observed when using the Qwen2.5-VL. By supporting dynamic high- resolution inputs (up to 3.2M pixels), Qwen2.5-VL provides more detailed visual representations, leading to more precise attention maps and more accurate cropping decisions. Efficiency vs. Spatial Reasoning Trade-off. V*-Bench heav- ily features spatial reasoning and visual search queries, such as determining the relative positions of distant objects. While LookWise successfully localizes multiple individual targets, it generates relatively tight bounding boxes around them. When objects are extremely far apart, localizing tightly around IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY12 them may inadvertently discard the surrounding contextual information necessary for inferring broad relational tasks. In contrast, search-based methods like ZoomEye utilize a recursive tree-search: they iteratively split the image into sub-regions, verify targets, and merge confident regions into a single large composite image. This exhaustive approach makes them more robust for complex spatial queries but incurs massive computational overhead (as demonstrated in our efficiency analysis, ZoomEye suffers from an approxi- mately 4.0× latency increase). Overall, LookWise maintains competitive average performance while operating significantly faster. The performance gap on specific V*-Bench spatial queries primarily reflects a deliberate algorithmic trade-off: LookWise prioritizes inference efficiency and precise target- level localization over exhaustive global search. 2) Case Studies on Adaptive Visual Reasoning: We provide qualitative examples (Figure 7–Figure 8) to demonstrate how LookWise knows where to look for challenging queries. Fine-Grained and Small Object Perception. Small visual details are easily compressed or lost in standard global low- resolution views. For instance, identifying a minute “spider” logo on a sail or a low-contrast bag in a dimly lit street (Fig- ure 7) is highly challenging for baseline models. LookWise addresses this by extracting the core textual targets (“blue sail”, “bag”) and using them to localize subtle query-relevant regions. The resulting high-resolution observations preserve the fine-grained features needed for accurate prediction. Multi-Target Spatial Reasoning. Questions requiring relative position comprehension often cause severe attention drift in standard models, leading them to focus on a single dominant object or irrelevant background. When asking whether a “baby stroller” is to the left or right of a “person in orange” (Fig- ure 8), LookWise explicitly decouples the two entities. Each textual target produces a corresponding bounding box, and the boxes are merged through a spatial union operation to obtain a local view that covers both subjects. This localized observation preserves their relative spatial relation while filtering out distractors, enabling more reliable spatial reasoning. V. CONCLUSION We propose LookWise, a training-free framework for visual reasoning in MLLMs, which can adaptively determines when to look more carefully and where the query-relevant regions are located. First, to mitigate the noise caused by blind cropping, we introduce a confidence-based module. In many cases, the global view alone is sufficient for answering simple queries. LookWise therefore estimates uncertainty from the token confidence produced by the initial prediction and triggers cropping only when needed. Second, to address attention drift in current localization methods, we introduce semantic-guided localization. The queried target is extracted from the question and used as the query in a text-to-image attention mechanism, allowing the model to locate relevant regions more accurately. Extensive experiments demonstrate that LookWise achieves strong performance across multiple challenging benchmarks while maintaining high inference efficiency. As a plug-and- play module that requires no parameter updates, it works across different MLLM backbones. Overall, LookWise reduces perceptual redundancy and alleviates spatial attention drift, highlighting the importance of adaptive visual reasoning for fine-grained multimodal perception. REFERENCES [1] S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang et al., “Qwen2.5-vl technical report,” arXiv preprint arXiv:2502.13923, 2025. [2] W. Wang, Z. Gao, L. Gu, H. Pu, L. Cui, X. Wei, Z. Liu, L. Jing, S. Ye, J. Shao et al., “Internvl3.5: Advancing open-source multimodal models in versatility, reasoning, and efficiency,” arXiv:2508.18265, 2025. [3] A. Hurst, A. Lerer, A. P. Goucher, A. Perelman, A. Ramesh, A. Clark, A. Ostrow, A. Welihinda, A. Hayes, A. Radford et al., “Gpt-4o system card,” arXiv:2410.21276, 2024. [4] G. Comanici, E. Bieber, M. Schaekermann, I. Pasupat, N. Sachdeva, I. Dhillon, M. Blistein, O. Ram, D. Zhang, E. Rosen et al., “Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities,” arXiv:2507.06261, 2025. [5] Z. Gao, X. Wang, X. Tan, and Y. Xie, “Tpru: Advancing temporal and procedural understanding in large multimodal models,” arXiv preprint arXiv:2602.18884, 2026. [6] B. Li, W. Huang, Z. Gao, Y. Wang, Y. Shen, J. Lin, L. You, Y. Shen, S. Lin, W. Ouyang et al., “Llava-radz: Can multimodal large lan- guage models effectively tackle zero-shot radiology recognition?” arXiv preprint arXiv:2503.07487, 2025. [7] S. Wang, B. Liu, Z. Gao, L. Ma, X. Wang, Y. Xie, and X. Tan, “Explore with long-term memory: A benchmark and multimodal llm- based reinforcement learning framework for embodied exploration,” arXiv preprint arXiv:2601.10744, 2026. [8] Y. Tang, J. Bi, S. Xu, L. Song, S. Liang, T. Wang, D. Zhang, J. An, J. Lin, R. Zhu et al., “Video understanding with large language models: A survey,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 36, no. 2, p. 1355–1376, 2026. [9] M. Li, K. Chen, Z. Bi, M. Liu, B. Peng, Q. Niu, J. Liu, J. Wang, S. Zhang, X. Pan et al., “Surveying the mllm landscape: A meta-review of current surveys,” arXiv preprint arXiv:2409.18991, 2024. [10] J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou et al., “Chain-of-thought prompting elicits reasoning in large language models,” Advances in neural information processing systems, vol. 35, p. 24 824–24 837, 2022. [11] Y. Li, Z. Liu, Z. Li, X. Zhang, Z. Xu, X. Chen, H. Shi, S. Jiang, X. Wang, J. Wang et al., “Perception, reason, think, and plan: A survey on large multimodal reasoning models,” arXiv preprint arXiv:2505.04921, 2025. [12] X. Wang, Z. Pan, H. Chen, and W. Zhu, “Divico: Disentangled visual token compression for efficient large vision-language model,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 36, no. 2, p. 1392–1405, 2026. [13] OpenAI,“Thinkingwithimages,”https://openai.com/index/ thinking-with-images/, 2025. [14] T. Yu, K. Fu, S. Wang, Q. Huang, and J. Yu, “Prompting video-language foundation models with domain-specific fine-grained heuristics for video question answering,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 35, no. 2, p. 1615–1630, 2025. [15] H. Zhao, G.-P. Ji, R. Yan, H. Xiong, and Z. Li, “Videoexpert: Augmented llm for temporal-sensitive video understanding,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 36, no. 5, p. 5893– 5908, 2026. [16] Z. Zheng, M. Yang, J. Hong, C. Zhao, G. Xu, L. Yang, C. Shen, and X. Yu, “Deepeyes: Incentivizing" thinking with images" via reinforce- ment learning,” arXiv:2505.14362, 2025. [17] A. Hosseini, X. Yuan, N. Malkin, A. Courville, A. Sordoni, and R. Agarwal, “V-star: Training verifiers for self-taught reasoners,” 2024. [Online]. Available: https://arxiv.org/abs/2402.06457 [18] S. Kim, R. Xiao, S. Alaniz, Y. Xian, and Z. Akata, “Training-free uncertainty guidance for complex visual tasks with mllms,” arXiv preprint arXiv:2510.00705, 2025. [19] J. Lee, Y. Choi, H. Choi, H. Kim, and S. Kim, “A training-free, task- agnostic framework for enhancing mllm performance on high-resolution images,” arXiv preprint arXiv:2507.10202, 2025. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY13 [20] W. Wang, L. Ding, M. Zeng, X. Zhou, L. Shen, Y. Luo, W. Yu, and D. Tao, “Divide, conquer and combine: A training-free framework for high-resolution image perception in multimodal large language models,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 39, no. 8, 2025, p. 7907–7915. [21] H. Shen, K. Zhao, T. Zhao, R. Xu, Z. Zhang, M. Zhu, and J. Yin, “Zoom- eye: Enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, p. 6613–6629. [22] J. Zhang, M. Khayatkhoei, P. Chhikara, and F. Ilievski, “Mllms know where to look: Training-free perception of small visual details with multimodal llms,” arXiv preprint arXiv:2502.17422, 2025. [23] J. Zhu, H. Wang, and M. Shi, “Multi-modal large language model enhanced pseudo 3d perception framework for visual commonsense reasoning,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 34, no. 11, p. 11 682–11 694, 2024. [24] H. Liu, C. Li, Q. Wu, and Y. J. Lee, “Visual instruction tuning,” Advances in neural information processing systems, vol. 36, p. 34 892– 34 916, 2023. [25] D. Zhu, J. Chen, X. Shen, X. Li, and M. Elhoseiny, “Minigpt-4: Enhancing vision-language understanding with advanced large language models,” arXiv preprint arXiv:2304.10592, 2023. [26] W. Dai, J. Li, D. Li, A. Tiong, J. Zhao, W. Wang, B. Li, P. N. Fung, and S. Hoi, “Instructblip: Towards general-purpose vision-language models with instruction tuning,” Advances in neural information processing systems, vol. 36, p. 49 250–49 267, 2023. [27] Z. Li, B. Yang, Q. Liu, Z. Ma, S. Zhang, J. Yang, Y. Sun, Y. Liu, and X. Bai, “Monkey: Image resolution and text label are important things for large multi-modal models,” in proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, p. 26 763–26 773. [28] P. Wu and S. Xie, “V?: Guided visual search as a core mechanism in multimodal llms,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024, p. 13 084–13 094. [29] H. Liu, C. Li, Y. Li, B. Li, Y. Zhang, S. Shen, and Y. J. Lee, “Llava-next: Improved reasoning, ocr, and world knowledge,” January 2024. [Online]. Available: https://llava-vl.github.io/blog/2024-01-30-llava-next/ [30] Z. Chen, J. Wu, W. Wang, W. Su, G. Chen, S. Xing, M. Zhong, Q. Zhang, X. Zhu, L. Lu et al., “Internvl: Scaling up vision foundation models and aligning for generic visual-linguistic tasks,” in Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2024, p. 24 185–24 198. [31] H. Wang, A. Su, W. Ren, F. Lin, and W. Chen, “Pixel reasoner: Incentivizing pixel-space reasoning with curiosity-driven reinforcement learning,” arXiv preprint arXiv:2505.15966, 2025. [32] E. Fadeeva, A. Rubashevskii, A. Shelmanov, S. Petrakov, H. Li, H. Mubarak, E. Tsymbalov, G. Kuzmin, A. Panchenko, T. Baldwin et al., “Fact-checking the output of large language models via token- level uncertainty quantification,” in Findings of the Association for Computational Linguistics: ACL 2024, 2024, p. 9367–9385. [33] T. Zhang, H. Shi, Y. Wang, H. Wang, X. He, Z. Li, H. Chen, L. Han, K. Xu, H. Zhang et al., “Tokur: Token-level uncertainty estimation for large language model reasoning,” arXiv preprint arXiv:2505.11737, 2025. [34] D. N. Yaldiz, Y. F. Bakman, B. Buyukates, C. Tao, A. Ramakrishna, D. Dimitriadis, J. Zhao, and S. Avestimehr, “Do not design, learn: A trainable scoring function for uncertainty estimation in generative llms,” in Findings of the Association for Computational Linguistics: NAACL 2025, 2025, p. 691–713. [35] B. Xu and Y. Lu, “Tecp: Token-entropy conformal prediction for llms,” Mathematics, vol. 13, no. 20, p. 3351, 2025. [36] A. Neubeck and L. Van Gool, “Efficient non-maximum suppression,” in 18th international conference on pattern recognition (ICPR’06), vol. 3. IEEE, 2006, p. 850–855. [37] J. Zhang, M. Khayatkhoei, P. Chhikara, and F. Ilievski, “Towards perceiving small visual details in zero-shot visual question answering with multimodal llms,” arXiv preprint arXiv:2310.16033, 2023. [38] D. Schwenk, A. Khandelwal, C. Clark, K. Marino, and R. Mottaghi, “A-okvqa: A benchmark for visual question answering using world knowledge,” in European conference on computer vision.Springer, 2022, p. 146–162. [39] Y. Li, Y. Du, K. Zhou, J. Wang, W. X. Zhao, and J.-R. Wen, “Evaluating object hallucination in large vision-language models,” arXiv preprint arXiv:2305.10355, 2023. Jie Zhang received the M.S. degree from the Hefei University of Technology, Hefei, China, in 2009, and the Ph.D. degree from the University of Science and Technology of China, Hefei, in 2014.,He is currently an Associate Professor with the Institute of Intelligent Machines, Hefei Institutes of Physical Science, Chinese Academy of Sciences, Hefei. His current research interests include image processing, pattern recognition, and artificial intelligence. Xuanhua He received the B.E. degree in soft- ware engineering from Xiamen University, Xiamen, China, in 2022. Then He received his M.S. degree at the University of Science and Technology of China, Hefei, China. He is currently pursuing his Ph.D de- gree with the Hong Kong University of Science and Technology, hongkong, China. His research interests include image restoration and video generation. Hailong Huang received the B.E. degree in com- puter science and technology from Xiamen Univer- sity, Xiamen, China, in 2022.Then He received his M.S. degree at the Zhejiang university, Hangzhou, China. Yuxiang Shen received the B.E. degree in Electronic Information Engineering from Xiamen University, Xiamen, China, in 2026. He is pursuing the M.S. degree at the University of Science and Technology of China, Hefei, China. Zhenkun Gao received the B.E. degree in soft- ware engineering from Xiamen University, Xiamen, China, in 2024. He is pursuing the M.S. degree at East China Normal University, Shanghai, China. IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY14 Xueheng Li received the B.E. degree in computer science and technology from Xiamen University, Xiamen, China, in 2024. He is pursuing the M.S. degree at the University of Science and Technology of China, Hefei, China. Haoxuan Che received the B.E. degree in North- western Polytechnical University (NWPU), Xi’an, China. Then he received Ph.D. in Computer Science and Engineering from The Hong Kong University of Science and Technology (HKUST), Hongkong, China. Chengjun Xie received the M.S. degree in software engineering from the Hefei University of Technol- ogy, Hefei, China, in 2008, and the Ph.D. degree from the Hefei University of Technology, Anhui, China, in 2014.,He is working with the Institute of Intelligent Machinery, Chinese Academy of Sci- ences, Beijing, as an Associate Researcher. His research interests include image processing, machine learning, and pattern recognition. Man Zhou received the Ph.D. degree from the Uni- versity of Science and Technology of China, Hefei, China, in 2022. He particularly focuses on geog- raphy information system, machine/deep-learning- based satellite image processing, multi-source infor- mation fusion. Dr. Zhou was the recipient of the Baidu Scholarship (top ten globally) in 2022 and the WAIC Yunfan Award in 2023 (top 15 globally).

Instant research discovery