Paper deep dive
IRSTD-Agent: Agentic Infrared Small Target Detection via Zoom-Guided Interaction Learning
Jiawen Xi, Yu Zhang, Tianyi Zhao, Zhu Liu, Maoxun Yuan, Xingxing Wei
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Infrared small-target detection plays an important role in maritime monitoring and aerial surveillance. Although multimodal large language models (MLLMs) offer promising capabilities for visual understanding, existing MLLM-based approaches struggle to precisely localize infrared small targets. In this paper, we propose IRSTD-Agent, an agentic framework for infrared small target detection through dynamic visual search. The framework enables an MLLM to adaptively determine where and at what scale to inspect an image and progressively gather fine-grained visual evidence for precise target localization. Five complementary visual tools (PROPOSAL, ZOOM, DETECT, DROP and REFINE) support object candidate discovery, adaptive observation, target localization, hypothesis rejection, and target extent refinement, together enabling a coordinated search process over original-resolution images. To teach the MLLMs to conduct this search, we introduce Zoom-guided Interaction Learning, which uses annotation-derived interaction trajectories to supervise tool selection and the corresponding arguments. Through extensive experiments on WideIRSTD-Full and IRSTD-1k datasets, we demonstrate that IRSTD-Agent outperforms the evaluated vision-language models and enhances the precise localization capabilities of MLLMs in IRSTD tasks.
Tags
Links
- Source: https://arxiv.org/abs/2610.05342v1
- Canonical: https://arxiv.org/abs/2610.05342v1
Trouble viewing inline? Open PDF directly →
Full Text
60,224 characters extracted from source content.
Expand or collapse full text
IRSTD-Agent: Agentic Infrared Small Target Detection via Zoom-Guided Interaction Learning Jiawen Xi Affiliation: Beihang University Yu Zhang Affiliation: Beihang University Tianyi Zhao Affiliation: Beihang University Zhu Liu Affiliation: Dalian University of Technology*Corresponding authors Maoxun Yuan Affiliation: Beihang University Xingxing Wei Affiliation: Beihang University Abstract Infrared small-target detection plays an important role in maritime monitoring and aerial surveillance. Although multimodal large language models (MLLMs) offer promising capabilities for visual understanding, existing MLLM-based approaches struggle to precisely localize infrared small targets. In this paper, we propose IRSTD-Agent, an agentic framework for infrared small target detection through dynamic visual search. The framework enables an MLLM to adaptively determine where and at what scale to inspect an image and progressively gather fine-grained visual evidence for precise target localization. Five complementary visual tools (PROPOSAL, ZOOM, DETECT, DROP and REFINE) support object candidate discovery, adaptive observation, target localization, hypothesis rejection, and target extent refinement, together enabling a coordinated search process over original-resolution images. To teach the MLLMs to conduct this search, we introduce Zoom-guided Interaction Learning, which uses annotation-derived interaction trajectories to supervise tool selection and the corresponding arguments. Through extensive experiments on WideIRSTD-Full and IRSTD-1k datasets, we demonstrate that IRSTD-Agent outperforms the evaluated vision-language models and enhances the precise localization capabilities of MLLMs in IRSTD tasks. 1 Introduction Infrared small target detection (IRSTD) aims to identify and accurately localize dim thermal targets in cluttered backgrounds, supporting applications such as maritime monitoring and aerial surveillance. Although specialized deep learning-based approaches have made substantial progress through feature enhancement, context aggregation, and background suppression (Li et al., 2023; Liu et al., 2024a; Yuan et al., 2026), they are typically developed as task-specific models, thereby limiting their generalization capabilities in visual understanding. With the development of multimodal large language models, it has become possible to possess capabilities similar to human visual intelligence and to handle complex visual understanding tasks (Li et al., 2025; Yuan et al., 2025). Existing efforts to improve the capability of MLLMs in IRSTD task can be broadly grouped into two categories: infrared-domain fine-tuning and cue-guided prompting. As shown in Figure 1(a), the first strategy fine-tunes MLLMs on infrared image–text pairs and task-specific instructions to improve their understanding of infrared imagery, such as Infrared-LLaVA and IRGPT (Jiang et al., 2024; Cao et al., 2025). However, they require substantial annotation effort and incur considerable computational overhead for model finetuning. More importantly, many state-of-the-art MLLMs (OpenAI et al., 2023; Team et al., 2023) are proprietary and cannot provide access to model parameters, making finetuning inapplicable to some of the strongest existing models and limiting its scalability to rapidly evolving MLLMs. Another strategy introduces auxiliary language cues to guide infrared small target localization, as shown in Figure 1(b). For example, MIRSAM (Zhang et al., 2025b) incorporates textual descriptions of target location and appearance through CLIP-based prompts to support infrared small target segmentation. Similarly, LGNet (Chib and Singh, 2026) leverages MLLMs to generate descriptions of infrared small target locations and incorporates these descriptions into a dedicated detection network to improve performance. Although these strategies offer promising directions for adapting MLLMs to IRSTD without parameter updates, the effectiveness of precisely localizing infrared small targets remains to be established. Figure 1: Motivation for IRSTD-Agent. (a) Full-supervision MLLMs adapt to infrared target localization through task-specific supervision, incurring annotation and training costs. (b) Prompt-guided MLLMs use auxiliary language cues without updating model parameters, but fail to precisely localize small infrared targets. (c) Our IRSTD-Agent employs dynamic visual search to acquire local evidence for target recognition and precise localization. To address these challenges, we propose an IRSTD-Agent, a framework that formulates IRSTD as a dynamic visual search process, as illustrated in Figure 1(c). Instead of relying on a single fixed observation, our approach enables an MLLM to actively acquire local visual evidence and adapt its search through sequential tool interactions. Given an infrared image, the agent identifies candidate regions and adaptively determines where and at what scale to inspect, gathering evidence to localize targets and reject background clutter. To support this process, we introduce five complementary visual tools: PROPOSAL for candidate discovery, ZOOM for adaptive observation, DETECT for target localization, DROP for clutter rejection, and REFINE for target extent refinement. By coordinating these tools, our IRSTD-Agent systematically examines object candidate regions and extracts fine-grained visual evidence from original-resolution images, thereby achieving the recognition and precise localization of infrared small targets through an adaptive search process. To teach the MLLM to guide this search, we introduce Zoom-guided Interaction Learning (ZIL), which uses annotation-derived interaction trajectories to supervise tool selection and the corresponding arguments. Through ZIL, the MLLM learns to select informative views, localize targets, and reject distractors based on the acquired visual evidence. Our contributions are summarized as follows: • We introduce IRSTD-Agent, the first agentic framework for infrared small target detection, enabling the model to actively acquire local observations and use visual tools for adaptive visual search and precise target localization in cluttered infrared scenes. • We design five active visual tools (PROPOSAL, ZOOM, DETECT, DROP and REFINE) for IRSTD-Agent to perform dynamic visual search and introduce zoom-guided interaction learning to teach MLLMs to select appropriate search tools and their parameters based on visual evidence, using annotation-derived interaction trajectories as supervision. • Extensive experiments conducted on the WideIRSTD-Full and IRSTD-1K datasets demonstrate that IRSTD-Agent consistently outperforms existing MLLM-based methods and accurately localizes infrared small targets across various infrared image scenarios. 2 Related Work Infrared Small-Target Detection. Modern IRSTD methods develop specialized architectures and supervision for weak-target representation and localization. DNANet (Li et al., 2023) strengthens hierarchical feature propagation, MSHNet (Liu et al., 2024a) introduces scale- and location-sensitive supervision, and NS-FPN (Yuan et al., 2026) improves multi-scale feature interaction through frequency-guided feature fusion. Other studies explore unified encoder–decoder architectures (Wu et al., 2023), target-adapted convolutions (Yang et al., 2025), and learning from sparse point annotations (Yu et al., 2025). AuxDet (Shi et al., 2026) further studies omni-domain IRSTD by incorporating auxiliary acquisition metadata.Recent work has also introduced vision-language priors into IRSTD. SAIST (Zhang et al., 2025a) integrates textual guidance with SAM for infrared small-target segmentation. Overall, these approaches form specialized solutions tailored to infrared small-target detection and are typically deployed through predefined detection pipelines. Vision-Language Models for Visual Localization. Recent VLMs have extended multimodal understanding to spatial localization and detection. General-purpose MLLMs such as Qwen3.5/3.8 (Qwen Team, 2026a; Qwen Team, 2026b) and InternVL3.5 (Wang et al., 2025) can follow visual localization instructions and produce coordinate-based predictions. Earlier grounding models explicitly align natural-language expressions with image regions. MDETR (Kamath et al., 2021) performs end-to-end text-conditioned object detection through joint vision-language reasoning, while GLIP (Li et al., 2022) unifies object detection and phrase grounding within a grounded language-image pre-training framework.Models such as LocateAnything (Wang et al., 2026), SenseNova-Vision (Han et al., 2026), Grounding DINO (Liu et al., 2024b), and Florence-2 (Xiao et al., 2024) further develop language-conditioned grounding and structured spatial prediction. Visual Chain-of-Thought and Structured Perception Pipelines. Recent work incorporates visual evidence acquisition into multimodal reasoning along two main directions. One line extends chain-of-thought reasoning with intermediate visual interactions, allowing models to identify, revisit, or zoom into relevant regions during reasoning in a more deliberate and iterative manner. Representative approaches include V*, Visual CoT, DeepEyes, and Pixel Reasoner (Wu and Xie, 2024; Shao et al., 2024; Zheng et al., 2026; Su et al., 2025), which study such capabilities across grounding, fine-grained perception, and multimodal reasoning tasks. Another line organizes visual processing through structured workflows. ZoomEye performs hierarchical image exploration through tree search (Shen et al., 2025), ZoomEarth develops an adaptive cropping framework for high-resolution remote-sensing tasks (Liu et al., 2026), and AgentDet coordinates specialized roles through a shared-blackboard detection pipeline (Li et al., 2026). 3 IRSTD-Agent IRSTD-Agent formulates infrared small-target detection as a dynamic visual search process over the original-resolution image. Rather than predicting all targets from a single resized observation, the agent repeatedly selects visual tools to discover candidate regions, acquire higher-resolution evidence, localize visible targets, and reject unsupported hypotheses. Figure 2 summarizes this process: the current view, candidate list, and interaction history inform each tool call, whose execution updates the state for subsequent decisions. Target localizations are accumulated across search branches, while unsupported candidates are rejected. We first describe how the agent selects and executes these tools, and then detail each tool and the annotation-derived interaction learning. Figure 2: Overview of IRSTD-Agent. The policy interacts with an infrared image through visual tools, acquiring local observations and accumulating target localizations in an evidence workspace. Annotation-derived interaction trajectories supervise tool selection and arguments. Candidate discovery and appearance-guided refinement support the search and localization process. 3.1 Agent Formulation Figure 2 summarizes the framework, linking tool-based visual search and the evidence workspace with annotation-derived interaction supervision. Given an infrared image I∈ℝH×WI ^H× W , IRSTD-Agent performs detection through successive visual-tool interactions. The agent is equipped with five tools, =,,,,.U=\ PROPOSAL, ZOOM, DETECT, DROP, REFINE\. (1) These tools serve different stages of visual search: Proposal discovers candidate locations, Zoom acquires a new local observation, Detect records visible targets, Drop rejects unsupported hypotheses, and Refine estimates the spatial extent of an existing localization. At interaction step t, the agent observes the current view I[Ct]I[C_t], where Ct=(Xt,Yt,Wt)C_t=(X_t,Y_t,W_t) specifies the view center (Xt,Yt)(X_t,Y_t) and window size WtW_t in the original image. The interaction state StS_t maintains the information required to continue the search, including the current view, candidate hypotheses, interaction history, accumulated detections, and remaining budget. The model-visible context mtm_t is obtained by serializing the information in StS_t that is exposed to the policy. Conditioned on the current view and context, the search policy predicts a structured tool call at∼πθ(⋅∣I[Ct],mt),a_t _θ(· I[C_t],m_t), (2) where ata_t specifies the selected search tool together with its required arguments. The search policy selects from search=,,,U_search=\ PROPOSAL, ZOOM, DETECT, DROP\. Refine operates on an existing detection and is handled separately. Once a tool is selected, it is executed on the original image and updates the interaction state: St+1=ℰ(I,St,at),S_t+1=E(I,S_t,a_t), (3) where ℰE denotes the tool executor. The updated state is then serialized into the context mt+1m_t+1, together with the latest interaction information, for the next agent decision. 3.2 Active Visual Tools Each visual tool operates directly on native-resolution pixels from the original image I. Given the current view CtC_t, the tools either update the view, introduce candidate hypotheses, produce target localizations, or refine existing detections, as detailed below. Figure 3 illustrates how successive tool calls update the observations and accumulated evidence to produce target localizations and refined extents. Proposal: candidate discovery. Proposal discovers potential target locations within the current search region during the active visual search process. Given the native-resolution crop I[Ct]I[C_t], it returns a ranked set of candidate hypotheses: t=(I[Ct])=((qtk,stk))k=1Kt,st1≥⋯≥stKt.Q_t= PROPOSAL\! (I[C_t] )= ((q_tk,s_tk) )_k=1^K_t, s_t1≥·s≥ s_tK_t. (4) Here qtkq_tk denotes the k-th candidate location mapped to original-image coordinates, and stks_tk is its proposal score. The returned candidates are search hypotheses rather than final detections. Internally, the tool combines complementary spatial and frequency responses. The spatial branch aggregates multi-scale morphological top-hat filtering, difference-of-Gaussians, local standardized contrast, and patch contrast, while the frequency branch uses FFT-based spectral-residual saliency. Their normalized responses are fused as Rt=λRsp,t+(1−λ)Rfft,t.R_t=λ R_sp,t+(1-λ)R_fft,t. (5) Candidate peaks are then ranked and spatially suppressed to form tQ_t. Figure 3: Visual-tool interaction in IRSTD-Agent. The agent iteratively selects tools based on the current view and accumulated interaction context, while tool execution updates the search process and acquired evidence. Detection and refinement produce the final target localizations and extents. Zoom: evidence acquisition. Zoom predicts a new crop-relative center (xt,yt)(x_t,y_t) and window size wtw_t for the next observation during the active visual search process. These parameters are applied to the current view CtC_t to update the search region: Ct+1=(Ct,xt,yt,wt).C_t+1= Zoom\! (C_t;\,x_t,y_t,w_t ). (6) Here (xt,yt)(x_t,y_t) specifies where to move within the current view, while wtw_t controls the scale of the next view. The updated region I[Ct+1]I[C_t+1] is then used in the next agent interaction. Detect: target localization. Detect enables IRSTD-Agent itself to localize the targets visible in the current view. Given the current image region I[Ct]I[C_t], the agent predicts a structured set of normalized target centers: t=(I[Ct])=ptjj=1nt,P_t= DETECT\! (I[C_t] )=\p_tj\_j=1^n_t, (7) where ptjp_tj denotes the normalized center of the j-th target predicted by IRSTD-Agent within the current view. The predicted centers are converted to the original-image coordinate system and recorded as image-level detections. The current search branch is then terminated. Drop: hypothesis rejection. Drop is an argument-free action selected when the current visual evidence does not support the presence of a target: at=⟹terminate the current search branch without producing a detection.a_t= DROP the current search branch without producing a detection. (8) The rejection is recorded in the interaction history, while no image-level detection is added. Refine: appearance-based extent refinement. Given a detected center pip_i, Refine uses its location to recover the target extent from the original image. It first initializes a local box around the center, suppresses the local background, and applies adaptive thresholding to isolate target-supporting responses. The connected region associated with the detected center is then extracted, and its spatial extent is used to obtain the refined bounding box. We denote this image-space transformation as b~i=(I,pi), b_i= Refine(I,p_i), (9) where b~i b_i denotes the refined bounding box. The foreground pixels of the extracted connected region define a binary mask MiM_i, which is mapped back to the original-image coordinates. Taking the union of these masks produces the image-level segmentation map used for visualization and the evaluation in Appendix B.2. Thus, the mask follows the image-derived target region rather than a filled detection box. If no valid region is obtained, the initialized local box is preserved and no mask is added to the segmentation map. 3.3 Zoom-Guided Interaction Learning We construct interaction supervision from standard target annotations to teach IRSTD-Agent how to select visual tools and their arguments during search in a supervised manner. Each training trajectory consists of a sequence of agent interactions, τ=(I[Ct],mt,at⋆)t=0T,τ= \ (I[C_t],m_t,a_t ) \_t=0^T, (10) where I[Ct]I[C_t] is the current view, mtm_t is the accumulated interaction context, and at⋆a_t is the supervised tool call at step t. Successive actions therefore connect what the agent currently observes with how it should continue the search. Annotation-derived interaction trajectories. The lower panel of Figure 2 illustrates supervision through positive and negative search branches: positive trajectories acquire evidence for target localization, whereas negative trajectories terminate with Drop after inspecting background candidates. For positive target hypotheses, we construct target-preserving Zoom actions that progressively expose the annotated target at finer spatial scales, followed by Detect supervision once sufficient target evidence is visible. Proposal-offset examples place the target away from the initial candidate location, preventing the agent from simply copying the proposal position and encouraging localization from the visual evidence itself. Negative hypotheses are supervised with Drop when the inspected region contains background rather than an annotated target. For views containing multiple visible targets, Detect supervision includes all valid targets in current view, allowing a single branch to return multiple detections. Proposal supervision is included in the same interaction to teach candidate discovery during search. The resulting interactions are organized into complete trajectories τ=(I[Ct],mt,at⋆)t=0Tτ=\(I[C_t],m_t,a_t )\_t=0^T. After trajectory construction and sampling, the retained trajectories form the supervised fine-tuning set SFT=τii=1N.D_SFT=\ _i\_i=1^N. (11) Together, these trajectories teach the agent when to propose, where and how far to zoom, when to detect, and when to reject a hypothesis. Policy optimization. The search policy is optimized over SFTD_SFT by ℒSFT=−∑τ∈SFT∑tlogπθ(at⋆∣I[Ct],mt).L_SFT=- _τ _SFT _t _θ (a_t I[C_t],m_t ). (12) This objective encourages the policy to assign higher probability to the correct tool call at⋆a_t given the current view and interaction context, thereby learning both tool selection and its corresponding arguments. Each supervised action is serialized as a structured tool call, while prompt tokens are excluded from the loss during supervised fine-tuning of the policy. 4 Experiments 4.1 Experimental Setup Datasets. We study two complementary settings. WideIRSTD-Full (Li and others, 2025; Shi et al., 2026) evaluates wide-area target search on its complete 2,000-image test set, preserving each image’s native resolution and allowing multiple targets or no targets per image. Its images span a wide range of resolutions and acquisition domains, introducing substantial variation in both scale and scene and making wide-area target search particularly challenging. IRSTD-1k (Zhang et al., 2022) is evaluated using its official split of 800 training images and 201 test images at 512×512512× 512 resolution. Because of overlapping in two datasets, the settings are evaluated independently. Implementation Details. IRSTD-Agent is built on Qwen3.5-27B, with separate checkpoints independently fine-tuned on WideIRSTD-Full and IRSTD-1k using the zoom-guided SFT. IRSTD-1k uses its independently trained checkpoint, with no weights transferred from WideIRSTD-Full. We apply LoRA with rank 16, α=32α=32, and dropout 0.05, while freezing the vision encoder and multimodal projector. Training is performed for one epoch with a learning rate of 1×10−51× 10^-5, cosine decay, 5% warmup, BF16 computation, and gradient checkpointing on NVIDIA A100 80GB GPUs. Evaluation Metrics. MLLM base models are evaluated using their native boxes without additional refinement or NMS. Grounding methods, Visual Reasoning and Active Search methods, and IRSTD-Agent use center-based refinement and box NMS. We report precision (P), recall (R), and F1-scores for target detection. Following prior IRSTD evaluations (Li et al., 2023; Chen et al., 2022), a prediction is matched to an annotation if their box centers are within 3 pixels in the original image. Matches are established one-to-one using the same criterion for all methods. We additionally report the mean box IoU over matched targets to evaluate localization extent. 4.2 Ablation Studies All ablations below cover the official IRSTD-1k datasets. Tool and budget experiments use the main-table Qwen3.5-27B checkpoint. Table 1: Visual tool ablation on IRSTD-1k datasets. Checkmarks indicate enabled tools. P, R, and F1-scores are reported in percentages. The best results are highlighted in bold. PROPOSAL ZOOM DETECT DROP REFINE P R F1 ✓ ✓ ✓ 3.61 77.44 6.90 ✓ ✓ ✓ ✓ 23.89 86.87 37.47 ✓ ✓ ✓ ✓ 63.78 69.36 66.45 ✓ ✓ ✓ ✓ ✓ 81.09 75.08 77.97 Visual tools. Table 1 shows that the tools play complementary roles in the search process. Without DROP, the agent retains many candidate hypotheses, resulting in high recall (77.44%) but very low precision (3.61%).Introducing DROP substantially suppresses false detections, increasing precision to 63.78% and F1-scores to 66.45%. Replacing adaptive ZOOM with a fixed-scale observation strategy yields higher recall (86.87%) but much lower precision (23.89%) and F1-scores (37.47%), indicating that adaptive evidence acquisition is important for reliable target verification. Finally, REFINE further improves the precision–recall balance, raising F1-scores from 66.45% to 77.97%. Table 2: Ablation of zoom-guided interaction learning on IRSTD-1k datasets, P/R/F1-scores are percentages. Training P R F1 w/o SFT 5.80 78.11 10.80 w/o negative branch 6.72 78.79 12.38 Ours 81.09 75.08 77.97 Figure 4: Inference-budget ablation on IRSTD-1k with B=8,16,32,64,96,128B=8,16,32,64,96,128, varying the maximum model-call budget. Zoom-guided Interaction learning. Table 4 compares supervision settings and reports the full system as a reference. Under the same native tool-call execution protocol, the model without SFT (w/o SFT) attains 5.80 precision, 78.11 recall, and 10.80 F1-scores; SFT without the negative branch (w/o negative branch) yields 6.72 precision, 78.79 recall, and 12.38 F1-scores. Positive-only supervision thus provides limited improvement in this setting. The full zoom-guided SFT achieves 81.09 precision and 77.97 F1-scores, delivering the best performance. Action budget. Figure 4 varies both model-call caps over B=8,16,32,64,96,128, with the checkpoint and proposal/detection caps fixed. F1-scores increases from 69.28 to 79.45 and recall from 56.57 to 77.44. However, F1-scores rises by 0.87 points from 64 to 96 despite a substantial increase in inference time,and by 0.61 points from 96 to 128,indicating diminishing returns.These results indicate that B=64B=64 provides a favorable trade-off between detection performance and action budget. 4.3 Comparison with SOTA Methods Baseline Setup. Open-Source MLLMs are evaluated by directly prompting them to detect infrared targets from the full image and return bounding boxes. For models with their own coordinate conventions, the outputs are normalized to a common image-coordinate format before evaluation; for example, Qwen predictions follow its native 0–1000 coordinate protocol and are mapped back to the original image size. VLM-Based Grounding Models support multi-object localization or open-vocabulary detection, so they are directly queried with an infrared-target prompt and evaluated on all returned target boxes in the full image. Visual Reasoning and Active Search Methods are adapted to IRSTD through method-specific search, cropping, or reasoning interfaces. In particular, ZoomEye uses Qwen3.5-27B for hierarchical visual search before producing detections. Other methods receive infrared-target queries and structured detection outputs; Appendix A details their training, inference, and output-processing differences. Table 3: Detection results on WideIRSTD-Full and IRSTD-1k. The shared Method and Venue columns are followed by dataset-specific precision, recall, F1-scores, and IoU; values are percentages. Method Venue WideIRSTD-Full IRSTD-1k P↑ R↑ F1↑F_1 IoU↑ P↑ R↑ F1↑F_1 IoU↑ Open-Source MLLMs InternVL3.5-38B ’25 10.92 4.04 5.89 34.32 6.49 4.04 4.98 41.84 Qwen3.5-9B ’26 7.14 5.16 5.99 36.07 23.85 20.88 22.26 40.84 Qwen3.5-27B ’26 13.77 6.85 9.15 38.68 41.04 34.68 37.59 44.54 Qwen3.8-27B ’26 30.03 15.02 20.02 43.14 63.79 49.83 55.95 48.54 LocateAnything-3B ’26 32.22 17.06 22.31 50.03 79.02 38.05 51.36 54.40 SenseNova-Vision-7B-MoT ’26 3.43 2.48 2.88 42.00 28.07 5.39 9.04 59.67 VLM-Based Grounding Models Florence-2-large-ft CVPR ’24 6.90 3.66 4.78 64.39 11.43 14.81 12.90 68.22 Grounding DINO-B ECCV ’24 9.33 5.50 6.92 66.85 10.93 11.45 11.18 69.85 YOLO-World CVPR ’24 13.48 6.57 8.83 67.99 6.82 75.08 12.51 69.65 Rex-Omni CVPR ’26 16.04 39.06 22.74 67.01 48.37 75.08 58.84 70.34 Visual Reasoning and Active Search Methods ZoomEye EMNLP ’25 19.01 34.74 24.58 65.56 48.40 81.48 60.73 69.07 RSCoVLM-det RS ’26 46.38 22.35 30.17 63.42 77.70 72.73 75.13 69.70 ZoomEarth CVPR ’26 15.97 4.37 6.87 64.72 44.62 29.29 35.37 69.79 DeepEyes ICLR ’26 4.34 4.30 4.32 62.28 26.90 28.62 27.73 69.01 VisionReasoner ICLR ’26 18.78 21.98 20.25 66.63 37.73 54.88 44.72 68.33 IRSTD-Agent Ours 71.63 34.83 46.87 67.07 81.09 75.08 77.97 71.56 WideIRSTD-Full datasets. Direct full-image perception remains challenging on WideIRSTD-Full because tiny targets occupy only a small fraction of large infrared images and their weak responses can be lost during visual compression. As shown in Table 3, Qwen3.5-27B and Qwen3.8-27B achieve 9.15 and 20.02 F1-scores, while LocateAnything-3B reaches 22.31. Grounding models show a similar limitation; Rex-Omni attains 39.06 recall but only 22.74 F1-scores. In contrast, search-based methods benefit from acquiring local evidence. ZoomEye reaches 24.58 F1-scores, RSCoVLM-det reaches 30.17, and IRSTD-Agent achieves 46.87 F1-scores with 71.63 precision. Compared with direct Qwen3.5-27B inference using the same backbone, this is a 37.72 F1-scores gain, supporting the importance of active local observation in wide-area IRSTD. IRSTD-1k datasets. On IRSTD-1k, the fixed 512×512512× 512 resolution substantially reduces the scale mismatch between the original image and the model input. Table 3 shows that direct full-image models therefore become much stronger: Qwen3.8-27B reaches 55.95 F1-scores and Rex-Omni reaches 58.84. Active search remains beneficial, with ZoomEye reaching 60.73 F1-scores and RSCoVLM-det 75.13. IRSTD-Agent further achieves 81.09 precision, 75.08 recall, and 77.97 F1-scores. Compared with the WideIRSTD-Full results, this contrast suggests that active evidence acquisition is especially important when large image scale makes a single global observation insufficient. 4.4 Qualitative analysis. Figure 5 presents representative multi-target detection results on WideIRSTD-Full (top two rows) and IRSTD-1k (bottom two rows). The white regions highlight target-containing areas and are enlarged in the bottom insets, where green boxes denote ground-truth targets and red boxes denote model predictions. Direct full-image MLLMs frequently miss faint targets or produce spatially offset detections. In contrast, IRSTD-Agent recovers more weak targets through local visual search and yields more accurate localization in these challenging regions. Nevertheless, the examples also show that precise localization remains difficult for extremely small and visually ambiguous targets. Figure 5: Selected qualitative multi-target detection examples from WideIRSTD-Full (top two rows) and IRSTD-1k (bottom two rows) datasets. Columns compare ground truth, six full-image VLM baselines, and our method. Green boxes indicate ground truth and red boxes indicate predictions. White boxes mark regions enlarged in the insets along the bottom of each image. 5 Conclusion In this work, we investigate the challenges of infrared small target detection with multimodal large language models and highlight the importance of actively acquiring fine-grained visual evidence for precise localization. We propose IRSTD-Agent, an agentic framework that formulates detection as a dynamic visual search process. Five complementary visual tools enable candidate discovery, adaptive observation, target localization, hypothesis rejection, and target extent refinement. We further introduce Zoom-guided Interaction Learning, which uses annotation-derived interaction trajectories to teach the model to select appropriate tools and their arguments. Extensive experiments on WideIRSTD-Full and IRSTD-1k demonstrate the effectiveness of our framework, demonstrating detection performance superior to other state-of-the-art MLLM-based methods. Limitations and Future Work. Sequential tool interactions introduce additional inference overhead, and extremely small or visually ambiguous targets in IRSTD task remain challenging. Future work could investigate more efficient search policies and adaptive computation allocation to improve the balance between detection accuracy and computational cost. References Cao et al. (2025) Z. Cao, J. Zhang, and R. Zhang Irgpt: understanding real-world infrared image with bi-cross-modal curriculum on large-scale benchmark. In 2025 IEEE/CVF International Conference on Computer Vision (ICCV), p. 166–176. Cited by: §1. Chen et al. (2022) G. Chen, W. Wang, and S. Tan IRSTFormer: a hierarchical vision transformer for infrared small target detection. Remote Sensing 14 (14), p. 3258. External Links: Document Cited by: §4.1. Chib and Singh (2026) P. S. Chib and P. Singh Leveraging language prior for infrared small target detection. Neurocomputing, p. 133512. Cited by: §1. Han et al. (2026) X. Han, J. Li, K. Deng, Z. Chen, X. Shi, S. Wang, B. Li, L. Wang, S. Xie, X. You, J. Quan, Z. Cai, H. Diao, Z. Liu, L. Yang, D. Lin, and Q. Wang Vision as unified multimodal generation. arXiv preprint arXiv:2607.06560. Cited by: §2. Jiang et al. (2024) S. Jiang, Z. Chen, J. Liang, Y. Zhao, M. Liu, and B. Qin Infrared-llava: enhancing understanding of infrared images in multi-modal large language models. In Findings of the Association for Computational Linguistics: EMNLP 2024, p. 8573–8591. Cited by: §1. Kamath et al. (2021) A. Kamath, M. Singh, Y. LeCun, G. Synnaeve, I. Misra, and N. Carion MDETR - modulated detection for end-to-end multi-modal understanding. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 1780–1790. Cited by: §2. Li et al. (2025) B. Li et al. ICPR 2024 competition on resource-limited infrared small target detection challenge: methods and results. In Pattern Recognition. Competitions, Lecture Notes in Computer Science, Vol. 15334, p. 62–77. External Links: Document Cited by: §4.1. Li et al. (2023) B. Li, C. Xiao, L. Wang, Y. Wang, Z. Lin, M. Li, W. An, and Y. Guo Dense nested attention network for infrared small target detection. IEEE Transactions on Image Processing 32, p. 1745–1758. External Links: Document Cited by: §1, §2, §4.1. Li et al. (2025) G. Li, J. Xu, Y. Zhao, and Y. Peng Dyfo: a training-free dynamic focus visual search for enhancing lmms in fine-grained visual understanding. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 9098–9108. Cited by: §1. Li et al. (2026) H. Li, Y. Wang, Z. Yan, L. Wen, and B. Huang AgentDet: a shared-blackboard multi-agent framework for zero-/few-shot object detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 41626–41635. Cited by: §2. Li et al. (2022) L. H. Li, P. Zhang, H. Zhang, J. Yang, C. Li, Y. Zhong, L. Wang, L. Yuan, L. Zhang, J. Hwang, K. Chang, and J. Gao Grounded language-image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10965–10975. Cited by: §2. Liu et al. (2024a) Q. Liu, R. Liu, B. Zheng, H. Wang, and Y. Fu Infrared small target detection with scale and location sensitivity. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 17490–17499. Cited by: §1, §2. Liu et al. (2026) R. Liu, B. Fu, J. Song, K. Li, W. Li, L. Xue, H. Qiao, W. Zhang, D. Meng, and X. Cao ZoomEarth: active perception for ultra-high-resolution geospatial vision-language tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 34877–34888. Cited by: §2. Liu et al. (2024b) S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang Grounding DINO: marrying DINO with grounded pre-training for open-set object detection. In European Conference on Computer Vision, p. 38–55. External Links: Document Cited by: §2. OpenAI et al. (2023) OpenAI, J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774. Cited by: §1. Qwen Team (2026a) Qwen Team Qwen3.5-27B model card. Note: Official model card, Hugging FaceAccessed September 20, 2026 Cited by: §2. Qwen Team (2026b) Qwen Team Qwen3.8-27B model card. Note: Official model card, Hugging FaceAccessed September 20, 2026 Cited by: §2. Shao et al. (2024) H. Shao, S. Qian, H. Xiao, G. Song, Z. Zong, L. Wang, Y. Liu, and H. Li Visual cot: advancing multi-modal language models with a comprehensive dataset and benchmark for chain-of-thought reasoning. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37, p. 8612–8642. External Links: Document Cited by: §2. Shen et al. (2025) H. Shen, K. Zhao, T. Zhao, R. Xu, Z. Zhang, M. Zhu, and J. Yin ZoomEye: enhancing multimodal llms with human-like zooming capabilities through tree-based image exploration. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 6602–6618. External Links: Document Cited by: §2. Shi et al. (2026) Y. Shi, Y. Zhu, R. He, L. Hui, M. Cai, M. Cheng, and Y. Dai AuxDet: auxiliary metadata matters for omni-domain infrared small target detection. IEEE Transactions on Geoscience and Remote Sensing 64, p. 1–16. External Links: Document Cited by: §2, §4.1. Su et al. (2025) A. Su, H. Wang, W. Ren, F. Lin, and W. Chen Pixel reasoner: incentivizing pixel space reasoning via curiosity-driven reinforcement learning. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38, Main Conference, p. 8222–8251. External Links: Document Cited by: §2. Team et al. (2023) G. Team, R. Anil, S. Borgeaud, J. Alayrac, J. Yu, R. Soricut, J. Schalkwyk, A. M. Dai, A. Hauth, K. Millican, et al. Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. Cited by: §1. Wang et al. (2026) S. Wang, S. Liu, Y. Kuang, X. Wei, Y. Liu, Z. Li, Y. Man, G. Chen, A. Tao, G. Liu, J. Kautz, L. Zhang, and Z. Yu LocateAnything: fast and high-quality vision-language grounding with parallel box decoding. arXiv preprint arXiv:2605.27365. Cited by: §2. Wang et al. (2025) W. Wang, Z. Gao, L. Gu, et al. InternVL3.5: advancing open-source multimodal models in versatility, reasoning, and efficiency. arXiv preprint arXiv:2508.18265. Cited by: §2. Wu and Xie (2024) P. Wu and S. Xie V*: guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 13084–13094. External Links: Document Cited by: §2. Wu et al. (2023) X. Wu, D. Hong, and J. Chanussot UIU-net: u-net in u-net for infrared small object detection. IEEE Transactions on Image Processing 32, p. 364–376. External Links: Document Cited by: §2. Xiao et al. (2024) B. Xiao, H. Wu, W. Xu, X. Dai, H. Hu, Y. Lu, M. Zeng, C. Liu, and L. Yuan Florence-2: advancing a unified representation for a variety of vision tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 4818–4829. Cited by: §2. Yang et al. (2025) J. Yang, S. Liu, J. Wu, X. Su, N. Hai, and X. Huang Pinwheel-shaped convolution and scale-based dynamic loss for infrared small target detection. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 9202–9210. External Links: Document Cited by: §2. Yu et al. (2025) C. Yu, J. Zhao, Y. Liu, S. Zhao, Y. Dai, and X. Yue From easy to hard: progressive active learning framework for infrared small target detection with single point supervision. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 2588–2598. Cited by: §2. Yuan et al. (2025) M. Yuan, B. Cui, T. Zhao, J. Wang, S. Fu, X. Yang, and X. Wei Unirgb-ir: a unified framework for visible-infrared semantic tasks via adapter tuning. In Proceedings of the 33rd ACM International Conference on Multimedia, p. 2409–2418. Cited by: §1. Yuan et al. (2026) M. Yuan, D. Meng, Z. Xi, T. Zhao, S. Zhao, Y. Dai, and X. Wei Seeing through the noise: improving infrared small target detection and segmentation from noise suppression perspective. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 27783–27792. Cited by: §1, §2. Zhang et al. (2025a) M. Zhang, X. Li, F. Gao, J. Guo, X. Gao, and J. Zhang SAIST: segment any infrared small target model guided by contrastive language-image pretraining. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 9549–9558. Cited by: §2. Zhang et al. (2025b) M. Zhang, Q. Xu, Y. Wang, X. Li, and H. Yuan MIRSAM: multimodal vision-language segment anything model for infrared small target detection. Visual Intelligence 3, p. 4. External Links: Document Cited by: §1. Zhang et al. (2022) M. Zhang, R. Zhang, Y. Yang, H. Bai, J. Zhang, and J. Guo ISNet: shape matters for infrared small target detection. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 877–886. Cited by: §4.1. Zheng et al. (2026) Z. Zheng, M. Yang, J. Hong, C. Zhao, G. Xu, L. Yang, C. Shen, and XingYu DeepEyes: incentivizing "thinking with images" via reinforcement learning. In International Conference on Learning Representations, C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (Eds.), Vol. 2026, p. 126775–126798. Cited by: §2. Appendix Appendix A Implementation Details A.1 Tools and Coordinate Interface For implementation, we represent the executed view by its boundary coordinates C=[u1,v1,u2,v2]C=[u_1,v_1,u_2,v_2], with width wC=u2−u1>0w_C=u_2-u_1>0 and height hC=v2−v1>0h_C=v_2-v_1>0. This is equivalent to the center-and-size view representation used in Section 3.1 after boundary handling. Crop-normalized and original-image coordinates are related by TC(x,y)=(u1+xwC,v1+yhC),TC−1(X,Y)=(X−u1wC,Y−v1hC).T_C(x,y)=(u_1+xw_C,v_1+yh_C), T_C^-1(X,Y)= ( X-u_1w_C, Y-v_1h_C ). (13) A requested zoom size denotes the side length of the next view in original-image pixels. Each observation is extracted directly from the original image after rounding and boundary handling, rather than recursively from a previously resized crop. In the main joint-policy implementation, Proposal is invoked as a policy-controlled tool, after which Zoom selects a candidate-relative location and crop size for the next observation. Some controlled experiments use an earlier fixed-queue configuration in which proposals are initialized before search. In both cases, tool execution updates the current view and branch-local interaction history while preserving accumulated image-level detections. Detect and Drop terminate the current branch, with Drop producing no detection. Detect outputs normalized target centers (xj,yj)\(x_j,y_j)\ in the current crop, which are mapped to original-image coordinates by TCT_C before refinement. A.2 Proposal Tool Given a grayscale native-resolution view J, we first normalize its intensity range to obtain J¯ J. The spatial branch combines four complementary responses: Rsp=14[(RTH)+(RDoG)+(RLC)+(RPC)],R_ sp= 14 [N(R_ TH)+N(R_ DoG)+N(R_ LC)+N(R_ PC) ], (14) where RTHR_ TH, RDoGR_ DoG, RLCR_ LC, and RPCR_ PC denote multi-scale top-hat, difference-of-Gaussians, local standardized contrast, and patch-contrast responses, respectively, and (⋅)N(·) denotes response normalization. For the frequency branch, we use spectral-residual saliency. Let A=|ℱ(J¯)|A=|F( J)| and ϕ=argℱ(J¯)φ= ( J). The response is Rfft=(G∗|ℱ−1(eL−L¯+iϕ)|2),L=log(A+ϵ),R_ fft=N\! (G* |F^-1 (e^\,L- L+iφ ) |^2 ), L= (A+ε), (15) where L¯ L is a locally smoothed log-amplitude spectrum and G denotes spatial smoothing. The two branches are fused as R=λRsp+(1−λ)Rfft.R=λ R_ sp+(1-λ)R_ fft. (16) Local peaks in R are then ranked and spatially suppressed to produce the proposal set tQ_t. All proposal operations are image-based and contain no learned parameters. A.3 Refinement Tool and Output Assembly Given a detected center pip_i, Refine extracts a local neighborhood from the original image and estimates its background appearance. Let B^i(p) B_i(p) denote the estimated local background and define the residual response as ri(p)=I(p)−B^i(p).r_i(p)=I(p)- B_i(p). (17) An adaptive threshold θi _i, estimated from the local residual statistics, is then used to obtain target-supporting pixels. Starting from the response nearest to pip_i, we retain the connected component Ωi=CCpi(p:ri(p)≥θi), _i=C_p_i (\p:r_i(p)≥ _i\ ), (18) where CCpiC_p_i denotes the connected region associated with the detected target center. The spatial extent of Ωi _i defines the refined bounding box, b~i=BBox(Ωi),Mi(p)=[p∈Ωi], b_i=BBox( _i), M_i(p)=1[p∈ _i], (19) where MiM_i is the corresponding foreground mask. If no valid connected region is obtained, the initialized local box around pip_i is retained. For multiple detections, the image-level segmentation map is formed by M(p)=⋁iMi(p).M(p)= _iM_i(p). (20) Bounding-box predictions are subsequently merged using NMS, while mask assembly preserves the image-derived foreground regions rather than filling the refined boxes. A.4 Zoom-Guided interaction learning Following Equations 10–11, each retained trajectory τi _i is constructed from a source image IiI_i and its training annotations: τi=(Ii[Ci,t],mi,t,ai,t⋆)t=0Ti,SFT=τii=1N. _i=\(I_i[C_i,t],m_i,t,a_i,t )\_t=0^T_i, _SFT=\ _i\_i=1^N. (21) Here i indexes trajectories rather than distinct images, since multiple trajectories may be constructed from the same source image. Connected components of the training mask define target boxes gjg_j and centers pj⋆p_j . For a view C, we distinguish fully visible targets from those that only intersect the view: V(C)=j:gj⊆C,U(C)=j:area(gj∩C)>0.V(C)=\j:g_j C\, U(C)=\j:area(g_j∩ C)>0\. (22) Positive trajectories consist of target-preserving Zoom transitions that progressively acquire more informative local observations. Offset-recovery examples deliberately displace the initial candidate from the target center, requiring localization from visual evidence rather than direct proposal copying. At a terminal positive view, Detect supervises the normalized centers of all fully visible targets: at⋆=(TCt−1(pj⋆):j∈V(Ct)).a_t = DETECT (\T_C_t^-1(p_j ):j∈ V(C_t)\ ). (23) Negative trajectories are constructed from candidate branches containing no annotated target. A Drop label is assigned only when U(Ct)=∅U(C_t)= , ensuring that partially visible targets are not treated as negatives. Multi-target views retain all valid targets in the corresponding Detect supervision. Proposal and Zoom actions are included along the trajectory according to the executed search process. Each structured action is serialized into a token sequence yi,t⋆y_i,t . The token-level SFT objective is ℒSFT=−∑i=1N∑t=0Ti∑ℓ=1|yi,t⋆|logπθ(yi,t,ℓ⋆∣Ii[Ci,t],mi,t,yi,t,<ℓ⋆).L_SFT=- _i=1^N _t=0^T_i _ =1^|y_i,t | _θ (y_i,t, I_i[C_i,t],m_i,t,y_i,t,< ). (24) Prompt tokens are masked from the loss. Separate trajectory sets and checkpoints are constructed for each dataset. A.5 Baseline Adaptation We follow the method groups in Table 3 and adapt each baseline to infrared small-target detection while preserving its original inference paradigm whenever possible. MLLM and grounding baselines. InternVL3.5, Qwen3.5/3.8, LocateAnything, and SenseNova-Vision use released checkpoints without infrared-specific fine-tuning. They directly process the full image and are prompted to return all visible infrared targets as bounding boxes, with model-specific coordinate formats mapped back to the original image. Florence-2, Grounding DINO, YOLO-World, and Rex-Omni likewise use released weights and perform full-image multi-target localization with an infrared-target query. Their native task interfaces and decoding procedures are retained wherever applicable. Visual reasoning and active search baselines. ZoomEye, ZoomEarth, DeepEyes, and VisionReasoner retain their original search, cropping, or reasoning procedures, while their task interfaces are adapted to produce infrared-target detections. ZoomEye uses Qwen3.5-27B as its underlying multimodal model and performs hierarchical visual search before localization. Predictions produced in local views are mapped back to the original image coordinates for evaluation. RSCoVLM-det is the only baseline in this group that receives dataset-specific training. Separate LoRA adapters are trained for WideIRSTD-Full and IRSTD-1k using annotations converted from the corresponding training masks. Specialized IRSTD detectors. The specialized detectors reported in Appendix B.1, including RDIAN, LW-IRSTNet, DNANet, MSHNet, and NSFPN, are trained separately on each dataset using their original supervision. Their full-image and sliding-window evaluation protocols are reported separately in the corresponding appendix section. Appendix B More Analysis B.1 Comparison with Specialized IRSTD Methods Table 4 compares specialized IRSTD detectors with IRSTD-Agent on the full WideIRSTD-Full test set and its large-image subset. The full set contains 2,000 images and 5,328 targets. The Large subset contains 44 images with original area A=W×H>10242A=W× H>1024^2, comprising 99 targets. For these 44 images, we report both the original full-image resize protocol and a native-resolution 256×256256× 256 sliding-window alternative for the specialized detectors. Table 4: Specialized IRSTD methods and IRSTD-Agent on WideIRSTD-Full. Full and Large (original) use full-image resize, while Large (sliding) uses native-resolution 256×256256× 256 sliding windows for the specialized detectors. The two Large settings contain the same 44 images and 99 targets. IRSTD-Agent is unchanged in both columns. Values are percentages; IoU is averaged over matched targets only. Method Full (2,000) Large: original (44) Large: sliding 256256 (44) P R F1F_1 IoU P R F1F_1 IoU P R F1F_1 IoU RDIAN 72.66 46.43 56.66 60.25 33.33 1.01 1.96 14.06 48.60 52.53 50.49 69.54 U-Net 78.35 49.32 60.54 60.27 20.00 1.01 1.92 14.06 54.93 39.39 45.88 75.39 ResUNet 77.44 50.06 60.81 59.80 20.00 1.01 1.92 8.33 58.67 44.44 50.57 65.99 AttU-Net 78.66 48.16 59.74 59.40 12.50 1.01 1.87 25.00 55.56 45.45 50.00 66.89 MLCL-Net 78.69 41.85 54.64 59.39 20.00 1.01 1.92 25.00 44.92 53.54 48.85 64.13 IRSTD-Agent 71.63 34.83 46.87 67.07 78.26 54.55 64.29 75.34 78.26 54.55 64.29 75.34 For Full and Large (original), the specialized detectors follow the same full-image 256×256256× 256 resize protocol used in their standard evaluation, with predictions mapped back to the original image. For Large (sliding), the same trained checkpoints are evaluated without retraining using native-resolution 256×256256× 256 windows with stride 128. Predictions from overlapping windows are merged on the original image. IRSTD-Agent retains its original active local-observation pipeline, and its Large result is repeated for reference. Specialized IRSTD detectors are highly task-specific and perform strongly under their conventional input setting. On the full test set, their F1 scores range from 54.64 to 60.81, exceeding the 46.87 F1 of IRSTD-Agent. However, this advantage does not transfer directly to large images. Under full-image resize, all specialized detectors recover only 1.01% of the targets in the Large subset, with F1 around 1.9, showing that aggressive resizing can largely remove the already weak small-target evidence. Native-resolution sliding-window inference substantially recovers their performance, increasing F1 to 45.88–50.57 and recall to 39.39–53.54. Nevertheless, a considerable fraction of targets remains missed, and all specialized detectors remain below IRSTD-Agent on this subset, which reaches 54.55 recall and 64.29 F1. These results highlight the strong dependence of specialized IRSTD models on the observation scale: preserving local resolution is essential for large-image detection, while a fixed sliding-window strategy still provides less flexible evidence acquisition than the active search used by IRSTD-Agent. B.2 Segmentation Results Table B.2 reports segmentation performance on WideIRSTD-Full (2,000 images) and IRSTD-1k (201 images). For the MLLM, grounding, visual-search, and IRSTD-Agent pipelines, detected target centers are converted to foreground masks using the same center-to-mask refinement procedure. Therefore, the full-image MLLM results in this table include mask generation beyond their native-box detection outputs reported in the main text. We report image-averaged mask IoU (nIoU), foreground Dice, component-level detection probability (PdP_d), and false-alarm rate (FaF_a). nIoU is averaged over images with a nonempty mask union, while Dice is computed over foreground pixels across the test set. PdP_d uses one-to-one component-centroid matching within three pixels. FaF_a measures foreground pixels belonging to unmatched predicted components relative to the total number of image pixels and is reported as 106Fa10^6F_a; lower values indicate fewer false alarms. Table 5: Segmentation performance on WideIRSTD-Full and IRSTD-1k. nIoU, Dice, and PdP_d are reported as percentages, while FaF_a is reported as 106Fa10^6F_a. All methods use the same center-to-mask refinement for segmentation evaluation. Method WideIRSTD-Full (2,000) IRSTD-1k (201) nIoU↑ Dice↑ Pd↑P_d Fa↓F_a nIoU↑ Dice↑ Pd↑P_d Fa↓F_a Open-Source MLLMs InternVL3.5-38B 12.50 20.95 13.59 8.91 20.49 32.88 35.35 7.38 Qwen3.5-9B 12.30 21.35 14.88 16.86 37.83 42.35 52.53 17.37 Qwen3.5-27B 19.06 31.17 22.07 11.31 42.71 45.47 58.92 9.62 Qwen3.8-27B 19.07 32.90 23.25 12.69 48.97 57.22 68.35 9.57 LocateAnything-3B 15.78 23.42 17.83 11.63 27.48 35.94 37.37 3.28 SenseNova-Vision-7B-MoT 2.45 4.96 2.85 12.72 4.62 6.75 5.39 5.67 VLM-Based Grounding Models Florence-2-large-ft 4.09 9.00 3.55 10.37 9.39 18.11 13.80 48.89 Grounding DINO-B 7.35 11.04 5.29 14.01 8.48 17.48 10.44 30.90 YOLO-World 4.92 13.24 6.55 19.18 25.62 30.78 73.06 379.23 Rex-Omni 28.58 34.32 38.95 67.64 49.09 54.52 74.07 53.52 Visual Reasoning and Active Search ZoomEye 25.14 34.47 34.98 48.98 54.36 57.81 80.13 41.54 RSCoVLM-det 26.34 29.34 22.22 12.98 51.93 57.46 70.71 12.41 ZoomEarth 4.03 8.78 4.43 5.74 20.38 33.71 28.96 16.06 DeepEyes 4.07 6.91 4.34 19.11 17.53 28.88 27.95 28.37 VisionReasoner 20.07 27.40 22.03 30.24 35.14 43.39 54.21 58.44 IRSTD-Agent 31.71 37.67 34.68 13.35 52.19 57.90 76.09 17.61 On WideIRSTD-Full, IRSTD-Agent achieves the highest nIoU (31.71) and Dice (37.67), indicating stronger foreground overlap under the large-image setting. Rex-Omni attains a higher PdP_d of 38.95, but with a substantially larger false-alarm rate (67.64 versus 13.35), suggesting that its higher target recovery is accompanied by more spurious foreground responses. On IRSTD-1k, IRSTD-Agent reaches 52.19 nIoU, 57.90 Dice, 76.09 PdP_d, and 17.61 FaF_a. ZoomEye achieves slightly higher nIoU and PdP_d (54.36 and 80.13), but also produces considerably more false alarms (Fa=41.54F_a=41.54). Overall, the results show that segmentation quality depends not only on target recovery, but also on the spatial accuracy and false foreground introduced by the complete detection-and-refinement pipeline. Appendix C Prompt Templates C.1 IRSTD-Agent The system and observation prompts below use the center-output interface. Line breaks are adjusted for layout. The proposal instruction is appended to the original system prompt in compact form. System prompt. You are an infrared-small-target gaze agent. Inspect only the supplied crop. Coordinates are normalized to the CURRENT crop. Use ZOOM when additional local evidence is needed. Use DETECT when the visible targets can be reliably localized, and DROP when the current hypothesis is unsupported. Emit exactly one JSON action and no prose: ZOOM type,x,y,size; DETECT type,targets:[x,y]; DROP type; or PROPOSAL type. PROPOSAL runs RF/FFT proposals on the current crop when proposal_allowed=true and additional candidate hypotheses are needed. Proposal points guide search; they are not detections. Observation prompt. Each user message contains the current crop and the following text. Placeholders are filled with runtime values; history contains parsed actions from the current branch.image_size=image_size; crop_original=crop_original; crop_size=crop_size; step=step; history=history. Choose the next action. Here image_size is [W,H][W,H], crop_original is [u1,v1,u2,v2][u_1,v_1,u_2,v_2], crop_size is [u2−u1,v2−v1][u_2-u_1,v_2-v_1], and step is the zero-based branch depth. When an anchor is active, the observation appends:proposal_anchor_original=anchor; Follow the current proposal anchor when choosing which local hypothesis to inspect; do not switch to a distant bright target belonging to another proposal. The anchor uses original-image coordinates. The observation also includes proposal_allowed and, after a local Proposal call, local_proposals_original, a list of original-image candidate centers. Detection action instruction. When Detect is selected, IRSTD-Agent localizes all visible targets in the current crop. The detection-specific instruction is summarized below:Detect every fully visible infrared small target in this crop. Return all target centers normalized to the CURRENT crop in [0,1], not box corners or original-image coordinates. Output one JSON action: "type":"DETECT","targets":["x":...,"y":...] If the crop is background, output "type":"DROP". IRSTD-Agent predicts the target centers, while Proposal only supplies candidate locations. The detection instruction belongs to the same policy prompt. The executor maps the predicted centers to the original image before subsequent localization refinement. Structured responses. Illustrative action formats are:"type":"PROPOSAL" "type":"ZOOM","x":0.5,"y":0.5,"size":64 "type":"DETECT","targets":["x":0.3,"y":0.4, "x":0.7,"y":0.6] "type":"DROP" Zoom coordinates are crop-normalized; size is the requested side length in original-image pixels. Detect and Drop end the current branch, after which the executor advances to the next candidate. C.2 Full-Image MLLM and Grounding Prompts Full-image MLLM prompt. The task instruction requests all infrared targets from the full image. A compact rendering is:Detect all compact infrared small targets, including dim targets. Exclude broad bright regions, edges, and background texture. Return one tight bounding box per target in the required coordinate format, and an empty list if no target is visible. Output only the requested structured result. Model-specific output conventions are retained rather than imposing one literal prompt on all models. For example, the Qwen native-coordinate interface requests JSON boxes with each axis independently normalized to [0,1000][0,1000] and supplies the image dimensions; parsed boxes are mapped back to original-image coordinates. Grounding prompt. These methods receive an infrared-target text query through their native grounding interfaces rather than the agent’s action prompt. The category query used by YOLO-World and Rex-Omni is:infrared small target Florence-2 uses its <CAPTION_TO_PHRASE_GROUNDING> task with an infrared-target query, while Grounding DINO receives an infrared-target caption. Their native task formatting and output parsers are preserved. Appendix D Large Language Model Usage AI tools were used for language polishing and assistance with LaTeX formatting, including figure and table layout. Portions of the code were implemented and modified with assistance from GPT-5 and GPT-6 Codex. The authors take full responsibility for the final manuscript, code, and all reported results.