Paper deep dive
GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed
Qize Yu, Lianrui Fan, Bowen Ping, Xini Ding, Zetian Song, Junbo Niu, Kaixuan Wang, Tianxing Chen, Yue Chen, Minghua He, Yuran Wang, Jie Huang, Haojun Zhang, Min Chen, Hao Li, Wenxuan Song, Ruihai Wu, Xianming Liu, Shilong Liu, Shuchang Zhou, Ping Luo, Shiyu Huang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising. We introduce GroundAnything, a 4B-parameter grounding foundation model that reconciles fast parallel decoding with precise localization through blockwise denoising. Training combines grounding pretraining from public datasets and dedicated data engines, direct AR-to-diffusion conversion with joint AR and diffusion objectives, supervised fine-tuning, and GRPO-based reinforcement post-training. Across 30 grounding benchmarks, our autoregressive variant, GroundAnything-VLM, establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything also surpasses the prior state of the art at this scale, averaging 61.75% versus 53.32% for the fast MTP-based LocateAnything model. We further explore decoding strategies, showing that an optional self-speculative mode achieves a $4.51\times$ speedup over the AR counterpart with a 0.74 percentage-point drop in COCO F1mIoU. Infrastructure experiments show that progressive inference optimizations translate parallel decoding into practical speedups. These support efficient visual grounding in latency-sensitive real-world systems.
Tags
Links
- Source: https://arxiv.org/abs/2609.39600v1
- Canonical: https://arxiv.org/abs/2609.39600v1
Trouble viewing inline? Open PDF directly →
Full Text
205,548 characters extracted from source content.
Expand or collapse full text
GroundAnything: Reconciling Parallel Decoding with Precise Visual Grounding at Flash Speed Qize Yu 1,2,∗,‡ , Lianrui Fan 1,∗ , Bowen Ping 2,∗ , Xini Ding 1,∗ , Zetian Song 1,2,∗ , Junbo Niu 2 , Kaixuan Wang 3 , Tianxing Chen 3 , Yue Chen 2 , Minghua He 2 , Yuran Wang 2,6 , Jie Huang 2 , Haojun Zhang 1 , Min Chen 1 , Hao Li 1 , Wenxuan Song 7 , Ruihai Wu 4 , Xianming Liu 1 , Shilong Liu 5 , Shuchang Zhou 1 , Ping Luo 3,† , Shiyu Huang 1,† 1 XPeng Inc., 2 Peking University, 3 The University of Hong Kong, 4 University of California, Berkeley, 5 Princeton University, 6 National University of Singapore, 7 HKUST (GZ) ∗ Equal contribution, ‡ Project lead, † Corresponding authors OCR GUI Layout Grounding Dense DenseGrounding Grounding Referring Spatial Referring Dense Point OCR <|object_ref_start|>flower<|object_ref_end|><|box_start|><12><220><93><352>, <19><879><142><999>,<52><816><99><869>,<54><330><145><472>,[MASK]<616><259>[MASK], <151><481><234><602>,[MASK][MASK]<206><401>,<153>[MASK]<218><149>,<221>[MASK][MASK]<606>, <229>[MASK][MASK][MASK][MASK][MASK][MASK][MASK][MASK][MASK][MASK][MASK][MASK][MASK][MASK] Outperforms Baselines Across Benchmarks Throughput and accuracy Grounding as Visual Evidence Extraction 1234 ⋮ M−3M−2M−1M 2D visual evidence Prompt “flower” AR: N decoding steps 1 2 N 1 12 1234 ⋯ N−1 N ⋯ DLM: K ≪N decoding steps Bidirectional attention K 1 0 14 ⋯ N 1234 ⋯ N−1 N 1234 ⋯ N−1 N Parallel decoding + precise grounding 푇 푑푒푐표푑푒푑 푇 푚푎푠푘푒푑 푇 푡푎푟푔푒푡 푇 푣푖푠푖표푛 Generated Text TP2 enforce-eager Speedup at only −0.74 p Precise GroundingParallel Decoding Figure 1 GroundAnything: broad, precise grounding with fast parallel decoding. Top: diverse tasks challenge localization in dense scenes and complex layouts. Bottom: given a 2D image and prompt, bidirectional diffusion extracts visual evidence into 1D structured grounding tokens in parallel. Self-speculative decoding achieves a 4.51× speedup over GroundAnything-VLM with only a 0.74 p COCO F1mIoU drop. Project Page: https://groundingpi.github.io/groundanything/ Code: https://github.com/groundingpi/GroundAnything Model: https://huggingface.co/GroundingPI/GroundAnything https://huggingface.co/GroundingPI/GroundAnything-VLM 1 arXiv:2609.39600v1 [cs.CV] 30 Sep 2026 Abstract Autoregressive (AR) grounding models serialize spatial predictions, introducing sequential latency and imposing a causal order on output tokens. We view grounding as visual evidence extraction: objects, locations, and spatial relations are jointly constrained by the image and query, yet their dependencies do not imply an intrinsic left-to-right generation order. This distinction makes bidirectional diffusion a natural fit, allowing spatial hypotheses to emerge in parallel and be jointly refined through iterative denoising. We introduce GroundAnything, a 4B-parameter grounding foundation model that reconciles fast parallel decoding with precise localization through blockwise denoising. Training combines grounding pretraining from public datasets and dedicated data engines, direct AR-to-diffusion conversion with joint AR and diffusion objectives, supervised fine-tuning, and GRPO-based reinforcement post-training. Across 30 grounding benchmarks, our autoregressive variant, GroundAnything-VLM, establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything also surpasses the prior state of the art at this scale, averaging 61.75% versus 53.32% for the fast MTP-based LocateAnything model. We further explore decoding strategies, showing that an optional self-speculative mode achieves a 4.51×speedup over the AR counterpart with a 0.74 percentage-point drop in COCO F1mIoU. Infrastructure experiments show that progressive inference optimizations translate parallel decoding into practical speedups. These support efficient visual grounding in latency-sensitive real-world systems. Contents Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2 1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4 2 Related Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.1 Visual Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.2 Diffusion Vision-Language Models . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 2.3 Parallel Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3 Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 5 3.1 Problem Formulation: Grounding as Structured Visual Evidence Extraction . . . . . . . . . . . . . . 5 3.2 Model Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.3 GroundAnything Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6 3.4 Training Design . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.4.1 Base VLM Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.4.2 AR-to-Diffusion Conversion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.4.3 Reinforcement Post-Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7 3.5 Efficient Parallel Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8 4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.1 Implementation Details and Evaluation Setup. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.2 Main Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9 4.3 Speed Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.3.1 Decoding Strategy Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 4.3.2 DLM vs. MTP . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 4.3.3 Infrastructure Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.4 Ablation Studies . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 4.5 Discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2 Appendix Contents A Model and Training Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 A.1 Input/Output Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 A.2 Architecture, Tokenizer, and Visual Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 A.3 Expert-Driven Data Engine . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 A.4 Training Pipeline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 A.5 Base VLM Training (Pretrain 1) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 A.6 Coordinate Alignment (Pretrain 2 / SFT) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 A.7 AR-to-Diffusion Conversion (GroundAnything Only) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 A.8 Reinforcement Learning (RL): Causal GRPO . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 A.9 GroundAnything-VLM Reinforcement Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 A.10 Task Rewards . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 A.10.1 Box Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 A.10.2 OCR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 A.10.3 Pointing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 B Parallel Inference . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 B.1 Direct Diffusion Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 B.2 Optional Self-Speculative Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 B.3 Execution Optimizations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 C DLM–MTP Comparison: Experimental Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 C.1 Controlled Model Comparisons . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 C.2 Draft Acceptance and Token Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 C.3 Refinement and Fixed-Deadline Decoding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 C.4 Structural and Geometric Diagnostics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 D Additional Experimental Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 D.1 Speed Reporting and Decoding Sensitivity . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 D.2 Infrastructure Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 D.3 Architecture and Training Ablations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 D.3.1 Multi-level Visual Injection and Coordinate Readout . . . . . . . . . . . . . . . . . . . . . . . 35 D.4 Output Token Efficiency and Generation Cost . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 E Comprehensive Grounding Benchmark Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36 E.1 Common and Long-tailed Object Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 E.2 Dense and Tiny Object Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37 E.3 Referring Object Detection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 E.4 Object Pointing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 E.5 Robot and Spatial Pointing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 E.6 OCR . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 E.7 GUI Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45 E.8 Layout Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 48 E.9 Visual Prompting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49 E.10 Potential Application Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50 E.11 Qualitative Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53 E.11.1 General Object Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 E.11.2 Dense Object Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 E.11.3 Referring Grounding and Complex Visual Configurations . . . . . . . . . . . . . . . . . . . . . 56 E.11.4 Referring Point-in-Mask Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57 E.11.5 Dense Point Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 E.11.6 Dense Point-in-Mask Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 E.11.7 GUI Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58 E.11.8 OCR and Artistic Text Understanding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 E.11.9 Visual-Prompt Grounding . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61 3 1 Introduction Visual grounding turns visual inputs into structured spatial predictions across object detection, referring expression comprehension, OCR, GUI interaction, and embodied perception. Next-token prediction has become a mainstream paradigm for generative grounding, with models such as Pix2Seq, Florence-2, and Rex-Omni expressing visual predictions as structured token sequences (Jiang et al., 2026; Chen et al., 2022; Xiao et al., 2024). However, autoregressive (AR) decoding serializes labels and coordinates, introducing sequential latency and restricting output interactions to the generated prefix(Chen et al., 2022; Cheng et al., 2024). These limitations are particularly costly in dense scenes and complex layouts. To reduce AR decoding latency, LocateAnything uses multi-token prediction (MTP) for parallel box decoding (Wang et al., 2026). Diffusion VLMs support parallel generation and bidirectional refinement, with advances in multimodal understanding (You et al., 2026b; Wu et al., 2026a), GUI grounding (Kumbhar et al., 2026), and document OCR (Dong et al., 2026). Yet these grounding applications remain domain-specific, leaving diffusion’s broader potential for unified structured prediction across diverse visual tasks largely untapped. To unlock this potential and reconcile fast parallel decoding with broad and precise visual grounding, we start from a familiar observation: we may notice objects or read separate signs in different orders, yet reach the same judgment about what is present and where. We therefore view grounding as structured visual evidence extraction: spatial variables are jointly constrained by the image and query, without an intrinsic left-to-right generation order. Bidirectional masked diffusion naturally supports this process, allowing hypotheses to emerge in parallel and be refined using shared visual evidence and partially recovered structure (Figure 1). We introduce GroundAnything, a 4B-parameter grounding foundation model combining broad perceptual capabilities with blockwise parallel decoding. A shared vocabulary with quantized coordinates unifies diverse grounding tasks. Its decoder combines bidirectional attention within blocks with causal dependencies across blocks, enabling parallel prediction and KV-cache reuse (Arriola et al., 2025). Grounding pretraining draws on public datasets and dedicated data engines. We build GroundAnything-VLM and GroundAnything from a shared pretrained checkpoint, applying AR-to-diffusion conversion with joint AR and diffusion objectives to the latter. Both variants then undergo supervised fine-tuning and GRPO-based reinforcement post-training. Across 30 grounding benchmarks, GroundAnything-VLM establishes a new overall state of the art among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). With entropy-guided decoding, GroundAnything is, to our knowledge, the first diffusion language model to surpass the prior overall AR state of the art at this scale, averaging 61.75% at over 2×the speed of GroundAnything-VLM. We use this mode throughout the main benchmarks to evaluate pure diffusion decoding without AR verification. Additional ablations examine key model and training design choices. We further conduct a systematic speed analysis of decoding strategies and their trade-offs, including comparisons with MTP-based generation (Wang et al., 2026). An optional self-speculative mode combines diffusion drafting with AR verification and improves both speed and accuracy over entropy-guided decoding on COCO, achieving a 4.51×speedup over GroundAnything-VLM with only a 0.74 p F1mIoU drop. Systematic infrastructure experiments with SGLang, CUDA Graph, and FP8 demonstrate further practical speedups through progressive inference optimizations, supporting efficient visual grounding in real-world applications. In summary, our major contributions are as follows: • Unlocking diffusion for broad visual grounding. We introduce GroundAnything, a 4B foundation model that unifies diverse grounding tasks with precise localization and parallel decoding, trained through grounding pretraining, AR-to-diffusion conversion with joint objectives, supervised fine-tuning, and GRPO-based reinforcement learning. •State-of-the-art grounding at 4B scale. Across 30 benchmarks, GroundAnything surpasses the prior overall state of the art among similarly sized AR models and outperforms the larger Qwen3.7-Max. GroundAnything-VLM establishes a new overall state of the art at this scale and remains competitive with GPT-6 Astra. 4 •Systematic acceleration studies. We analyze decoding strategies in detail, compare with MTP-based generation, and evaluate progressive infrastructure optimizations for practical deployment. 2 Related Work 2.1 Visual Grounding Object detectors such as YOLO establish efficient localization within predefined category vocabularies (Redmon et al., 2016). Grounded vision–language pretraining extends detection to language-conditioned, open-set localization (Li et al., 2022; Liu et al., 2024). Generative approaches formulate detection as autoregressive next-token prediction (Chen et al., 2022) and integrate spatial grounding into multimodal language modeling (Peng et al., 2023). Structured generation unifies diverse perception tasks (Xiao et al., 2024), while quantized coordinates, large-scale grounding data, and reinforcement post-training strengthen localization (Jiang et al., 2026). However, AR generation serializes spatial predictions, increasing decoding costs for dense outputs. MTP-based parallel box decoding reduces this overhead by treating boxes and points as atomic generation units (Wang et al., 2026). This progression motivates exploring alternative parallel generation mechanisms for broad and precise grounding. 2.2 Diffusion Vision-Language Models Masked diffusion offers a complementary route to parallel generation through bidirectional conditioning and iterative token recovery. Diffusion VLMs demonstrate its potential for visual instruction following and multimodal understanding (You et al., 2026b; Li et al., 2026b). Direct AR-to-diffusion conversion adapts pretrained VLMs for efficient blockwise generation (Wu et al., 2026a). Beyond general understanding, specialized applications include GUI grounding with geometry-aware masking (Kumbhar et al., 2026) and structured document OCR with blockwise denoising (Dong et al., 2026). These studies demonstrate the promise of diffusion in both general multimodal understanding and specialized visual tasks. Nevertheless, a unified diffusion grounding foundation that preserves precise localization across heterogeneous tasks remains underexplored. GroundAnything addresses this gap through grounding-focused training and a shared structured output interface, combining broad perceptual capabilities with efficient parallel decoding. 2.3 Parallel Decoding Prior work explores blockwise generation (Arriola et al., 2025), adaptive parallel sampling (Wu et al., 2026b; Ben-Hamu et al., 2026), and diffusion-based speculative decoding (Liu et al., 2026; Wu et al., 2026a). Further work emphasizes practical acceleration through optimized inference infrastructure (Liu et al., 2025). Motivated by these advances, we examine decoding choices for GroundAnything in detail, studying their speed–accuracy trade-offs and extending the analysis to comparisons with MTP-based generation and progressive infrastructure optimizations. 3 Method 3.1 Problem Formulation: Grounding as Structured Visual Evidence Extraction Given an imageIand a task queryQ, grounding extracts semantic identifiers and their spatial evidence, represented by boxes, points, or ordered point sequences. We serialize this evidence asY= (y 1 ,...,y L ) using a shared vocabulary of text, structural markers, and quantized coordinates (Jiang et al., 2026); input/output conventions are detailed in Section A.1. These variables are jointly constrained by the image and query, but need not be recovered in a fixed left-to-right order (Figure 1). We therefore use blockwise masked diffusion (Arriola et al., 2025; Wu et al., 2026a; You et al., 2026a; Yu et al., 2026), modeling p θ (Y | I,Q) = K Y k=1 p θ Y (k) | I,Q,Y (<k) (1) 5 overKconsecutive response blocks. Bidirectional attention supports parallel token recovery within each block, while completed blocks form a causal prefix. Semantic ordering, including trajectory time order, is preserved. 3.2 Model Architecture GroundAnything is a diffusion grounding model obtained by directly converting GroundAnything-VLM, an autoregressive grounding model trained through multimodal and spatial pretraining, supervised fine-tuning, and reinforcement post-training. It combines a MoonViT-V2 (Kimi K3) visual encoder (Team et al., 2026), a projector with 2×2 spatial aggregation and a two-layer MLP, and a Qwen3-4B language decoder (Yang et al., 2025) (Figure 2). Its variable-length output interface uses a shared vocabulary for semantic labels, protocol markers, and 1,000 quantized coordinate tokens (<0>–<999>); absent queried categories receive a None payload. The conversion introduces a mask token and jointly trains causal prediction and blockwise denoising withB= 32; both modes share the decoder and vocabulary head, enabling parallel grounding and optional self-speculative decoding (Section 3.4.2). MoonViT-V2 (Kimi K3) MLP Query Locate all the instances that match the following categories: Minions</c>Human</c>Car. Clean multimodal vp y 1 y 2 y 3 y 4 y 5 y 6 Noisy y 1 M y 4 M y 6 Random mask Qwen3-4B Joint AR / diffusion AR lossMasked-token loss Parallel denoising M y 1 M y 3 y 1 y 2 y 3 Output token sequence <|object_ref_start|> Minions <|object_ref_end|> <|box_start|><36><230><421><942> , <538><338><929><955><|box_end|> , <|object_ref_start|> Human <|object_ref_end|> <|box_start|><384><337><567><842><|box_end|> , <|object_ref_start|> Car <|object_ref_end|> <|box_start|> None <|box_end|><|im_end|> tokens pretrain Figure 2 GroundAnything architecture. Clean multimodal inputs and response-corrupted text views share one visual encoding and jointly supervise causal prediction and masked denoising. One noisy view is illustrated, with its uncorrupted prompt copy omitted for clarity. The bottom-left panel illustrates parallel denoising, and the grounding example includes two minions and one human and an absent car category. Architecture specifications and parameter counts are provided in Section A.2. 3.3 GroundAnything Data Training data combines public datasets with annotations produced by our data engine. Our data engine (Figure 3) fuses multi-teacher annotations at the field level and applies task-specific validation. Accepted labels train a unified grounding expert for iterative annotation; complementary teachers and local observations resolve uncertain cases. This process expands supervision while refining existing labels. Field validity and query-level coverage are checked separately, so an empty result alone does not establish absence. Fusion, validation, and expert iteration are detailed in Section A.3. Candidate generationField fusionTask validationVersioned supervision Image + task query Teacher ensemble Category-level queries DiscoverLocalize ItemBoxMaskAttr. o 1 o 2 ? o 3 ?? Pass ? PendingFail Required fields Coverage, if required Same rules in every iteration GroundingReferring PointingDense Visual PromptingOCR GUILayout Revise / retire invalid labels Pass Grounding expert Iterative annotation On-demand evidence Complementary teachersLocal views / rollout PendingRe-fuse Train & evaluate Figure 3 Data engine. Field-level teacher fusion and task-specific validation guide expert iteration, with targeted observations resolving uncertain annotations. 6 3.4 Training Design 3.4.1 Base VLM Training Base pretraining has three stages. Vision–language connector alignment trains the projector on image– text pairs with both backbones frozen. Joint multimodal pretraining updates all modules using text and multimodal supervision. General visual and video understanding develops instruction following and image/video understanding. This autoregressive foundation supports subsequent spatial specialization through supervised fine-tuning (SFT) and reinforcement post-training. GroundAnything additionally undergoes the conversion below; training settings are provided in Sections A.4 and A.5. 3.4.2 AR-to-Diffusion Conversion Noisy responseClean multimodal y 1 t y 1 t y 2 t y 2 t y 3 t y 3 t y 4 t y 4 t y 5 t y 5 t y 6 t y 6 t v v p p y 1 y 1 y 2 y 2 y 3 y 3 y 4 y 4 y 5 y 5 y 6 y 6 Q N C Blocked K N → N → C → CBlocked Figure 4 Conversion attention mask. Rows/- columns are queries/keys. Purple denotes within-block bidirectionality, light green the permitted clean prefix, and green clean-stream causality. Other entries are blocked. The sym- bolsvandpcondense visual and prompt tokens. One noisy view with two-token blocks is shown; noisy prompt copies are omitted. Training uses B = 32. Following multimodal AR-to-diffusion conversion (Wu et al., 2026a; Dong et al., 2026), we retain the pretrained backbone and grounding vocabulary and add a mask token. For a clean sequencex= [V,P,Y] of visual embeddings, text prompt, and response, we partitionYintoB= 32 token blocks, indepen- dently of geometric-tuple boundaries. Samplingt b ∼ U(0,1) independently per block, we corrupt only response positions to form complementary noisy viewsw (a) = [P, e Y (a) ],a ∈ 1,2; EOS is masked in both. The joint input [w (1) ;w (2) ;x] contains visual embeddings only in x. Under the mask in Figure 4, each noisy response block attends bidirectionally within itself and reads the clean image/prompt context and strictly preceding clean response blocks. Clean tokens use causal attention and cannot read noisy views. The shared decoder minimizes L = 0.5L MDM + 0.5L AR ,(2) whereL MDM supervises masked response targets andL AR su- pervises clean next-token prediction. Both retain the pretrained token shift. Full loss definitions and training settings appear in Sections A.4 and A.7. 3.4.3 Reinforcement Post-Training We apply GRPO (Shao et al., 2024; Ping et al., 2026) using causal rollouts and exact causal token likelihoods (Figure 5). Box grounding combines set-completenessR set and strict localization R strict ; OCR scores annotation-aware text–geometry agreement; pointing scores location, count, label, and format. For each prompt group, advantages areA i =Z G ( P c w c Z G (R c,i )), whereZ G standardizes within the group. Box- reward weights are (0.7,0.3); OCR and pointing each provide one scalar reward. We optimize the clipped objective L GRPO =−E i h |T i | −1 X t∈T i min r i,t A i , clip(r i,t , 1− ε, 1 + ε)A i i ,(3) wherer i,t is the current-to-old causal token-probability ratio at the rollout temperature,T i contains valid response tokens including EOS, andε= 0.2. One update per rollout trains the projector and language parameters with the vision encoder frozen and no reference-policy KL term. The updated weights are reused for diffusion inference. Reward definitions and variant-specific reinforcement settings appear in Sections A.8 to A.10. 7 Stage 1: SFTStage 2: GRPO GTPred. GT: cup<400><335><582><558><EOS> Pred: cup<385><341><582><565><EOS> GroundAnything Token-level cross-entropy GT: <BOS>cup<400><335><582><558> Image + instruction “Locate the cup.” AR training π old y 1 y 2 y 3 y 4 Sequential Causal mask G = 8 Grounding MissingDuplicateAligned ... Set completeness Z set Strict localization Z strict A=0.7Z set +0.3Z strict OCR “GOODHOU5E” Word-levelLine-level ... Text + geometry Format + penalties A=Z(R OCR ) Diffusion inference Bidirectional Parallel unmasking M y 1 M y 3 M y 1 y 2 y 3 y 4 Exact causal log-probs Clipped GRPOUpdated shared weights θ Same weights Figure 5 Training and inference pipeline. SFT and causal GRPO update shared parameters reused for blockwise diffusion inference. The schematic illustrates token-level supervision and grounding/OCR reward aggregation; the final advantage includes the additional group normalization in Equation (9). Rollouts and unmasking states are illustrative. 3.5 Efficient Parallel Inference Parallel diffusion decoding. After image/query prefill, GroundAnything denoises response blocks with bidirectional attention and caches the completed prefix. Sub-blocks are processed from left to right. Our default Entropy-Guided Decoding (Liu et al., 2025) commits masked positions withH j ≤ τ, where H j =− P v p j (v)logp j (v) is the entropy of the unmodified token distribution. If none qualifies, the lowest- entropy position is committed to ensure progress. Committed tokens remain fixed, and a causal pass constructs the completed block’s cache without AR verification. Dynamic and Static Decoding instead use confidence thresholds and fixed quotas; commitment and cache construction are detailed in Section B.1. Optional self-speculative decoding. The shared weights also support diffusion drafting with causal verifica- tion (Wu et al., 2026a), accepting the longest matching prefix (Figure 6). Linear decoding uses two network function evaluations (NFEs) and 2Bquery tokens per round; quadratic decoding fuses verification and proposal generation in one NFE after initialization, usingB(B+ 1) query tokens. These counts describe queries, not Transformer FLOPs. Acceptance and proposal reuse are detailed in Section B.2; CUDA Graph and selective FP8 optimizations appear in Section B.3. (a) Linear Self-Speculation(b) Quadratic Self-Speculation 1. Diffusion draft2. AR verify sMMMsd 1 d 2 d 3 Bidirectional maskCausal mask sd 1 d 2 d 3 d 1 d 2 rd 3 Draft block 2 matches + correction 2 NFEs / iteration Query tokens: 2B (O(B)) Current draft d 0 d 1 d 2 d 3 B acceptance points 0d 0 M 1d 1 M 2d 2 M 3d 3 M B groups of B + 1 query tokens Mixed attention mask RowAR comparison Next proposals 0 a 0 = d 1 p 01 p 02 p 03 p 04 1 a 1 = d 2 p 11 p 12 p 13 p 14 2 a 2 ≠ d 3 p 21 p 22 p 23 p 24 3not checked p 31 p 32 p 33 p 34 Accepted d 0 d 1 d 2 Next draft (Row 2) p 21 p 22 p 23 p 24 1 NFE / iteration* Query tokens: B(B + 1) (O(B²)) Fused verify + propose Figure 6 Self-speculative decoding with shared model weights (B= 4). Linear decoding separates drafting from causal verification:d 1 andd 2 are accepted, while the mismatchedd 3 is replaced by the AR correctionr. Quadratic decoding comparesa i withd i+1 :a 0 =d 1 ,a 1 =d 2 , anda 2 ̸=d 3 , thereby committing [d 0 ,d 1 ,d 2 ] and reusing Row 2 as the next still-unverified proposal. Purple denotes bidirectional proposals and green causal verification. Matrix rows and columns denote queries and keys, respectively; the cached prefix is omitted.Mdenotes[MASK],sthe last accepted seed, and p ij proposal token j from row i. One NFE per quadratic round applies after initialization. 8 4 Experiments In this section, we ask: Question 1: Grounding capability. How far can GroundAnything push the breadth and precision of visual grounding? Question 2: Speed. How far can parallel grounding accelerate inference with minimal loss of precision, and what makes its speedups practical? Question 3: Design. Which model, training, and output choices support precise and efficient parallel grounding? 4.1 Implementation Details and Evaluation Setup. We evaluate GroundAnything-VLM and GroundAnything against 44 baselines on 30 benchmarks spanning 11 perceptual capabilities, covering specialized detectors, general-purpose VLMs, grounding specialists, and embodied foundations. All numerical results and conclusions involving our models are based on the mean of ten runs, using five random seeds with two runs per seed. Owing to the high computational cost, other baselines that we evaluate locally for the main leaderboard are averaged over three runs, using three random seeds with one run per seed. GPT-6 Astra uses thinking effortHigh. We report F1mIoU for box grounding and OCR, F1@Point for object pointing, and task-specific accuracy for spatial and GUI grounding. For a fair comparison of diffusion language model (DLM) decoding, all GroundAnything results in the main benchmark tables (Tables 1–4) use entropy-guided diffusion decoding, without self-speculative decoding. The self-speculative setting provides higher throughput and quality at the reported operating point, but combines diffusion drafting with causal autoregressive (AR) verification and correction; it is therefore not pure DLM decoding. We analyze this setting separately in Section 4.3 and highlight it in the teaser (Figure 1). 4.2 Main Results Benchmark suite. Detection covers common and long-tailed objects on COCO (Lin et al., 2014) and LVIS (Gupta et al., 2019), and dense and tiny objects on Dense200 (Jiang et al., 2026) and VisDrone (Zhu et al., 2018). Referring grounding uses RefCOCO, RefCOCO+, and RefCOCOg (Yu et al., 2016; Mao et al., 2016; Nagaraja et al., 2016); object pointing covers the four detection datasets and RefCOCOg. Spatial and GUI grounding use RoboSpatial (Song et al., 2025), RefSpatial (Zhou et al., 2026), ScreenSpot-V2 (Wu et al., 2025), ScreenSpot-Pro (Li et al., 2025), and OSWorld-G (Xie et al., 2026). OCR uses HierText (Long et al., 2022), ICDAR2015 (Karatzas et al., 2015), TotalText (Ch’Ng & Chan, 2017), and SROIE (Huang et al., 2019); layout grounding uses DocLayNet (Pfitzmann et al., 2022) and M6Doc (Cheng et al., 2023). Exemplar-based visual prompting on FSC147 (Ranjan et al., 2021) and Dense200 is also included in the main results. GroundAnything-VLM establishes a new overall best among similarly sized models at 72.42%, remaining competitive with GPT-6 Astra (71.35%). GroundAnything reaches 61.75% with pure diffusion decoding, surpassing the prior overall AR state of the art at comparable scale. Figure 7 summarizes this breadth; the comparisons below examine localization precision and task-dependent diffusion–AR gaps. 9 74.23 57.97 65.84 56.52 54.85 V i s u a l P r o m p t F S C 1 4 7 - t e s t , D e n s e 2 0 0 60.16 56.66 58.86 58.03 56.80 G r o u n d i n g C O C O , L V I S 58.14 51.90 55.24 51.17 51.08 D e n s e D e n s e 2 0 0 , V i s D r o n e 84.21 87.34 82.52 79.02 78.56 R e f e r r i n g R e f C O C O g * 5 68.29 64.20 81.08 66.61 66.39 R o b o P o i n t R o b o S p a t i a l , R e f S p a t i a l * 3 90.30 85.74 85.51 84.56 82.92 R e f e r r i n g P o i n t R e f C O C O g * 5 82.37 67.95 79.66 74.89 70.94 G r o u n d i n g P o i n t C O C O , L V I S 76.10 67.54 76.10 70.54 70.26 D e n s e P o i n t D e n s e 2 0 0 , V i s D r o n e 78.29 84.26 92.58 77.73 72.24 G U I S c r e e n s p o t - P r o , S c r e e n S p o t - V 2 , O S W o r l d - G 51.00 40.46 48.89 47.34 45.20 O C R H i e r T e x t , I C D A R 2 0 1 5 , T o t a l T e x t , S R O I E 80.27 62.54 74.27 71.64 69.06 L a y o u t D o c L a y N e t , M 6 D o c B o u n d i n g B o x e s P o i n t T e x t & B o u n d i n g B o x e s GroundAnything GroundAnything-VLM GroundAnything Best Three models Across 45 Baselines Including GPT-6 Astra Figure 7 Grounding across task groups. GroundAnything-VLM, GroundAny- thing, and leading baselines across box, point, text, and exemplar-conditioned tasks. Complete results: Section E. Reporting conventions. Scores are percentages; bold marks column bests, including ties (lower is better only for parse error). Model-name stars denote externally reported rows; entry-level stars denote source or task-interface exceptions.–,N/A, andUNKindicate unreported val- ues, unsupported or unresolved eval- uations, and unspecified zero-shot status, respectively. Daggers flag uncertain prompt/protocol align- ment, so affected scores are descrip- tive. Under our protocols, Ground- ingDINO lacks GUI/OCR/layout interfaces, and Kimi-K3 lacks com- patible pointing/OCR/layout out- puts; SenseNova-Vision’s GUI eval- uation remains unresolved. Lo- cateAnything lacks a supported visual-prompt interface. Starred SenseNova-Vision HierText/IC- DAR2015 scores follow Han et al. (2026); its other OCR scores are local. Complete sources and excep- tions appear in Section E. Detection and referring ground- ing. GroundAnythingreaches 70.04 F1mIoU on Dense200 and 91.61/91.10 on RefCOCOg val/test (Table 1), extending precise diffusion grounding to crowded scenes and language-conditioned targets, although tiny-object localization remains challenging. RefCOCO avg is the unweighted mean of RefCOCO, RefCOCOg, and RefCOCO+. Robot, spatial, andGUIgrounding. GroundAnything matches its AR counterpart on RefSpatial Unseen (62.34%) and improves ScreenSpot-Pro from 65.34% to 75.96% (Table 2), while Astra remains stronger on RefSpatial and GUI tasks. RefSpatial avg averages the Location and Placement splits. OCR, layout, and visual prompting. The shared interface extends to text, document regions, and exemplar- matched instances: GroundAnything-VLM reaches 85.78 on DocLayNet and 74.76 on M6Doc, while Groun- dAnything scores 63.52 on Dense200 visual prompting (Table 3). OCR and layout exhibit larger diffusion–AR gaps than referring and GUI grounding, identifying where parallel generation still sacrifices precision. Object pointing. Following Jiang et al. (2026), SAM-derived masks (Kirillov et al., 2023) determine point correctness, with F1@Point balancing misses and false positives. GroundAnything-VLM leads five of the six selected comparisons. Finding 1: Across 30 benchmarks, GroundAnything extends broad and precise grounding to pure diffusion, surpassing the prior overall AR state of the art at comparable scale. GroundAnything-VLM sets a new overall best at this scale, remaining competitive with GPT-6 Astra. 10 Table 1 Detection and referring grounding (F1mIoU). External rows follow Jiang et al. (2026). Full results appear in Sections E.1 to E.3. ModelCommonLong-tailedDense & TinyReferring grounding COCOLVISDense200VisDrone RefCOCOg val RefCOCOg test RefCOCO avg Closed-set Specialized Detectors DINO-R50 * (Zhang et al., 2022)55.60– DETR-R50 * (Carion et al., 2020)48.30– Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)60.5652.6124.9234.4749.7750.4345.15 Vision-Language Models (<10B) Qwen3-VL-4B (Bai et al., 2025a)46.5349.8614.0231.1475.2775.8875.24 Qwen3.5-9B (Qwen Team, 2026a)51.9948.8730.0232.8476.2076.2876.08 Rex-Omni (Jiang et al., 2026)56.2846.7453.2927.1973.9074.7669.29 LocateAnything Fast (Wang et al., 2026)53.0642.9020.709.8275.3076.5076.62 LocateAnything Hybrid (Wang et al., 2026)59.1249.5650.0728.5776.4377.6778.06 SenseNova-Vision (Han et al., 2026)57.4956.1268.1342.3578.6979.4877.55 RynnBrain1.1 (Li et al., 2026a)35.1526.010.128.2967.6668.2463.63 GroundAnything-VLM63.7056.6375.5540.7484.3783.5683.43 GroundAnything60.7252.5970.0433.7691.6191.1085.33 Vision-Language Models (10B–1T) SEED1.5-VL * (Guo et al., 2025)51.4046.7053.2027.4071.9073.20– Qwen3.8-27B (Qwen Team, 2026d)60.8449.9934.3033.3376.0377.3777.89 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)62.7953.2831.2041.5980.5481.7180.20 Kimi-K2.661.1651.1942.6729.9570.3371.8769.66 Kimi-K3 (Team et al., 2026)60.8947.7351.6430.3173.3973.9972.86 GPT-6 Astra62.7554.9765.0437.1274.9878.9177.81 4.3 Speed Evaluation 4.3.1 Decoding Strategy Analysis Question 2.1: Which decoding choices govern the speed–accuracy trade-off? 481632 Block Size 0 1 2 3 TPF 1.41 2.01 2.21 2.18 2.33 2.99 3.173.17 (a) 481632 Block Size 0 50 100 150 200 250 TPS (tokens/s) 157.7 215.9 226.6 223.8 69.2 83.5 88.0 49.9 (b) Linear Self-SpeculationQuadratic Self-SpeculationGroundAnything-VLM (49.6 TPS) Figure 8 Self-speculative schedules. TPF and TPS across block sizes; dashed lines denote GroundAnything-VLM. We report tokens per forward (TPF), output tokens per second (TPS), and COCO F1mIoU. Decoding protocols and additional analysis appear in Section D.1. Forward efficiency versus latency. AtB=32, quadratic self- speculation reaches higher TPF than the linear schedule (3.17 versus 2.18), yet substantially lower throughput (49.9 versus 223.8 TPS; Figure 8). ItsO(B 2 ) query-token cost offsets the reduction in forward calls, favoring the linear schedule at this operating point. Entropy threshold and block size. Relaxing the entropy threshold increases parallel commitment, but COCO quality falls sharply beyondτ= 0.8 (Figure 9). AtB= 32,τ= 0.8 achieves the highest F1mIoU in this threshold sweep (60.72) at 127.0 TPS, or 2.56×GroundAnything-VLM throughput. Block size also changes this balance: entropy-guided quality peaks atB= 16 in this COCO sweep, whereas linear self-speculation 11 Table 2 Robot, spatial, and GUI grounding (accuracy). External RefSpatial and JEDI/UI-R1 scores follow Jiang et al. (2026); GUI-Owl scores follow Wang et al. (2026). Full results appear in Sections E.5 and E.7. ModelRobot and spatial pointingGUI grounding RefSpatial (avg) RefSpatial Unseen RoboSpatial Context ScreenSpot-ProScreenSpot-V2OSWorld-G Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)14.254.334.92N/A * N/A * N/A * Vision-Language Models (<10B) JEDI * (Xie et al., 2026)–36.1088.60– UI-R1 * (Lu et al., 2026)–17.8085.40– Qwen3-VL-4B (Bai et al., 2025a)49.0027.2764.7556.7492.3056.91 Qwen3.5-9B (Qwen Team, 2026a)55.9237.0160.6653.1390.5760.99 Rex-Omni (Jiang et al., 2026)51.7537.0159.0236.7588.2946.10 LocateAnything Fast (Wang et al., 2026)36.0016.8815.5756.0488.4459.93 LocateAnything Hybrid (Wang et al., 2026)36.1720.7814.7557.0589.9460.46 SenseNova-Vision (Han et al., 2026)20.388.540.82N/A † N/A † N/A † RynnBrain1.1 (Li et al., 2026a)50.6036.9054.1034.6670.4433.33 RoboRefer * (Zhou et al., 2026)50.0039.00– GroundAnything-VLM69.0062.3472.1365.3494.8974.65 GroundAnything63.0062.3469.6775.9695.6081.21 Vision-Language Models (10B–1T) RoboPoint * (Yuan et al., 2024)16.108.40– GUI-Owl * (Ye et al., 2025)–58.00– Gemini-2.5-Pro * (Comanici et al., 2025)35.6027.10– Molmo-72B * (Deitke et al., 2025)30.2521.20– Qwen3.8-27B (Qwen Team, 2026d)60.0046.7563.9358.7694.5063.48 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)68.7557.1469.6755.0681.1349.29 Kimi-K2.641.3341.5630.336.07 † 52.36 † 10.11 † Kimi-K3 (Team et al., 2026)58.9254.9854.9225.36 † 82.70 † 68.26 † GPT-6 Astra85.9381.9365.6993.1797.8886.70 peaks at B = 32 (Figure 10); larger blocks are therefore not uniformly better. 0.20.40.60.81.01.2 Entropy Threshold τ 1.0 1.2 1.4 1.6 TPF (a) 1.02 1.03 1.11 1.24 1.37 1.60 0.20.40.60.81.01.2 Entropy Threshold τ 50 75 100 125 150 175 TPS (tokens/s) (b) 107.3 113.0 118.2 127.0 134.3 148.7 49.6 0.20.40.60.81.01.2 Entropy Threshold τ 52 55 58 61 64 COCO F1mIoU (%) (c) 59.97 59.92 59.82 60.72 54.06 56.14 63.70 Entropy-Guided DecodingGroundAnything-VLM Figure 9 Entropy-threshold sensitivity. TPF, TPS, and COCO F1mIoU atB= 32 with sub-block size 4. Dashed lines denote GroundAnything-VLM. Finding 2.1: In direct DLM decoding, the entropy threshold controls the speed–precision trade-off, providing a decoding-effort knob analogous to thinking effort. We favorτ= 0.8 in theB= 32 COCO sweep. Optional self-speculation improves both quality and throughput over this direct-decoding point, reaching 4.51× AR throughput with a 0.74 p F1mIoU gap. 12 Table 3 OCR, layout, and visual prompting (F1mIoU). External rows follow Jiang et al. (2026). SenseNova-Vision uses published HierText/ICDAR2015 scores (Han et al., 2026) and locally evaluated TotalText/SROIE scores. Full results appear in Sections E.6, E.8 and E.9. ModelOCRLayout groundingVisual prompting HierTextICDAR2015TotalTextSROIEDocLayNetM6DocFSC147Dense200 Closed-set Specialized Detectors DocLayout-YOLO * (Zhao et al., 2024)–81.10– PaddleOCRv5 * (Cui et al., 2025)30.5025.6025.7058.60– Vision-Language Models (<10B) Qwen3-VL-4B (Bai et al., 2025a)23.4828.4138.3540.4140.8124.7318.902.49 Qwen3.5-9B (Qwen Team, 2026a)29.6329.9037.2626.7434.6517.3520.5626.93 Rex-Omni (Jiang et al., 2026)34.4645.6552.3548.3568.0654.9557.1555.50 LocateAnything Fast (Wang et al., 2026)22.5926.7344.4024.8949.3747.03N/A * N/A * LocateAnything Hybrid (Wang et al., 2026)26.6527.4845.4930.0577.3465.94N/A * N/A * SenseNova-Vision (Han et al., 2026)31.20 * 49.50 * 11.4036.2685.5335.6262.5162.84 RynnBrain1.1 (Li et al., 2026a)2.0717.8116.903.596.024.534.781.45 GroundAnything-VLM41.3342.5049.0371.1585.7874.7660.2975.26 GroundAnything33.3141.8143.0843.6468.3856.6952.4363.52 Vision-Language Models (10B–1T) SEED1.5-VL * (Guo et al., 2025)12.0018.7019.5028.1028.7028.00– Qwen3.8-27B (Qwen Team, 2026d)32.9539.7241.5737.2442.2119.9216.8028.19 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)42.9543.0748.6438.5446.9332.3121.6231.81 Kimi-K2.625.26 † 32.25 † 40.99 † 46.83 † 15.59 † 13.50 † 32.4732.41 GPT-6 Astra39.5848.8753.5553.5777.5460.5961.0468.94 Table 4 Object pointing (F1@Point). External rows follow Jiang et al. (2026), where Molmo denotes Molmo-7B-D. Full results appear in Section E.4. ModelReferring object pointingCommon / long-tailedDense / tiny RefCOCOg valRefCOCOg testCOCOLVISDense200VisDrone Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)49.3449.9770.4155.0732.9339.45 Vision-Language Models (<10B) Qwen3-VL-4B (Bai et al., 2025a)76.4377.6465.3355.0821.7223.50 Qwen3.5-9B (Qwen Team, 2026a)77.5977.8572.2164.0065.3544.73 Rex-Omni (Jiang et al., 2026)84.9685.3279.7470.0476.6651.97 LocateAnything Fast (Wang et al., 2026)73.8474.9471.5362.1365.6356.36 LocateAnything Hybrid (Wang et al., 2026)75.8976.6573.7864.8978.0757.30 SenseNova-Vision (Han et al., 2026)74.6375.4272.9662.6678.7161.81 RynnBrain1.1 (Li et al., 2026a)74.4274.1725.7017.563.9513.18 Molmo-7B * (Deitke et al., 2025)83.7083.6077.3040.3033.1029.20 GroundAnything-VLM90.4491.0384.9279.8184.1668.03 GroundAnything85.7585.7371.3264.5777.1457.94 Vision-Language Models (10B–1T) SEED1.5-VL * (Guo et al., 2025)83.6084.2078.2070.7072.1056.70 Qwen3.8-27B (Qwen Team, 2026d)75.8475.8674.0167.6574.5551.16 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)71.2172.4072.1366.9466.2859.04 Kimi-K2.639.59 † 39.60 † 30.65 † 25.51 † 35.18 † 13.63 † GPT-6 Astra87.8084.9082.1777.1486.5765.62 13 481632 Block Size 1.0 1.5 2.0 2.5 TPF (a) 1.41 2.01 2.21 2.18 1.18 1.21 1.23 1.24 481632 Block Size 50 100 150 200 250 TPS (tokens/s) (b) 157.7 215.9 226.6 223.8 134.8 131.7 136.6 127.0 49.6 481632 Block Size 52 55 58 61 64 COCO F1mIoU (%) (c) 57.75 60.39 60.71 62.96 53.91 60.93 61.58 60.72 63.70 Self-Speculative DecodingEntropy-Guided DecodingGroundAnything-VLM Figure 10 Decoding strategy and block size. Linear self-speculation and entropy guidance (τ= 0.8, sub-block size 4) compared on COCO. Dashed lines denote GroundAnything-VLM. 4.3.2 DLM vs. MTP Question 2.2: Why use diffusion rather than MTP when both can draft for the same AR verifier? We match visual inputs, adaptation data, tokenization, and prefixes, and freeze the same converted model’s causal branch as verifier. The primary MTP baseline is an in-house causal drafter; bidirectional block-MTP and retrained causal-DLM controls test the role of attention (Section C). We address four questions: Question 2.2.1: At the same candidate length, which drafter achieves higher acceptance? Question 2.2.2: How much wall-clock time does each accepted draft token cost? Question 2.2.3: Can additional DLM refinement repay its extra forward-pass cost? Question 2.2.4: Which grounding failures benefit from bidirectional attention? 481632 Candidate length K 0 20 40 Draft acceptance (%) (a) 47.4 30.6 16.8 8.5 45.2 37.4 21.8 10.3 481632 Candidate length K 2 3 Accepted draft tokens per round (b) 1.89 2.45 2.69 2.72 1.81 2.99 3.48 3.30 Causal MTPDLM (1 step) Figure 11 Acceptance at matched candidate length. One-step DLM and causal MTP share prefixes and a frozen AR verifier. Accepted tokens exclude verifier corrections and bonuses. 481632 Candidate length K 4 6 8 10 12 Time per accepted draft token (ms) (a) 7.15 6.58 7.71 11.14 9.31 5.89 5.38 6.88 MTPDLM 1 step DLM 2 steps DLM 4 steps 0 2 4 6 8 Time per accepted draft token (ms) (b) 7.71 5.38 4.70 6.84 Causal MTPDLM (1 step)DraftAR verifyCommit / cache Figure 12 Cost per accepted draft token. Full-round latency includes rejected work; the stage breakdown uses K = 16. Draft acceptance. AtK= 16, one-step DLM accepts 3.48 draft tokens per round ver- sus 2.69 for causal MTP (Fig- ure 11), yielding more useful proposals at the same candi- date length. Cost per accepted draft token. Including drafting, verification, and rejected work, one-step DLM reduces full-round cost per accepted draft token from 7.71 to 5.38 ms (Figure 12). Higher acceptance therefore translates into lower wall-clock cost at this operating point. Refinement versus latency. At K= 16, a second DLM re- finement step lowers this cost to 4.70 ms; further refinement increases it. With validation- selected policies on the long- output subset, DLM commits 97.1 versus 84.6 tokens within 400 ms, although MTP leads at 50 ms (Figure 13). Refinement is thus useful when its acceptance gain repays its additional latency. 14 Bidirectional attention and grounding failures. Bidirectional DLM reduces Dense200’s unmatched valid-proposal rate from 24.89% for causal MTP to 14.79%; its retrained causal counterpart scores 21.74% (Figure 14). This supports bidirectional attention for drafting jointly constrained spatial predictions. Bidirectional block-MTP also improves draft quality, so the benefit is not exclusive to diffusion. These gains concern draft efficiency and reliability; exact shared verification preserves the AR verifier’s completed output. 1248 DLM denoising steps 10 20 30 40 50 Draft acceptance (%) (a) 21.8 37.2 41.9 43.1 1248 DLM denoising steps 5.0 7.5 10.0 12.5 Time per accepted draft token (ms) (b) 5.38 4.70 6.84 11.83 2550100200400 Decode budget (ms) 0 50 100 Committed tokens (c) 3.7 10.6 21.0 42.0 84.6 4.2 8.6 22.4 47.9 97.1 DLMCausal MTP Figure 13 Refinement and available decode time. (a,b) Varying refinement depth atK= 16. (c) Committed tokens under equal deadlines on long-output prefixes, using validation-selected policies. COCODense200 DocLayNet GUI 0 2 4 6 Draft format error (%) (a) 5.01 4.13 3.82 0.55 1.80 1.64 1.48 0.49 COCODense200 DocLayNet GUI 0 10 20 30 Valid-box miss rate (%) (b) 12.8 24.9 19.0 8.0 9.5 14.8 11.5 7.5 COCODense200 DocLayNet GUI 0 20 40 60 Draft acceptance (%) (c) 35.6 28.1 31.7 49.6 55.8 58.5 59.6 54.1 Causal MTPBlock MTP (bidirectional)DLM (causal)DLM (bidirectional) Figure 14 Draft diagnostics before verification. Comparisons atK= 8; both DLMs use two refinement steps. Valid-box misses count unmatched proposals, not missed ground-truth instances. GUI denotes ScreenSpot-Pro. Finding 2.2: Bidirectional diffusion with a limited number of refinement steps improves draft-generation efficiency over the tested causal MTP baseline. AtK= 16, two passes reduce cost per accepted draft token to 4.70 ms; further refinement raises the cost. 4.3.3 Infrastructure Analysis Question 2.3: How can parallel decoding translate into practical inference speedups? Decode Method NVIDIA H800 inference Speed Up Dynamic Decoding 6.59 3.48 × Static Decoding (4) 6.76 3.45 × Entropy-Guided Decoding 7.15 3.51 × Self-Speculative Decoding 3.73 4.06 × Time per Output Token (ms/token, ↓)0510152025 Native PyTorch (Eager) → + SGLang (Eager) → + CUDA Graph → + BF16 to FP8 Figure15 Progressiveinferenceoptimization. Time per output token (↓) and cumulative speedup relative to each decoding mode’s native PyTorch implementation. Progressive integration with SGLang, CUDA Graph, and FP8 reduces time per output token across all four decoding modes (Figure 15). The complete stack yields 3.45–4.06×speedups over each mode’s native PyTorch implementation, with self-speculation reaching 3.73 ms/token. Implementation gains are measured within each decoding mode, separately from algorithmic speedups (Section D.2). 4.4 Ablation Studies Coordinate representation. Quantized coordi- nates improve the reported ablation average by 1.43 p for GroundAnything-VLM and 5.68 p for Groun- dAnything over textual coordinates (Figure 16). 15 020406080 Grounding Score (30-Bench Avg.) w/o Stage I Full Quantized + MoonViT-V2 (Kimi K3) + Stage I Textual (0.25× speed) MoonViT (Kimi-VL) Qwen3-ViT (Qwen3-VL-4B) 71.60 72.81 71.12 71.74 55.98 61.03 60.12 45.04 GroundAnything-VLMGroundAnything 72.42 61.75 Figure 16 Architecture and training ablations. Coordinate representation, visual encoder, and Stage I under the ablation protocol (Section D.3). Table 5 Output token efficiency. Mean boxes and tokens per image, and tokens per box. SEED1.5-VL values are external references from Jiang et al. (2026); output-length ratios are not runtime speedups. COCODense200 ModelBoxes/imgTokens/imgTokens/boxBoxes/imgTokens/imgTokens/box SEED1.5-VL (Guo et al., 2025)4.2631.0148.873.15446.374.5 GroundAnything-VLM6.146.47.687.6446.85.1 GroundAnything6.146.47.687.6446.85.1 ViT choice. MoonViT-V2 gives the strongest AR and DLM scores among the tested encoders. Section D.3.1 analyzes when multi-level visual evidence can help coordinate prediction; the encoder abla- tion supports our choice under this recipe. Training. Removing Stage I causes the largest DLM drop (16.62 p). Compact outputs and dense scenes. The shared output format uses 7.6 tokens/box on COCO and 5.1 on Dense200, compared with 148.8 and 74.5 for the SEED1.5-VL reference reported by Jiang et al. (2026) (Table 5). Figure 17 shows how gener- ation time and output length vary with predicted object count: GroundAnything reduces generation time across the dis- played ranges, with larger absolute sav- ings for longer outputs (Section D.4). 0-9 20-2940-4960-6980-89 100-109120-129140-149160-169180-189200-209230-239250-259270-279300-309350-359410-419 Number of Predicted Boxes (Selected Ranges) 0 5 10 15 20 Average Generation Time (s) GroundAnything-VLM time GroundAnything time Output tokens (both models) 0 500 1000 1500 2000 2500 Average Output Tokens Figure 17 Generation cost versus predicted object count. Average generation time for GroundAnything-VLM and GroundAnything, with output-token counts across box-count ranges. 4.5 Discussion Takeaway 1: Grounding as visual evidence extraction naturally suits bidirectional diffusion: related spatial fields share the same image evidence and can be filled jointly, then selectively refined before commitment. This connects the task’s structure to parallel decoding. Takeaway 2: Diffusion is a generation strategy within a Transformer backbone (Wu et al., 2026a). Its distinctive value lies in suitable workloads, such as parallel filling and structured outputs, where joint prediction and limited refinement can repay their computational cost. 16 5 Conclusion We introduced GroundAnything, reconciling fast parallel decoding with broad and precise visual grounding. Structured grounding naturally suits DLMs: its predictions are jointly constrained by shared visual evidence. Diffusion complements the Transformer backbone through a generation process with distinctive strengths in parallel filling and structured outputs. Causal conditioning supports long outputs across blocks, while diffusion accelerates prediction within them. GroundAnything thus highlights a broader principle: generation should follow task structure, and spatial evidence need not be inferred in the order it is serialized. AI Use Statement In this work, we used generative AI tools to assist with code development, to polish the writing, and to produce some of the figures. We also used generative AI models as part of the data annotation pipeline to generate pseudo-labels for model training. We did not use generative AI tools to develop the research ideas or methodology, to design or interpret the experiments, or to outline the paper. We have reviewed all AI-assisted work. We take responsibility for the final content of this work, including text, claims, data annotations, or artifacts produced with the aid of generative AI. Ethics Statement This work does not involve human-subject studies. All datasets and evaluation benchmarks used in this work were obtained from publicly available or appropriately licensed sources in accordance with their respective terms and licenses. The experiments focus on visual grounding and structured visual understanding and do not involve real-world robotic control, autonomous driving, or interaction with people. GroundAnything is developed as a general-purpose visual grounding model for research rather than for safety-critical real-world deployment. Any future integration into embodied or autonomous systems should be subject to appropriate safety evaluation and human oversight. The authors declare no conflicts of interest. Reproducibility Statement We will publicly release the code, model weights of GroundAnything and GroundAnything-VLM, and evaluation data to support systematic and reproducible evaluation. The release will include training and decoding configurations, inference implementations, and evaluation scripts with standardized task prompts, output parsers, and metric implementations. The shared architecture, tokenizer, visual processing, and input/output protocol are specified in Sections A.1 and A.2; the data engine and validation pipeline in Section A.3; and base VLM training, AR-to-diffusion conversion, and continued supervised fine-tuning in Sections A.4, A.5 and A.7. Variant-specific GRPO configurations and task rewards are detailed in Sections A.8 to A.10. Direct diffusion decoding, optional self-speculative decoding, and execution optimizations are described in Sections B.1 to B.3. Controlled DLM–MTP comparisons are documented in Section C, while speed metrics, decoding sensitivity, and infrastructure comparisons appear in Sections D.1 and D.2. Architecture and training ablations and output-token efficiency analyses are provided in Sections D.3 and D.4. The benchmark suite, evaluation metrics, reporting conventions, and complete task-level results are given in Section E. Research Scope, Data Use, and Institutional Disclaimer This work originated from exploratory academic research undertaken by the project leader Qize Yu during their internship at Xpeng Inc. The project was conducted exclusively for scientific investigation and academic publication and does not involve commercial applications, product development, or commercial deployment. All data used in this project were used solely for academic research and maintained under strict segregation from the company’s commercial model development and deployment activities. No project data were used to train, fine-tune, evaluate, or otherwise support commercial models, products, or services. Internal legal review of the dataset materials was completed on September 21, 2026. This work neither uses nor discloses business data containing users’ private or personally identifiable information. The research-only scope described here does not modify or supersede the applicable terms and licenses of the source datasets. 17 The views, methods, findings, and conclusions presented in this paper are those of the authors and do not represent the official positions, technical direction, product roadmap, or commercial commitments of Xpeng Inc. The company’s support for this research should not be construed as endorsement of any commercial application. Neither the research findings nor their publication constitute a claim of readiness, safety, or suitability for commercial deployment. Acknowledgments We thank Xpeng Inc. for providing computational and data resources in support of this academic research, and the data team for their assistance with data preparation and research support. We are particularly grateful to Professor Ping Luo for his guidance on the research ideas and manuscript writing. We also thank Xinghang Li, Qing Li, Baiqiao Yin, Xinyu Wei, Jiadi You, Linhao Zhou, Qiman Wu, Ziteng Cui, Yi Zou, Wei Wei, Hanzhen Zhang and Zhuo Li for their valuable suggestions and constructive feedback. References [1] Arriola, M., Gokaslan, A., Chiu, J., Yang, Z., Qi, Z., Han, J., Sahoo, S., and Kuleshov, V. Block diffusion: Interpolating between autoregressive and diffusion language models. In International Conference on Learning Representations, volume 2025, p. 50726–50753, 2025. [2] Bai, S., Cai, Y., Chen, R., Chen, K., Chen, X., Cheng, Z., Deng, L., Ding, W., Gao, C., Ge, C., Ge, W., Guo, Z., Huang, Q., Huang, J., Huang, F., Hui, B., Jiang, S., Li, Z., Li, M., Li, M., Li, K., Lin, Z., Lin, J., Liu, X., Liu, J., Liu, C., Liu, Y., Liu, D., Liu, S., Lu, D., Luo, R., Lv, C., Men, R., Meng, L., Ren, X., Ren, X., Song, S., Sun, Y., Tang, J., Tu, J., Wan, J., Wang, P., Wang, P., Wang, Q., Wang, Y., Xie, T., Xu, Y., Xu, H., Xu, J., Yang, Z., Yang, M., Yang, J., Yang, A., Yu, B., Zhang, F., Zhang, H., Zhang, X., Zheng, B., Zhong, H., Zhou, J., Zhou, F., Zhou, J., Zhu, Y., and Zhu, K. Qwen3-vl technical report, 2025a. URL https://arxiv.org/abs/2511.21631. [3] Bai, S., Chen, K., Liu, X., Wang, J., Ge, W., Song, S., Dang, K., Wang, P., Wang, S., Tang, J., Zhong, H., Zhu, Y., Yang, M., Li, Z., Wan, J., Wang, P., Ding, W., Fu, Z., Xu, Y., Ye, J., Zhang, X., Xie, T., Cheng, Z., Zhang, H., Yang, Z., Xu, H., and Lin, J. Qwen2.5-vl technical report, 2025b. URL https://arxiv.org/abs/2502.13923. [4] Ben-Hamu, H., Gat, I., Severo, D., Nolte, N. S., and Karrer, B. Accelerated sampling from masked diffusion models via entropy bounded unmasking. Advances in Neural Information Processing Systems, 38:55981–56007, 2026. [5] Carion, N., Massa, F., Synnaeve, G., Usunier, N., Kirillov, A., and Zagoruyko, S. End-to-end object detection with transformers. In European conference on computer vision, p. 213–229. Springer, 2020. [6] Chen, T., Saxena, S., Li, L., Fleet, D. J., and Hinton, G. Pix2seq: A language modeling framework for object detection, 2022. URL https://arxiv.org/abs/2109.10852. [7] Cheng, H., Zhang, P., Wu, S., Zhang, J., Zhu, Q., Xie, Z., Li, J., Ding, K., and Jin, L. M 6 doc: A large-scale multi-format, multi-type, multi-layout, multi-language, multi-annotation category dataset for modern document layout analysis, 2023. URL https://arxiv.org/abs/2305.08719. [8] Cheng, Z., Li, K., Jin, P., Li, S., Ji, X., Yuan, L., Liu, C., and Chen, J. Parallel vertex diffusion for unified visual grounding. In Proceedings of the AAAI conference on artificial intelligence, volume 38, p. 1326–1334, 2024. [9] Ch’Ng, C. K. and Chan, C. S. Total-text: A comprehensive dataset for scene text detection and recognition. In 2017 14th IAPR international conference on document analysis and recognition (ICDAR), volume 1, p. 935–942. IEEE, 2017. [10] Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., Dhillon, I., Blistein, M., Ram, O., Zhang, D., Rosen, E., et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities. arXiv preprint arXiv:2507.06261, 2025. [11] Cui, C., Sun, T., Lin, M., Gao, T., Zhang, Y., Liu, J., Wang, X., Zhang, Z., Zhou, C., Liu, H., Zhang, Y., Lv, W., Huang, K., Zhang, Y., Zhang, J., Zhang, J., Liu, Y., Yu, D., and Ma, Y. Paddleocr 3.0 technical report, 2025. URL https://arxiv.org/abs/2507.05595. [12] Dai, X., Chen, Y., Xiao, B., Chen, D., Liu, M., Yuan, L., and Zhang, L. Dynamic head: Unifying object detection heads with attentions. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 7369–7378. ieee, 2021. 18 [13] Dang, R., Guo, J., Hou, B., Leng, S., Li, K., Li, X., Liu, J., Mao, Y., Wang, Z., Yuan, Y., Zhu, M., Lin, X., Bai, Y., Jiang, Q., Zhao, Y., Zeng, M., Gao, J., Jiang, Y., Cen, J., Huang, S., Wang, L., Zhang, W., Liu, C., Yang, J., Lu, S., and Zhao, D. Rynnbrain: Open embodied foundation models, 2026. URL https://arxiv.org/abs/2602.14979. [14] Deitke, M., Clark, C., Lee, S., Tripathi, R., Yang, Y., Park, J. S., Salehi, M., Muennighoff, N., Lo, K., Soldaini, L., et al. Molmo and pixmo: Open weights and open data for state-of-the-art vision-language models. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 91–104. IEEE, 2025. [15] Deng, C., Zhu, D., Li, K., Gou, C., Li, F., Wang, Z., Zhong, S., Yu, W., Nie, X., Song, Z., Shi, G., and Fan, H. Emerging properties in unified multimodal pretraining, 2025. URL https://arxiv.org/abs/2505.14683. [16] Dong, H., Niu, J., Wang, B., Zeng, W., Zhang, W., and He, C. Mineru-diffusion: Rethinking document ocr as inverse rendering via diffusion decoding. In European Conference on Computer Vision, p. 439–456. Springer, 2026. [17] Guo, D., Wu, F., Zhu, F., Leng, F., Shi, G., Chen, H., Fan, H., Wang, J., Jiang, J., Wang, J., et al. Seed1. 5-vl technical report. arXiv preprint arXiv:2505.07062, 2025. [18] Gupta, A., Dollar, P., and Girshick, R. Lvis: A dataset for large vocabulary instance segmentation. In 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 5351–5359. IEEE, 2019. [19] Han, X., Li, J., Deng, K., Chen, Z., Shi, X., Wang, S., Li, B., Wang, L., Xie, S., You, X., Quan, J., Cai, Z., Diao, H., Liu, Z., Yang, L., Lin, D., and Wang, Q. Vision as unified multimodal generation, 2026. URL https://arxiv.org/abs/2607.06560. [20] Huang, Z., Chen, K., He, J., Bai, X., Karatzas, D., Lu, S., and Jawahar, C. Icdar2019 competition on scanned receipt ocr and information extraction. In 2019 International Conference on Document Analysis and Recognition (ICDAR), p. 1516–1520. IEEE, 2019. [21] Jiang, Q., Huo, J., Chen, X., Xiong, Y., Zeng, Z., Chen, Y., Ren, T., Yu, J., and Zhang, L. Detect anything via next point prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 25472–25483, 2026. [22] Karatzas, D., Gomez-Bigorda, L., Nicolaou, A., Ghosh, S., Bagdanov, A., Iwamura, M., Matas, J., Neumann, L., Chandrasekhar, V. R., Lu, S., Shafait, F., Uchida, S., and Valveny, E. Icdar 2015 competition on robust reading. In 2015 13th International Conference on Document Analysis and Recognition (ICDAR), p. 1156–1160, 2015. doi: 10.1109/ICDAR.2015.7333942. [23] Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., Xiao, T., Whitehead, S., Berg, A. C., Lo, W.-Y., et al. Segment anything. In 2023 IEEE/CVF international conference on computer vision (ICCV), p. 3992–4003. IEEE, 2023. [24] Kumbhar, S., Liao, H., Appalaraju, S., and Singh, K. Y. Towards gui agents: Vision-language diffusion models for gui grounding, 2026. URL https://arxiv.org/abs/2603.26211. [25] Li, K., Meng, Z., Lin, H., Luo, Z., Tian, Y., Ma, J., Huang, Z., and Chua, T.-S. Screenspot-pro: Gui grounding for professional high-resolution computer use. In Proceedings of the 33rd ACM International Conference on Multimedia, p. 8778–8786, 2025. [26] Li, K., Hou, B., Zhu, M., Zhang, T., Cheng, Z., Wang, Z., Leng, S., Li, X., Lin, X., Yao, B., Zeng, M., Liu, J., Dang, R., Guo, J., Huang, S., Zhao, H., Ping, H., Zhao, Y., Zhao, T., Wang, K., Lu, T., Xue, S., Tang, J., Wang, Y., Wang, Z., Gao, J., Lu, S., Liu, C., Yang, J., Chen, M., and Zhao, D. Rynnbrain 1.1: Towards more capable and generalizable embodied foundation model, 2026a. URL https://arxiv.org/abs/2607.17977. [27] Li, L. H., Zhang, P., Zhang, H., Yang, J., Li, C., Zhong, Y., Wang, L., Yuan, L., Zhang, L., Hwang, J.-N., et al. Grounded language-image pre-training. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10955–10965. IEEE, 2022. [28] Li, S., Kallidromitis, K., Bansal, H., Gokul, A., Kato, Y., Kozuka, K., Kuen, J., Lin, Z., Chang, K.-W., and Grover, A. Lavida: A large diffusion language model for multimodal understanding. Advances in Neural Information Processing Systems, 38:105101–105134, 2026b. [29] Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. Microsoft coco: Common objects in context. In European conference on computer vision, p. 740–755. Springer, 2014. [30] Liu, A., He, M., Zeng, S., Zhang, S., Zhang, L., Wu, C., Jia, W., Liu, Y., Zhou, X., and Zhou, J. Wedlm: Reconciling diffusion language models with standard causal attention for fast inference, 2025. URLhttps://arxiv. org/abs/2512.22737. 19 [31] Liu, J., Dong, X., Ye, Z., Mehta, R., Fu, Y., Singh, V., Zhang, C., and Molchanov, P. Tidar: Think in diffusion, talk in autoregression. Proceedings of Machine Learning and Systems, 8:748–762, 2026. [32] Liu, S., Zeng, Z., Ren, T., Li, F., Zhang, H., Yang, J., Jiang, Q., Li, C., Yang, J., Su, H., et al. Grounding dino: Marrying dino with grounded pre-training for open-set object detection. In European conference on computer vision, p. 38–55. Springer, 2024. [33] Long, S., Qin, S., Panteleev, D., Bissacco, A., Fujii, Y., and Raptis, M. Towards end-to-end unified scene text detection and layout analysis. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 1039–1049. IEEE, 2022. [34] Lu, Z., Chai, Y., Guo, Y., Yin, X., Liu, L., Wang, H., Xiao, H., Ren, S., Zhao, P., Liu, G., et al. Ui-r1: Enhancing efficient action prediction of gui agents by reinforcement learning. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, p. 17608–17616, 2026. [35] Mao, J., Huang, J., Toshev, A., Camburu, O., Yuille, A. L., and Murphy, K. Generation and comprehension of unambiguous object descriptions. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 11–20, 2016. [36] Meng, L., Yang, J., Tian, R., Dai, X., Wu, Z., Gao, J., and Jiang, Y.-G. Deepstack: Deeply stacking visual tokens is surprisingly simple and effective for lmms. Advances in Neural Information Processing Systems, 37:23464–23487, 2024. [37] Nagaraja, V. K., Morariu, V. I., and Davis, L. S. Modeling context between objects for referring expression understanding. In European conference on computer vision, p. 792–807. Springer, 2016. [38] Peng, Z., Wang, W., Dong, L., Hao, Y., Huang, S., Ma, S., and Wei, F. Kosmos-2: Grounding multimodal large language models to the world, 2023. URL https://arxiv.org/abs/2306.14824. [39] Pfitzmann, B., Auer, C., Dolfi, M., Nassar, A. S., and Staar, P. Doclaynet: A large human-annotated dataset for document-layout segmentation. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, p. 3743–3751, 2022. [40] Ping, B., Chen, Z., Hui, T., Yu, Q., Li, C., Yan, J., and Chang, B. LongAct: Harnessing intrinsic activation patterns for long-context reinforcement learning, 2026. URL https://arxiv.org/abs/2604.14922. [41] Qin, Y., Ye, Y., Fang, J., Wang, H., Liang, S., Tian, S., Zhang, J., Li, J., Li, Y., Huang, S., Zhong, W., Li, K., Yang, J., Miao, Y., Lin, W., Liu, L., Jiang, X., Ma, Q., Li, J., Xiao, X., Cai, K., Li, C., Zheng, Y., Jin, C., Li, C., Zhou, X., Wang, M., Chen, H., Li, Z., Yang, H., Liu, H., Lin, F., Peng, T., Liu, X., and Shi, G. Ui-tars: Pioneering automated gui interaction with native agents, 2025. URL https://arxiv.org/abs/2501.12326. [42] Qwen Team. Qwen3.5: Towards native multimodal agents, February 2026a. URLhttps://qwen.ai/blog?id= qwen3.5. [43] Qwen Team. Qwen3.6-27B: Flagship-level coding in a 27B dense model, April 2026b. URLhttps://qwen.ai/blog? id=qwen3.6-27b. [44] Qwen Team. Qwen3.7: The agent frontier, May 2026c. URL https://qwen.ai/blog?id=qwen3.7. [45] Qwen Team. Qwen3.8-Max: A new bar for coding and cowork, August 2026d. URLhttps://qwen.ai/blog?id= qwen3.8. [46] Ranjan, V., Sharma, U., Nguyen, T., and Hoai, M. Learning to count everything. In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 3393–3402. IEEE, 2021. [47] Redmon, J., Divvala, S., Girshick, R., and Farhadi, A. You only look once: Unified, real-time object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition, p. 779–788, 2016. [48] Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y. K., Wu, Y., and Guo, D. Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024. URLhttps: //arxiv.org/abs/2402.03300. [49] Song, C. H., Blukis, V., Tremblay, J., Tyree, S., Su, Y., and Birchfield, S. Robospatial: Teaching spatial understanding to 2d and 3d vision-language models for robotics. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 15768–15780. IEEE, 2025. [50] Team, K., Bai, T., Bai, Y., Bao, Y., Cai, J., Cai, X., Cao, P., Cao, Y., Chai, Z., Charles, Y., et al. Kimi k3: Open frontier intelligence. arXiv preprint arXiv:2607.24653, 2026. 20 [51] Wang, S., Liu, S., Kuang, Y., Wei, X., Liu, Y., Li, Z., Man, Y., Chen, G., Tao, A., Liu, G., et al. Locateanything: Fast and high-quality vision-language grounding with parallel box decoding. In European Conference on Computer Vision, p. 336–357. Springer, 2026. [52] Wu, C., Lan, S., Fu, Y., Gao, S., Wang, J., Yu, J., Alvarez, J. M., Molchanov, P., Luo, P., Han, S., Zhu, L., and Xie, E. Fast-dvlm: Efficient block-diffusion vlm via direct conversion from autoregressive vlm, 2026a. URL https://arxiv.org/abs/2604.06832. [53] Wu, C., Zhang, H., Xue, S., Liu, Z., Diao, S., Zhu, L., Luo, P., Han, S., and Xie, E. Fast-dllm: Training-free acceleration of diffusion llm by enabling kv cache and parallel decoding. In International Conference on Learning Representations, volume 2026, p. 57027–57051, 2026b. [54] Wu, Z., Chen, X., Pan, Z., Liu, X., Liu, W., Dai, D., Gao, H., Ma, Y., Wu, C., Wang, B., et al. Deepseek-vl2: Mixture-of-experts vision-language models for advanced multimodal understanding. arXiv preprint arXiv:2412.10302, 2024. [55] Wu, Z., Wu, Z., Xu, F., Wang, Y., Sun, Q., Jia, C., Cheng, K., Ding, Z., Chen, L., Liang, P. P., et al. Os-atlas: Foundation action model for generalist gui agents. In International Conference on Learning Representations, volume 2025, p. 5090–5108, 2025. [56] Xiao, B., Wu, H., Xu, W., Dai, X., Hu, H., Lu, Y., Zeng, M., Liu, C., and Yuan, L. Florence-2: Advancing a unified representation for a variety of vision tasks. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 4818–4829. IEEE, 2024. [57] Xie, T., Deng, J., Li, X., Yang, J., Wu, H., Chen, J., Hu, W., Wang, X., Xu, Y., Wang, Z., et al. Scaling computer-use grounding via user interface decomposition and synthesis. Advances in Neural Information Processing Systems, 38, 2026. [58] Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. [59] Ye, J., Zhang, X., Xu, H., Liu, H., Wang, J., Zhu, Z., Zheng, Z., Gao, F., Cao, J., Lu, Z., Liao, J., Zheng, Q., Huang, F., Zhou, J., and Yan, M. Mobile-agent-v3: Fundamental agents for gui automation, 2025. URL https://arxiv.org/abs/2508.15144. [60] You, J., Yu, Q., Chen, Y., Cai, M., Zhong, Z., Wang, Y., Ping, B., Liang, J., Shen, Z., Yan, H., Li, Y., Wu, R., Qi, X., and Chen, Y. AffordanceWAM: Affordance-aware joint world-action modeling for robot manipulation, 2026a. URL https://arxiv.org/abs/2609.22332. [61] You, Z., Nie, S., Zhang, X., ZHOU, J., Lu, Z., Wen, J.-R., and Li, C. Llada-v: Large language diffusion models with visual instruction tuning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 10093–10105, 2026b. [62] Yu, L., Poirson, P., Yang, S., Berg, A. C., and Berg, T. L. Modeling context in referring expressions. In European conference on computer vision, p. 69–85. Springer, 2016. [63] Yu, Q., You, J., Wang, Y., Liang, J., Ping, B., Tian, Y., Chen, Y., Cai, M., Gong, Z., Wu, R., Li, Y., Liang, J., and Chen, Y. AffordanceVLA: A vision-language-action model empowering action generation through affordance-aware understanding, 2026. URL https://arxiv.org/abs/2606.06155. [64] Yuan, W., Duan, J., Blukis, V., Pumacay, W., Krishna, R., Murali, A., Mousavian, A., and Fox, D. Robopoint: A vision-language model for spatial affordance prediction for robotics, 2024. URLhttps://arxiv.org/abs/2406.10721. [65] Yue, Z., Lin, Z., Song, Y., Wang, W., Ren, S., Gu, S., Li, S., Li, P., Zhao, L., Li, L., et al. Mimo-vl technical report. arXiv preprint arXiv:2506.03569, 2025. [66] Zhang, H., Li, F., Liu, S., Zhang, L., Su, H., Zhu, J., Ni, L. M., and Shum, H.-Y. Dino: Detr with improved denoising anchor boxes for end-to-end object detection, 2022. URL https://arxiv.org/abs/2203.03605. [67] Zhao, Z., Kang, H., Wang, B., and He, C. Doclayout-yolo: Enhancing document layout analysis through diverse synthetic data and global-to-local adaptive perception, 2024. URL https://arxiv.org/abs/2410.12628. [68] Zhou, E., An, J., Chi, C., Han, Y., Rong, S., Zhang, C., Wang, P., Wang, Z., Huang, T., Sheng, L., et al. Roborefer: Towards spatial referring with reasoning in vision-language models for robotics. Advances in Neural Information Processing Systems, 38:28404–28481, 2026. [69] Zhu, P., Wen, L., Bian, X., Ling, H., and Hu, Q. Vision meets drones: A challenge, 2018. URLhttps: //arxiv.org/abs/1804.07437. 21 Appendix A Model and Training Details A.1 Input/Output Protocol GroundAnything and GroundAnything-VLM share a structured interface: the query specifies what evidence to extract, and each response entry associates a semantic identifier with spatial coordinates. The geometry is task-dependent, while the enclosing grammar remains unchanged, following the unified grounding formulation of Jiang et al. (2026). Query templates. Table 6 lists the task instructions. Bracketed terms are placeholders. A category list joins the requested names with</c>, without intervening spaces; for example,person</c>car. Visual reference boxes use the same coordinate tokens as outputs. The image and instruction form the user turn; the structured response forms the assistant turn. Table 6 Task-specific input instructions. Category names and referring expressions are preserved in the response identifiers. TaskInstruction Category groundingLocate all the instances that match the following categories: [CATEGORIES]. Referring groundingLocate the target referred to by the following description: [EXPRESSION]. Category pointingPoint to: [CATEGORIES]. Referring pointingPoint to the target referred to by the following description: [EXPRESSION]. GUI groundingPoint to the UI element to click for the following instruction: [INSTRUCTION]. OCROCR task detect all the text in box format. Layout groundingDetect all document layout elements that match the following categories: [CATE- GORIES]. Visual promptingGiven reference boxes<|box_start|>[BOXES]<|box_end|>indicating one or more objects, find all similar objects in the image and output their bounding boxes. Response grammar. The entry template and illustrative payloads are shown below. Displayed line breaks are for readability and are omitted in the serialized response. Entry template <|object_ref_start|>[IDENTIFIER]<|object_ref_end|> <|box_start|>[PAYLOAD]<|box_end|> Boxes: two instances of one category <|object_ref_start|>person<|object_ref_end|> <|box_start|><10><20><30><40>,<50><60><70><80><|box_end|> Point: one target location <|object_ref_start|>button<|object_ref_end|> <|box_start|><512><384><|box_end|> OCR: transcription as identifier <|object_ref_start|>OPEN<|object_ref_end|> <|box_start|><100><200><300><260><|box_end|> Explicitly queried but absent category <|object_ref_start|>car<|object_ref_end|> <|box_start|>None<|box_end|> Boxes use four atomic coordinate tokens in (x 1 ,y 1 ,x 2 ,y 2 ) order; points use two in (x,y) order. Instances sharing an identifier occupy one wrapper, with tuples separated by commas without spaces. Distinct entries are separated by a comma and a space. Ordered point sequences retain their temporal order. The ordinary 22 textNonedenotes an absent queried target, not a padding element. OCR transcriptions occupy the identifier field rather than a separate text channel. Coordinates and canonicalization. Coordinates are normalized by image width and height and quantized to <0>–<999>. For annotations on a [0, 1000] grid, the conversion is q(v) = 999v + 500 1000 .(4) Inputxywhboxes are converted toxyxy; polygons used for box supervision become axis-aligned enclosing boxes. Non-finite or out-of-range coordinates and inverted or degenerate boxes fail validation. Box instances are stably sorted byx 1 , and unordered points byx; semantic sequence order is never replaced by spatial sorting. Structural tokens and termination. Table 7 distinguishes entry boundaries from response termination. Struc- tural and coordinate tokens are retained when decoding text for the parser. The diffusion mask is an internal input token and is excluded from the generatable vocabulary. Table 7 Grounding and multimodal token conventions. Token(s)Function <|object_ref_start|>, <|object_ref_end|>Delimit a semantic identifier <|box_start|>, <|box_end|>Delimit a geometric payload <0>–<999>Atomic quantized coordinates </c>Separate requested categories <|vision_start|>, <|vision_end|>Delimit visual input <|image_pad|>Image placeholder in the input template <|im_start|>, <|im_end|>Turn boundaries; the latter terminates the response |<MASK>|Internal diffusion mask A.2 Architecture, Tokenizer, and Visual Processing The variants share a MoonViT-V2 (Kimi K3) visual encoder, a multimodal projector, and a Qwen3-4B language architecture. All language layers use full attention and one-dimensional rotary position indices. Dynamic image processing supplies at most 1024 projected visual tokens, corresponding to 4096 patches before 2×2 aggregation; channel means and standard deviations are both (0.5,0.5,0.5). Table 8 gives the module specifications. Table 8 Shared backbone architecture. Positional capacity is independent of the training sequence lengths. ComponentConfiguration Vision encoder27 layers; width 1024; FFN width 4096; 12 heads; QKV width 1536; patch size 14 ProjectorLayerNorm(1024), 2×2 aggregation, bias-free linear layers 4096→4096→2560 with GELU, then RMSNorm(2560); normalization ε = 10 −5 Language backbone36 layers; width 2560; FFN width 9728; 32 query heads and 8 KV heads; head dimension 128 Language numericsSiLU; attention dropout 0; RMSNorm ε = 10 −6 ; RoPE base 5,000,000 Positional capacity262,144 positions 23 Table 9 Parameter counts. The 4B designation refers to the language backbone. ModuleGroundAnything-VLM GroundAnything Vision encoder401,214,464401,214,464 Projector27,267,58427,267,584 Decoder excluding vocabulary3,633,511,9363,633,511,936 Input embedding390,835,200390,837,760 Output head390,835,200390,837,760 Total4,843,664,3844,843,669,504 Vocabulary and parameter shar- ing. GroundAnything-VLM ex- tends the 151,669-entry base vo- cabulary with 1000 coordinate to- kens and</c>, giving 152,670 en- tries. Coordinate IDs are 151669– 152668; the category delimiter has ID 152669. GroundAnything adds |<MASK>|at ID 152670, giving 152,671 entries. EOS and padding have distinct IDs, 151645 and 151643; the image, vision-start, and vision-end IDs are 151655, 151652, and 151653. Input embeddings and output heads are untied. Within GroundAnything, causal and diffusion modes share all model weights. The new mask row is initialized separately in each vocabulary matrix using the FP32 mean of the rows for<|im_start|>,<|im_end|>,<|vision_start|>, and<|vision_end|>, then cast to the model dtype. This adds 5120 parameters. The complete multimodal model has approximately 4.844B parameters (Table 9). A.3 Expert-Driven Data Engine Field-level fusion. Complementary teachers provide semantic, localization, segmentation, text, interface, and layout evidence. Category discovery precedes category-conditioned localization; category names are normalized while attributes needed for referring expressions are retained. Predictions are mapped back to the original image and aligned only when their query scope and annotation granularity agree. Fusion operates on individual fields, allowing reliable text or geometry to be retained without accepting an entire inconsistent annotation. Validation and coverage. Task-specific checks assess geometric validity, semantic consistency, text–region agreement, and cross-view support. Each dependency has a validation statea(f)∈ pass, fail, pending; query-level coveragec(I,Q) uses the same state space. Coverage checks whether the required query scope has been examined and relevant unresolved proposals and matching conflicts have been addressed. Unresolved dependencies trigger complementary teachers or targeted local observations. Required dependencies and acceptance. LetF(I,Q,Y) contain the fields and semantic facts required to validate candidateY, including query–target correspondence. The task protocol and query semantics determine these requirements, which are then instantiated for the candidate. Missing required fields remain inFwith statuspending: omitting a field cannot remove its validation requirement. Leth(Q)∈0,1indicate whether the query requires exhaustive instance coverage. Acceptance requires A(I,Q,Y ) = 1[∀f ∈F (I,Q,Y ), a(f ) = pass]1[h(Q) = 0 ∨ c(I,Q) = pass],(5) so every required dependency must pass, and exhaustive queries additionally require validated coverage. For example, validating the returned boxes does not establish that every queried instance has been found; an all-instance query remains unaccepted while coverage is pending. Empty-target claims. An answer with no instances still requires verification of its query–answer relation. Its dependency set explicitly includes a query-scoped absence factf abs ∈F(I,Q,Y), so an empty prediction never yields an empty dependency set. An unsupported absence claim remains pending; contrary evidence makes it fail. It passes only when absence is verified andc(I,Q) =pass, even whenh(Q) = 0. Missing outputs, failed annotation runs, and unresolved empty predictions do not constitute evidence of absence. These requirements prevent empty predictions from being accepted through a vacuously true check over an empty set. Derived box, point, referring, and visual-prompt supervision preserves its dependencies on the validated source evidence; crops and coordinate transforms remain consistent with visible content. 24 Expert iteration. Accepted annotations train a unified grounding expert, which then proposes annotations for further validation. Difficult cases receive targeted crops or complementary teacher evidence. New evidence can correct or retire previous labels, with versioned annotation snapshots keeping each training input fixed. This loop improves both coverage and field consistency rather than merely accumulating predictions. A.4 Training Pipeline GroundAnything follows four training phases: Base VLM Training (Pretrain 1), Coordinate Alignment (Pretrain 2, also termed SFT), AR-to-Diffusion conversion, and reinforcement learning (RL). GroundAnything- VLM follows three phases: Base VLM Training, Coordinate Alignment, and RL. The Stage 1–4 numbering below refers specifically to pretraining and alignment: Stages 1–3 belong to Pretrain 1, while Stage 4 is Pretrain 2. Table 10 summarizes the two pipelines and their relation to Figure 5. Table 10 Training pipelines. GroundAnything has four phases; GroundAnything-VLM has three and proceeds directly from coordinate alignment to RL. Training phaseGroundAnything-VLMGroundAnything Base VLM Training (Pretrain 1)Stages 1–3; causal cross-entropyStages 1–3; causal cross-entropy Coordinate Alignment (Pretrain 2 / SFT) Stage 4; assistant-only spatial cross- entropy Stage 4; assistant-only spatial cross- entropy AR-to-Diffusion conversion—0.5L MDM + 0.5L AR ; shared weights Reinforcement learning (RL)Causal GRPOCausal GRPO; shared AR/diffusion weights A.5 Base VLM Training (Pretrain 1) Stage 1: vision–language connector alignment. The vision encoder and language backbone are frozen, and only the projector is trained on image-caption data. Assistant-only causal cross-entropy aligns the visual features with the language input space. Stage 2: joint multimodal pretraining. All modules are unfrozen and jointly trained on pure text and general image–text data. Pure-text examples supervise all valid causal text targets; multimodal examples supervise assistant responses, excluding visual inputs, prompts, and masked template positions. The text and multimodal losses are normalized separately over their valid target tokens and then summed,L mix =L text +L vlm . Counts are aggregated across data-parallel workers so that each domain contributes its global token mean. Packing preserves independent subsequence boundaries and supervision masks. Stage 3: general visual and video understanding. All modules continue training on general visual question answering and instruction-following data, image captions, and video data with assistant-only causal cross- entropy. The configured context and packing lengths are both 32K. This stage has no separate pure-text quota or dedicated spatial-specialization mixture; spatial supervision is concentrated in Pretrain 2. Aggregate base-pretraining exposure exceeds 700B tokens. A.6 Coordinate Alignment (Pretrain 2 / SFT) Stage 4: spatial specialization. Initialized from the Stage 3 checkpoint, coordinate alignment updates the vision encoder, projector, and language parameters on eight spatial task groups: detection, GUI grounding, referring grounding, referring pointing, OCR, document layout, dense pointing and counting, and visual prompting. Boxes, points, semantic identifiers, and protocol markers are supervised jointly as assistant-response tokens. The coordinate tokens<0>through<999>use the same causal cross-entropy objective as the other valid response tokens. This single alignment phase is the supervised fine-tuning (SFT) phase referred to in the main text. Its context and packing lengths are 8K. Table 11 gives the configured recipe for both pretraining phases. After coordinate alignment, GroundAnything- VLM proceeds to RL, while GroundAnything first undergoes the AR-to-Diffusion conversion in Section A.7. 25 Table11 Configured pretraining and alignment recipe. Base VLM Training (Pretrain 1) comprises Stages 1–3; Coordinate Alignment (Pretrain 2 / SFT) is Stage 4. Context and packing lengths count complete multimodal sequences. Base VLM Training (Pretrain 1)Pretrain 2 SettingStage 1Stage 2Stage 3Stage 4 Trainable modulesProjectorAllAllAll Language peak LRFrozen10 −5 10 −5 10 −5 Vision peak LRFrozen10 −6 10 −6 10 −6 Projector peak LR5× 10 −5 10 −6 10 −6 10 −6 Per-device batch / accumulation6 / 12 / 12 / 12 / 1 GPUs / global batch128 / 768128 / 256128 / 256128 / 256 Context / packing8K / 8K8K / 8K32K / 32K8K / 8K Weight decay / gradient clipping0 / 1.00.1 / 1.00.1 / 1.00.1 / 1.0 LR schedule / minimum ratio Cosine / 2%Cosine / 10%Cosine / 10%Cosine / 10% Warmup / configured budget 3% / 1 epoch3% / 1 epoch3% / 1 epoch3% / 1 epoch All four pretraining/alignment stages use BF16, ZeRO-1, AdamW with (β 1 ,β 2 ) = (0.9,0.95), FlashAttention, and activation checkpointing. Global batches count packed sequences and equal the number of GPUs times the per-device batch times gradient accumulation. Here 8K and 32K denote 8192 and 32768 tokens, respectively, and one epoch denotes the configured training budget. Language parameters include the untied input embedding and output head. A.7 AR-to-Diffusion Conversion (GroundAnything Only) GroundAnything is initialized from the autoregressive checkpoint after coordinate alignment. Direct multimodal conversion (Wu et al., 2026a) retains its grounding vocabulary and output grammar. Letx= [V,P,Y] contain visual embeddingsV, the text promptPencoding queryQ, and responseY. Supervised response spans are partitioned into consecutive blocks ofB= 32 tokens. Blocks may cross geometric-tuple boundaries, stop at response-span boundaries, and have a shorter final block; they require no semantic padding. Complementary corruption. For each response blockb, samplet b ∼U(0,1). For every supervised non-EOS positioniin that block, drawm (1) i ∼ Bernoulli(t b ) and setm (2) i = 1− m (1) i . Viewareplaces the target by |<MASK>|ifm (a) i = 1, givingw (a) = [P, e Y (a) ]. EOS is masked in both views. Images, prompts, headers, unsupervised turns, and empty thinking prefixes are not corrupted. Attention visibility. The joint input [w (1) ;w (2) ;x] shares one clean multimodal stream and one visual encoding; each noisy view retains an uncorrupted prompt copy. Within a view, a noisy response block attends bidirectionally to itself and reads the clean visual/prompt context and strictly preceding clean response blocks. It cannot read current or future clean response targets or the other noisy view. Clean tokens use token-level causal attention and cannot attend to either noisy view. Figure 4 illustrates one view with two-token blocks for clarity. Joint objective. For masked target setsM (1) andM (2) , and valid clean response targetsT, the two losses are L MDM =− P 2 a=1 P i∈M (a) logp θ (y i | x,w (a) ;A a ) |M (1) | +|M (2) | ,(6) L AR =− 1 |T| X i∈T logp θ (y i | V,P,y <i ).(7) HereA a enforces the visibility rules above, so writingxas an input does not expose the clean target. The MDM mean includes both masked EOS targets and has no inverse-noise weighting. Both losses retain the next-token shift: hidden positioni−1 predicts targeti. Prompts, visual positions, headers, padding, unsupervised turns, and empty thinking prefixes are excluded. Valid target losses are averaged across the training batch and combined as in Equation (2). 26 Conversion optimization. Within this conversion phase, optimization has two substages: Phase I freezes the vision encoder and trains the projector and language parameters; Phase I updates all modules. Both retain the fixed block size and joint objective. The optimizer and scheduler restart between phases. Table 12 reports the settings. Table 12 GroundAnything conversion configuration. SettingPhase IPhase I GPUs256256 Per-device / global batch32 / 819232 / 8192 Clean sequence length40964096 Trainable modulesProjector and languageAll Language / projector peak LR2× 10 −5 / 2× 10 −5 2× 10 −5 / 2× 10 −5 Vision peak LRFrozen2× 10 −6 Gradient checkpointingLanguageLanguage and vision Seed / data seed32 / 3242 / 42 Conversion uses BF16, ZeRO-1, one gradient-accumulation step, and AdamW with betas (0.9,0.95), epsilon 10 −8 , weight decay 0.1, and gradient clipping at 1.0. Both phases use cosine decay with 3% warmup. A.8 Reinforcement Learning (RL): Causal GRPO RL is the final training phase for both variants, following coordinate alignment for GroundAnything-VLM and AR-to-Diffusion conversion for GroundAnything. GroundAnything uses causal attention for both rollout generation and likelihood evaluation. For each prompt,G= 8 responses are sampled at temperatureT= 0.7. Before updating, a no-gradient teacher-forcing pass records detached old-policy log-probabilities; a second pass computes current-policy gradients. Both use FP32 log-softmax at the rollout temperature. These are exact likelihoods of the causal policy, not likelihoods of a diffusion denoising trajectory. Advantage normalization. For a scalar component u within one prompt group, define Z G (u i ) = u i − ̄u q G −1 P G j=1 (u j − ̄u) 2 + 10 −6 , ̄u = G −1 G X j=1 u j .(8) Each active reward component is normalized before weighting, and their sum is normalized again within the same group: A i = Z G X c∈C task w c Z G (R c,i ) ! .(9) Box rewards have weights (0.7,0.3); OCR and pointing each supply one complete scalar reward with unit weight. Internal OCR subterms are not normalized separately. Constant groups produce zero advantage. Policy objective. Letπ θ,T denote the causal policy at temperatureT, and letT i contain response tokens including EOS, excluding prompts and post-EOS padding. Expanding the response-wise averaging in Equation (3), the update minimizes r i,t (θ) = π θ,T (y i,t | I,Q,y i,<t ) π old,T (y i,t | I,Q,y i,<t ) ,(10) L GRPO =− 1 G G X i=1 1 |T i | X t∈T i min(r i,t A i , clip(r i,t , 1− ε, 1 + ε)A i ),(11) withε= 0.2. There is one update per rollout and no reference-policy KL term. The vision encoder is frozen; the projector and language parameters are updated. Causal and diffusion attention modes subsequently reuse these parameters without changing the output protocol. 27 A.9 GroundAnything-VLM Reinforcement Settings GroundAnything-VLM uses the same task-reward definitions but a distinct optimization configuration. It freezes both the vision encoder and projector and retains a frozen supervised reference with KL coefficient 0.02. Active reward components are standardized within each prompt group using the sample standard deviation and epsilon 10 −8 ; after weighted summation, advantages are standardized across the complete response batch. GroundAnything instead uses population standard deviations and a second within-group normalization. Table 13 keeps these recipes separate. Table 13 Variant-specific GRPO settings. Global batches count generated responses, not distinct prompts. SettingGroundAnythingGroundAnything-VLM GPUs256256 Per-device response batch / accumulation1 / 11 / 1 Responses per prompt88 Prompt / response batch32 / 25632 / 256 Updates per rollout11 Learning rate2× 10 −6 5× 10 −7 Schedule / warmupConstant / noneConstant / none AdamW betas / epsilon(0.9, 0.999) / 10 −8 (0.9, 0.999) / 10 −8 Weight decay / gradient clipping0 / 1.00.01 / 1.0 Policy clipping0.20.2 Reference-policy KLNo reference0.02 Trainable modulesProjector and languageLanguage Precision / shardingBF16 / ZeRO-1BF16 / ZeRO-2 Seed2026090220260812 GroundAnything maintains FP32 optimizer states. Its rollout and gradient passes use the same causal policy; the inference implementations described below do not alter this training factorization. A.10 Task Rewards The rewards below evaluate complete structured responses. We writenandmfor the numbers of predictions and reference instances, respectively, andclip [0,1] for clipping to the unit interval. Content scores below assume nonempty references; missing predictions receive zero content agreement. A.10.1 Box Grounding Set completeness. For each reference boxg j , select the predicted box with maximum IoU before checking its label: i ∗ (j) = arg max i IoU(b i ,g j ), s j = IoU(b i ∗ (j) ,g j )1[ℓ i ∗ (j) = ℓ j ].(12) With S = P j s j , soft precision and recall are P = S/n and R = S/m, and R set = 2PR P + R + 10 −8 .(13) This GT-wise matching permits a prediction to support multiple references. Consequently,R set is a complete- ness proxy rather than a bounded one-to-one F1 metric. It is combined with a stricter reward that penalizes duplicate and oversized predictions. Strict localization. Let ̄ Faverage localization F1 at IoU thresholds 0.50, 0.75, and 0.95. The strict reward is R strict = clip [0,1] 0.10R fmt + 0.10R count + 0.50 ̄ F + 0.25R IoU + 0.05R sort − 0.07P big − 0.03P dup . (14) The auxiliary terms score protocol validity, instance-count agreement, matched IoU, and canonical ordering. Penalties discourage unsupported large boxes and duplicate predictions. The set and strict rewards are normalized separately before combination, rather than treating their raw weighted sum as a single reward. 28 A.10.2 OCR Text–geometry matching. OCR uses one-to-one Hungarian matching. The main text key applies Unicode NFKC normalization and case folding and retains alphanumeric characters; symbol-only strings retain codepoint-based keys. LetE ij be one minus normalized Levenshtein distance between the resulting keys, and I ij the box IoU. For an affinity matrix A, define F (A) = 2 n + m max M X (i,j)∈M A ij ,(15) whereMis a one-to-one assignment. Hard affinity requires equal text keys andI ij ≥ τ. Write its score asH τ and its average over τ ∈0.50, 0.55,..., 0.95 as ̄ H. The soft affinity and single-reference score are A soft ij = p I ij E ij 1[I ij ≥ 0.10, E ij ≥ 0.20],(16) V (P,T ) = clip [0,1] 0.45 ̄ H + 0.35F (A soft ) + 0.10H 0.50 + 0.10C ,(17) with count agreement C = min(n,m)/ max(n,m). Word/line consistency. LetV hi andV lo be the higher and lower scores against complete word- and line- level references. For a region with alternative granularities, local agreement selects the better complete representation, not independently chosen fragments. Singleton consensus uses the maximum text and box similarities among the reference alternatives, followed by one-to-one assignment, with score 0.60 ̄ H ∗ + 0.40S ∗ . Aggregating local agreements gives Γ, with lower weights for text-conflict regions. The global and local terms are combined as D = αV hi + (1− α)V lo , Q g = βD + (0.90− β)Γ,(18) with (α,β) = (0.90, 0.35) for conflicting granularities and (0.75, 0.50) otherwise. Define m g = max(|T word |,|T line |, 1), C g = min(n,m g ) m g . The reward is R g = clip [0,1] (Q g + 0.10f g C g )f 2 g − 0.05P dup − 0.10P over − 0.05P invalid .(19) Complementary-reference consistency. For complex text, each reference score additionally preserves surface- form accuracy:W= 0.85V+ 0.15U, whereUis exact-string F1 after NFKC and case folding, retaining punctuation and requiring IoU at least 0.50. Two primary references and a complementary reference yield W text = 0.82(0.70 max(W a ,W b ) + 0.30 min(W a ,W b )) + 0.18W c .(20) When auxiliary geometry is available,Q c = 0.95W text + 0.05G aux ; otherwiseQ c =W text . The geometry-only score isG aux = 0.60 ̄ H geo + 0.30S geo + 0.10C geo , excluding transcription agreement. Letm c be the rounded median reference count and C c = min(n,m c )/m c . Then R c = clip [0,1] (0.90Q c + 0.10f c C c )f 2 c − 0.05P dup − 0.10P over − 0.05P invalid − 0.08P large . (21) Format gating and penalties. The parse grades in Table 14 gate the full content score quadratically. Penalties measure duplicated, excessive, or invalid predictions;P large additionally penalizes boxes covering over 80% of the image without geometric reference support. The annotation structure selectsR g orR c , which enters GRPO as one scalar reward. 29 Table 14 OCR format grades used for squared reward gating. BranchParse statusGrade Word/lineComplete, valid structure with no invalid instances1.00 Incomplete structure with reliably parsed instances0.60 Only some valid instances can be recovered0.30 No reliable parse0.00 ComplementaryStrictly valid structure with no invalid content1.00 referencesComplete main structure with extraneous content0.45 Incomplete structure with parseable instances0.35 Recoverable instances without a compliant overall structure0.15 No reliable parse0.00 A.10.3 Pointing Point predictions are matched one-to-one within each label. For distancedon the normalized 0–999 coordinate grid, similarity is s(d) = ( exp[− 1 2 (d/75) 2 ], d≤ 200, 0,d > 200. (22) The matched similarities define soft precisionPand recallR. WithF 2 = 5PR/(4P+R), defined as zero when the denominator vanishes, the reward is R point = 0.55F 2 + 0.25R count + 0.10F 1,label + 0.10R fmt .(23) Count agreement is the smaller-to-larger count ratio when the larger count is positive. Format validity is 1 for a valid response, 0.5 when valid points remain recoverable from a malformed response, and 0 otherwise. B Parallel Inference B.1 Direct Diffusion Decoding Block state and token alignment. Image/query prefill establishes a causal prefix cache and the first anchor token. A physical block contains this known anchor followed byB−1 mask positions. Predictions retain the training token shift. Sub-blocks are completed from left to right; every denoising forward still evaluates the full physical block. Completed tokens remain visible to subsequent masked positions, and the prefix cache is reused throughout. Entropy-guided commitment. Following the use of entropy as a reliability signal in diffusion decoding (Liu et al., 2025), we compute p j = softmax(z j ), H j =− X v∈V gen p j (v) logp j (v),(24) over the generatable vocabulary, excluding the mask token. For the still-masked positionsUof the active sub-block, commit all positions inS=j ∈ U:H j ≤ τ. IfSis empty, commit onlyarg min j∈U H j . Candidate sampling and reliability assessment are separate operations. The procedure repeats until the sub-block is complete; committed tokens are neither remasked nor revised. Causal cache construction. Bidirectional hidden states depend on positions to their right and cannot serve directly as causal history. After completing a block, one causal forward recomputes its authoritative KV entries and predicts the next anchor. This pass does not verify, reject, or replace the completed block. Thus, a block requiringDdenoising passes usesD+ 1 NFEs, excluding initial prefill. The number of passes depends on commitment decisions rather than a fixed per-response denoising count. 30 Other commitment policies. Dynamic Decoding uses a confidence threshold with a forced best-position fallback. Static Decoding commits the highest-confidence remaining positions under a prescribed per- pass quota, distributing the remaining positions across the remaining passes. These policies change token commitment, while preserving blockwise generation. They are distinct from AR verification; their confidence scores should not be identified with the unmodified-logit entropy used above. B.2 Optional Self-Speculative Decoding Self-speculative decoding (Wu et al., 2026a) uses GroundAnything’s shared weights for bidirectional drafting and causal verification, as illustrated in Figure 6. Verification accepts only the longest consecutive draft prefix agreeing with the causal predictions. The first disagreement ends acceptance; later coincidental matches are discarded. Rejected suffix states are removed from the cache. Linear schedule. Given a committed seed, a bidirectional pass predicts a candidate block, and a causal pass verifies it in parallel. Accepted draft tokens are committed with the verifier correction at the first mismatch. Each round uses two NFEs and 2Bquery tokens. The seed is counted once and is excluded from draft-acceptance statistics. Quadraticschedule. The fused query containsBgroups ofB+1 tokens, with groupiarranged as [d i ,M,...,M] at overlapping positions [t+i,...,t+i+B]. Row leaders support causal verification; each row’s mask positions form a bidirectional proposal group. All queries read the cached prefix. Compare row predictiona i with d i+1 , starting the accepted count at one for the known leading tokend 0 . At the first mismatch, select that row’s proposal as the next draft. This selection reuses computation but does not verify the selected proposal. After initialization, each round uses one NFE andB(B+ 1) query tokens. The linear/quadratic terminology concerns query-token counts, not total Transformer FLOPs or a fixed number of accepted tokens. These schedules use greedy causal comparisons. Their verification semantics concern the converted model’s causal branch and do not establish equivalence to sampling an arbitrary autoregressive distribution or to an earlier model’s outputs. B.3 Execution Optimizations The implementation progresses from Native PyTorch (Eager) to SGLang (Eager), CUDA Graph replay, and selective FP8 execution. Denoising and causal passes retain their respective attention and cache semantics. FP8 applies to eligible language-model linear operations, with the remaining components kept in BF16. These changes reduce execution overhead or arithmetic cost; quantization can still alter logits and decoding decisions. The corresponding latency results are analyzed in Section D.2. C DLM–MTP Comparison: Experimental Details C.1 Controlled Model Comparisons The drafters share visual preprocessing, the projector, coordinate vocabulary, serialization, and autoregressive backbone initialization. A single frozen causal branch of GroundAnything verifies all proposals. Adaptation matches ordered examples, supervised-token exposure, optimizer updates, effective batch size, and candidate- length sampling. These controls do not imply equal trainable parameter counts or training FLOPs. The primary in-house MTP baseline uses a causal recurrent future-token module with weights shared across prediction horizons. It is distinct from LocateAnything, which uses bidirectional attention within box-aligned MTP blocks and autoregressive fallback in Hybrid Mode (Wang et al., 2026). An additional in-house block- MTP control predicts a fully masked candidate block in one bidirectional pass; it shares our data and token format but is not a reproduction of LocateAnything. The attention ablation compares separately trained causal and bidirectional DLMs with the same architecture, initialization, denoising objective, corruption distribution, training exposure, and refinement schedule. Only candidate-block attention differs during training and inference. All candidate positions can read the committed 31 prefix; the prefix cannot read candidates. This pair isolates attention, whereas comparing block MTP with DLM also changes the objective and refinement procedure. C.2 Draft Acceptance and Token Cost All configurations receive the same bank of verifier-generated prefixes from held-out COCO prompts, with validation and test prefixes separated by image. The deadline comparison uses long-output prefixes and therefore characterizes decode progress on that subset rather than full-dataset request throughput. Candidate lengthKcounts newly proposed tokens, excluding the committed anchor, padding, and verifier correction or bonus. Greedy verification accepts the longest matching prefix, then emits a correction at the first mismatch or a bonus after complete acceptance, subject to response termination. For roundr, letK r ,A r , andC r denote proposed, accepted-draft, and committed-output token counts, and let T r be full-round latency in milliseconds. We report R draft = P r A r P r K r , ̄ A = 1 N r X r A r , ̄ C = 1 N r X r C r ,(25) c A = P r T r P r A r ,c C = P r T r P r C r , ̄ T = 1 N r X r T r ,(26) whereN r >0 is the number of rounds; each ratio requires a positive denominator. Ratios of totals retain the cost of zero-acceptance and malformed-proposal rounds. For a full nonterminal round,C r =A r + 1; terminal rounds use actual emitted counts. Output throughput is 1000/c C , while 1000/c A measures accepted draft tokens per second. Round timing begins after the shared prefix cache is ready and ends after output and cache commitment. It includes drafting, verification, rejected work, candidate selection, scheduling, and cache maintenance; shared visual encoding and prefix prefill are outside this decode-only cost. All methods use the same execution conditions. Acceptance and latency results. AtK= 16, one-step DLM accepts 21.76% of candidates (3.48 tokens per round), compared with 16.81% (2.69 tokens) for causal MTP. This reducesc A from 7.71 to 5.38 ms, a 30.2% reduction at matched candidate length. Across the tested lengths, DLM’s lowest cost is 5.38 ms atK= 16, while MTP’s is 6.58 ms atK= 8. However, MTP is preferable atK= 4: its cost is 7.15 ms versus 9.31 ms for DLM. AtK= 32, DLM’s accepted prefix decreases to 3.30 tokens despite the longer proposal. These results favor selecting candidate length by accepted work and elapsed time rather than nominal parallelism. C.3 Refinement and Fixed-Deadline Decoding The refinement comparison fills allKcandidate positions on the first pass. Before passj ∈ 2,...,S, it remasks the lowest-confidence m j = K(S− j + 1) S (27) positions and predicts them jointly. Both DLM attention variants use this schedule, followed by causal verification. This controlled draft-refinement experiment is distinct from monotone entropy-guided commitment in direct generation. For two refinement depthsSandS ′ with positive mean latency and accepted-token counts, extra refinement lowers accepted-token cost precisely when ̄ T S ′ ̄ T S < ̄ A S ′ ̄ A S .(28) The condition for committed-token cost replaces ̄ Aby ̄ C. Thus, improved draft acceptance alone does not establish a wall-clock benefit. For a deadlineD, candidate length and denoising depth are selected on validation prefixes and fixed for testing. Methods may execute repeated rounds, and only rounds completed by the deadline contribute: ̄ N (D) = 1 N N X i=1 X r C i,r 1[t finish i,r ≤ D].(29) 32 This measures progress at equal elapsed time, counting verifier corrections and bonuses. Refinement and deadline results. AtK= 16, two DLM passes reducec A from 5.38 to 4.70 ms, whereas four and eight passes increase it to 6.84 and 11.83 ms. Cost per committed output token decreases by only 3.8%, because the verifier contributes a correction or bonus token in each full round. At 400 ms, DLM commits 97.1 tokens versus 84.6 for MTP; at 50 ms, MTP leads with 10.6 versus 8.6. Thus, refinement is beneficial over part of the latency range, but additional passes and very short budgets can reverse the advantage. C.4 Structural and Geometric Diagnostics Each drafter receives the same diagnostic windows: category transitions from COCO, crowded objects from Dense200, document regions from DocLayNet, and GUI targets from ScreenSpot-Pro. Candidate length is matched, both DLM variants use two refinement steps, and the bidirectional block-MTP control uses one pass. Diagnostic windows are selected from the common reference serialization before inspecting proposals. Raw format error detects illegal structural transitions, token types, or coordinate arity, using the parser state inherited from the prefix. Ending partway through an otherwise valid tuple at the candidate boundary is not itself a format error. Diagnostics precede grammar filtering, fallback, and verification. The valid-box miss rate is M box = # unmatched complete, syntactically valid proposed boxes # complete, syntactically valid proposed boxes .(30) Matching is one-to-one against relevant remaining references, constrained by category or referring target and IoU at least 0.5. Unmatched duplicates, reversed corners, and zero-area boxes count as geometric misses. The numerator counts unmatched complete, syntactically valid proposed boxes; the denominator counts all complete, syntactically valid proposed boxes. The rate is undefined when this denominator is zero. It measures the fraction of eligible proposals that remain unmatched and is interpreted alongside format errors; it is not a ground-truth miss rate. Verifier agreement, structural validity, and geometric accuracy measure different properties. Exact greedy verification produces the same sequence as the shared causal verifier; a finite deadline can yield different-length prefixes of that sequence. These diagnostics therefore assess draft reliability and rejected computation, rather than improved final localization under a fixed exact verifier. Where bidirectional refinement helps. Relative to causal MTP, bidirectional DLM reduces COCO format errors from 5.01% to 1.80%. Its valid-box miss rate decreases by 10.10 p on Dense200 and 7.52 p on DocLayNet, compared with only 0.46 p on GUI. These differences are consistent with greater benefits for proposals containing several spatially related elements. The retrained DLM pair provides the attention control: bidirectional attention lowers the Dense200 rate from 21.74% to 14.79% at the same refinement depth. Bidirectional block-MTP also improves over causal MTP, so bidirectionality itself is not a uniquely diffusion-based advantage. Its one-pass comparison with two-step DLM changes objective and refinement cost; it does not isolate diffusion training at equal elapsed time. The separate deadline analysis provides the matched-time comparison. Relation to the grounding formulation. Together, the acceptance, cost, and deadline results support Finding 2.2: bidirectional diffusion with limited refinement makes draft generation more efficient than the tested causal MTP baseline over the reported operating range. The attention control supports joint access to visual evidence, while the refinement sweep identifies when an additional denoising pass repays its cost; neither establishes superiority over every MTP design. The geometric diagnostic measures proposal reliability before verification, separately from ScreenSpot accuracy, F1mIoU, and ground-truth recall. Exact verification preserves the shared causal model’s completed output. 33 D Additional Experimental Analysis D.1 Speed Reporting and Decoding Sensitivity Metrics and scope. TPF denotes committed output tokens per model forward; TPS denotes output tokens per second. Forward counts must include the causal cache-construction or verification passes required by the relevant schedule. The physical block sizeBin the decoding sweeps includes the known anchor, whereas candidate lengthKin the DLM–MTP comparison counts newly proposed tokens. The two quantities should not be identified. All quality values in the decoding sweeps are COCO F1mIoU, and their speed–accuracy trade-offs should not be extrapolated to a 30-benchmark quality average. Reference operating points. The AR reference is 49.6 TPS with 63.70 F1mIoU. AtB= 32, throughput is 223.8 TPS for linear self-speculation, 49.9 TPS for quadratic self-speculation, and 127.0 TPS for entropy guidance atτ= 0.8. The corresponding COCO quality is 62.96 for linear self-speculation and 60.72 for entropy guidance. Thus, the two main speedups are 223.8/49.6 = 4.51×and 127.0/49.6 = 2.56×, with quality gaps of 0.74 and 2.98 p. Interpreting the Decoding Sweeps Linear versus quadratic self-speculation. Linear TPF rises from 1.41 atB= 4 to 2.21 atB= 16, then changes little atB= 32 (2.18). Quadratic TPF reaches 3.17 at bothB= 16 andB= 32. AtB= 32, its 45.4% higher TPF nevertheless accompanies much lower TPS than the linear schedule. Fusing verification and proposal reduces forward-call count but increases the number of query tokens, illustrating why TPF alone is not a latency metric. FromB= 16 toB= 32, linear throughput changes from 226.6 to 223.8 TPS, whereas quadratic throughput falls from 88.0 to 49.9 TPS. Entropy threshold. Asτincreases from 0.2 to 1.2, TPF increases from 1.02 to 1.60 and TPS from 107.3 to 148.7. COCO F1mIoU remains near 60 forτ ≤0.8, peaks at 60.72 atτ= 0.8, falls to 54.06 atτ= 1.0, and partly recovers to 56.14 atτ= 1.2. The non-monotonic quality curve supports a speed–accuracy trade-off rather than a universal monotonic accuracy law. Increasingτpermits more uncertain tokens to be committed together; the observed quality drop is consistent with the risk of premature commitment. Block size and decoding mode. Entropy-guided TPF changes only from 1.18 to 1.24 over the tested block sizes, while its best COCO score is 61.58 atB= 16. Linear self-speculation attains its best tested COCO score atB= 32. Consequently, theB= 32 operating point used for the headline comparison is not the maximum-quality choice for every decoding strategy. Block-size selection therefore depends on task quality and output length; the COCO sweep alone does not establish a universal optimum. What exact verification preserves. Self-speculation verifies against the converted model’s causal branch. Exact greedy verification preserves that branch’s outputs, not necessarily the outputs of the separately trained GroundAnything-VLM. The 0.74 p gap to GroundAnything-VLM therefore does not imply that the verifier accepts mismatching tokens. Direct entropy-guided decoding has no AR acceptance test and remains the appropriate main-table setting for assessing diffusion generation itself. D.2 Infrastructure Analysis Figure 15 adds SGLang (Eager), CUDA Graph, and FP8 sequentially to each decoding mode. The final time per output token is 6.59 ms for Dynamic Decoding, 6.76 ms for Static Decoding, 7.15 ms for Entropy-Guided Decoding, and 3.73 ms for Self-Speculative Decoding. Relative to each mode’s Native PyTorch (Eager) implementation, the cumulative speedups are 3.48×, 3.45×, 3.51×, and 4.06×, respectively. The consistent reductions support the claim that parallel decoding requires corresponding execution support to realize practical acceleration. These ratios compare implementations within a decoding mode. They are not multiplied by the separate 4.51×algorithmic comparison, whose denominator is GroundAnything-VLM at a different reported operating 34 point. The figure reports latency rather than a matched quality ablation for every infrastructure stage. In particular, it does not establish that FP8 preserves accuracy; selective quantization can change logits and thus commitment or verification decisions. The implementation-level distinction is described in Section B.3. D.3 Architecture and Training Ablations Coordinate representation. In Figure 16, replacing quantized coordinates with textual coordinates reduces the AR score from 73.03 to 71.60 and the DLM score from 61.66 to 55.98. The respective 1.43 and 5.68 p drops support the compact coordinate vocabulary as a useful part of both variants, with a larger effect in the diffusion setting. The figure’s 0.25×annotation concerns the textual-representation comparison; it should not be identified with the external SEED1.5-VL token-count ratios. Visual encoder. With MoonViT (Kimi-VL), the AR and DLM scores are 72.81 and 61.03; with Qwen3-ViT, they are 71.12 and 60.12. MoonViT-V2 is consistently strongest among these tested choices. These comparisons support the selected encoder within this training recipe; they do not establish a universal ranking across vision architectures. Section D.3.1 analyzes the task-dependent value of multi-level visual injection at the coordinate-token readout; this is a separate design question from the encoder comparison. Training-phase ablation. The ablation labeled “Stage I” in Figure 16 gives 71.74 for GroundAnything-VLM and 45.04 for GroundAnything, drops of 1.29 and 16.62 p. We retain the figure’s ablation label here; it is distinct from the Stage 1–4 pretraining/alignment numbering above. The larger DLM sensitivity indicates that the ablated training phase contributes substantially to useful grounding behavior after conversion. The experiment changes a complete phase and does not isolate individual reward components or optimization choices within it. Comparison scope. The full ablation scores (73.03/61.66) differ from the final 30-benchmark summary (72.42/61.75). We retain the ablation values and interpret only within-comparison changes, without replacing either set of scores or assuming identical aggregation. D.3.1 Multi-level Visual Injection and Coordinate Readout Connection to the encoder choice. GroundAnything uses MoonViT-V2 with an input-level projector and no intermediate visual injection. DeepStack supplies additional visual evidence at intermediate language layers (Meng et al., 2024); Qwen3-VL adds three projected ViT feature levels to its first three language layers and reports fine-grained understanding gains (Bai et al., 2025a). The relevant question here is whether grounding adaptation makes this evidence useful to the coordinate-token readout under both causal and masked contexts. Our encoder ablation supports MoonViT-V2 under the current recipe but does not isolate the injection scheme. Readout sensitivity. Fix model parameters, image, query, and one decoding context: an AR prefix or a DLM block with fixed masks and visible tokens. For vectorized hidden states, writeh ℓ+1 =F ℓ (h ℓ ) +S ℓ u ℓ and q=g(h L ), whereu ℓ contains projected visual features,S ℓ inserts them at visual-token positions, andq ∈ R V contains logits over a fixed vocabulary at one coordinate position. The readoutgincludes final normalization and the output head; I indexes injection layers. With h 0 fixed, changing injected features by δu ℓ gives d = X ℓ∈I B ℓ δu ℓ + O(∥δu∥ 2 2 ), B ℓ = Dg(h L )J L−1 ·J ℓ+1 S ℓ , J j = DF j (h j ). (31) Heredis the exact logit change,δuconcatenates the feature changes, and empty products are identities; all derivatives are evaluated at the unchanged interface. The chain rule gives each path contribution; locally bounded second derivatives give the remainder. This is sensitivity relative to the specified interface, not prediction error relative to an ideal alignment. 35 Task alignment. Letydenote the reference coordinate token andL y (q) =− logp y its cross-entropy loss. Writep=softmax(q) andp ′ =softmax(q+d) for the original and updated token distributions. For the exact change d, L y (q + d)−L y (q) = log V X v=1 p v e d v ! − d y = (p− e y ) ⊤ d + D KL (p∥p ′ ), (32) wheree y is the one-hot target andD KL (p∥p ′ ) = P v p v log(p v /p ′ v ). The first equality follows from log-softmax; expanding the KL divergence gives the second. Additional evidence lowers coordinate loss precisely when its contribution opposing the current loss gradient exceeds the nonnegative KL remainder. Averaging over matched reference coordinate positions and decoding contexts gives the corresponding token-loss comparison. The loss identity is exact; substituting the linearized change from Equation (31) requires controlling its remainder. If additional branches can be zeroed with all other components unchanged, the multi-injection model retains the single-interface model as a special case; path count alone cannot raise optimal task loss. The testable issue is whether finite grounding data and optimization learn useful contributions across causal and masked contexts. A controlled comparison should fix encoder and language-backbone initialization, coordinate vocabulary, data, and training budget, then compare input-only and multi-level interfaces with comparable capacity. Coordinate loss across adaptation budgets and dense/tiny-object localization would test this hypothesis; entropy-based commitment and final box metrics require separate evaluation beyond the fixed-context analysis. Disabling branches after training measures sensitivity only. Thus, the current ablation supports our encoder choice, while attributing its advantage to the absence of DeepStack requires an injection-specific experiment. D.4 Output Token Efficiency and Generation Cost Compact serialization. Table 5 follows the output-length analysis of Jiang et al. (2026). Its SEED1.5-VL row is taken from that work. The GroundAnything variants use the same structured vocabulary and have identical reported output statistics: 7.6 tokens/box on COCO and 5.1 on Dense200. Compared with 148.8 and 74.5 tokens/box for SEED1.5-VL, these are approximately 19.6×and 14.6×lower token counts per box. The ratios reflect serialization length, not matched-hardware latency; differences in detected instance counts and model execution preclude reading them as speedups. Generation cost across object counts. Figure 17 groups outputs by predicted box count and reports average generation time alongside output-token count. Longer coordinate lists increase decoding work for both variants, while GroundAnything reduces generation time across the displayed ranges, with larger absolute savings for longer outputs. The token-efficiency table characterizes serialization length; this figure characterizes generation cost as the number of predicted instances grows. The speed–quality comparisons in Section D.1 complement this output-length analysis. E Comprehensive Grounding Benchmark Results We expand the selected main-text comparisons with complete baseline tables, threshold-specific localization metrics, and task-level analysis. GroundAnything uses Entropy-Guided Decoding throughout these tables; GroundAnything-VLM uses AR decoding. External results and unresolved evaluations retain their original qualifications. Benchmark Suite and Metrics Tasks and datasets. The suite covers common and long-tailed detection on COCO (Lin et al., 2014) and LVIS (Gupta et al., 2019); dense and tiny-object detection on Dense200 (Jiang et al., 2026) and VisDrone (Zhu et al., 2018); referring grounding on RefCOCO, RefCOCO+, and RefCOCOg (Yu et al., 2016; Mao et al., 2016; Nagaraja et al., 2016); and object pointing on COCO, LVIS, Dense200, VisDrone, and RefCOCOg val/test. Spatial pointing uses RefSpatial (Zhou et al., 2026) and RoboSpatial (Song et al., 2025), while GUI grounding uses ScreenSpot-V2 (Wu et al., 2025), ScreenSpot-Pro (Li et al., 2025), and OSWorld-G (Xie et al., 36 2026). OCR covers HierText (Long et al., 2022), ICDAR2015 (Karatzas et al., 2015), TotalText (Ch’Ng & Chan, 2017), and SROIE (Huang et al., 2019); layout grounding uses DocLayNet (Pfitzmann et al., 2022) and M6Doc (Cheng et al., 2023); visual prompting uses FSC147 (Ranjan et al., 2021) and Dense200. Metrics and aggregation. Following the grounding evaluation convention of Jiang et al. (2026), box tasks report recall, precision, and F1 at IoU 0.50 and 0.95, together with their reported aggregates over IoU thresholds from 0.50 to 0.95. F1mIoU denotes the threshold-aggregated F1 score, not mean matched-box IoU. OCR additionally requires transcription agreement under the loose-match evaluation protocol and reports parse-error rates separately. Object pointing uses point-in-mask F1; spatial pointing uses point-in-mask accuracy, ScreenSpot uses action accuracy, and OSWorld-G uses exact accuracy. RefCOCO avg is the unweighted mean of the three family entries; RefSpatial avg averages Location and Placement. Reported benchmark entries are preserved, and missing metrics are not reconstructed from other scores. Reporting conventions and external scores. The notation in Section 4.2 applies throughout. R and P denote recall and precision; lower is better only for parse error. Model-name stars identify external-only rows, while entry-level stars identify individual substitutions or task-interface exceptions. The relevant captions specify their sources. In particular, selected detector, SEED1.5-VL, Molmo, spatial, and GUI references are taken from Jiang et al. (2026); GUI-Owl references follow Wang et al. (2026). Size groups refer to language-backbone configurations; the 7B understanding branch is used for MoT models and total language parameters for MoE models. Qwen3.7-Max, Kimi-K2.6, Kimi-K3, and GPT-6 Astra are grouped separately in the>1T category. Interpreting unsupported or uncertain evaluations. N/A means that the required task interface or a reliable prompt/parser combination was unavailable; it does not establish that a model intrinsically lacks the capability. The † marker identifies uncertain prompt/protocol alignment, including the affected Kimi, DeepSeek, and SenseNova-Vision evaluations. Repeated DeepSeek GUI attempts did not yield a suitable prompt/parser combination. Starred BAGEL and MiMo observations under suspected protocol incompatibility are retained descriptively. Where external scores replace unreproduced local results, only the matching model and available metrics are used: DeepSeek-VL2-Small scores are never substituted for the full model, and missing parse-error rates remain unreported. The following comparisons use the reported compatible entries. E.1 Common and Long-tailed Object Detection COCO. GroundAnything-VLM achieves 63.70 F1mIoU, while direct diffusion retains 60.72, exceeding Rex- Omni by 4.44 p. The threshold breakdown qualifies this result: GroundAnything’s F1 at IoU 0.95 is 22.39, below LocateAnything Hybrid’s 27.61, despite its higher aggregate. Thus, the overall gain does not imply uniformly tighter boxes at the strictest threshold. LVIS. GroundAnything reaches 52.59 F1mIoU, above Rex-Omni (46.74) and LocateAnything Fast (42.90), while GroundAnything-VLM reaches 56.63. This extends competitive grounding beyond common categories, although SenseNova-Vision remains stronger than the DLM variant at 56.12. The AR variant’s 75.46 F1 at IoU 0.50 versus 23.23 at IoU 0.95 also shows that rare-category coverage and very tight localization remain distinct challenges. E.2 Dense and Tiny Object Detection Dense200. GroundAnything-VLM and GroundAnything reach 75.55 and 70.04 F1mIoU, respectively, both above GPT-6 Astra (65.04), Rex-Omni (53.29), and LocateAnything Hybrid (50.07). GroundAnything also scores 26.86 at IoU 0.95 versus 12.64 for GPT-6 Astra, so its advantage is not restricted to coarse overlap. These results directly support precise visual evidence extraction when many neighboring instances must be represented in one response. VisDrone. GroundAnything improves over Rex-Omni from 27.19 to 33.76 F1mIoU, but remains below GroundingDINO (34.47) and SenseNova-Vision (42.35). GroundAnything-VLM scores 40.74. Both variants have low F1 at IoU 0.95 (3.42 and 2.24), identifying precise tiny-object boundaries as a remaining limitation despite the stronger Dense200 results. 37 Table 15 COCO. Complete box-grounding metrics. Model-name stars denote external scores from Table 2 of Jiang et al. (2026). Model Zero-shot IoU 0.50IoU 0.95mIoU RPF1RPF1RPF1 Closed-set Specialized Detectors DINO-R50 * (Zhang et al., 2022)NO62.6076.5068.8017.8025.8021.1050.0062.4055.60 DETR-R50 * (Carion et al., 2020)NO59.6073.9065.9010.6019.0013.6042.9055.3048.30 DyHead-R50 * (Dai et al., 2021)NO58.1076.6066.1011.9020.6015.0044.8060.1051.30 Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)YES79.6083.2381.3724.6628.0726.2561.1363.6560.56 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)YES69.9080.9975.0417.9321.7319.6553.9259.3656.28 LocateAnything Fast (Wang et al., 2026)YES66.1573.3469.5624.7326.9025.7750.8959.2653.06 LocateAnything Hybrid (Wang et al., 2026)YES74.6780.8077.6126.6428.6627.6152.4065.1659.12 LocateAnything Slow NTP (Wang et al., 2026)YES75.0281.8778.3024.9727.1926.0354.7065.0959.37 Qwen2.5-VL-7B (Bai et al., 2025b)UNK57.1862.8759.8912.1613.2312.6744.6749.0546.27 Qwen3-VL-2B (Bai et al., 2025a)UNK63.5067.1465.2721.7722.5722.1642.6444.8642.48 Qwen3-VL-4B (Bai et al., 2025a)UNK66.0570.6468.2723.6924.8724.2744.8747.7646.53 Qwen3-VL-8B (Bai et al., 2025a)UNK65.7069.4867.5423.9125.1124.4944.8047.2946.30 Qwen3.5-4B (Qwen Team, 2026a)UNK65.8667.7566.7922.9423.6723.3048.4045.7147.58 Qwen3.5-9B (Qwen Team, 2026a)UNK73.1768.1370.5623.8324.4824.1558.5046.3051.99 RynnBrain1.1 (Li et al., 2026a)UNK36.4960.5145.5311.5616.5313.6028.7245.3135.15 SenseNova-Vision (Han et al., 2026)UNK79.0780.5479.8027.1930.1928.6163.1355.3757.49 MiMo-VL-7B-SFT (Yue et al., 2025)UNK65.4872.1868.6710.0211.1110.5445.7250.4747.98 MiMo-VL-7B-RL (Yue et al., 2025)UNK65.6474.5269.799.1710.299.6944.4850.2447.18 BAGEL (Deng et al., 2025)UNK65.5775.2070.0611.0413.0111.9446.3047.1545.98 RynnBrain (Dang et al., 2026)UNK21.2749.8929.824.8910.876.7415.4635.3721.51 GroundAnything-VLMYES81.5381.4381.4823.9123.8323.8763.7563.6463.70 GroundAnythingYES81.0270.3875.3326.3819.4522.3963.7054.9160.72 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)UNK78.1979.0078.5925.0225.6825.3561.2662.1961.72 Qwen3-VL-32B (Bai et al., 2025a)UNK76.4777.7977.1223.0523.8423.4459.5060.8460.16 Qwen3.8-27B (Qwen Team, 2026d)UNK77.4879.5178.4823.6024.4424.0159.9761.7260.84 Qwen3.5-35B-A3B (Qwen Team, 2026a)UNK78.3779.6579.0025.2025.9225.5661.4562.7162.07 DeepSeek-VL2-Small-16B (Wu et al., 2024)UNK20.8645.2228.550.851.731.1410.1221.1913.70 DeepSeek-VL2-27B (Wu et al., 2024)UNK34.5884.6549.1012.8928.1217.6728.6067.9740.25 SEED1.5-VL * (Guo et al., 2025)YES65.3078.6071.3012.7016.4014.3046.8056.9051.40 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)UNK78.9882.6280.7624.1925.1524.6660.9764.7262.79 Kimi-K2.6UNK72.2780.1075.9925.0427.1926.0758.2864.3461.16 Kimi-K3 (Team et al., 2026)UNK72.6984.3378.0823.4026.1324.6956.9965.3760.89 GPT-6 AstraUNK82.6577.2279.8126.5025.7426.1164.6161.0362.75 Why strict overlap is difficult for small objects. For equal axis-aligned boxes of widthw >0 and heighth >0 separated only horizontally by δ, with |δ| < w, IoU(δ) = w−|δ| w +|δ| ,IoU≥ η ⇐⇒ |δ|≤ w 1− η 1 + η ,0 < η < 1.(33) The intersection and union areas are (w−|δ|)hand (w+|δ|)h; IoU is zero for|δ| ≥ w. Atη= 0.75, the permissible shift is onlyw/7. Under this translation-only model, the failure rate at a given width equals the probability of exceeding this displacement threshold, a prediction testable from measured offsets. The calculation explains size sensitivity without identifying its architectural cause; size errors and missed instances require separate analysis. 38 Table 16 LVIS. Complete box-grounding metrics. The starred SEED1.5-VL scores are from Table 3 of Jiang et al. (2026). Model Zero-shot IoU 0.50IoU 0.95mIoU RPF1RPF1RPF1 Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)YES52.6482.6164.3023.3426.5524.8444.3059.5852.61 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)YES58.5073.1164.9917.7620.0418.8344.0246.5846.74 LocateAnything Fast (Wang et al., 2026)YES48.0661.5853.9820.3725.1222.5038.4348.5542.90 LocateAnything Hybrid (Wang et al., 2026)YES55.9871.9862.9822.4327.7224.7944.3056.2549.56 LocateAnything Slow NTP (Wang et al., 2026)YES57.9875.0365.4120.9626.0323.2244.9557.4650.44 Qwen2.5-VL-7B (Bai et al., 2025b)UNK49.5266.5256.788.169.878.9333.3743.6037.80 Qwen3-VL-2B (Bai et al., 2025a)UNK55.2068.9361.3016.5019.1217.7140.6649.7144.73 Qwen3-VL-4B (Bai et al., 2025a)UNK59.6076.1666.8718.7122.2520.3244.8456.1549.86 Qwen3-VL-8B (Bai et al., 2025a)UNK59.4774.3966.1018.0021.1919.4644.2554.4948.84 Qwen3.5-4B (Qwen Team, 2026a)UNK58.1970.4163.7217.8720.5219.1042.6550.9546.43 Qwen3.5-9B (Qwen Team, 2026a)UNK60.8672.0365.9819.3322.0020.5845.2853.0748.87 RynnBrain1.1 (Li et al., 2026a)UNK26.8652.2735.488.7613.5610.6420.1836.6426.01 SenseNova-Vision (Han et al., 2026)UNK64.5075.9769.7627.0930.9428.8852.0860.8456.12 MiMo-VL-7B-SFT (Yue et al., 2025)UNK42.8860.7850.286.268.507.2128.3939.8933.17 MiMo-VL-7B-RL (Yue et al., 2025)UNK43.5061.8751.095.557.526.3827.5338.7432.18 BAGEL (Deng et al., 2025)UNK34.6851.4741.448.7411.589.9624.8835.7629.34 RynnBrain (Dang et al., 2026)UNK15.0643.2922.353.248.904.7510.3428.9815.25 GroundAnything-VLMYES72.9178.1875.4622.7523.7223.2354.9458.4256.63 GroundAnythingYES69.9867.6868.8127.0119.7822.8453.5054.7352.59 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)UNK62.8975.2568.5219.8722.5421.1247.1555.6751.05 Qwen3-VL-32B (Bai et al., 2025a)UNK61.2172.4466.3618.0120.4419.1545.4153.2048.99 Qwen3.8-27B (Qwen Team, 2026d)UNK62.8674.1768.0518.7921.1419.9046.4354.1549.99 Qwen3.5-35B-A3B (Qwen Team, 2026a)UNK62.8774.8268.3320.0422.7521.3147.1455.4150.94 DeepSeek-VL2-Small-16B (Wu et al., 2024)UNK12.8028.4117.650.501.200.716.1013.058.31 DeepSeek-VL2-27B (Wu et al., 2024)UNK26.1370.9638.1910.4024.0514.5221.0654.6230.39 SEED1.5-VL * (Guo et al., 2025)YES54.7082.0065.6015.0028.1019.5038.5059.3046.70 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)UNK63.8679.0670.6520.2723.4921.7648.5459.0453.28 Kimi-K2.6UNK57.7976.4465.8221.5826.9123.9545.2358.9551.19 Kimi-K3 (Team et al., 2026)UNK57.1776.6665.5017.4621.5419.2942.0855.1447.73 GPT-6 AstraUNK71.3573.8572.5823.4724.6024.0254.2355.7354.97 E.3 Referring Object Detection RefCOCOg val/test. On RefCOCOg, GroundAnything reaches 91.61 F1mIoU on validation and 91.10 on test, versus 84.37 and 83.56 for its AR counterpart. LocateAnything Hybrid scores 76.43 and 77.67. The reported gains persist at IoU 0.95, where GroundAnything scores 88.63/87.15. Within this evaluation, the improvement therefore reflects precise language-conditioned localization rather than only successful target identification. RefCOCO family. GroundAnything scores 88.49, 83.06, and 84.43 on RefCOCO, RefCOCOg, and RefCOCO+, giving an 85.33 mean versus 83.43 for GroundAnything-VLM. All three exceed LocateAnything Fast, although the larger DeepSeek-VL2-27B remains stronger on these family entries. These results support broad referring competence without claiming a universal lead over every model size or evaluation split. E.4 Object Pointing Following Jiang et al. (2026), SAM (Kirillov et al., 2023) converts ground-truth boxes to object masks. A predicted point is correct when it lies inside the corresponding mask; F1@Point balances point precision and 39 Table 17 Dense200. Complete box-grounding metrics. Local BAGEL and DeepSeek runs did not reproduce the reference results, so only available external scores are reported for these models. The starred BAGEL scores are taken from Table 1 of Han et al. (2026). No matching external result is available for full DeepSeek-VL2; Small and Tiny checkpoint results are not substituted. The starred DeepSeek-VL2-Small and SEED1.5-VL scores are from Table 4 of Jiang et al. (2026), also reported in Table 2 of Wang et al. (2026). ModelIoU 0.50IoU 0.95mIoU RPF1RPF1RPF1 Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)22.4336.3027.7310.9218.3813.7020.1632.6224.92 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)70.6175.1372.808.669.178.9151.7954.8853.29 LocateAnything Fast (Wang et al., 2026)22.4526.1224.159.1610.509.7819.3022.3120.70 LocateAnything Hybrid (Wang et al., 2026)58.0663.3260.5721.5222.9022.1948.1352.1850.07 LocateAnything Slow NTP (Wang et al., 2026)74.8981.4678.0320.6422.1121.3559.6564.6262.04 Qwen2.5-VL-7B (Bai et al., 2025b)0.500.570.530.000.000.000.220.240.23 Qwen3-VL-2B (Bai et al., 2025a)9.9313.2811.361.271.931.537.189.658.23 Qwen3-VL-4B (Bai et al., 2025a)17.6122.9419.922.683.763.1312.3316.2314.02 Qwen3-VL-8B (Bai et al., 2025a)15.4216.4015.902.262.582.4111.1311.7711.44 Qwen3.5-4B (Qwen Team, 2026a)43.4448.4745.828.209.018.5832.2036.1234.04 Qwen3.5-9B (Qwen Team, 2026a)39.0242.0040.467.378.197.7628.8231.3330.02 RynnBrain1.1 (Li et al., 2026a)0.104.000.190.020.500.040.062.400.12 SenseNova-Vision (Han et al., 2026)79.1886.8482.8324.5225.7225.1065.3171.2068.13 MiMo-VL-7B-SFT (Yue et al., 2025)11.7512.3812.060.140.140.145.625.955.78 MiMo-VL-7B-RL (Yue et al., 2025)13.1014.1413.600.080.070.076.316.886.58 BAGEL * (Deng et al., 2025)–42.40 RynnBrain (Dang et al., 2026)0.031.500.060.000.000.000.020.800.03 GroundAnything-VLM90.3892.9391.6427.7528.4528.0974.5876.5475.55 GroundAnything83.1689.2586.1026.3027.4526.8669.7370.3570.04 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)51.7456.0953.839.9110.7610.3239.1242.5640.77 Qwen3-VL-32B (Bai et al., 2025a)24.9728.3526.562.894.043.3717.7820.6919.12 Qwen3.8-27B (Qwen Team, 2026d)44.3147.4445.828.929.569.2333.2835.4034.30 Qwen3.5-35B-A3B (Qwen Team, 2026a)46.4349.5847.959.2210.009.6035.1837.3636.24 DeepSeek-VL2-Small-16B * (Wu et al., 2024)–16.00–3.90–12.70 DeepSeek-VL2-27B * (Wu et al., 2024)– SEED1.5-VL * (Guo et al., 2025)–76.90–5.30–53.20 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)37.5643.6940.407.208.657.8628.9433.8431.20 Kimi-K2.655.6459.0057.279.219.919.5541.3844.0442.67 Kimi-K3 (Team et al., 2026)65.9578.9971.8810.4011.8811.0947.6156.4051.64 GPT-6 Astra89.0184.6386.7612.9012.3912.6466.5463.6165.04 recall. Referring pointing. On RefCOCOg val/test, GroundAnything-VLM reaches 90.44 and 91.03 F1@Point. GroundAnything scores 85.75/85.73, modestly exceeding Rex-Omni (84.96/85.32) while remaining below its AR counterpart. Unlike referring boxes, referring points do not improve after conversion in these results, demonstrating that the diffusion–AR trade-off depends on the output primitive. 40 Table 18 VisDrone. Complete box-grounding metrics. Local BAGEL and DeepSeek runs did not reproduce the reference results, so only available external scores are reported for these models. The starred BAGEL scores are taken from Table 1 of Han et al. (2026). No matching external result is available for full DeepSeek-VL2; Small and Tiny checkpoint results are not substituted. The starred DeepSeek-VL2-Small and SEED1.5-VL scores are from Table 4 of Jiang et al. (2026), also reported in Table 2 of Wang et al. (2026). ModelIoU 0.50IoU 0.95mIoU RPF1RPF1RPF1 Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)32.3887.0947.212.817.424.0823.6563.5434.47 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)42.4752.3346.891.071.271.1624.7830.1227.19 LocateAnything Fast (Wang et al., 2026)13.5815.4414.450.850.970.909.2710.449.82 LocateAnything Hybrid (Wang et al., 2026)40.6646.9943.592.342.642.4826.7830.6228.57 LocateAnything Slow NTP (Wang et al., 2026)59.2870.0664.222.733.132.9137.3243.8140.31 Qwen2.5-VL-7B (Bai et al., 2025b) 27.6342.8333.590.440.630.5215.4823.5518.67 Qwen3-VL-2B (Bai et al., 2025a)37.1646.1541.171.161.461.2923.0128.2325.36 Qwen3-VL-4B (Bai et al., 2025a)47.8552.3650.001.781.861.8229.9232.4731.14 Qwen3-VL-8B (Bai et al., 2025a)46.6144.8445.711.741.701.7228.8027.8128.29 Qwen3.5-4B (Qwen Team, 2026a)54.9350.7852.772.602.532.5734.9132.7133.77 Qwen3.5-9B (Qwen Team, 2026a)57.5245.0450.522.872.482.6636.7529.6932.84 RynnBrain1.1 (Li et al., 2026a)7.4246.4112.800.382.050.644.8329.238.29 SenseNova-Vision (Han et al., 2026)65.1069.9967.452.863.072.9640.9143.8842.35 MiMo-VL-7B-SFT (Yue et al., 2025)26.4228.7327.530.140.150.1512.6013.8813.21 MiMo-VL-7B-RL (Yue et al., 2025)27.9432.8630.200.280.340.3113.7916.6215.07 BAGEL * (Deng et al., 2025)–23.00 RynnBrain (Dang et al., 2026)8.7832.8513.860.200.750.324.9718.527.84 GroundAnything-VLM57.8169.1662.983.173.713.4237.5544.5240.74 GroundAnything46.9454.6050.482.092.422.2432.5235.1033.76 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)57.3656.3256.832.612.732.6736.4436.4336.44 Qwen3-VL-32B (Bai et al., 2025a)48.9446.1447.501.481.461.4729.5128.2428.86 Qwen3.8-27B (Qwen Team, 2026d)56.4449.0152.462.762.622.6935.4831.4233.33 Qwen3.5-35B-A3B (Qwen Team, 2026a)59.7850.8154.932.922.762.8437.9732.8635.23 DeepSeek-VL2-Small-16B * (Wu et al., 2024)–35.80–1.70–23.30 DeepSeek-VL2-27B * (Wu et al., 2024)– SEED1.5-VL * (Guo et al., 2025)–55.90–0.60–27.40 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)60.2771.4065.362.843.453.1138.2845.5241.59 Kimi-K2.641.3665.7950.791.201.731.4224.4238.7329.95 Kimi-K3 (Team et al., 2026)41.5176.4953.810.981.591.2123.5742.4730.31 GPT-6 Astra64.6065.8365.182.692.802.7436.7037.5837.12 Common and long-tailed pointing. GroundAnything-VLM attains 84.92 on COCO and 79.81 on LVIS; Groun- dAnything decreases to 71.32 and 64.57. For the DLM variant, precision is lower than recall on both datasets: 66.42 versus 77.00 on COCO and 58.67 versus 71.80 on LVIS. This imbalance points to excess or incorrectly localized point predictions as an important source of the gap, rather than missed instances alone. 41 Table 19 RefCOCOg validation and test. Complete recorded F1 metrics at IoU 0.50, 0.95, and mIoU. The starred BAGEL scores are taken from Table 1 of Han et al. (2026), because local runs did not reproduce the reported performance. The starred SEED1.5-VL scores are from Table 5 of Jiang et al. (2026). ModelRefCOCOg valRefCOCOg test F1@.50F1@.95F1mIoUF1@.50F1@.95F1mIoU Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)58.3722.4749.7758.5224.3850.43 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)87.0135.2373.9087.3636.5174.76 LocateAnything Fast (Wang et al., 2026)87.9939.3975.3088.3442.0876.50 LocateAnything Hybrid (Wang et al., 2026)88.5040.4076.4388.9142.6377.67 LocateAnything Slow NTP (Wang et al., 2026)88.1834.8174.9388.6436.9976.46 Qwen2.5-VL-7B (Bai et al., 2025b)78.1915.1061.8678.4916.3162.93 Qwen3-VL-2B (Bai et al., 2025a)85.2532.1371.8085.8733.8872.64 Qwen3-VL-4B (Bai et al., 2025a)88.5336.6175.2788.6936.6175.88 Qwen3-VL-8B (Bai et al., 2025a)88.9235.6175.7989.2637.0376.26 Qwen3.5-4B (Qwen Team, 2026a)89.2434.8975.0389.0035.1475.54 Qwen3.5-9B (Qwen Team, 2026a)89.8936.6376.2089.4236.8976.28 RynnBrain1.1 (Li et al., 2026a)83.2124.3667.6683.7525.1468.24 SenseNova-Vision (Han et al., 2026)89.9443.5878.6989.8544.9179.48 MiMo-VL-7B-SFT (Yue et al., 2025)86.5114.7266.4486.5915.2666.78 MiMo-VL-7B-RL (Yue et al., 2025)88.2812.5664.8987.6912.9264.19 BAGEL * (Deng et al., 2025)–76.40–77.80 RynnBrain (Dang et al., 2026)73.7217.3257.9973.7718.2958.38 GroundAnything-VLM94.1151.4684.3793.7548.3283.56 GroundAnything92.6688.6391.6192.7887.1591.10 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)91.6638.1977.7691.3638.8078.43 Qwen3-VL-32B (Bai et al., 2025a)86.2734.1773.7986.0334.8374.02 Qwen3.8-27B (Qwen Team, 2026d)90.1435.6976.0390.8636.9877.37 Qwen3.5-35B-A3B (Qwen Team, 2026a)90.9238.4677.3290.8839.0978.38 DeepSeek-VL2-Small-16B (Wu et al., 2024)59.860.9026.1861.060.8927.49 DeepSeek-VL2-27B (Wu et al., 2024)92.4655.3682.7392.1958.3384.07 SEED1.5-VL * (Guo et al., 2025)84.7030.9071.9085.2032.1073.20 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)91.5648.4580.5492.3648.7581.71 Kimi-K2.680.1541.1770.3381.1942.3671.87 Kimi-K3 (Team et al., 2026)88.0234.3973.3987.9235.0673.99 GPT-6 Astra92.1335.8974.9889.9340.2578.91 Dense and tiny-object pointing. GroundAnything scores 77.14 on Dense200 and 57.94 on VisDrone, exceeding Rex-Omni on both. GroundAnything-VLM reaches 84.16 and 68.03, while GPT-6 Astra remains stronger on Dense200 at 86.57. The remaining gap to the AR variant confirms that strong dense box grounding does not automatically guarantee equally strong point-set prediction. 42 Table 20 RefCOCO family. The three datasets are reported separately; their F1mIoU arithmetic mean is RefCOCO avg in the main text. ModelRefCOCORefCOCOgRefCOCOplus F1@.50F1@.95F1mIoUF1@.50F1@.95F1mIoUF1@.50F1@.95F1mIoU Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)51.1923.2844.3357.9923.1749.6949.2020.8941.42 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)84.3732.8571.4384.7434.7472.3476.9229.5364.10 LocateAnything Fast (Wang et al., 2026)92.4843.9580.6188.4740.6976.1784.4339.5373.08 LocateAnything Hybrid (Wang et al., 2026)92.7344.3281.3789.4241.4677.7385.8040.5775.08 LocateAnything Slow NTP (Wang et al., 2026)92.4338.4380.2188.6535.6376.0885.2235.6973.90 Qwen2.5-VL-7B (Bai et al., 2025b)82.4217.7166.9375.2815.5660.2873.0416.3959.27 Qwen3-VL-2B (Bai et al., 2025a)88.3326.6572.3985.8925.7369.8181.2124.3466.47 Qwen3-VL-4B (Bai et al., 2025a)92.0932.7577.6388.7932.2574.7386.7630.9473.36 Qwen3-VL-8B (Bai et al., 2025a)91.1129.8775.8388.7428.8273.3386.4329.0271.98 Qwen3.5-4B (Qwen Team, 2026a)90.1129.0774.4688.4226.9871.6184.1327.7969.48 Qwen3.5-9B (Qwen Team, 2026a)91.9735.0978.3189.5333.4775.2787.4533.9074.64 RynnBrain1.1 (Li et al., 2026a)82.6120.8865.8084.2821.4867.5772.4519.1857.52 SenseNova-Vision (Han et al., 2026)90.1245.0779.6689.1844.7378.6984.4041.6874.28 MiMo-VL-7B-SFT (Yue et al., 2025)86.0514.4566.2684.8714.3364.7676.7913.1358.86 MiMo-VL-7B-RL (Yue et al., 2025)90.4414.1568.4587.6913.1164.8084.8313.5564.42 BAGEL (Deng et al., 2025)79.3925.0965.2377.6922.2062.3068.3420.4554.45 RynnBrain (Dang et al., 2026)73.9913.6055.5976.4914.2957.9165.5111.7248.88 GroundAnything-VLM95.5448.0185.5392.6643.7881.6292.6548.0183.15 GroundAnything91.5278.6488.4988.0068.7883.0687.7373.9484.43 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)93.4639.8380.7891.3338.6978.3789.5738.3677.44 Qwen3-VL-32B (Bai et al., 2025a)90.3334.5077.2184.4931.5471.7585.3433.0873.17 Qwen3.8-27B (Qwen Team, 2026d)92.9338.4479.8390.3737.1576.9189.0737.7676.91 Qwen3.5-35B-A3B (Qwen Team, 2026a)92.5736.4479.3090.6234.7776.4987.6134.3875.06 DeepSeek-VL2-Small-16B (Wu et al., 2024)66.951.1030.9564.471.2929.6064.931.2530.58 DeepSeek-VL2-27B (Wu et al., 2024)94.7374.3590.4394.0265.7487.3291.3371.6287.19 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)95.1145.5582.9992.7844.8780.3891.7832.4177.23 Kimi-K2.682.5539.9172.2782.0640.1871.8376.0033.8764.89 Kimi-K3 (Team et al., 2026)88.4635.7875.1288.5734.5174.1481.8633.7769.31 GPT-6 Astra93.6137.1479.4587.2144.2776.1590.4536.3077.84 E.5 Robot and Spatial Pointing Relational localization and placement. GroundAnything retains the AR variant’s 62.34% RefSpatial Unseen accuracy and reaches 69.67% on RoboSpatial Context, above GPT-6 Astra’s 65.69%. Its RefSpatial Loca- tion/Placement scores are 59.00/67.00, below the AR variant’s 67.00/71.00 and GPT-6 Astra’s 86.00/85.86. Parallel grounding therefore transfers to relational and free-space targets, while more difficult spatial reasoning remains a clear source of headroom. The external RoboRefer comparison uses the setting without a depth prior. E.6 OCR HierText and ICDAR2015. GroundAnything obtains 33.31/41.81 F1mIoU, compared with 41.33/42.50 for GroundAnything-VLM. Both variants have zero reported parse errors on these two datasets, so their quality gap cannot be explained by malformed responses alone. The DLM variant exceeds LocateAnything Fast and Hybrid on both datasets but trails Rex-Omni; the external SenseNova-Vision entries provide F1mIoU only and do not support conclusions about its parsing reliability. 43 Table 21 Referring object pointing. Starred BAGEL scores are retained despite unreliable support for the unified pointing protocol. Kimi-K3, both MiMo variants, and both DeepSeek variants are N/A under that protocol. These outcomes do not establish intrinsic pointing capability. The starred Molmo and SEED1.5-VL scores are from Table 7 of Jiang et al. (2026); its Molmo checkpoint is Molmo-7B-D. ModelRefCOCOg valRefCOCOg test F1@PointF1@Point Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)49.3449.97 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)84.9685.32 LocateAnything Fast (Wang et al., 2026)73.8474.94 LocateAnything Hybrid (Wang et al., 2026)75.8976.65 LocateAnything Slow NTP (Wang et al., 2026)77.1777.59 Qwen2.5-VL-7B (Bai et al., 2025b)81.6582.48 Qwen3-VL-2B (Bai et al., 2025a)76.1776.01 Qwen3-VL-4B (Bai et al., 2025a)76.4377.64 Qwen3-VL-8B (Bai et al., 2025a)81.9782.09 Qwen3.5-4B (Qwen Team, 2026a)79.3579.31 Qwen3.5-9B (Qwen Team, 2026a)77.5977.85 RynnBrain1.1 (Li et al., 2026a)74.4274.17 SenseNova-Vision (Han et al., 2026)74.6375.42 MiMo-VL-7B-SFT (Yue et al., 2025)N/A * N/A * MiMo-VL-7B-RL (Yue et al., 2025)N/A * N/A * BAGEL (Deng et al., 2025)55.92 * 54.57 * RynnBrain (Dang et al., 2026)73.3473.67 Molmo-7B * (Deitke et al., 2025)83.7083.60 GroundAnything-VLM90.4491.03 GroundAnything85.7585.73 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)82.7382.88 Qwen3-VL-32B (Bai et al., 2025a)76.5276.23 Qwen3.8-27B (Qwen Team, 2026d)75.8475.86 Qwen3.5-35B-A3B (Qwen Team, 2026a)75.3576.09 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * SEED1.5-VL * (Guo et al., 2025)83.6084.20 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)71.2172.40 Kimi-K2.639.59 † 39.60 † Kimi-K3 (Team et al., 2026)N/A * N/A * GPT-6 Astra87.8084.90 TotalText and SROIE. GroundAnything scores 43.08 on TotalText and 43.64 on SROIE, versus 49.03 and 71.15 for its AR counterpart. The 27.51 p SROIE gap is substantially larger than the 5.95 p TotalText gap, showing that diffusion’s quality cost varies markedly across OCR settings. Although GroundAnything exceeds LocateAnything Hybrid on SROIE, it remains below Rex-Omni and the dedicated PaddleOCRv5 reference. Its parse-error rates are unreported on these datasets and must not be treated as zero. 44 Table 22 Object pointing on COCO and LVIS. Starred BAGEL scores are retained despite unreliable support for the unified pointing protocol. Kimi-K3, both MiMo variants, and both DeepSeek variants are N/A under that protocol. These outcomes do not establish intrinsic pointing capability. The starred Molmo and SEED1.5-VL scores are from Table 7 of Jiang et al. (2026); its Molmo checkpoint is Molmo-7B-D. ModelCOCOLVIS R@PointP@PointF1@PointR@PointP@PointF1@Point Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)68.9271.9770.4144.9171.1555.07 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)77.8181.7779.7463.4678.1470.04 LocateAnything Fast (Wang et al., 2026)68.9974.2671.5355.8170.0562.13 LocateAnything Hybrid (Wang et al., 2026)73.8073.7773.7860.9669.3664.89 LocateAnything Slow NTP (Wang et al., 2026)74.6875.0574.8662.3672.4267.01 Qwen2.5-VL-7B (Bai et al., 2025b)61.2465.7663.4246.4656.8251.12 Qwen3-VL-2B (Bai et al., 2025a)55.6554.9855.3143.7050.1146.69 Qwen3-VL-4B (Bai et al., 2025a)63.1367.6965.3349.4562.1755.08 Qwen3-VL-8B (Bai et al., 2025a)64.9266.7665.8352.2561.4156.46 Qwen3.5-4B (Qwen Team, 2026a)70.1568.8569.5055.6164.7259.82 Qwen3.5-9B (Qwen Team, 2026a)71.8772.5672.2158.9070.0764.00 RynnBrain1.1 (Li et al., 2026a)21.0632.9625.7013.3525.6317.56 SenseNova-Vision (Han et al., 2026)70.8575.2172.9657.2169.2662.66 MiMo-VL-7B-SFT (Yue et al., 2025)N/A * N/A * N/A * N/A * N/A * N/A * MiMo-VL-7B-RL (Yue et al., 2025)N/A * N/A * N/A * N/A * N/A * N/A * BAGEL (Deng et al., 2025)33.57 * 39.16 * 36.15 * 22.73 * 31.65 * 26.46 * RynnBrain (Dang et al., 2026)9.4617.7312.335.3813.737.73 Molmo-7B * (Deitke et al., 2025)–77.30–40.30 GroundAnything-VLM84.7785.0884.9276.5683.3579.81 GroundAnything77.0066.4271.3271.8058.6764.57 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)74.0171.9872.9862.9872.6167.45 Qwen3-VL-32B (Bai et al., 2025a)72.7070.2471.4560.4466.4263.29 Qwen3.8-27B (Qwen Team, 2026d)72.6975.3874.0161.9874.4767.65 Qwen3.5-35B-A3B (Qwen Team, 2026a)72.2171.4271.8161.0570.9965.65 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * SEED1.5-VL * (Guo et al., 2025)–78.20–70.70 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)71.6172.6572.1360.9774.2266.94 Kimi-K2.629.83 † 31.52 † 30.65 † 23.20 † 28.32 † 25.51 † Kimi-K3 (Team et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * GPT-6 Astra86.5678.1482.1774.9279.5077.14 E.7 GUI Grounding ScreenSpot-Pro. GroundAnything improves overall action accuracy from 65.34% for GroundAnything-VLM to 75.96%, with gains in every reported text/icon domain subset. For example, CAD icon accuracy rises from 48.44% to 71.88%, and scientific-interface icon accuracy from 61.82% to 81.82%. These gains support precise parallel localization in complex interfaces, although GPT-6 Astra remains stronger overall at 93.17%. Uncertain prompt/protocol results are not used to infer intrinsic GUI capability. 45 Table 23 Object pointing on Dense200 and VisDrone. Starred BAGEL scores are retained despite unreliable support for the unified pointing protocol. Kimi-K3, both MiMo variants, and both DeepSeek variants are N/A under that protocol. These outcomes do not establish intrinsic pointing capability. The starred Molmo and SEED1.5-VL scores are from Table 7 of Jiang et al. (2026); its Molmo checkpoint is Molmo-7B-D. ModelDense200VisDrone R@PointP@PointF1@PointR@PointP@PointF1@Point Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)22.0065.4532.9327.2071.7639.45 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)75.5977.7676.6647.8156.9251.97 LocateAnything Fast (Wang et al., 2026)64.6366.6565.6355.2257.5556.36 LocateAnything Hybrid (Wang et al., 2026)77.4378.7378.0759.1855.5457.30 LocateAnything Slow NTP (Wang et al., 2026)78.8781.3980.1161.1760.7760.97 Qwen2.5-VL-7B (Bai et al., 2025b)12.1236.4218.1912.8118.2115.04 Qwen3-VL-2B (Bai et al., 2025a)14.0615.9514.9510.6411.9811.27 Qwen3-VL-4B (Bai et al., 2025a)14.1346.9321.7221.2726.2523.50 Qwen3-VL-8B (Bai et al., 2025a)20.6132.9625.3617.6818.5418.10 Qwen3.5-4B (Qwen Team, 2026a)56.8260.3058.5132.7231.5732.13 Qwen3.5-9B (Qwen Team, 2026a)61.0270.3365.3544.4245.0444.73 RynnBrain1.1 (Li et al., 2026a)2.1034.503.957.6846.5513.18 SenseNova-Vision (Han et al., 2026)74.8682.9878.7159.3664.4861.81 MiMo-VL-7B-SFT (Yue et al., 2025)N/A * N/A * N/A * N/A * N/A * N/A * MiMo-VL-7B-RL (Yue et al., 2025)N/A * N/A * N/A * N/A * N/A * N/A * BAGEL (Deng et al., 2025)8.48 * 16.84 * 11.28 * 5.09 * 7.33 * 6.01 * RynnBrain (Dang et al., 2026)0.511.000.685.6434.469.69 Molmo-7B * (Deitke et al., 2025)–33.10–29.20 GroundAnything-VLM81.9386.5184.1661.7775.7068.03 GroundAnything76.5077.7977.1456.2759.7257.94 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)70.5275.6973.0146.6843.9445.27 Qwen3-VL-32B (Bai et al., 2025a)44.0446.1145.0527.8825.7126.75 Qwen3.8-27B (Qwen Team, 2026d)73.9775.1574.5551.0751.2551.16 Qwen3.5-35B-A3B (Qwen Team, 2026a)68.5472.8970.6545.6443.9244.76 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * SEED1.5-VL * (Guo et al., 2025)–72.10–56.70 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)57.5678.1266.2855.0463.6559.04 Kimi-K2.636.14 † 34.27 † 35.18 † 11.22 † 17.37 † 13.63 † Kimi-K3 (Team et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * GPT-6 Astra89.5283.7486.5764.3966.9965.62 ScreenSpot-V2 and OSWorld-G. GroundAnything reaches 95.60% ScreenSpot-V2 accuracy and 81.21% OSWorld-G exact accuracy, improving over GroundAnything-VLM by 0.71 and 6.56 p. The smaller gain on ScreenSpot-V2 reflects a setting where the AR baseline already scores 94.89%; OSWorld-G leaves more room for improvement. GroundAnything does not improve every individual text/icon subset, and GPT-6 Astra still leads both overall metrics. Unreported DLM parse-error rates remain unavailable. 46 Table 24 Robot and spatial pointing. Point-in-mask accuracy is reported for each dataset. The starred RefSpatial baselines follow Table 11 of Jiang et al. (2026); RoboRefer uses the setting without a depth prior. Values retain the precision recorded in our evaluation tables. ModelPoint-in-mask accuracy RefSpatial Location RefSpatial Placement RefSpatial Unseen RoboSpatial Context Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)26.502.004.334.92 Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)51.0052.5037.0159.02 LocateAnything Fast (Wang et al., 2026)53.0019.0016.8815.57 LocateAnything Hybrid (Wang et al., 2026)55.0017.3320.7814.75 LocateAnything Slow NTP (Wang et al., 2026)54.0027.2016.8816.39 Qwen2.5-VL-7B (Bai et al., 2025b)43.0015.5018.1826.23 Qwen3-VL-2B (Bai et al., 2025a)42.0024.0011.6932.79 Qwen3-VL-4B (Bai et al., 2025a)48.0050.0027.2764.75 Qwen3-VL-8B (Bai et al., 2025a)51.0045.0028.5759.02 Qwen3.5-4B (Qwen Team, 2026a)65.0038.0038.9650.82 Qwen3.5-9B (Qwen Team, 2026a)65.0046.8337.0160.66 RynnBrain1.1 (Li et al., 2026a)49.7051.5036.9054.10 SenseNova-Vision (Han et al., 2026)34.516.258.540.82 MiMo-VL-7B-SFT (Yue et al., 2025)1.00 † 8.03 † 4.64 † 4.10 † MiMo-VL-7B-RL (Yue et al., 2025)1.00 † 2.20 † 2.61 † 4.92 † BAGEL (Deng et al., 2025)51.7912.0123.3813.93 RynnBrain (Dang et al., 2026)42.0037.0022.0828.69 Molmo-7B * (Deitke et al., 2025)21.9012.8012.20– RoboRefer * (Zhou et al., 2026)51.0049.0039.00– GroundAnything-VLM67.0071.0062.3472.13 GroundAnything59.0067.0062.3469.67 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)72.0066.0061.0463.93 Qwen3-VL-32B (Bai et al., 2025a)62.0052.0041.5663.93 Qwen3.8-27B (Qwen Team, 2026d)64.0056.0046.7563.93 Qwen3.5-35B-A3B (Qwen Team, 2026a)70.0058.0053.2571.31 DeepSeek-VL2-Small-16B (Wu et al., 2024)2.11 † 1.33 † 0.32 † 5.74 † DeepSeek-VL2-27B (Wu et al., 2024)3.75 † 0.66 † 0.72 † 5.74 † SpaceLLaVA * 5.804.304.00– RoboPoint * (Yuan et al., 2024)22.909.308.40– Molmo-72B * (Deitke et al., 2025)45.8014.7021.20– Gemini-2.5-Pro * (Comanici et al., 2025)47.0024.2027.10– Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)71.5066.0057.1469.67 Kimi-K2.654.0928.5741.5630.33 Kimi-K3 (Team et al., 2026)67.8550.0054.9854.92 GPT-6 Astra86.0085.8681.9365.69 47 Table 25 OCR on HierText and ICDAR2015. Each dataset reports four loose-match F1 measures and parse-error rate. GroundingDINO is N/A because OCR is unsupported. Kimi-K3 and both DeepSeek variants are N/A because their outputs do not satisfy the evaluation protocol; this does not establish a lack of OCR capability. SenseNova-Vision uses the HierText and ICDAR2015 F1mIoU scores of 31.20 and 49.50 reported in Table 1 of Han et al. (2026), because its local evaluation prompts could not be aligned. Other metrics for these datasets are unavailable; TotalText and SROIE use local results. The starred PaddleOCRv5 and SEED1.5-VL scores use the BBOX results in Table 10 of Jiang et al. (2026). ModelHierTextICDAR2015 F1@.50F1@.75F1@.95F1mIoU Parse err. F1@.50F1@.75F1@.95F1mIoU Parse err. Closed-set Specialized Detectors PaddleOCRv5 * (Cui et al., 2025)45.20–3.4030.50–38.20–1.2025.60– Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)54.1635.672.1034.461.9273.3950.070.9645.650.00 LocateAnything Fast (Wang et al., 2026)29.9125.833.1322.5931.4644.5128.320.4026.733.43 LocateAnything Hybrid (Wang et al., 2026)35.5030.433.6126.6519.0445.6129.410.4027.481.21 LocateAnything Slow NTP (Wang et al., 2026)58.3448.655.2842.941.1053.0733.290.6531.810.00 Qwen2.5-VL-7B (Bai et al., 2025b)29.6114.280.5115.490.0055.5023.371.1927.720.00 Qwen3-VL-2B (Bai et al., 2025a)25.0811.380.3312.651.7449.4724.521.9327.430.40 Qwen3-VL-4B (Bai et al., 2025a)41.0623.420.9723.480.9351.5226.371.6728.410.20 Qwen3-VL-8B (Bai et al., 2025a)42.3223.470.9923.890.2954.2029.711.4030.820.00 Qwen3.5-4B (Qwen Team, 2026a)30.4716.500.7617.0012.6541.1616.200.3719.610.00 Qwen3.5-9B (Qwen Team, 2026a)48.6030.101.9029.6313.9356.0226.650.8729.900.00 RynnBrain1.1 (Li et al., 2026a)3.552.040.112.0730.2436.7815.630.2017.816.05 SenseNova-Vision * (Han et al., 2026)–31.20–49.50– MiMo-VL-7B-SFT (Yue et al., 2025)24.7712.820.3613.441.3364.3130.580.4634.331.01 MiMo-VL-7B-RL (Yue et al., 2025)26.1313.010.3213.880.2956.8424.320.3728.797.86 BAGEL (Deng et al., 2025)11.913.300.054.870.7536.2910.410.1715.480.20 RynnBrain (Dang et al., 2026)4.081.920.062.130.6436.0515.150.4717.810.60 GroundAnything-VLM60.1845.453.8241.330.0064.5246.352.0242.500.00 GroundAnything46.9739.474.9733.310.0058.5350.802.0941.810.00 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)36.2725.161.9423.494.1253.8332.091.7531.640.20 Qwen3-VL-32B (Bai et al., 2025a)15.559.080.499.0054.3242.2626.111.6925.4225.40 Qwen3.8-27B (Qwen Team, 2026d)51.5035.062.4632.950.0668.6739.980.9539.720.00 Qwen3.5-35B-A3B (Qwen Team, 2026a)49.8333.222.1631.383.7151.8525.411.2228.043.02 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * SEED1.5-VL * (Guo et al., 2025)27.10–0.2012.00–38.60–0.0018.70– Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)62.9847.453.9342.950.0069.0945.151.9743.070.00 Kimi-K2.645.89 † 24.25 † 1.25 † 25.26 † 0.12 † 56.81 † 31.14 † 1.95 † 32.25 † 0.00 † Kimi-K3 (Team et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * GPT-6 Astra61.3541.893.3839.580.2379.0255.720.8748.870.00 E.8 Layout Grounding Layout grounding predicts labeled document regions and uses the box evaluation convention of Jiang et al. (2026). DocLayNet. GroundAnything-VLM reaches 85.78 F1mIoU, slightly above SenseNova-Vision (85.53) and above the external DocLayout-YOLO reference (81.10). GroundAnything scores 68.38, close to Rex-Omni (68.06) but below LocateAnything Hybrid (77.34). Its 37.78 F1 at IoU 0.95 nevertheless exceeds GPT-6 Astra’s 34.93, illustrating that strict boundary accuracy and overall region detection can rank models differently. 48 Table 26 OCR on TotalText and SROIE. Each dataset reports four loose-match F1 measures and parse-error rate. GroundingDINO is N/A because OCR is unsupported. Kimi-K3 and both DeepSeek variants are N/A because their outputs do not satisfy the evaluation protocol; this does not establish a lack of OCR capability. The starred PaddleOCRv5 and SEED1.5-VL scores use the BBOX results in Table 10 of Jiang et al. (2026). ModelTotalTextSROIE F1@.50F1@.75F1@.95F1mIoU Parse err. F1@.50F1@.75F1@.95F1mIoU Parse err. Closed-set Specialized Detectors PaddleOCRv5 * (Cui et al., 2025)40.20–0.7025.70–77.70–5.6058.60– Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)74.0459.214.4752.350.0072.2847.652.4948.353.61 LocateAnything Fast (Wang et al., 2026)60.9649.645.4844.403.0033.1829.731.8624.8945.83 LocateAnything Hybrid (Wang et al., 2026)62.4051.075.6645.491.3340.5335.842.1230.0534.17 LocateAnything Slow NTP (Wang et al., 2026)70.7953.404.8849.130.3388.2377.854.4965.491.11 Qwen2.5-VL-7B (Bai et al., 2025b)56.6228.472.5431.130.0030.7614.040.3515.580.00 Qwen3-VL-2B (Bai et al., 2025a)60.7629.256.4533.550.0034.8612.510.1515.930.00 Qwen3-VL-4B (Bai et al., 2025a)65.0236.497.7538.350.0068.7341.960.9840.410.00 Qwen3-VL-8B (Bai et al., 2025a)61.4035.447.4036.700.0049.4124.330.4625.880.00 Qwen3.5-4B (Qwen Team, 2026a)54.4028.834.7531.560.0045.9620.700.3023.211.11 Qwen3.5-9B (Qwen Team, 2026a)63.7835.224.5137.261.0051.5220.590.8126.741.94 RynnBrain1.1 (Li et al., 2026a)30.3813.044.2916.9010.677.233.240.063.590.83 SenseNova-Vision (Han et al., 2026)18.3011.211.0211.400.0046.9741.904.2336.260.00 MiMo-VL-7B-SFT (Yue et al., 2025)64.2442.311.7939.760.0030.1313.020.3615.015.28 MiMo-VL-7B-RL (Yue et al., 2025)65.2440.222.1939.550.6739.7715.710.4719.000.28 BAGEL (Deng et al., 2025)51.1919.021.0224.250.0018.634.940.077.741.11 RynnBrain (Dang et al., 2026)35.5817.411.5718.900.007.082.370.023.194.72 GroundAnything-VLM72.6653.354.9549.030.0091.8583.607.0671.150.00 GroundAnything60.4646.015.4143.08–49.3144.853.0243.64– Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)63.5242.495.3841.212.3352.0529.370.6329.370.00 Qwen3-VL-32B (Bai et al., 2025a)44.5128.938.2728.8418.6710.126.310.206.0273.89 Qwen3.8-27B (Qwen Team, 2026d)65.7342.573.9941.570.0061.2738.971.7037.240.28 Qwen3.5-35B-A3B (Qwen Team, 2026a)59.8634.725.3135.951.6765.8336.310.9236.640.28 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * SEED1.5-VL * (Guo et al., 2025)35.00–0.3019.50–51.90–0.8028.10– Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)72.5852.187.9248.640.0057.7244.251.5038.540.00 Kimi-K2.667.66 † 40.90 † 5.52 † 40.99 † 0.00 † 76.94 † 49.43 † 1.92 † 46.83 † 0.00 † Kimi-K3 (Team et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * GPT-6 Astra74.6162.134.9653.550.0070.8162.545.2753.570.00 M6Doc. GroundAnything-VLM obtains 74.76 F1mIoU, exceeding LocateAnything Slow NTP (68.35). Groun- dAnything reaches 56.69, above Rex-Omni (54.95) but below LocateAnything Hybrid (65.94). Together with DocLayNet, these results establish coverage of document-region grounding while identifying a substantial diffusion–AR gap in structured layout extraction. E.9 Visual Prompting FSC147. GroundAnything-VLM and GroundAnything achieve 60.29 and 52.43 F1mIoU, respectively, below SenseNova-Vision’s 62.51. The DLM variant has higher F1 at IoU 0.95 than its AR counterpart (9.37 versus 4.48), despite the lower aggregate. This threshold dependence again cautions against equating a single strict-overlap score with overall grounding quality. LocateAnything’s N/A entries indicate an unavailable visual-prompt interface, not zero accuracy. 49 Table 27 ScreenSpot-Pro. Action accuracy is broken down by domain and target type, followed by overall action accuracy and parse-error rate. GroundingDINO is N/A because GUI grounding is unsupported. BAGEL is N/A because its outputs do not satisfy the unified protocol. The starred JEDI, UI-R1, and UI-TARS scores are from Table 8 of Jiang et al. (2026); GUI-Owl-32B scores are from Table 3 of Wang et al. (2026). ModelDevCreativeCADSciOfficeOSOverall TextIconTextIconTextIconTextIconTextIconTextIcon Action acc. Parse err. Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)61.049.6653.0312.5923.359.3857.6426.3665.5424.5342.0613.4836.75– LocateAnything Fast (Wang et al., 2026)68.1844.8359.6032.8758.3835.9471.5353.6475.1456.6051.4037.0856.04– LocateAnything Hybrid (Wang et al., 2026)70.1344.8359.6036.3658.8837.5071.5351.8272.8858.4958.8840.4557.05– LocateAnything Slow NTP (Wang et al., 2026)70.7848.2861.1139.8660.4139.0675.6951.8274.5854.7257.9440.4558.57– Qwen2.5-VL-7B (Bai et al., 2025b)40.913.4536.369.0918.273.1247.226.3656.5013.2134.5811.2426.57– Qwen3-VL-2B (Bai et al., 2025a)47.407.5929.298.3923.867.8138.8918.1849.1522.6437.3820.2227.77– Qwen3-VL-4B (Bai et al., 2025a)72.7331.7266.6725.1757.8726.5677.0835.4584.7547.1778.5034.8356.74– Qwen3-VL-8B (Bai et al., 2025a)75.3230.3471.2120.9860.9126.5676.3940.0083.6237.7473.8333.7156.86– Qwen3.5-4B (Qwen Team, 2026a)79.2242.0767.6825.1765.9935.9474.3134.5582.4950.9469.1643.8259.27– Qwen3.5-9B (Qwen Team, 2026a)70.7837.9364.1430.7740.1028.1272.9233.6474.5849.0669.1638.2053.13– RynnBrain1.1 (Li et al., 2026a)52.6015.8645.9612.5921.3210.9452.0821.8260.4520.7547.6620.2234.660.38 SenseNova-Vision (Han et al., 2026)N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † MiMo-VL-7B-SFT (Yue et al., 2025)28.574.8331.824.2019.297.8152.7812.7340.1120.7524.306.7423.216.83 MiMo-VL-7B-RL (Yue et al., 2025)27.922.0738.384.2027.419.3857.6417.2753.1124.5329.917.8727.589.30 BAGEL (Deng et al., 2025)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * RynnBrain (Dang et al., 2026)44.817.5935.355.5915.237.8140.2813.6454.8018.8742.0611.2427.072.47 JEDI * (Xie et al., 2026)61.0013.8053.508.4027.409.4054.2018.2064.4032.1038.309.0036.10– UI-R1 * (Lu et al., 2026)22.704.1027.303.5011.206.3042.4011.8032.2011.3013.104.5017.80– UI-TARS * (Qin et al., 2025)47.404.1042.906.3017.804.7056.9017.3050.3017.0021.505.6027.70– GroundAnything-VLM75.9755.8672.7358.0455.8448.4486.1161.8274.5862.2662.6248.3165.340.06 GroundAnything77.9270.3485.8666.4371.5771.8890.9781.8283.6266.0464.4960.6775.96– Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)87.0153.7976.7744.7675.1346.8881.2548.1887.5764.1579.4457.3069.64– Qwen3-VL-32B (Bai et al., 2025a)73.3820.0076.2624.4859.3934.3886.1130.9183.6241.5160.7522.4755.660.13 Qwen3.8-27B (Qwen Team, 2026d)85.7164.8345.4541.9650.7614.0667.3636.3687.5756.6074.7747.1958.760.76 Qwen3.5-35B-A3B (Qwen Team, 2026a)82.4744.8356.5727.9754.8214.0653.4729.0961.5841.5153.2720.2249.080.38 DeepSeek-VL2-Small-16B (Wu et al., 2024)0.00 † 0.00 † 0.00 † 0.00 † 0.00 † 0.00 † 0.00 † 0.00 † 0.00 † 1.89 † 0.93 † 0.00 † 0.13 † 25.62 † DeepSeek-VL2-27B (Wu et al., 2024)0.00 † 0.00 † 0.00 † 0.00 † 0.51 † 0.00 † 0.00 † 0.00 † 0.00 † 0.00 † 0.00 † 0.00 † 0.06 † 34.28 † GUI-Owl * (Ye et al., 2025)84.4039.3065.2018.2062.4028.1082.6039.1081.4039.6070.1036.0058.00– Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)79.2237.5062.6334.2746.1931.2577.0831.8278.5347.1761.6838.2055.0610.57 Kimi-K2.6–6.07 † 8.22 † Kimi-K3 (Team et al., 2026)29.87 † 20.00 † 43.94 † 25.17 † 25.38 † 3.12 † 29.86 † 12.73 † 27.12 † 13.21 † 28.97 † 8.99 † 25.36 † 3.73 † GPT-6 Astra96.1084.8395.4589.5195.4385.9494.4487.2798.8796.2397.2089.8993.170.00 Dense200 with exemplars. GroundAnything-VLM scores 75.26 F1mIoU, while GroundAnything reaches 63.52, above Rex-Omni (55.50) and SenseNova-Vision (62.84). GroundAnything’s F1 at IoU 0.95 is 25.97 versus 12.29 for GPT-6 Astra, but its IoU-0.50 recall is lower than the AR variant’s (67.13 versus 88.68). Thus, the DLM model can localize exemplar-matched instances precisely, while recovering the full instance set remains more difficult. E.10 Potential Application Scenarios The following prospective workflows combine changing queries with substantial spatial output; they are opportunities for evaluation rather than demonstrated deployments. High-throughputindustrialinspection. Language or exemplar queries can specify defects, components, assembly checks, and labels across changing product lines. Parallel output is particularly relevant when each image requires many localized results. Embodied and driving data annotation. Batch processing of robot views, manipulation keyframes, and road images can produce candidate object, part, text, and relation-conditioned spatial annotations. Lower decoding 50 Table 28 ScreenSpot-V2 and OSWorld-G. ScreenSpot-V2 includes text/icon results for mobile, desktop, and web environments, overall action accuracy, and parse-error rate; OSWorld-G reports exact accuracy and parse-error rate. GroundingDINO is N/A because GUI grounding is unsupported. BAGEL is N/A because its outputs do not satisfy the unified protocol. The starred ScreenSpot-V2 scores are from Table 8 of Jiang et al. (2026). ModelScreenSpot-V2OSWorld-G Mobile text Mobile icon Desktop text Desktop icon Web text Web icon Action acc. Parse err. Exact acc. Parse err. Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)96.9082.4697.9480.7189.7476.3588.29–46.10– LocateAnything Fast (Wang et al., 2026)94.4881.0493.8186.4388.8983.2588.44–59.93– LocateAnything Hybrid (Wang et al., 2026)96.2183.8992.7889.2988.8986.2189.94–60.46– LocateAnything Slow NTP (Wang et al., 2026)95.8683.4193.3089.2990.6085.2290.02–61.17– Qwen2.5-VL-7B (Bai et al., 2025b)98.2882.9491.7567.8692.7479.3187.34–34.22– Qwen3-VL-2B (Bai et al., 2025a)93.1070.6279.9061.4379.9162.0776.49–33.51– Qwen3-VL-4B (Bai et al., 2025a)97.9387.2096.9187.1494.4486.2192.30–56.91– Qwen3-VL-8B (Bai et al., 2025a)98.6289.5798.4587.8694.8789.1693.71–56.91– Qwen3.5-4B (Qwen Team, 2026a)98.9788.1597.9489.2995.7390.1593.95–54.61– Qwen3.5-9B (Qwen Team, 2026a)96.5588.1597.9493.5787.6178.8290.57–60.99– RynnBrain1.1 (Li et al., 2026a)82.0771.0975.2652.8673.0857.6470.444.2533.334.96 SenseNova-Vision (Han et al., 2026)N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † N/A † MiMo-VL-7B-SFT (Yue et al., 2025)91.0368.2586.6050.0063.6856.6571.548.1834.046.38 MiMo-VL-7B-RL (Yue et al., 2025)90.6967.3085.5749.2978.2162.5674.695.6637.062.30 BAGEL (Deng et al., 2025)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * RynnBrain (Dang et al., 2026)89.3169.1980.9356.4379.9160.1074.695.2735.991.77 JEDI * (Xie et al., 2026)96.6081.5096.9078.6088.5083.7088.60– UI-R1 * (Lu et al., 2026)84.3096.2075.4089.2063.6092.3085.40– UI-TARS * (Qin et al., 2025)95.2079.1090.7068.6087.2078.3084.70– GroundAnything-VLM97.2489.1098.4594.2995.7393.6094.890.0074.650.00 GroundAnything98.6291.0097.9495.0094.8795.0795.60–81.21– Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)98.9792.4298.4592.8696.5889.1695.13–68.44– Qwen3-VL-32B (Bai et al., 2025a)98.6290.0598.4586.4396.5891.6394.340.0860.280.71 Qwen3.8-27B (Qwen Team, 2026d)97.9390.5298.4594.2994.8789.6694.500.2463.480.00 Qwen3.5-35B-A3B (Qwen Team, 2026a)97.5989.1096.3992.1494.0286.7093.001.4262.230.18 DeepSeek-VL2-Small-16B (Wu et al., 2024)2.07 † 0.00 † 4.64 † 1.43 † 0.85 † 0.49 † 1.57 † 41.82 † 0.89 † 11.52 † DeepSeek-VL2-27B (Wu et al., 2024)6.90 † 1.42 † 2.58 † 0.00 † 2.56 † 1.48 † 2.91 † 25.63 † 0.00 † 32.62 † Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)89.3182.9471.1376.4381.6279.8081.1313.2149.2924.47 Kimi-K2.6–52.36 † 7.15 † 10.11 † 7.98 † Kimi-K3 (Team et al., 2026)95.86 † 88.63 † 97.42 † 89.29 † 63.68 † 61.08 † 82.70 † 1.57 † 68.26 † 4.96 † GPT-6 Astra98.9795.2698.9798.5798.2997.0497.880.2486.700.18 cost could expand annotation and review capacity as images, queries, and instances multiply. Dense remote sensing. Image tiles of ports, car parks, airports, and urban areas require locating many queried instances. Open queries and long coordinate lists make this a useful setting for evaluating parallel spatial extraction. Batch OCR and document processing. Receipts, forms, archives, and complex pages require text together with region locations. Long transcriptions and multi-region outputs make this a practical test of decoding savings at matched extraction quality. Medical and microscopy annotation. Candidate boxes or points for cells, nuclei, and repeated structures could assist research counting and expert review. Domain-specific reliability requires separate evaluation before 51 Table 29 DocLayNet. Complete box-grounding metrics. GroundingDINO lacks a compatible document-region interface; it, Kimi-K3, and both DeepSeek variants are N/A. Starred MiMo and BAGEL scores retain observations under suspected output-protocol incompatibility and are not formal capability measurements. The starred DocLayout-YOLO and SEED1.5-VL scores are from Table 9 of Jiang et al. (2026). ModelIoU 0.50IoU 0.95mIoU RPF1RPF1RPF1 Closed-set Specialized Detectors DocLayout-YOLO * (Zhao et al., 2024)–91.20–52.10–81.10 Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)83.4888.8586.0827.1327.9527.5366.2569.9868.06 LocateAnything Fast (Wang et al., 2026)55.1059.2657.1125.0326.5525.7747.7351.1249.37 LocateAnything Hybrid (Wang et al., 2026)87.9590.9089.4038.7239.8239.2676.1478.5777.34 LocateAnything Slow NTP (Wang et al., 2026)91.5594.5893.0439.5640.6440.0978.9881.4580.19 Qwen2.5-VL-7B (Bai et al., 2025b)30.2329.0729.642.522.922.7016.4116.7216.55 Qwen3-VL-2B (Bai et al., 2025a)40.3835.7037.892.802.832.8121.1019.1920.09 Qwen3-VL-4B (Bai et al., 2025a)64.8669.5067.108.408.688.5439.9041.7640.81 Qwen3-VL-8B (Bai et al., 2025a)60.0957.8058.927.137.187.1537.7636.5837.16 Qwen3.5-4B (Qwen Team, 2026a)59.3747.0252.485.044.474.7435.0328.5131.43 Qwen3.5-9B (Qwen Team, 2026a)63.4449.1255.376.966.096.5039.1231.1034.65 RynnBrain1.1 (Li et al., 2026a)5.5024.018.961.526.682.483.7016.216.02 SenseNova-Vision (Han et al., 2026)95.4595.7895.6149.2349.4349.3385.3885.6785.53 MiMo-VL-7B-SFT (Yue et al., 2025)3.37 * 5.01 * 4.03 * 0.17 * 0.24 * 0.20 * 1.21 * 1.99 * 1.50 * MiMo-VL-7B-RL (Yue et al., 2025)5.43 * 8.01 * 6.47 * 0.11 * 0.19 * 0.14 * 1.98 * 3.15 * 2.43 * BAGEL (Deng et al., 2025)10.83 * 10.93 * 10.88 * 0.60 * 0.69 * 0.64 * 4.34 * 4.64 * 4.48 * RynnBrain (Dang et al., 2026)7.1617.7410.200.981.881.294.019.405.62 GroundAnything-VLM96.2996.2296.2644.8044.7544.7785.8385.7385.78 GroundAnything78.7772.7675.6538.0737.5037.7871.2565.7468.38 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)73.8572.1472.9810.4410.1010.2747.1546.1046.62 Qwen3-VL-32B (Bai et al., 2025a)40.0736.2238.054.084.224.1521.4920.3720.91 Qwen3.8-27B (Qwen Team, 2026d)69.2570.2169.738.628.748.6841.9742.4642.21 Qwen3.5-35B-A3B (Qwen Team, 2026a)63.8356.7360.077.406.997.1940.0236.0337.92 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * SEED1.5-VL * (Guo et al., 2025)–54.90–4.30–28.70 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)75.8565.6070.3512.7612.0812.4149.9044.3146.93 Kimi-K2.629.23 † 28.21 † 28.71 † 2.29 † 2.43 † 2.36 † 15.64 † 15.55 † 15.59 † Kimi-K3 (Team et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * GPT-6 Astra97.4595.7296.5835.0434.8334.9378.2476.8677.54 using these annotations in specialist workflows. Sports and video analysis. Sampled frames can be queried for players, balls, officials, and jersey numbers, including appearance or spatial conditions. Cheaper frame-level extraction could support denser sampling; temporal association remains a downstream task. Retail inventory and agricultural counting. Crowded shelves, logistics bins, orchards, and nurseries combine repeated instances with changing appearance or region queries. Exemplar conditioning provides a useful interface when category names alone are insufficient. 52 Table 30 M6Doc. Complete box-grounding metrics. GroundingDINO lacks a compatible document-region interface; it, Kimi-K3, and both DeepSeek variants are N/A. Starred MiMo and BAGEL scores retain observations under suspected output-protocol incompatibility and are not formal capability measurements. The starred SEED1.5-VL scores are from Table 9 of Jiang et al. (2026). ModelIoU 0.50IoU 0.95mIoU RPF1RPF1RPF1 Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026) 73.0978.2775.5918.1618.7818.4653.3656.6454.95 LocateAnything Fast (Wang et al., 2026)59.8261.5460.6718.0618.5018.2846.3947.6947.03 LocateAnything Hybrid (Wang et al., 2026)83.5484.9484.2326.1826.5926.3965.3866.5065.94 LocateAnything Slow NTP (Wang et al., 2026)88.1689.6488.8925.6226.0825.8567.7868.9468.35 Qwen2.5-VL-7B (Bai et al., 2025b)19.2022.9120.892.242.652.4311.8614.2612.95 Qwen3-VL-2B (Bai et al., 2025a)22.3124.6823.442.242.582.4012.6114.0813.31 Qwen3-VL-4B (Bai et al., 2025a)37.9444.1340.804.885.625.2223.0326.6924.73 Qwen3-VL-8B (Bai et al., 2025a)35.8436.6436.234.735.014.8721.5622.3621.95 Qwen3.5-4B (Qwen Team, 2026a)17.3215.6716.451.471.541.508.448.138.28 Qwen3.5-9B (Qwen Team, 2026a)30.9228.0829.434.084.174.1217.8016.9417.35 RynnBrain1.1 (Li et al., 2026a)4.1126.777.130.896.201.562.6017.514.53 SenseNova-Vision (Han et al., 2026)50.0163.1555.828.8710.609.6632.1239.9935.62 MiMo-VL-7B-SFT (Yue et al., 2025)2.88 * 5.77 * 3.84 * 0.03 * 0.10 * 0.05 * 0.91 * 2.14 * 1.28 * MiMo-VL-7B-RL (Yue et al., 2025)4.63 * 9.36 * 6.19 * 0.07 * 0.12 * 0.09 * 1.53 * 3.24 * 2.07 * BAGEL (Deng et al., 2025)10.62 * 15.75 * 12.68 * 0.40 * 0.62 * 0.48 * 4.94 * 7.61 * 5.99 * RynnBrain (Dang et al., 2026)3.9821.546.720.322.060.552.0010.883.38 GroundAnything-VLM91.9193.1592.5331.5731.9031.7474.2975.2474.76 GroundAnything69.6573.2971.4225.9624.0924.9955.9157.4956.69 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)45.1947.3246.236.696.916.8027.9229.1728.53 Qwen3-VL-32B (Bai et al., 2025a)46.8948.8547.857.027.377.1929.6631.0230.32 Qwen3.8-27B (Qwen Team, 2026d)34.4235.8835.134.194.474.3319.4320.4319.92 Qwen3.5-35B-A3B (Qwen Team, 2026a)44.9046.6645.766.636.996.8128.0729.3228.68 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * SEED1.5-VL * (Guo et al., 2025)–48.00–3.40–28.00 Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)48.2350.8449.509.369.879.6131.5033.1632.31 Kimi-K2.621.96 † 23.87 † 22.87 † 2.80 † 3.16 † 2.97 † 12.89 † 14.16 † 13.50 † Kimi-K3 (Team et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * GPT-6 Astra78.3083.5180.8419.6821.5420.5858.5062.7860.59 Interactive annotation and GUI grounding. Users can revise descriptions and confirm candidate boxes, points, or click locations through repeated queries. Short responses should be assessed by end-to-end interaction latency, including visual processing and prefill. E.11 Qualitative Analysis We examine selected visualizations across grounding tasks, emphasizing the spatial and semantic demands visible in each example. Source images and their displayed annotations are preserved. Panels marked “User- curated candidate” are curated illustrations; panels marked “GT-completed display” include ground-truth completion and are not presented as raw model predictions. These examples provide qualitative context, not additional estimates of accuracy or recall. A panel marked N/A denotes an unavailable comparison. 53 Table 31 FSC147 visual prompting. Complete box-grounding metrics. LocateAnything variants lack a supported visual-prompt interface; both DeepSeek variants have incompatible output protocols. Their entries are N/A. Starred GroundingDINO scores come from an unsupported visual-prompt task interface; starred MiMo and BAGEL scores are retained observations under suspected output-protocol incompatibility. ModelIoU 0.50IoU 0.95mIoU RPF1RPF1RPF1 Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)7.94 * 20.16 * 11.39 * 1.56 * 4.94 * 2.37 * 6.34 * 16.01 * 9.08 * Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)78.9777.9278.449.139.019.0757.5656.7557.15 LocateAnything Fast (Wang et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * LocateAnything Hybrid (Wang et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * LocateAnything Slow NTP (Wang et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Qwen2.5-VL-7B (Bai et al., 2025b)14.7034.2920.570.091.200.167.5417.3910.49 Qwen3-VL-2B (Bai et al., 2025a)22.5539.8528.800.300.730.4211.2119.0714.11 Qwen3-VL-4B (Bai et al., 2025a)18.7483.9830.650.591.600.8611.6051.1618.90 Qwen3-VL-8B (Bai et al., 2025a)19.8074.9331.330.390.850.5311.0940.6717.41 Qwen3.5-4B (Qwen Team, 2026a)27.0641.9832.910.140.210.1711.7018.6214.37 Qwen3.5-9B (Qwen Team, 2026a)29.3760.2939.490.641.210.8415.3131.3020.56 RynnBrain1.1 (Li et al., 2026a)5.2547.709.470.020.140.032.6822.334.78 SenseNova-Vision (Han et al., 2026)78.5688.8883.4010.2510.7810.5159.4665.8962.51 MiMo-VL-7B-SFT (Yue et al., 2025)13.59 * 16.68 * 14.98 * 0.04 * 0.04 * 0.04 * 4.65 * 5.79 * 5.15 * MiMo-VL-7B-RL (Yue et al., 2025)13.58 * 21.88 * 16.76 * 0.01 * 0.02 * 0.01 * 4.51 * 7.23 * 5.56 * BAGEL (Deng et al., 2025)10.94 * 52.39 * 18.09 * 0.31 * 1.06 * 0.48 * 6.62 * 31.67 * 10.95 * RynnBrain (Dang et al., 2026)2.0437.233.870.020.170.031.0017.121.88 GroundAnything-VLM87.4088.5287.954.504.464.4860.0960.4960.29 GroundAnything66.7372.0969.319.249.519.3751.0353.9052.43 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)28.5653.7837.311.603.452.1919.0535.4824.78 Qwen3-VL-32B (Bai et al., 2025a)29.3379.2442.821.251.901.5118.2447.6526.36 Qwen3.8-27B (Qwen Team, 2026d)14.8086.5125.270.602.420.979.8457.5716.80 Qwen3.5-35B-A3B (Qwen Team, 2026a)37.4880.3451.121.442.311.7723.3649.1331.65 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)19.9190.3032.620.821.921.1513.3157.9821.62 Kimi-K2.652.1755.6053.832.753.062.8931.3333.6932.47 Kimi-K3 (Team et al., 2026)67.2976.0371.393.984.164.0740.5644.8342.59 GPT-6 Astra89.8387.1288.488.348.038.1861.8360.2561.04 E.11.1 General Object Grounding Figure 18 Multi-scale object grounding in a cluttered indoor scene. 54 Table 32 Dense200 visual prompting. Complete box-grounding metrics. LocateAnything variants lack a supported visual-prompt interface; both DeepSeek variants have incompatible output protocols. Their entries are N/A. Kimi-K3 is N/A because the required prompt format is unsupported. Starred GroundingDINO scores come from an unsupported visual-prompt task interface; starred MiMo and BAGEL scores are retained observations under suspected output- protocol incompatibility. ModelIoU 0.50IoU 0.95mIoU RPF1RPF1RPF1 Open-set Specialized Detectors GroundingDINO (Liu et al., 2024)2.03 * 9.98 * 3.38 * 1.35 * 5.64 * 2.18 * 1.90 * 9.00 * 3.14 * Vision-Language Models (<10B) Rex-Omni (Jiang et al., 2026)73.0574.8173.9210.8911.1211.0054.9356.0855.50 LocateAnything Fast (Wang et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * LocateAnything Hybrid (Wang et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * LocateAnything Slow NTP (Wang et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Qwen2.5-VL-7B (Bai et al., 2025b)0.7523.661.460.000.000.000.4413.330.86 Qwen3-VL-2B (Bai et al., 2025a)7.7660.8213.760.691.781.005.0833.858.79 Qwen3-VL-4B (Bai et al., 2025a)1.7693.003.450.218.000.411.2762.902.49 Qwen3-VL-8B (Bai et al., 2025a)8.2687.7115.100.993.911.585.7955.0510.45 Qwen3.5-4B (Qwen Team, 2026a)2.8278.695.440.347.710.652.0154.253.87 Qwen3.5-9B (Qwen Team, 2026a)26.6665.6137.914.1910.335.9619.0046.2726.93 RynnBrain1.1 (Li et al., 2026a)1.1662.502.280.074.000.140.7438.251.45 SenseNova-Vision (Han et al., 2026)69.5684.6876.3819.1522.0620.5057.2169.7062.84 MiMo-VL-7B-SFT (Yue et al., 2025)2.87 * 20.40 * 5.03 * 0.01 * 0.00 * 0.00 * 1.08 * 6.38 * 1.83 * MiMo-VL-7B-RL (Yue et al., 2025)2.58 * 14.41 * 4.38 * 0.01 * 0.01 * 0.01 * 0.89 * 5.40 * 1.51 * BAGEL (Deng et al., 2025)1.28 * 56.29 * 2.50 * 0.02 * 1.25 * 0.05 * 0.74 * 29.87 * 1.45 * RynnBrain (Dang et al., 2026)1.2664.502.470.021.000.030.7436.351.44 GroundAnything-VLM88.6892.8590.7228.5130.0129.2473.5977.0075.26 GroundAnything67.1379.3872.7424.9727.0625.9758.1669.9663.52 Vision-Language Models (10B–1T) Qwen3.6-27B (Qwen Team, 2026b)31.9259.8741.645.797.696.6123.8145.8931.33 Qwen3-VL-32B (Bai et al., 2025a)18.9655.7328.302.516.053.5513.8441.1120.69 Qwen3.8-27B (Qwen Team, 2026d)25.0868.0636.654.6114.627.0119.1553.4028.19 Qwen3.5-35B-A3B (Qwen Team, 2026a)41.0971.3552.155.857.146.4329.9551.0437.73 DeepSeek-VL2-Small-16B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * DeepSeek-VL2-27B (Wu et al., 2024)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * Vision-Language Models (>1T) Qwen3.7-Max (Qwen Team, 2026c)28.0572.4640.445.409.516.8922.3755.1631.81 Kimi-K2.643.1451.1946.825.486.846.0929.9035.3932.41 Kimi-K3 (Team et al., 2026)N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * N/A * GPT-6 Astra95.4488.7192.0412.8411.7512.2971.4866.4468.94 Grounding in a cluttered indoor scene. The indoor scene in Figure 18 combines people, furniture, books, containers, and small accessories. Large objects provide scene context, while partially visible items and accessories require finer spatial discrimination. The displayed annotations illustrate the need to retain instance identity across scale and occlusion, rather than replacing a collection of nearby objects with one broad region. In particular, distinguishing an accessory from its wearer requires semantic association as well as localization. The supplied Ours panel is explicitly marked as a GT-completed display and is treated as an illustrative rendering, not as an unedited prediction set. 55 Figure 19 Dense instance grounding of fruit decorations. E.11.2 Dense Object Grounding Dense instances under occlusion. The repeated fruit decorations in Figure 19 vary in apparent size and overlap along a central support and surrounding garlands. The comparison illustrates a distinction between localizing individual fruits and enclosing an entire decorated structure. Several broad boxes in the comparison panels merge multiple instances, whereas the displayed Ours panel retains finer instance granularity. Adjacent fruits with similar color make both duplicate suppression and boundary separation difficult. The curated visualization illustrates these error modes without assigning additional detection scores. Figure 20 Dense grounding of repeated bricks. Repeated boundaries and fine spatial structure. Figure 20 uses a brick wall to probe repeated structures with small appearance differences. The displayed Ours boxes follow individual bricks and preserve the staggered rows, whereas crossing or oversized boxes in the comparison panels mix neighboring units. Success here requires aligning the queried unit with local mortar boundaries while maintaining consistency across a large set of nearly interchangeable instances. The figure is a qualitative illustration of spatial granularity; an unannotated comparison panel alone does not identify the underlying cause of a missing display. E.11.3 Referring Grounding and Complex Visual Configurations Figure 21 Separating overlapping birds with deceptive silhouettes. Complex case: visually deceptive overlap. Figure 21 presents a visually deceptive configuration: two birds overlap so closely that their similar feather textures and aligned body contours can resemble a single two- headed bird. A third bird is only partly visible at the lower image boundary. The Rex-Omni panel encloses the overlapping pair in one large box and marks the foreground bird separately; LocateAnything shows one broad box; the Ours panel distinguishes the two overlapping instances and the truncated foreground instance. 56 The relevant evidence includes two distinct heads and beaks, differently oriented necks, and the relationship between each head and its body contour. A purely salient-region interpretation can merge those cues into one object. Resolving the ambiguity calls for instance-level semantic understanding, occlusion reasoning, and spatial part association, motivating a strong vision–language foundation rather than localization based on isolated texture or outline alone. This example illustrates a demanding capability; a single selected case does not establish a particular internal reasoning mechanism or a universal advantage over all competing models. Figure 22 Grounding a pair specified by a visual relationship. Relational referring beyond visual salience. The query in Figure 22 asks for two people who can make eye contact. The relevant pair occupies a small area near the bottom of a poster dominated by much larger portraits. The displayed Ours regions select that pair, while other panels emphasize the larger faces. The challenge is to interpret a relation between two instances and their orientation, rather than rank people by size or salience. This example concerns the visible relational configuration, not an inference about the depicted people’s identity or mental state. Figure 23 Grounding a spatially specified, partially visible person. Referring at the image boundary. In Figure 23, the phrase “the person on the far right” refers to a partially truncated spectator at the image boundary, rather than the prominent foreground participant. The scene contains many people with similar hats and clothing, making category recognition alone insufficient. The displayed Ours box follows the boundary instance. The example illustrates the need to resolve a relative spatial expression over all visible candidates, including small and partly occluded instances, before predicting the target extent. E.11.4 Referring Point-in-Mask Grounding Figure 24 Point grounding within an occluded person’s visible mask. 57 Referring points inside visible target regions. Figure 24 separates semantic target selection from the geometric requirement that a point lie inside the target’s visible mask. The child is partially hidden by a float, so the enclosing box center can fall on an occluder. The displayed Ours point lies on the visible face, while the comparison points fall on the float or another person. A valid point therefore requires both identifying the intended instance and selecting visible evidence belonging to it, rather than using a generic scene center or box-center heuristic. E.11.5 Dense Point Grounding Figure 25 Dense point grounding of overlapping balloons. Dense pointing with repeated appearances. Figure 25 contains many balloons with repeated colors, touching outlines, partial occlusions, and instances cut by the image boundary. The displayed points span the arch rather than collapsing the group to a few salient locations. The task combines set coverage with one-point- per-instance consistency: missing small balloons and placing repeated points on the same balloon are distinct errors. Marker size is part of the visualization and should not be interpreted as localization uncertainty or predicted object size. E.11.6 Dense Point-in-Mask Grounding Figure 26 Dense point grounding of vehicles in a night scene. Dense point placement in low light. Figure 26 shows vehicles from above under low illumination, including moving vehicles on the road and a compact parked group. Ground-truth masks identify the visible vehicle regions, while point predictions must remain within those regions despite shadows and bright headlights. The displayed Ours points cover both isolated and clustered vehicles. The example illustrates why coverage and interior-point placement must be evaluated together: a point on a light streak or adjacent road surface can be close to a vehicle while missing its visible mask. E.11.7 GUI Grounding Figure 27 GUI target grounding in a multi-window desktop. 58 GUI grounding among overlapping windows. The desktop in Figure 27 contains an editor, a terminal, a plot window, and a property inspector. The marked reference region is the small plot-title area, surrounded by nearby menu and toolbar controls. The displayed Ours point lies in that region, whereas the comparison points are displaced toward neighboring controls. This case requires assigning a target to the correct window and interpreting local interface structure. It illustrates visual target grounding, not successful execution of an unshown action sequence. E.11.8 OCR and Artistic Text Understanding Figure 28 OCR of artistic lettering embedded in illustration. Complex case: artistic letters and decorative texture. The word “DRAW” in Figure 28 is distributed across staggered, tilted letters embedded in brush strokes and dense line art. Decorative contours resemble character strokes, while the letters do not share a conventional horizontal baseline. The Ours panel reads the two spatial groups as “DR” and “AW”; LocateAnything shows “DR” and “AN”, and Rex-Omni uses a single “DRAW” region. This distinction matters: a complete word box and two correct fragments can both be reasonable annotation granularities, so the split alone is not evidence of superior recognition. The informative feature is maintaining the correct letter identity and its spatial support despite stylization, especially the final W. Such reading requires integrating local strokes with word-level context and distinguishing typography from illustration. It motivates strong visual–linguistic representations while also showing why OCR comparisons must specify both transcription and grouping conventions. Complex case: shape-based typography. In Figure 29, the poster combines small conventional words with the oversized, heavily deformed word “FAN” and an illustrated electric fan. Rounded, interlocking glyphs and decorative stars blur the boundary between text and drawing. The Ours panel preserves the large “FAN” region while also identifying smaller words such as “BE”, “YOUR”, and “OWN”. Reading the slogan requires tracking character order across sizes and styles rather than treating all blue shapes as one object. The fan illustration supplies useful semantic context, but a grounded transcription must still be supported by visible letter strokes. This is a demanding vision–language reading example; it does not imply that contextual plausibility can substitute for character evidence or that every small printed line is recovered perfectly. Scene text with perspective and scale variation. Figure 30 includes large lettering along a curved entrance, a hanging business sign, and smaller roadside text. Perspective changes character orientation and spacing, while foliage and uneven contrast complicate localization. The displayed Ours regions retain separate text groups across these scales; comparison panels illustrate merged phrases and transcription changes. The case 59 Figure 29 OCR of highly stylized poster lettering. Figure 30 OCR of outdoor signs with perspective distortion. highlights joint text–geometry consistency: a plausible phrase in a region that spans multiple signs is different from a transcription aligned to the intended word or line. Figure 31 OCR of a low-contrast receipt with aligned fields. Document text and tabular alignment. The faded receipt in Figure 31 contains headings, item descriptions, quantities, unit prices, totals, and tax fields. The displayed Ours regions preserve separate text cells and 60 Figure 32 Exemplar-guided grounding of repeated rounded objects. Figure 33 Exemplar-guided grounding of traffic cones. repeated numerical entries across the document, while comparison panels illustrate merged headers and missed or fragmented lines. Useful OCR must retain both character identity and the layout that associates each value with its row and column. The visualization concerns recognition and localization; it does not establish arithmetic verification or downstream financial correctness. E.11.9 Visual-Prompt Grounding Visual exemplars in highly repetitive scenes. In Figure 32, a boxed visual exemplar specifies the repeated rounded objects without relying on a category name. The target instances form dense rows with mild shape and size variation. The displayed Ours boxes cover instances across the tray, while the Rex-Omni panel contains sparse and repeated edge-aligned proposals. The LocateAnything panel is explicitly marked N/A and is not treated as a scored failure. The example illustrates transferring an exemplar’s visual identity to many instances while avoiding duplicate localization. Visual prompting with perspective variation. Figure 33 uses a local visual reference to specify cones distributed around a running course. Their apparent size changes strongly with depth, and the scene includes both orange and blue cones. The displayed Ours boxes follow instances from the foreground toward the distant gate, illustrating visual category transfer beyond the exact reference crop. The comparison highlights partial coverage and repeated localization. The panel marked N/A denotes an unavailable comparison, not an observed zero score. 61