Paper deep dive
Credit the Right Box: Marginal Contribution Assignment for Structured Visual Perception
Xinheng Han, Jianfei Wang, Yu Chen, Xiang Wang, Shuai Li, Weixing Li, Feng Pan
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multimodal Large Language Models (MLLMs) are increasingly expected to solve structured perception tasks that require visual recognition, language-to-object binding, object cardinality preservation, and precisely localized grounding and segmentation outputs. However, existing group-relative reinforcement learning methods provide only response-level supervision, creating a granularity mismatch for structured multi-object prediction: a single advantage is broadcast to all tokens in a response, without distinguishing individual box contributions. To address this mismatch, we propose MCR-GRPO, a marginal contribution assignment framework that derives box-level credit directly from each sampled response. Specifically, Marginal Contribution Reward (MCR) estimates each predicted box's contribution through a leave-one-out comparison, measuring how the matched set value changes when the box is removed from the response. After within-response normalization, records that improve the set value receive positive credit, while redundant or harmful ones are suppressed. To make marginal attribution stable and informative, we further introduce a Continuous Matched Set Value Evaluator that integrates permutation-invariant matching, count-aware normalization, and graded localization. MCR-GRPO maps normalized box-level marginal advantages to the token spans that generated each box, preserving GRPO's response-level comparison while enabling box-aware optimization of structured multi-object grounding. Experiments across REC, DOD, segmentation, and counting benchmarks show state-of-the-art performance over prior GRPO-based baselines.
Tags
Links
- Source: https://arxiv.org/abs/2608.01055v1
- Canonical: https://arxiv.org/abs/2608.01055v1
Trouble viewing inline? Open PDF directly →
Full Text
76,373 characters extracted from source content.
Expand or collapse full text
Supplementary Material Credit the Right Box: Marginal Contribution Assignment for Structured Visual Perception Xinheng Han1,2, Jianfei Wang2, Yu Chen2, Xiang Wang2 Shuai Li2, Weixing Li1, Feng Pan1 Work done during internship at Amap, Alibaba Group.Project Leader Xinheng Han1,2, Jianfei Wang2, Yu Chen2, Xiang Wang2 Shuai Li2, Weixing Li1, Feng Pan1 Work done during internship at Amap, Alibaba Group.Project Leader Abstract Multimodal Large Language Models (MLLMs) are increasingly expected to solve structured perception tasks that require visual recognition, language-to-object binding, object cardinality preservation, and precisely localized grounding and segmentation outputs. However, existing group-relative reinforcement learning methods provide only response-level supervision, creating a granularity mismatch for structured multi-object prediction: a single advantage is broadcast to all tokens in a response, without distinguishing individual box contributions. To address this mismatch, we propose MCR-GRPO, a marginal contribution assignment framework that derives box-level credit directly from each sampled response. Specifically, Marginal Contribution Reward (MCR) estimates each predicted box’s contribution through a leave-one-out comparison, measuring how the matched set value changes when the box is removed from the response. After within-response normalization, records that improve the set value receive positive credit, while redundant or harmful ones are suppressed. To make marginal attribution stable and informative, we further introduce a Continuous Matched Set Value Evaluator that integrates permutation-invariant matching, count-aware normalization, and graded localization. MCR-GRPO maps normalized box-level marginal advantages to the token spans that generated each box, preserving GRPO’s response-level comparison while enabling box-aware optimization of structured multi-object grounding. Experiments across REC, DOD, segmentation, and counting benchmarks show state-of-the-art performance over prior GRPO-based baselines. Introduction Multimodal Large Language Models (MLLMs) (Liu et al. 2023b; Bai et al. 2025a) have made rapid progress in general visual understanding, but many perception problems require more than a fluent textual answer. In grounding, segmentation, and counting tasks, a model cannot merely judge whether the queried objects are present, but must organize its prediction into a set of structured object records, each of which identifies a distinct target instance, preserves its role in the required cardinality, and associates it with a precise spatial output such as a point, box, or mask prompt. This requirement motivates structured visual perception, where MLLMs are expected to produce object-level visual records that jointly align language, instance identity, cardinality, and spatial localization. Recent work has explored Group Relative Policy Optimization (GRPO) (Shao et al. 2024), a reinforcement learning method, to enhance the structured visual perception ability of MLLMs (Bai et al. 2025b; Shen et al. 2025). Representative methods such as VisionReasoner (Liu et al. 2026b) adopt GRPO to train models to generate structured point-and-box outputs, and further combine these outputs with SAM2 (Ravi et al. 2025) to unify detection (Yu et al. 2026), segmentation (Huang et al. 2026), and counting (Deitke et al. 2025) within a single perception framework. However, GRPO optimizes sampled responses with a single response-level advantage, which is effective for selecting better answers at the response level but is too coarse for structured perception. A structured perception response is not a monolithic decision, since one response may contain a correctly localized object, a duplicated prediction, a missing instance, and a near-miss box. Broadcasting one response-level advantage to all generated tokens therefore creates a mismatch between the reward signal and the object record structure of the actual prediction, because the model is not told which individual box or object record improves the structured output and which one degrades it. As a result, correct object spans and harmful object spans can be reinforced together. Figure 1: Three Paradigms of Supervision for Structured Visual Perception. Response supervision scores the whole structured answer with a single advantage. Trajectory supervision revises entire generated paths, recovering feedback at the cost of extra rollouts, and still updates at sequence granularity. Object-record Supervision (MCR-GRPO) supervises at the object-record level, providing native response-internal credit assignment without auxiliary trajectories or extra rollouts. To resolve this mismatch, the core question is whether object-local credit can be derived directly from the original structured response, without introducing additional rollouts, auxiliary trajectories, or a separate box-level objective. We answer this question with MCR-GRPO, a GRPO framework that assigns credit to individual boxes by reusing the matched set value of each response. Its core component, Marginal Contribution Reward (MCR), estimates each prediction’s contribution through a leave-one-out comparison. Given a structured response, MCR treats it as a set of object records, evaluates the matched set value of the whole set, removes one predicted record at a time, and measures how the matched set value changes. After within-response normalization, this produces signed, box-specific credit, where beneficial boxes receive positive credit, while redundant, distracting, or harmful boxes receive negative credit. To make these marginal contributions smoother and more informative, we further introduce a Continuous Matched Set Value Evaluator that respects the unordered and count-sensitive nature of structured multi-object prediction. It computes pairwise scores between predicted and ground-truth objects, solves a Hungarian matching, and normalizes the matched score by the larger set size. It also incorporates continuous IoU scores rather than binarized localization labels, reducing the instability caused by borderline boxes near a hard threshold. The resulting value provides a permutation-invariant, count-aware, and localization-sensitive substrate for measuring marginal contribution. These marginal credits are normalized within each response and mapped to the token spans that generated the corresponding boxes. In this way, MCR-GRPO preserves GRPO’s response-level comparison while enabling box-aware optimization for unified structured multi-object grounding. The main contributions are summarized as follows: • We propose MCR-GRPO, a GRPO framework for box-level credit assignment, which derives response-internal credit without auxiliary trajectories while preserving response-level comparison. • We introduce Marginal Contribution Reward (MCR) to estimate signed box-specific contributions by measuring the leave-one-out change in set value, capturing duplication and substitution effects that independent pairwise scoring cannot express. • We design a Continuous Matched Set Value Evaluator, providing a smooth substrate that makes leave-one-out marginal attribution stable and informative. • Experiments on REC, DOD, segmentation, and counting demonstrate that MCR-GRPO achieves state-of-the-art performance over prior methods in unified structured visual perception, while preserving general VQA ability. Figure 2: Overview of MCR-GRPO. Given an image-query pair, the policy samples structured responses and parses each response into box-indexed object records with their generated token spans. MCR-GRPO matches the predicted records to ground-truth objects, computes the matched set value, and estimates each object-record’s leave-one-out contribution to box-level MCR advantages. The advantages are mapped back to the token spans and combined with the response-level GRPO advantage, reinforcing helpful records while suppressing harmful or redundant ones. Related Work Structured Visual Perception with MLLMs. Structured visual perception has progressed from language-conditioned detection, grounding, and segmentation to MLLMs that directly express boxes, regions, points, or masks in a language interface (Li et al. 2022; Liu et al. 2025a; Peng et al. 2024; Ma et al. 2024). Pixel-level and promptable MLLMs further connect language reasoning to dense masks through special tokens, visual prompts, and segmentation decoders (Lai et al. 2024; Wang et al. 2025b; Qian et al. 2025; Kirillov et al. 2023; Ravi et al. 2025). Recent unified models extend this interface across grounding, segmentation, OCR, counting, and VQA-style perception by sharing backbones, output formats, or task mixtures (Xiao et al. 2024; Bai et al. 2025a; Li et al. 2024). The remaining bottleneck is the structure of the prediction itself: a response is an unordered, count-sensitive object set where correct instances, duplicates, omissions, and near-threshold boxes may coexist (Liu et al. 2023a; Xie et al. 2023). Thus, beyond enabling MLLMs to emit coordinates, structured visual perception requires training signals that can identify which object records preserve instance identity, cardinality, and localization quality. Learning Structured Perception. Supervised fine-tuning (SFT) teaches output formats and visual tool interfaces, but strong structured-perception systems still rely on large curated grounding, mask, and instruction mixtures (Liu et al. 2023b; Ren et al. 2024; Ma et al. 2024). Reinforcement fine-tuning (RFT) reduces this dependence by replacing dense annotations with verifiable rewards, and recent visual RL methods use IoU, point, mask, format, or external-segmenter feedback for grounding, detection, segmentation, OCR, and counting (Liu et al. 2025b; Shen et al. 2025; Bai et al. 2025b; Yu et al. 2026; Liu et al. 2026a; Huang et al. 2026; You and Wu 2025). VisionReasoner advances toward unified visual perception, but it still follows GRPO’s response-level supervision: one scalar reward and one relative advantage update the whole sampled response, even when the reward is computed by multi-object matching (Liu et al. 2026b). This granularity mismatch can reinforce both correct and harmful boxes within the same structured answer. Trajectory feedback provides another route by scoring intermediate steps, revised candidates, or generated paths (Lightman et al. 2024; Wang et al. 2024). Group Revision adapts this idea to object grounding by revising an initial response and using revision gains for reward shaping and advantage scaling (Liu et al. 2026c). It recovers supervision on hard cases, but its feedback unit is still a revised trajectory rather than an individual box, and the extra revision prompts and rollouts increase training cost. MCR-GRPO instead keeps the original GRPO responses and derives response-internal box-level credit from each predicted box’s leave-one-out marginal contribution to the matched set value, without auxiliary trajectories, extra rollouts, or a separate box-level objective. Method Problem Formulation Given an image x∈x and a language query q∈q , the policy model πθ _θ is optimized to generate a structured visual perception response. The ground truth is an unordered object set G=gjj=1NG=\g_j\_j=1^N, where N is the number of target objects. Each ground-truth object gjg_j contains spatial supervision, such as a bounding box bj⋆∈ℝ4b_j ^4 and a mask mj⋆m_j . During RL training, multiple responses are sampled for the same image-query pair. Following GRPO, the old policy πθold _ _old samples a group of responses ymm=1M∼πθold(⋅∣x,q)\y_m\_m=1^M _ _old(· x,q), where M is the group size. For each sampled response, we omit the response index when discussing its internal predicted set. A deterministic parser extracts the predicted object set S=dii=1KS=\d_i\_i=1^K, where K is the number of predicted object records. Each predicted record did_i contains a predicted bounding box b^i∈ℝ4 b_i ^4, a predicted point p^i∈ℝ2 p_i ^2, and the generated token span τi _i that produces this record. A conventional response-level reward RmseqR^seq_m is assigned to each sampled response ymy_m, and the group-normalized advantage is computed as: Amseq=Rmseq−μx,qσx,q+ϵ,A^seq_m= R^seq_m- _x,q _x,q+ε, (1) where μx,q _x,q and σx,q _x,q are the mean and standard deviation of the response rewards within the sampled group. This advantage compares different responses, but it is applied uniformly to all valid tokens in the same response. Such response-level optimization cannot distinguish the object records inside one structured answer, even though different records may be correct, duplicated, mislocalized, or harmful. Therefore, our goal is to augment response-level GRPO with response-internal credit assignment over the predicted object set S. Continuous Matched Set Value Evaluator To measure the quality of a structured prediction, we introduce a continuous matched set value evaluator. Given a predicted object set S=dii=1KS=\d_i\_i=1^K and the ground-truth set G=gjj=1NG=\g_j\_j=1^N, the evaluator produces a scalar value V(S,G)V(S,G) that reflects set-level grounding quality. For each predicted record did_i and ground-truth object gjg_j, we compute a continuous pair score: s(di,gj)=αsiou(b^i,bj⋆) s(d_i,g_j)=α s_iou( b_i,b_j ) +βsbox(b^i,bj⋆) +β s_box( b_i,b_j ) (2) +γspoint(p^i,mj⋆). +γ s_point( p_i,m_j ). where sious_iou measures box overlap, sboxs_box measures coordinate-level localization closeness, and spoints_point measures point quality. The weights α,β,γα,β,γ control the relative importance of these spatial cues. Unlike prior binary reward designs that threshold localization correctness, our evaluator keeps the spatial quality continuous whenever possible. Specifically, the IoU term is directly defined as: siou(b^i,bj⋆)=IoU(b^i,bj⋆)s_iou( b_i,b_j )=IoU( b_i,b_j ). For coordinate-level box quality, we first compute the mean absolute coordinate distance in pixel space: ℓijbox=14‖b^i−bj⋆‖1 ^box_ij= 14 \| b_i-b_j \|_1, and convert it into a truncated exponential score: sbox(b^i,bj⋆)=exp(−ℓijbox/10),ℓijbox≤40,0,otherwise.s_box( b_i,b_j )= cases (- ^box_ij/10),& ^box_ij≤ 40,\\ 0,&otherwise. cases (3) For point quality, we do not use the distance to a reference point as the main criterion. In the downstream SAM-based segmentation setting, a point is useful when it lies on the target object, while a point closer to a reference point is not necessarily better. Therefore, we define the point score using the ground-truth object mask mj⋆m_j and bounding box bj⋆b_j : spoint(p^i,mj⋆)=1.0,p^i∈bj⋆ and p^i∈mj⋆,0.3,p^i∈bj⋆ and p^i∉mj⋆,0,otherwise.s_point( p_i,m_j )= cases1.0,& p_i∈ b_j and p_i∈ m_j ,\\ 0.3,& p_i∈ b_j and p_i∉ m_j ,\\ 0,&otherwise. cases (4) Among the three terms, IoU provides the primary estimate of region-level set quality, while the box-distance and point-validity terms provide auxiliary localization and promptability cues. We therefore set α=2α=2, β=γ=1β=γ=1 in our evaluator. Since both S and G are unordered sets, we use Hungarian matching to compute an optimal one-to-one assignment between predicted records and ground-truth objects: M⋆=argmaxM′∈ℳ(S,G)∑(i,j)∈M′s(di,gj),M = _M (S,G) _(i,j)∈ M s(d_i,g_j), (5) where ℳ(S,G)M(S,G) denotes the set of valid one-to-one matchings. In implementation, this maximum-weight bipartite matching is solved by the Hungarian algorithm. The continuous matched set value is defined as: V(S,G)=1max(K,N)∑(i,j)∈M⋆s(di,gj).V(S,G)= 1 (K,N) _(i,j)∈ M s(d_i,g_j). (6) This evaluator is permutation-invariant since it depends on Hungarian matching rather than object order. It is count-aware because the denominator penalizes extra predictions and missing targets and is continuous because near-correct boxes can receive partial credit instead of being collapsed into a binary failure. This value V(S,G)V(S,G) provides the set-level basis for computing marginal contribution rewards. Marginal Contribution Reward Given the continuous matched set value V(S,G)V(S,G), we estimate response-internal object credit through leave-one-out comparison. The core idea is to measure how the set-level value changes when one predicted object record is removed from the structured prediction. For each predicted record di∈Sd_i∈ S, we compute the leave-one-out value difference as: ΔVi=V(S,G)−V(S∖di,G). V_i=V(S,G)-V(S \d_i\,G). (7) Here, ΔVi V_i is the raw marginal contribution of did_i before normalization. When ΔVi≫0 V_i 0, removing did_i substantially decreases the set value, so did_i is a helpful record that contributes to the structured prediction. When ΔVi≪0 V_i 0, removing did_i improves the set value, so did_i is harmful, often due to duplication, false positives, count inflation, or misleading localization. This can occur only when the response over-predicts (K>N)K>N), whereas forK≤NK≤ N every raw contribution is non-negative and poorly localized records are instead suppressed in relative terms after the normalization. When ΔVi≈0 V_i≈ 0, did_i has little marginal effect under the current evaluator, either because it is weakly matched or because its contribution is redundant with other records. Method Referring Expression Comprehension (Acc@0.5) Avg. DOD (AP) ReasonG RefCOCO RefCOCO+ RefCOCOg D3D^3 Val Test Val TestA TestB Val TestA TestB Val Test Full Pres. Abs. Supervised Fine-Tuning Based Methods PerceptionGPT-7B - - 88.6 92.5 84.6 82.1 88.6 74.2 84.1 85.2 - - - - VistaLLM-7B - - 88.1 91.5 83.0 82.9 89.8 74.8 83.6 84.4 - - - - Elysium-7B - - 89.1 92.1 85.0 82.9 88.9 75.6 82.9 83.6 - - - - Groma-7B - - 89.5 92.1 86.3 83.9 88.9 78.1 86.3 87.0 - 16.0 15.9 16.3 Open-Source MLLM and Reinforcement Fine-Tuning Based Methods Qwen2.5-VL-7B 68.9 59.8 88.8 91.7 81.4 82.3 88.2 69.2 84.7 85.7 80.1 19.6 19.4 20.3 SegZero-7B 69.3 64.6 89.3 91.5 81.9 82.0 87.6 74.7 86.1 86.3 81.3 - - - VisionReasoner-7B 80.1 78.5 88.6 90.6 84.7 83.6 87.9 80.2 86.1 87.5 84.8 22.0 21.2 24.1 GroupRevision-7B 83.7 81.2 89.5 91.6 85.1 84.4 88.3 77.6 88.4 88.3 85.8 - - - MCR-GRPO (ours) 84.0 83.3 90.2 91.6 86.2 85.2 87.8 80.6 89.1 89.9 86.8 23.1 22.5 24.8 Table 1: Comparison with SOTA Methods on REC and DOD Tasks. The best results are highlighted in bold. This leave-one-out formulation captures interactions that independent pairwise scoring cannot. A duplicated prediction may obtain a reasonable local matching score, but it can still reduce the set value by increasing the predicted count. Conversely, a near-correct prediction can receive positive marginal credit if it improves the optimal matching. Thus, ΔVi V_i attributes each record according to its effect on the whole predicted set S, rather than its isolated similarity to a single ground-truth object. A single Hungarian matching costs O(max(K,N)3)O( (K,N)^3). Since MCR computes one full-set value and K leave-one-out values, the per-response matching cost is O((K+1)max(K,N)3)O((K+1) (K,N)^3), which is small compared with VLM rollout generation and policy optimization. The raw sign of ΔVi V_i is regime-dependent. Under over-prediction (K>N)(K>N), an unmatched record provably receives strictly negative raw credit, so duplicate suppression follows from the count-aware normalization rather than a hand-designed penalty. Under K≤NK≤ N every raw contribution is non-negative, so harmful must be defined relative to a reference point. Absolute set quality is already carried by V(S,G)V(S,G) through the response-level reward, and centering makes MCR zero-sum and hence orthogonal to that signal. The z-score is strictly increasing, so it preserves the ranking by marginal contribution, and unmatched or perfectly substitutable records provably occupy the lowest credits. Finally, |A¯imcr|≤K−1 | A_i^mcr |≤ K-1, so the composite advantage keeps the sign of AseqA^seq whenever λmcrK−1<|Aseq| _mcr K-1< |A^seq |. MCR therefore guarantees a strict within-response ordering rather than an absolute sign per box, which is analyzed in Appendix. We then normalize the raw value differences within the same rollout. This normalization is computed only over the predicted object records in S; tokens outside the structured object records, such as reasoning tokens or other non-structural text, do not participate in this normalization. For K>1K>1, we compute: μΔ=1K∑i=1KΔVi,σΔ=1K∑i=1K(ΔVi−μΔ)2, _ = 1K _i=1^K V_i, _ = 1K _i=1^K( V_i- _ )^2, (8) and obtain the normalized object-level MCR credit as: A¯imcr=ΔVi−μΔσΔ+ϵ. A_i^mcr= V_i- _ _ +ε. (9) The normalization is rollout-local, so MCR compares object records inside the same structured response rather than across different sampled responses. For the special case where a rollout contains only one predicted object record, there is no response-internal comparison to perform. We therefore set its normalized MCR credit to zero: A¯1mcr=0 A_1^mcr=0, if K=1K=1. With this convention, the normalized object-level MCR credits within each rollout have zero algebraic sum ∑i=1KA¯imcr=0 _i=1^K A_i^mcr=0. MCR is thus a zero-sum redistribution at the object-record level. It injects no net preference for or against the whole response, but reallocates credit among the records inside it, while the ranking across rollouts remains determined by the response advantage AseqA^seq. After mapping to token spans, exact cancellation holds at the record level rather than the token level, since spans differ in length; in practice structured records have near-uniform span lengths. Finally, the normalized MCR credit is delivered to the tokens that generate the corresponding object record. Let τi _i denote the generated token span of did_i. We define the token-level MCR term for record did_i as: At,imcr=A¯imcr,t∈τi,0,t∉τi.A_t,i^mcr= cases A_i^mcr,&t∈ _i,\\ 0,&t∉ _i. cases (10) Thus, only the tokens belonging to the structured span of did_i receive its MCR credit. Tokens that are not assigned to any predicted object record have zero MCR term and are affected only by the response-level GRPO advantage in the final policy update. MCR-GRPO Optimization MCR-GRPO combines the response-level GRPO signal with the token-span MCR residual. For each sampled response ymy_m, we first compute its response-level reward. Let Fm∈0,1F_m∈\0,1\ denote whether ymy_m satisfies all required structural rules, including the reasoning tags, answer tags, valid JSON format, and valid bbox_2dbbox\_2d and point_2dpoint\_2d fields. These checks follow the format-reward design in VisionReasoner, but we use an all-pass rule: if any condition fails, Fm=0F_m=0. The response-level reward is: Rmseq=Fm(4+1.5Rmnr+V(S,G)),R_m^seq=F_m (4+1.5\,R_m^nr+V(S,G) ), (11) where Rmnr∈0,1R_m^nr∈\0,1\ is the non-repetition reward, and V(S,G)V(S,G) is the accuracy reward computed by the continuous matched set value evaluator. Since α=2,β=γ=1α=2,β=γ=1, the maximum value of V(S,G)V(S,G) is 44. The constant 4 provides a fixed base reward for format-valid responses so that, after group normalization, any response failing the structural checks receives a strongly negative advantage. Following VisionReasoner, the non-repetition reward RmnrR_m^nr is set to 1 unless the response contains repeated predictions. We then obtain the response-level advantage AmseqA_m^seq by applying the group z-score normalization in Eq. (1) to RmseqR_m^seq. The final token-level advantage is: Am,tMCR-GRPO=Amseq+λmcrAt,imcr,t∈τi,A_m,t^MCR -GRPO=A_m^seq+ _mcrA_t,i^mcr, t∈ _i, (12) where AmseqA_m^seq preserves GRPO’s response-level comparison, while At,imcrA_t,i^mcr adds localized residual credit only to the token spans that generate predicted object records. We optimize the policy with a clipped GRPO objective. The token-level likelihood ratio is: ρm,t(θ)=πθ(ym,t∣x,q,ym,<t)πθold(ym,t∣x,q,ym,<t). _m,t(θ)= _θ(y_m,t x,q,y_m,<t) _ _old(y_m,t x,q,y_m,<t). (13) Method Segmentation (gIoU & cIoU) Avg. Counting (Acc) Avg. ReasonSeg RefCOCO RefCOCO+ RefCOCOg Pixmo Count Val Test TestA TestB TestA TestB Test Val Test Test Supervised Fine-Tuning Based Methods LLaVA-OV-7B - - 58.1 - 47.1 - 55.6 - - - - - LISA-7B 44.4 36.8 79.1 72.3 70.8 58.1 70.6 - - - - - PixelLM-7B - - 78.6 68.2 71.7 58.3 70.5 - - - - - PerceptionGPT-7B - - 78.6 71.7 73.9 61.3 71.7 - - - - - SEGLLM 57.2 52.4 81.5 75.4 73.0 62.5 73.6 67.9 - - - - Read-7B 59.8 56.8 80.2 73.2 73.7 60.4 71.4 67.9 - - - - Open-Source MLLM and Reinforcement Fine-Tuning Based Methods Qwen2.5-VL-7B 56.9 52.1 77.9 66.5 74.0 55.6 70.9 64.8 63.3 67.9 76.0 69.1 Seg-R1-7B 58.6 56.7 78.7 67.6 70.9 57.9 71.4 66.0 - - - - Seg-Zero-7B 62.6 57.5 80.3 72.2 76.2 62.3 72.6 69.1 - - - - VisionReasoner-7B 66.3 63.6 77.4 67.6 71.1 55.8 68.3 67.2 70.1 69.5 87.6 75.7 GroupRevision-7B 67.5 66.7 78.0 69.5 73.3 59.3 71.1 69.4 75.9 73.0 91.0 80.0 MCR-GRPO (ours) 69.4 67.2 78.0 69.2 72.1 59.5 71.6 69.6 75.7 76.4 92.7 81.6 Table 2: Comparison with SOTA on Segmentation and Counting Tasks. We use SAM2 if necessary in segmentation tasks. The best results are highlighted in bold. For compact notation, we define the clip and KL terms: Lm,t(θ) L_m,t(θ) =min(ρm,t(θ)Am,tMCR-GRPO, = ( _m,t(θ)A_m,t^MCR -GRPO, (14) clip(ρm,t(θ),1−ϵc,1+ϵc)Am,tMCR-GRPO). ( _m,t(θ),1- _c,1+ _c)A_m,t^MCR -GRPO ). Dm,tKL=DKL(πθ(⋅∣x,q,ym,<t)∥πref(⋅∣z,q,ym,<t)).D_m,t^KL=D_KL ( _θ(· x,q,y_m,<t)\;\|\; _ref(· z,q,y_m,<t) ). (15) The final objective of MCR-GRPO is: MCR _MCR (θ)-GRPO= _ -GRPO(θ)= (16) [1M∑m=1M1|ym|∑t=1|ym|(Lm,t(θ)−βDm,tKL)]. [ 1M _m=1^M 1|y_m| _t=1^|y_m| (L_m,t(θ)-β D_m,t^KL ) ]. Experiment Training Data. We train MCR-GRPO on VisionReasoner7K (Liu et al. 2026b), a compact multi-object visual perception corpus comprising 7,099 object-record-supervised examples that span category-level localization, referring comprehension, and reasoning instructions. To avoid introducing additional supervision, we use the ground-truth boxes and points as SAM2 prompts to generate masks, rather than relying on human-annotated masks. These sources do not include the held-out DOD, counting, or VQA sets used below. Evaluation Benchmarks. We evaluate the models across five task families: referring expression comprehension (REC), described object detection (DOD), segmentation, object counting, and visual question answering (VQA). For REC, we report results on RefCOCO(+/g) (Yu et al. 2016) and additionally evaluate on ReasonG, which is derived by converting ReasonSeg (Lai et al. 2024) masks into bounding boxes. For described object detection, we use the D3D^3 benchmark (Xie et al. 2023). For segmentation, we evaluate referring segmentation on RefCOCO(+/g) and reasoning segmentation on ReasonSeg. For counting, we use PixMo-Count (Deitke et al. 2025) and CountBench (Paiss et al. 2023). We further include standard VQA benchmarks as auxiliary probes to examine whether structured-perception training preserves general multimodal ability. Implementation Details. We initialize the MLLM from Qwen2.5-VL-7B-Instruct (Bai et al. 2025a) and use pretrained SAM2 (Ravi et al. 2025) weights to generate segmentation masks when mask outputs are required. Training is implemented with the VeRL (Sheng et al. 2025) framework, and rollout generation is accelerated by vLLM (Kwon et al. 2023). Unless otherwise specified, we train with a learning rate of 1×10−61× 10^-6, a KL coefficient of 5×10−35× 10^-3, gradient accumulation of 2, and a global batch size of 16. For each prompt, we sample 8 rollouts for group-relative optimization. Evaluation Metrics. Following standard evaluation protocols, we adopt Acc@0.5 for REC, where a prediction is correct if its prediction has an IoU of least 0.5. For DOD, we report the standard detection AP. We use generalized Intersection over Union (gIoU) for reasoning segmentation and cumulative IoU (cIoU) for referring segmentation. For counting, we report accuracy based on predicted box count. REC and DOD. We compare MCR-GRPO with current state-of-the-art methods on REC and DOD tasks, including PerceptionGPT-7B (Pi et al. 2024), VistaLLM-7B (Pramanick et al. 2024), Elysium-7B (Wang et al. 2025a), Groma-7B (Ma et al. 2024), Qwen2.5-VL-7B (Bai et al. 2025a), SegZero-7B (Liu et al. 2026a), VisionReasoner-7B (Liu et al. 2026b), and GroupRevision-7B (Liu et al. 2026c). As shown in Table 1, our method achieves the best average REC accuracy of 86.8%86.8\%. On the DOD benchmark D3D^3, MCR-GRPO also consistently outperforms previous methods, achieving the best AP with 23.1%23.1\%, 22.5%22.5\%, and 24.8%24.8\%, respectively. Model OCRBench ChartQA SimpleVQA Num Acc Acc Qwen2.5-VL-7B 858 83.8 26.4 MCR-GRPO (ours) 874 87.7 26.5 Model DUDE MMStar MME-Realworld Acc Acc Acc (Lite) Qwen2.5-VL-7B 47.9 62.0 43.3 MCR-GRPO (ours) 48.9 62.6 48.2 Table 3: Evaluation results on VQA benchmarks. MCR Cont REC DOD Seg Count RefCOCOg D3D^3 ReasonSeg Pixmo Val Test Full Val Val × × 86.1 87.5 22.0 66.3 70.1 ✓ × 88.9 89.4 22.9 68.4 67.1 × ✓ 88.1 88.3 22.0 68.5 69.5 ✓ ✓ 89.1 89.9 23.1 69.4 75.7 Table 4: Ablation Study of Our Approach on the VisionReasoner baseline with Marginal Contribution Reward (MCR) and Continuous Matched Set Value Evaluator (Cont). Segmentation and Counting. We further compare MCR-GRPO with state-of-the-art methods on segmentation and counting tasks, including LLaVA-OV-7B (Li et al. 2024), LISA-7B (Lai et al. 2024), PixelLM-7B (Ren et al. 2024), PerceptionGPT-7B (Pi et al. 2024), SEGLLM (Wang et al. 2025b), Read-7B (Qian et al. 2025), Qwen2.5-VL-7B (Bai et al. 2025a), and Seg-R1-7B (You and Wu 2025) besides the baselines discussed above. As shown in Table 2, MCR-GRPO achieves the best average performance on both segmentation and counting, reaching 69.6%69.6\% on segmentation and 81.6%81.6\% on counting. These indicate that, by assigning marginal contribution rewards to box-level spatial structures, our method strengthens the model’s ability to preserve object identity, distinguish target instances, and maintain count consistency without introducing additional trajectory. Visual QA Ability. To evaluate whether MCR-GRPO preserves and improves general visual question answering ability, we compare it with Qwen2.5-VL-7B on six widely used VQA benchmarks, including DUDE (Van Landeghem et al. 2023), ChartQA (Masry et al. 2022), SimpleVQA (Cheng et al. 2025), OCRBench (Liu et al. 2024), MMStar (Chen et al. 2024), and MME-Realworld-Lite (Zhang et al. 2025). As shown in Table 3, MCR-GRPO consistently improves the model across all evaluated benchmarks. These results indicate that our training does not degrade the model’s general VQA ability. Instead, by improving structured visual perception, MCR-GRPO brings consistent gains to VQA scenarios beyond the tasks directly optimized during training. Ablation of the Key Components. We conduct ablation studies on four representative tasks, including REC, DOD, segmentation, and counting, to analyze the effect of the two key designs in MCR-GRPO, namely Marginal Contribution Reward (MCR) and the Continuous Matched Set Value Evaluator (Cont). We use VisionReasoner as the baseline. When Cont is removed, the value function used by MCR is VisionReasoner’s original Accuracy Reward. Table 4 disentangles the two designs. Adding the continuous evaluator alone improves REC (+2.0%+2.0\% / +0.8%+0.8\% on RefCOCOg) and reasoning segmentation (+2.2%+2.2\%) by making response-level ranking smoother, but it leaves DOD unchanged and slightly lowers counting (69.5%69.5\% vs. 70.1%70.1\%): a smoother scalar is still broadcast over all boxes and cannot tell the model which box to drop. Applying MCR alone on top of the thresholded value improves REC and DOD, yet degrades counting below the baseline: on a discretized value, removing a box either leaves the score unchanged or changes it abruptly, so marginal contributions become sparse and noisy, and the resulting credit misleads cardinality-sensitive behavior. Only the combination improves all four tasks, with the largest margin exactly on counting (+5.6%+5.6\% over baseline and +8.6%+8.6\% over MCR alone). The two designs are therefore complementary rather than independently additive: the continuous matched set value is the substrate that makes leave-one-out attribution informative, and box-level attribution is the mechanism that converts graded set-level feedback into cardinality-aware learning. MCR Ratio REC DOD Seg Count RefCOCOg D3D^3 ReasonSeg Pixmo Val Test Full Val Val 0.05 88.1 88.7 22.9 68.8 72.9 0.10 89.1 89.9 23.1 69.4 75.7 0.20 89.6 89.6 22.9 67.0 75.9 Table 5: Ablation of MCR Ratio (λmcr _mcr). Ablation of MCR Ratio. We further study the effect of the MCR ratio (λmcr _mcr), which controls the strength of the box-level MCR advantage in the final policy update. As shown in Table 5, a small ratio of 0.05 already brings competitive performance, but the MCR signal is not strong enough to fully exploit response-internal box-level credit. Increasing the ratio to 0.10 gives the best overall trade-off, achieving the highest REC accuracy, DOD AP, and Segmentation gIoU, while maintaining strong counting accuracy. When the ratio is further increased to 0.20, the model obtains slightly higher RefCOCOg Val and PixMo Val results, but its performance drops on D3D^3 and ReasonSeg. This suggests that overly emphasizing box-level residual credit may weaken the balance with the original response-level GRPO objective. Therefore, we set the MCR ratio to 0.10 by default, as it provides a stable balance between response-level preference optimization and fine-grained object-record credit assignment. Additional qualitative results are provided in the Appendix. Conclusion This paper addresses the response-internal credit assignment problem in GRPO-based structured visual perception. We propose MCR-GRPO, a GRPO framework that assigns box-level marginal contribution rewards to structured object records while preserving the original response-level preference comparison. The core idea is to evaluate the predicted object set with a continuous matched set value and then estimate each box’s contribution through leave-one-out comparison, so that helpful, redundant, and harmful object records can receive different learning signals. Experiments on REC, DOD, segmentation, counting, and VQA show that MCR-GRPO improves structured visual perception across multiple benchmarks. MCR-GRPO enables object-level visual records to serve as the basic units for learning, credit assignment, and multi-object understanding in MLLMs, which offers a new perspective for structured visual perception. References S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin (2025a) Qwen2.5-vl technical report. External Links: 2502.13923, Link Cited by: Introduction, Structured Visual Perception with MLLMs., Implementation Details., REC and DOD., Segmentation and Counting.. S. Bai, M. Li, Y. Liu, J. Tang, H. Zhang, L. Sun, X. Chu, and Y. Tang (2025b) UniVG-r1: reasoning guided universal visual grounding with reinforcement learning. External Links: 2505.14231, Link Cited by: Introduction, Learning Structured Perception.. L. Chen, J. Li, X. Dong, P. Zhang, Y. Zang, Z. Chen, H. Duan, J. Wang, Y. Qiao, D. Lin, et al. (2024) Are we on the right way for evaluating large vision-language models?. Advances in Neural Information Processing Systems 37, p. 27056–27087. Cited by: Visual QA Ability.. X. Cheng, W. Zhang, S. Zhang, J. Yang, X. Guan, X. Wu, X. Li, G. Zhang, J. Liu, Y. Mai, Y. Zeng, Z. Wen, K. Jin, B. Wang, W. Zhou, Y. Lu, H. Ji, T. Li, W. Huang, and Z. Li (2025) SimpleVQA: multimodal factuality evaluation for multimodal large language models. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 4637–4646. Cited by: Visual QA Ability.. M. Deitke, C. Clark, S. Lee, R. Tripathi, Y. Yang, J. S. Park, M. Salehi, N. Muennighoff, K. Lo, L. Soldaini, et al. (2025) Molmo and pixmo: open weights and open data for state-of-the-art vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 91–104. Cited by: Introduction, Evaluation Benchmarks.. J. Huang, Z. Xu, J. Zhou, T. Liu, Y. Xiao, M. Ou, B. Ji, X. Li, and K. Yuan (2026) SAM-r1: leveraging sam for reward feedback in multimodal segmentation via reinforcement learning. Advances in Neural Information Processing Systems 38, p. 138362–138383. Cited by: Introduction, Learning Structured Perception.. A. Kirillov, E. Mintun, N. Ravi, H. Mao, C. Rolland, L. Gustafson, T. Xiao, S. Whitehead, A. C. Berg, W. Lo, P. Dollar, and R. Girshick (2023) Segment anything. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 4015–4026. Cited by: Structured Visual Perception with MLLMs.. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica (2023) Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, p. 611–626. Cited by: Implementation Details.. X. Lai, Z. Tian, Y. Chen, Y. Li, Y. Yuan, S. Liu, and J. Jia (2024) LISA: reasoning segmentation via large language model. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9579–9589. Cited by: Structured Visual Perception with MLLMs., Evaluation Benchmarks., Segmentation and Counting.. B. Li, Y. Zhang, D. Guo, R. Zhang, F. Li, H. Zhang, K. Zhang, P. Zhang, Y. Li, Z. Liu, and C. Li (2024) LLaVA-onevision: easy visual task transfer. External Links: 2408.03326, Link Cited by: Structured Visual Perception with MLLMs., Segmentation and Counting.. L. H. Li, P. Zhang, H. Zhang, J. Yang, C. Li, Y. Zhong, L. Wang, L. Yuan, L. Zhang, J. Hwang, K. Chang, and J. Gao (2022) Grounded language-image pre-training. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10965–10975. Cited by: Structured Visual Perception with MLLMs.. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe (2024) Let's verify step by step. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024, p. 39578–39601. External Links: Link Cited by: Learning Structured Perception.. C. Liu, H. Ding, and X. Jiang (2023a) GRES: generalized referring expression segmentation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 23592–23601. Cited by: Structured Visual Perception with MLLMs.. H. Liu, C. Li, Q. Wu, and Y. J. Lee (2023b) Visual instruction tuning. Advances in neural information processing systems 36, p. 34892–34916. Cited by: Introduction, Learning Structured Perception.. S. Liu, Z. Zeng, T. Ren, F. Li, H. Zhang, J. Yang, Q. Jiang, C. Li, J. Yang, H. Su, J. Zhu, and L. Zhang (2025a) Grounding dino: marrying dino withăgrounded pre-training forăopen-set object detection. In Computer Vision – ECCV 2024, Cham, p. 38–55. External Links: ISBN 978-3-031-72970-6 Cited by: Structured Visual Perception with MLLMs.. Y. Liu, Z. Li, M. Huang, B. Yang, W. Yu, C. Li, X. Yin, C. Liu, L. Jin, and X. Bai (2024) OCRBench: on the hidden mystery of ocr in large multimodal models. Science China Information Sciences 67 (12). External Links: ISSN 1869-1919, Link, Document Cited by: Visual QA Ability.. Y. Liu, B. Peng, Z. Zhong, Z. Yue, F. Lu, B. Yu, and J. Jia (2026a) Seg-zero: reasoning-chain guided segmentation via cognitive reinforcement. External Links: 2503.06520, Link Cited by: Learning Structured Perception., REC and DOD.. Y. Liu, T. Qu, Z. Zhong, B. Peng, S. Liu, B. Yu, and J. Jia (2026b) VisionReasoner: unified reasoning-integrated visual perception via reinforcement learning. In The Fourteenth International Conference on Learning Representations, Cited by: Introduction, Learning Structured Perception., Training Data., REC and DOD.. Y. Liu, Y. Ji, A. Le, J. Zhu, J. Pan, C. Peng, J. Deng, F. Liu, and J. Wu (2026c) From failure to feedback: group revision unlocks hard cases in object-level grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 4816–4828. Cited by: Learning Structured Perception., REC and DOD.. Z. Liu, Z. Sun, Y. Zang, X. Dong, Y. Cao, H. Duan, D. Lin, and J. Wang (2025b) Visual-rft: visual reinforcement fine-tuning. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), p. 2034–2044. Cited by: Learning Structured Perception.. C. Ma, Y. Jiang, J. Wu, Z. Yuan, and X. Qi (2024) Groma: localized visual tokenization for grounding multimodal large language models. In European Conference on Computer Vision, p. 417–435. Cited by: Structured Visual Perception with MLLMs., Learning Structured Perception., REC and DOD.. A. Masry, J. Q. Tan, S. Joty, E. Hoque, et al. (2022) ChartQA: a benchmark for question answering about charts with visual and logical reasoning. In Findings of the association for computational linguistics: ACL 2022, p. 2263–2279. Cited by: Visual QA Ability.. R. Paiss, A. Ephrat, O. Tov, S. Zada, I. Mosseri, M. Irani, and T. Dekel (2023) Teaching clip to count to ten. In Proceedings of the IEEE/CVF international conference on computer vision, p. 3170–3180. Cited by: Evaluation Benchmarks.. Z. Peng, W. Wang, L. Dong, Y. Hao, S. Huang, S. Ma, Q. Ye, and F. Wei (2024) Grounding multimodal large language models to the world. In International Conference on Learning Representations, Vol. 2024, p. 51575–51598. External Links: Link Cited by: Structured Visual Perception with MLLMs.. R. Pi, L. Yao, J. Gao, J. Zhang, and T. Zhang (2024) PerceptionGPT: effectively fusing visual perception into llm. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 27124–27133. Cited by: REC and DOD., Segmentation and Counting.. S. Pramanick, G. Han, R. Hou, S. Nag, S. Lim, N. Ballas, Q. Wang, R. Chellappa, and A. Almahairi (2024) Jack of all tasks master of many: designing general-purpose coarse-to-fine vision-language model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 14076–14088. Cited by: REC and DOD.. R. Qian, X. Yin, and D. Dou (2025) Reasoning to attend: try to understand how <seg> token works. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 24722–24731. External Links: Document Cited by: Structured Visual Perception with MLLMs., Segmentation and Counting.. N. Ravi, V. Gabeur, Y. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, et al. (2025) SAM 2: segment anything in images and videos. In International Conference on Learning Representations, Vol. 2025, p. 28085–28128. Cited by: Introduction, Structured Visual Perception with MLLMs., Implementation Details.. Z. Ren, Z. Huang, Y. Wei, Y. Zhao, D. Fu, J. Feng, and X. Jin (2024) PixelLM: pixel reasoning with large multimodal model. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 26374–26383. Cited by: Learning Structured Perception., Segmentation and Counting.. Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo (2024) DeepSeekMath: pushing the limits of mathematical reasoning in open language models. External Links: 2402.03300, Link Cited by: Introduction. H. Shen, P. Liu, J. Li, C. Fang, Y. Ma, J. Liao, Q. Shen, Z. Zhang, K. Zhao, Q. Zhang, R. Xu, and T. Zhao (2025) VLM-r1: a stable and generalizable r1-style large vision-language model. External Links: 2504.07615, Link Cited by: Introduction, Learning Structured Perception.. G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu (2025) Hybridflow: a flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, p. 1279–1297. Cited by: Implementation Details.. J. Van Landeghem, R. Tito, Ł. Borchmann, M. Pietruszka, D. Jurkiewicz, R. Powalski, P. Józiak, S. Biswas, M. Coustaty, and T. Stanisławek (2023) ICDAR 2023 competition on document understanding of everything (dude). In Document Analysis and Recognition - ICDAR 2023, Cham, p. 420–434. Cited by: Visual QA Ability.. H. Wang, Y. Ye, Y. Wang, Y. Nie, and C. Huang (2025a) Elysium: exploring object-level perception in videos via mllm. In Computer Vision – ECCV 2024, Cham, p. 166–185. External Links: ISBN 978-3-031-72670-5 Cited by: REC and DOD.. P. Wang, L. Li, Z. Shao, R. Xu, D. Dai, Y. Li, D. Chen, Y. Wu, and Z. Sui (2024) Math-shepherd: verify and reinforce llms step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 9426–9439. Cited by: Learning Structured Perception.. X. Wang, S. Zhang, S. Li, K. Li, K. Kallidromitis, Y. Kato, K. Kozuka, et al. (2025b) SegLLM: multi-round reasoning segmentation with large language models. In International Conference on Learning Representations, Vol. 2025, p. 56526–56547. External Links: Link Cited by: Structured Visual Perception with MLLMs., Segmentation and Counting.. B. Xiao, H. Wu, W. Xu, X. Dai, H. Hu, Y. Lu, M. Zeng, C. Liu, and L. Yuan (2024) Florence-2: advancing a unified representation for a variety of vision tasks. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 4818–4829. Cited by: Structured Visual Perception with MLLMs.. C. Xie, Z. Zhang, Y. Wu, F. Zhu, R. Zhao, and S. Liang (2023) Described object detection: liberating object detection with flexible expressions. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, p. 79095–79107. External Links: Document, Link Cited by: Structured Visual Perception with MLLMs., Evaluation Benchmarks.. Z. You and Z. Wu (2025) Seg-r1: segmentation can be surprisingly simple with reinforcement learning. External Links: 2506.22624, Link Cited by: Learning Structured Perception., Segmentation and Counting.. E. Yu, K. Lin, L. Zhao, Y. Wei, Y. Peng, H. Wei, J. Sun, C. Han, Z. Ge, X. Zhang, et al. (2026) Perception-r1: pioneering perception policy with reinforcement learning. Advances in Neural Information Processing Systems 38, p. 94827–94853. Cited by: Introduction, Learning Structured Perception.. L. Yu, P. Poirson, S. Yang, A. C. Berg, and T. L. Berg (2016) Modeling context in referring expressions. In European conference on computer vision, p. 69–85. Cited by: Evaluation Benchmarks.. Y. Zhang, H. Zhang, H. Tian, C. Fu, S. Zhang, J. Wu, F. Li, K. Wang, Q. Wen, Z. Zhang, L. Wang, and R. Jin (2025) MME-realworld: could your multimodal llm challenge high-resolution real-world scenarios that are difficult for humans?. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025, p. 89655–89701. External Links: Link Cited by: Visual QA Ability.. Appendix A Implementation Details We initialize the MLLM from Qwen2.5-VL-7B-Instruct (Bai et al. 2025) and use pretrained SAM2 (Ravi et al. 2025) weights to generate segmentation masks when mask outputs are required. Training is implemented with the VeRL (Sheng et al. 2025) framework, and rollout generation is accelerated by vLLM (Kwon et al. 2023). We train on a single node with eight NVIDIA H20 GPUs, each with 141141 GB of memory, and set vLLM’s gpu memory utilization to 0.60.6. We train for one epoch on the 7,0997,099-sample VisionReasoner7K (Liu et al. 2026a) training set with a rollout batch size of 1616 and drop last is true, resulting in 443443 steps. We empirically find that checkpoints around 400400 training steps are sufficiently optimized and yield stronger performance. Therefore, our evaluation mainly focuses on this training stage. Unless otherwise specified, we use AdamW as the optimizer, training with a learning rate of 1×10−61× 10^-6, a KL coefficient of 5×10−35× 10^-3, gradient accumulation of 22, a global batch size of 1616, and gradient clipping with a maximum norm of 1.01.0. For each prompt, we sample 88 rollouts for group-relative optimization. The maximum number of generated tokens per rollout is 20002000. Rollout sampling uses a temperature of 1.01.0, top-p of 1.01.0, and disabled top-k sampling. Notably, all images are resized to 840×840840× 840 before inference in both training and testing. Appendix B Reward and Evaluator Details Before computing the response-level reward, we first validate whether a sampled response can be deterministically parsed into structured object records. This validation is necessary because both the continuous matched set value and the MCR attribution require a well-defined predicted object set S. If the response format is invalid, the object records cannot be reliably extracted, and the subsequent matching-based reward computation becomes undefined. A response is treated as structurally valid only when all required components are present. Specifically, it must contain a complete reasoning region and a complete answer region, delimited by <think>...</think> and <answer>...</answer>, respectively. The content inside <answer>...</answer> must be parseable as a JSON array, where each element is a dictionary representing one predicted object record. For every predicted object record, the bbox_2d field must be present and contain exactly four numerical values, and the point_2d field must be present and contain exactly two numerical values. These requirements ensure that each predicted item can be converted into a box-indexed object record with a valid point. We use a conjunctive all-pass rule for format validation. Let Fm∈0,1F_m∈\0,1\ denote the structural validity indicator for the sampled response ymy_m. We set Fm=1F_m=1 only if the reasoning-answer structure, JSON parsing, array structure, dictionary format, and all required object fields are valid simultaneously. If any requirement fails, we set Fm=0F_m=0. This differs from partial format scoring: an invalid response does not receive separate partial rewards for individual fields, because a single malformed component can prevent the construction of the predicted object set S and thus block the evaluator and MCR computation. Given this structural gate, the response-level reward is defined as: Rmseq=Fm(4+1.5Rmnr+V(S,G)),R_m^seq=F_m (4+1.5R_m^nr+V(S,G) ), (B1) where Rmnr∈0,1R_m^nr∈\0,1\ is the non-repetition reward and V(S,G)V(S,G) is the continuous matched set value. Following VisionReasoner, RmnrR_m^nr is set to 1 unless the response contains repeated predictions. The constant 4 serves two purposes. First, it creates a clear reward separation between structurally valid and invalid responses after group-relative normalization, so responses that cannot be parsed receive a strongly negative advantage. Second, it aligns the base reward scale with the accuracy term. Since the evaluator uses α=2α=2 and β=γ=1β=γ=1, the maximum pair score is 4, and the matched set value satisfies V(S,G)≤4V(S,G)≤ 4. Therefore, the base reward has the same scale as the maximum accuracy reward, making format validity and structured prediction quality comparable within the response-level reward. Thus, format-valid responses are rewarded on a scale that can still be refined by non-repetition and matched-set accuracy, while format-invalid responses receive zero response-level reward and no object-level MCR attribution. Appendix C Protocol and Prompt Comparison Method Supervision Stages Test-Time Qwen2.5-VL base MLLM 1 Initial VisionReasoner Response 1 Initial Struct GroupRevision Trajectory 2 Revised Struct MCR-GRPO Object-record 1 Initial Struct Table C1: Comparison of Supervision Strategies and Inference Stages. VisionReasoner uses response supervision, GroupRevision introduces trajectory supervision through an additional revision pass, and MCR-GRPO performs object-record supervision within a single structured response. Qwen2.5-VL Prompt Locate "query", report the bboxes coordinates in JSON format. MCR-GRPO & VisionReasoner Prompt Please find “query” with bboxs and points. Compare the difference between object(s) and find the most closely matched object(s). Output the thinking process in <think> </think> and final answer in <answer> </answer> tags. Output the bbox(es) and point(s) inside the interested object(s) in JSON format. i.e. <think> thinking process here </think> <answer> example </answer> GroupRevision Stage I (Init) Prompt Please find "query" with bounding boxes and points. Compare the difference between object(s) and identify the most closely matched one(s). Output the thinking process in <think> ... </think> and the final answer in <answer> ... </answer> tags. Return the bbox(es) and point(s) of the referenced object(s) in JSON format. i.e., <think> thinking process here </think> <answer> example </answer> GroupRevision Stage I (Revision) Prompt You previously predicted bounding box(es) (box) and point(s) (points) for the question "query", with the reasoning think. Rethink whether the previous bbox(es) and point(s) match the target object(s). – If yes: keep the same object(s); tighten each bbox to the object boundary and set the point to the object center. – If no: discard previous predictions and output the correct object(s), each with one bbox and one point inside it. Respond with exactly: <think> thinking process here </think> <answer> example </answer> Table C2: Prompt Comparison Across Methods. MCR-GRPO, Qwen2.5-VL, and VisionReasoner use a single initial prompt, while GroupRevision adds a second-stage revision prompt. We compare the supervision strategies and inference protocols of different models. As shown in Table C1, Qwen2.5-VL (Bai et al. 2025) serves as the base MLLM and directly generates the initial response. VisionReasoner (Liu et al. 2026a) follows response supervision, where a single response-level signal is assigned to the whole structured answer. GroupRevision (Liu et al. 2026b) uses trajectory supervision by adding a second-stage revision pass conditioned on the initial prediction. In contrast, MCR-GRPO performs object-record supervision within the original structured response, assigning box-level credit during training while preserving a single-stage inference protocol. Table C2 lists the prompt templates used in our comparison. VisionReasoner and MCR-GRPO use the same structured-output prompt, while GroupRevision additionally uses a second-stage revision prompt. Appendix D Additional Experiments on COCO We further evaluate MCR-GRPO on COCO to examine whether response-internal box-level credit assignment generalizes to a standard object detection benchmark. We compare our model with Qwen2.5-VL (Bai et al. 2025), VisionReasoner (Liu et al. 2026a), and GroupRevision (Liu et al. 2026b). We evaluate on COCO val2017 using the standard COCO detection protocol. All methods are evaluated with the same image resizing, decoding configuration, prompt format, and post-processing rules. As shown in Table D3, MCR-GRPO achieves the best COCO AP among the compared methods. Compared with Qwen2.5-VL, MCR-GRPO improves AP from 29.2%29.2\% to 39.7%39.7\%. Compared with VisionReasoner, MCR-GRPO improves AP from 37.7%37.7\% to 39.7%39.7\%. Notably, MCR-GRPO also slightly outperforms GroupRevision, while using only a single inference stage and requiring no additional revision pass. Method Extra Stage COCO AP50:95 Qwen2.5-VL-7B × 29.2 VisionReasoner-7B × 37.7 GroupRevision-7B ✓ 39.2 MCR-GRPO (Ours) × 39.7 Table D3: Additional COCO Results. MCR-GRPO achieves the best COCO AP without introducing an extra inference stage. The best results are highlighted in bold. To further compare inference protocols, Table D4 reports detailed COCO metrics for GroupRevision and MCR-GRPO. GroupRevision uses a two-stage inference procedure, where the model first generates an initial response and then performs an additional revision stage. In contrast, MCR-GRPO keeps the inference pipeline single-stage: it derives box-level supervision during training, but does not require an additional response generation or revision trajectory at test time. Despite this simpler inference protocol, MCR-GRPO obtains higher AP, AP50, AP75, AR10, and AR100 than GroupRevision-7B. Method Extra Stage AP AP50 AP75 GroupRevision-7B ✓ 39.2 57.9 40.7 MCR-GRPO (Ours) × 39.7 59.2 40.8 Method Extra Stage AR1 AR10 AR100 GroupRevision-7B ✓ 35.5 49.5 50.1 MCR-GRPO (Ours) × 35.0 49.7 50.2 Table D4: Detailed COCO Comparison Between GroupRevision-7B and MCR-GRPO. MCR-GRPO uses a single-stage inference protocol and achieves stronger overall detection performance. The best results are highlighted in bold. These results suggest that the gain of MCR-GRPO does not rely on test-time revision. Instead, MCR-GRPO improves the model during training by assigning marginal contribution rewards to box-level object records, allowing the final model to produce stronger detections in a single inference pass. Appendix E Qualitative Comparison We qualitatively compare our MCR-GRPO with VisionReasoner (Liu et al. 2026a) across Described Object Detection (DOD), Segmentation, Referring Expression Comprehension (REC), and Counting. Figures E1–E3 show the query, the prediction from each model, and the ground truth under the same task-specific visualization protocol. These examples complement the aggregate results in the main paper by illustrating differences in instance selection, localization, and cardinality. Described object detection. Figure E1 presents examples from the D3D^3 benchmark (Xie et al. 2023), whose queries contain attributes, relations, and negated descriptions. In the displayed cases, MCR-GRPO more accurately identifies the complete set of objects satisfying the description while excluding visually similar distractors. This distinction is especially visible for multi-object queries, which require both description-consistent localization and correct cardinality. Segmentation and REC. Figure E2 shows reasoning segmentation examples from ReasonSeg (Lai et al. 2024) and referring segmentation examples from RefCOCO, RefCOCO+, and RefCOCOg (Yu et al. 2016). All samples can also be used as REC examples. These queries require spatial-relation reasoning, ordinal disambiguation, or recognition from an indirect description. In the displayed cases, MCR-GRPO more reliably selects the intended instance and produces spatial outputs that align more closely with the target extent. For segmentation, both methods use the same SAM2-based mask-generation pipeline described in the main paper. Counting. Figure E3 compares the two models on PixMo-Count (Deitke et al. 2025) and CountBench (Paiss et al. 2023). As in the main experiments, the predicted count is obtained from the number of generated object records, without additional counting-specific supervision. In the displayed scenes, MCR-GRPO more closely matches the ground-truth cardinality by reducing missed targets and redundant predictions while retaining a localized box for each counted instance. Figure E1: Qualitative Comparison with VisionReasoner on Described Object Detection. Examples are drawn from the D3D^3 benchmark. The displayed queries cover attribute-rich, relational, and negated descriptions. We present visualizations of the model-predicted boxes and points overlaid on the images. Figure E2: Qualitative comparison with VisionReasoner on Segmentation and Referring Expression Comprehension. Examples are drawn from ReasonSeg and RefCOCO(+/g). The displayed queries involve reasoning-dependent descriptions, spatial relations, and ordinal cues. We present visualizations of the model-predicted boxes and points, as well as the masks generated by SAM2, overlaid on the images. Figure E3: Qualitative Comparison with VisionReasoner on Counting. Examples are drawn from PixMo-Count and CountBench. The predicted count is the number of localized object records. We present visualizations of the model-predicted boxes and points, as well as the masks generated by SAM2, overlaid on the images. In addition, we visualize the model’s counting process and the resulting counts on the images. Appendix F Formal Analysis of Marginal Contributions In this section, we provide a formal analysis of the leave-one-out MCR credit used in the main text. The analysis separates the raw value difference ΔVi V_i from the normalized residual credit A¯imcr A_i^mcr. The raw value captures how removing one predicted record changes the count-normalized matched set value, while the subsequent within-response normalization converts these value differences into signed object-record credits for policy optimization. This analysis explains why unmatched records receive negative raw credit under over-prediction, and why redundant or weak records can be suppressed by negative normalized credit within the same response. Setting. Let S=dii=1KS=\d_i\_i=1^K be the predicted object records and G=gjj=1NG=\g_j\_j=1^N be the ground-truth objects. Pair scores satisfy s(di,gj)≥0s(d_i,g_j)≥ 0. For any predicted subset T⊆ST S, let ℳ(T,G)M(T,G) denote the set of valid one-to-one matchings between T and G. Following the matching objective in the main text, define: MT⋆=argmaxM′∈ℳ(T,G)∑(r,j)∈M′s(dr,gj).M_T = _M (T,G) _(r,j)∈ M s(d_r,g_j). (F2) The matched set value for T is: V(T,G)=1max(|T|,N)∑(r,j)∈MT⋆s(dr,gj).V(T,G)= 1 (|T|,N) _(r,j)∈ M_T s(d_r,g_j). (F3) In particular, the raw leave-one-out value used by MCR is: ΔVi=V(S,G)−V(S∖di,G). V_i=V(S,G)-V(S \d_i\,G). (F4) For compactness, we write MS⋆M_S for the optimal matching on S, and M−i⋆M_-i for the optimal matching on S∖diS \d_i\. A predicted record did_i is called unmatched if there exists an optimal matching MS⋆M_S that does not include did_i. Lemma 1 (Removal bounds). For any di∈Sd_i∈ S: (a) Removing did_i cannot increase the optimal matched sum: ∑(r,j)∈M−i⋆s(dr,gj)≤∑(r,j)∈MS⋆s(dr,gj). _(r,j)∈ M_-i s(d_r,g_j)≤ _(r,j)∈ M_S s(d_r,g_j). (F5) (b) If did_i is matched to gj(i)g_j(i) in an optimal matching MS⋆M_S , then: ∑(r,j)∈M−i⋆s(dr,gj)≥[∑(r,j)∈MS⋆s(dr,gj)]−s(di,gj(i)). _(r,j)∈ M_-i s(d_r,g_j)≥ [ _(r,j)∈ M_S s(d_r,g_j) ]-s(d_i,g_j(i)). (F6) (c) If did_i is unmatched in some optimal matching MS⋆M_S , then: ∑(r,j)∈M−i⋆s(dr,gj)=∑(r,j)∈MS⋆s(dr,gj). _(r,j)∈ M_-i s(d_r,g_j)= _(r,j)∈ M_S s(d_r,g_j). (F7) Proof. (a) Any matching after removing did_i remains feasible for the original predicted set by leaving did_i unused. Since all pair scores are non-negative, the maximum matched sum cannot increase after removal. (b) If did_i is matched, deleting the pair (i,j(i))(i,j(i)) from MS⋆M_S gives a feasible matching after removing did_i with the stated weight. (c) If did_i is unmatched in an optimal matching, the same matching remains feasible after removal, and the bound (a) gives equality. ∎ Proposition 1 (Regime-dependent raw sign). (a) If K≤NK≤ N, then ΔVi≥0 V_i≥ 0 for every di∈Sd_i∈ S. (b) If K>NK>N and did_i is unmatched in some optimal matching, then: ΔVi=−1K(K−1)∑(r,j)∈MS⋆s(dr,gj)≤0. V_i=- 1K(K-1) _(r,j)∈ M_S s(d_r,g_j)≤ 0. (F8) The inequality is strict whenever the matched sum is positive. Proof. (a) When K≤NK≤ N, both values use denominator N, so: ΔVi=1N(∑(r,j)∈MS⋆s(dr,gj)−∑(r,j)∈M−i⋆s(dr,gj))≥0 V_i= 1N ( _(r,j)∈ M_S s(d_r,g_j)- _(r,j)∈ M_-i s(d_r,g_j) )≥ 0 (F9) by Lemma 1. (b) When K>NK>N and did_i is unmatched, the matched sum is unchanged after removal, while the denominator changes from K to K−1K-1. Hence: ΔVi V_i =1K∑(r,j)∈MS⋆s(dr,gj)−1K−1∑(r,j)∈MS⋆s(dr,gj) = 1K _(r,j)∈ M_S s(d_r,g_j)- 1K-1 _(r,j)∈ M_S s(d_r,g_j) (F10) =−1K(K−1)∑(r,j)∈MS⋆s(dr,gj), =- 1K(K-1) _(r,j)∈ M_S s(d_r,g_j), with strict negativity whenever the matched sum is positive. Finally, within-response normalization preserves the ordering of raw marginal contributions. When the within-response standard deviation is nonzero, A¯imcr A_i^mcr is a strictly increasing affine transformation of ΔVi V_i. Therefore, for any two records did_i and djd_j, ΔVi>ΔVj⇒A¯imcr>A¯jmcr. V_i> V_j A_i^mcr> A_j^mcr. (F11) Thus, MCR guarantees response-relative ordering rather than an absolute helpful-or-harmful sign per box. If all records have identical raw marginal contributions, the variance is zero and we set all normalized MCR credits to zero. ∎ Proposition 2 (Relative Credit After Normalization). For K>1K>1 and σΔ>0 _ >0 in Eqs. (8)–(9), normalized MCR credit is: A¯imcr=ΔVi−μΔσΔ+ϵ. A_i^mcr= V_i- _ _ +ε. (F12) Thus A¯imcr A_i^mcr preserves the ordering of ΔVi V_i, and its sign is determined by whether ΔVi V_i is above or below the within-response mean μΔ _ . Proof. The denominator σΔ+ϵ _ +ε is positive when σΔ>0 _ >0 and ϵ≥0ε≥ 0. Therefore normalization is a strictly increasing affine map of ΔVi V_i. Centering makes records below the response mean receive negative residual credit, even when their raw ΔVi V_i is non-negative. If σΔ=0 _ =0, MCR sets all normalized credits to zero. ∎ Proposition 3 (Lowest Credits Under Over-Prediction). Let K>NK>N. For any record dk∈Sd_k∈ S and any unmatched record did_i, ΔVk≥ΔVi. V_k≥ V_i. (F13) Equality holds iff: ∑(r,j)∈M−k⋆s(dr,gj)=∑(r,j)∈MS⋆s(dr,gj). _(r,j)∈ M_-k s(d_r,g_j)= _(r,j)∈ M_S s(d_r,g_j). (F14) Thus unmatched records and perfectly substitutable records occupy the lowest normalized MCR credits whenever σΔ>0 _ >0. Proof. For the unmatched record did_i, ΔVi=1K∑(r,j)∈MS⋆s(dr,gj)−1K−1∑(r,j)∈MS⋆s(dr,gj). V_i= 1K _(r,j)∈ M_S s(d_r,g_j)- 1K-1 _(r,j)∈ M_S s(d_r,g_j). (F15) For any record dkd_k, ΔVk=1K∑(r,j)∈MS⋆s(dr,gj)−1K−1∑(r,j)∈M−k⋆s(dr,gj). V_k= 1K _(r,j)∈ M_S s(d_r,g_j)- 1K-1 _(r,j)∈ M_-k s(d_r,g_j). (F16) Lemma 1 gives: ∑(r,j)∈M−k⋆s(dr,gj)≤∑(r,j)∈MS⋆s(dr,gj), _(r,j)∈ M_-k s(d_r,g_j)≤ _(r,j)∈ M_S s(d_r,g_j), (F17) so ΔVk≥ΔVi V_k≥ V_i. Since normalization is strictly increasing when σΔ>0 _ >0, the same ordering holds after normalization. ∎ Proposition 4 (Bounded Residual Credit). For K≥2K≥ 2 and ϵ=0ε=0 in Eq. (9), |A¯imcr|≤K−1.| A_i^mcr|≤ K-1. (F18) With ϵ>0ε>0, the magnitude can only decrease. Therefore, for object tokens t∈τit∈ _i with Am,tMCR-GRPO=Amseq+λmcrAt,imcr,A_m,t^MCR -GRPO=A_m^seq+ _mcrA_t,i^mcr, (F19) the composite token advantage has the same sign as AmseqA_m^seq whenever λmcrK−1<|Amseq|. _mcr K-1<|A_m^seq|. (F20) Proof. For object tokens t∈τit∈ _i, Eq. (10) gives At,imcr=A¯imcrA_t,i^mcr= A_i^mcr. Let zi=A¯imcrz_i= A_i^mcr with ϵ=0ε=0. By the population-standard-deviation normalization in Eq. (8), we have ∑izi=0 _iz_i=0 and ∑izi2=K _iz_i^2=K. Hence: zi2=(∑k≠izk)2≤(K−1)∑k≠izk2=(K−1)(K−zi2),z_i^2= ( _k≠ iz_k )^2≤(K-1) _k≠ iz_k^2=(K-1)(K-z_i^2), (F21) which gives |zi|≤K−1|z_i|≤ K-1. If ϵ>0ε>0, each normalized value is multiplied by σΔ/(σΔ+ϵ)≤1 _ /( _ +ε)≤ 1. The sign statement follows from the triangle inequality. ∎ These results explain how MCR turns leave-one-out value changes into box-level training signals. The raw value difference ΔVi V_i measures how much a predicted record contributes to the count-normalized matched set value. Under over-prediction, removing an unmatched or redundant record can improve the set value, yielding negative raw credit. When K≤NK≤ N, raw contributions are non-negative, but within-response centering can assign negative normalized residual credit to records whose marginal contributions fall below the response mean. Thus, weak, redundant, or poorly localized records can still be suppressed in the policy update, while records with stronger marginal contributions receive larger MCR credits and are encouraged. In this way, MCR realizes the positive-and-negative box-level credit assignment described in the main text while preserving the original response-level GRPO comparison. Appendix G Additional Qualitative Examples Figure G4 visualizes representative MCR-GRPO inference outputs across the task families evaluated in the main paper. The examples cover DOD on D3D^3 (Xie et al. 2023), segmentation and REC on ReasonSeg (Lai et al. 2024) and RefCOCO(+/g) (Yu et al. 2016), and counting on PixMo-Count (Deitke et al. 2025) and CountBench (Paiss et al. 2023). For each input, the figure displays an excerpt from the generated <think> content together with the final prediction. The reasoning identifies visual evidence used to distinguish the target from distractors or enumerate the queried category, while the final output expresses the decision as localized boxes, segmentation masks, or a count. These examples show how the same structured inference interface supports different task-specific outputs. Figure G4: Cross-Task Reasoning and Prediction Examples from MCR-GRPO. Representative examples cover DOD on D3D^3, segmentation and REC on ReasonSeg and RefCOCO(+/g), and counting on PixMo-Count and CountBench. Each example pairs an excerpt from the generated <think> content with the final prediction, showing how attribute, relational, and cardinality reasoning is connected to task-appropriate boxes, masks, and counts. Supplementary References S. Bai, K. Chen, X. Liu, J. Wang, W. Ge, S. Song, K. Dang, P. Wang, S. Wang, J. Tang, H. Zhong, Y. Zhu, M. Yang, Z. Li, J. Wan, P. Wang, W. Ding, Z. Fu, Y. Xu, J. Ye, X. Zhang, T. Xie, Z. Cheng, H. Zhang, Z. Yang, H. Xu, and J. Lin (2025) Qwen2.5-vl technical report. External Links: 2502.13923, Link Cited by: Appendix A, Appendix C, Appendix D. M. Deitke, C. Clark, S. Lee, R. Tripathi, Y. Yang, J. S. Park, M. Salehi, N. Muennighoff, K. Lo, L. Soldaini, et al. (2025) Molmo and pixmo: open weights and open data for state-of-the-art vision-language models. In Proceedings of the Computer Vision and Pattern Recognition Conference, p. 91–104. Cited by: Appendix E, Appendix G. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica (2023) Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, p. 611–626. Cited by: Appendix A. X. Lai, Z. Tian, Y. Chen, Y. Li, Y. Yuan, S. Liu, and J. Jia (2024) LISA: reasoning segmentation via large language model. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 9579–9589. Cited by: Appendix E, Appendix G. Y. Liu, T. Qu, Z. Zhong, B. Peng, S. Liu, B. Yu, and J. Jia (2026a) VisionReasoner: unified reasoning-integrated visual perception via reinforcement learning. In The Fourteenth International Conference on Learning Representations, Cited by: Appendix A, Appendix C, Appendix D, Appendix E. Y. Liu, Y. Ji, A. Le, J. Zhu, J. Pan, C. Peng, J. Deng, F. Liu, and J. Wu (2026b) From failure to feedback: group revision unlocks hard cases in object-level grounding. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 4816–4828. Cited by: Appendix C, Appendix D. R. Paiss, A. Ephrat, O. Tov, S. Zada, I. Mosseri, M. Irani, and T. Dekel (2023) Teaching clip to count to ten. In Proceedings of the IEEE/CVF international conference on computer vision, p. 3170–3180. Cited by: Appendix E, Appendix G. N. Ravi, V. Gabeur, Y. Hu, R. Hu, C. Ryali, T. Ma, H. Khedr, R. Rädle, C. Rolland, L. Gustafson, et al. (2025) SAM 2: segment anything in images and videos. In International Conference on Learning Representations, Vol. 2025, p. 28085–28128. Cited by: Appendix A. G. Sheng, C. Zhang, Z. Ye, X. Wu, W. Zhang, R. Zhang, Y. Peng, H. Lin, and C. Wu (2025) Hybridflow: a flexible and efficient rlhf framework. In Proceedings of the Twentieth European Conference on Computer Systems, p. 1279–1297. Cited by: Appendix A. C. Xie, Z. Zhang, Y. Wu, F. Zhu, R. Zhao, and S. Liang (2023) Described object detection: liberating object detection with flexible expressions. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, p. 79095–79107. External Links: Document, Link Cited by: Appendix E, Appendix G. L. Yu, P. Poirson, S. Yang, A. C. Berg, and T. L. Berg (2016) Modeling context in referring expressions. In European conference on computer vision, p. 69–85. Cited by: Appendix E, Appendix G.