Paper deep dive
Getting Space Right: Agentic Rewards for Image Generation
Xiaofeng Tan, Jiaxin Lin, Haobo Yuan, Tiancheng Shen, Zihan Su, Rui Chen, Hongsong Wang, Ming-Hsuan Yang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Reinforcement learning (RL) is promising for spatial text-to-image generation, but existing work largely focuses on policy optimization, assuming accurate rewards and effective online rollouts. We observe in-domain gains with out-of-domain degradation, exposing two limitations: unreliable spatial rewards in open-world settings and rollouts lacking basic-to-complex progression. We propose a curriculum-guided RL framework. First, we introduce SpatialCritic, an agentic reward model equipped with a spatial perception harness that combines reusable verification guidance with specialized vision tools. We then construct spatial preference datasets through targeted counterfactual perturbations and perform agent-harness alignment to improve tool selection and evidence integration efficiently. Second, curriculum learning organizes shared foundational capabilities and their compositional dependencies in a capability graph for mastery-guided rollouts, periodic review, and targeted remediation. On SpaRW-Eval, SpatialCritic achieves 95.1% verification accuracy, outperforming 320B GLM-5.3-Flash with a 40x smaller 8B backbone and raising average baseline scores from 78.4 to 83.9 on SD3.5-Medium and 83.2 to 86.5 on BAGEL. Project page: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2610.14852v1
- Canonical: https://arxiv.org/abs/2610.14852v1
Trouble viewing inline? Open PDF directly ā
Full Text
63,003 characters extracted from source content.
Expand or collapse full text
GETTING SPACE RIGHT: AGENTIC REWARDS FOR IMAGE GENERATION Xiaofeng Tan 1,2ā Jiaxin Lin 2 Haobo Yuan 3 Tiancheng Shen 3 Zihan Su 4 Rui Chen 2 Hongsong Wang 1ā Ming-Hsuan Yang 3 1 Southeast University 2 Tencent HY 3 UC Merced 4 Tsinghua University ABSTRACT Reinforcement learning (RL) is promising for spatial text-to-image generation, but existing work largely focuses on policy optimization, assuming accurate re- wards and effective online rollouts. We observe in-domain gains with out-of- domain degradation, exposing two limitations: unreliable spatial rewards in open- world settings and rollouts lacking basic-to-complex progression. We propose a curriculum-guided RL framework. First, we introduce SPATIALCRITIC, an agentic reward model equipped with a spatial perception harness that combines reusable verification guidance with specialized vision tools. We then construct spatial preference datasets through targeted counterfactual perturbations and per- form agentāharness alignment to improve tool selection and evidence integration efficiently. Second, curriculum learning organizes shared foundational capabili- ties and their compositional dependencies in a capability graph for mastery-guided rollouts, periodic review, and targeted remediation. On SpaRW-Eval, SPATIAL- CRITIC achieves 95.1% verification accuracy, outperforming 320B GLM-5.3- Flash with a 40Ć smaller 8B backbone and raising average baseline scores from 78.4 to 83.9 on SD3.5-Medium and 83.2 to 86.5 on BAGEL. See the project page. 1INTRODUCTION Enhancing spatial intelligent in text-to-image (T2I) generation, i.e., generating accurate spatial re- lationships faithful to text prompts (Ghosh et al., 2023; Wang et al., 2026c), remains a critical and challenging task in T2I post-training. Recently, reinforcement learning (RL) has emerged as a pre- vailing paradigm for aligning pre-trained models with task-specific goals. RL aligns a T2I model by generating images from training prompts, evaluating rewards, and up- dating the policy. Thus, its effectiveness depends on three aspects: (1) rollout organization, (2) reward accuracy, and (3) policy optimization. Existing work mainly advances the third (Black et al., 2024; Clark et al., 2024; Liu et al., 2025a; Zheng et al., 2026; Tan et al., 2026a;b), often assum- ing that rollouts support effective learning and rewards provide reliable feedback. This leaves two questions: (Q1) Are reward models accurate and robust enough in open-world scenarios? (Q2) Can rollout organization support progressive learning from basic spatial capabilities to complex spatial compositions? 0120240360 Training step 0.4 0.6 0.8 1.0 GenEval score (ID) GenEval (ID) SpatialGenEval (OOD) 0.66 0.68 0.70 0.72 0.74 Spatial accuracy (OOD) Pretrained Model -2.48 Figure 1: Generalization col- lapse.ID accuracy improves while OOD performance falls. Unfortunately, these assumptions fail in spatial text-to-image generation: RL-aligned models improve steadily on in-domain (ID) benchmarks but generalize poorly out of domain (OOD), eventually falling below the pre-trained model (Fig. 1). This failure stems from repurposing benchmarks such as GenEval as training prompts and their rule-based scoring pipelines as on- line rewards, exposing two critical flaws: (1) inaccurate and non-robust rewards, as benchmark-specific scoring generalizes poorly to open-world scenarios, especially under object occlu- sion (Tang et al., 2026); and (2) rollouts targeting complex compositions without progressive learning. Reused benchmark prompts emphasize joint constraint satisfaction, neglecting fun- damental spatial capabilities and their progressive composition into complex ones. To overcome the brittleness of rule-based rewards, recent efforts pursue learned reward modeling across two paradigms: regression-based models that map dual-encoder or VLM representations ā Work done at Tencent HY. ā Corresponding author. 1 arXiv:2610.14852v1 [cs.CV] 10 Oct 2026 (a) Regression Reward Models (b) Generative Reward Models(c) Agentic RewardModel CLIP-based dual encoder VLM + Regression Head Distillate Prompt Emb Image Emb Cosine VLM Hidden State Black-box Spatial Evidence Models: ImageReward Ā· VisionReward Ā· SpatialScore MultimodalContext TeacherVLM StudentVLM Evaluation Reasoning </ans> Yes / No Reward Score ā=ķ ! (āķ¦ķķ ā|ā ) Teacher-dependent Supervision Models: RewardDanceĀ· RubricRMĀ· Z-Reward 1. Claim Decomposition Statue and cactus present Contains an extra plant Exact corner placement Statue orientation 2. Tool Planning qDetect relevant objects qVerify visual attributes qEstimate orientation 4. Tool Results 3. Vision Toolbox VLM Reasoning &Specialized Vision Tools Reward Score ā Prompt Image Prompt Image Reward Score ā Evaluation Reasoning </ans> Yes / No Prompt Image āAmarblestatueisintheback- leftcorneroftheroomalongwith apottedcactus.ā MultimodalContext VLM Planner Reason,Plan,Decide Reward Score ā Verify Encoder Encoder Regression Head Our SpatialCtritc: GeneralizableandGrounded Reward Figure 2: Comparison of reward-modeling paradigms. (a) Regression-based scoring; (b) teacher- guided generative evaluation; and (c) our agentic reward model, SPATIALCRITIC. into scalar preference scores (Xu et al., 2026; Tang et al., 2026), and generative models that em- ulate deliberative evaluation via reasoning rationales or dynamic rubrics (Wang et al., 2026a; Kan et al., 2026), as shown in Fig. 2. However, neither ensures reliable spatial verification. Regression models treat foundation models as black-box feature extractors, underutilizing native reasoning and geometric depth priors. Meanwhile, generative models fundamentally rely on ungrounded teacher generation, and struggle to explicitly and reliably resolve geometric quantities such as occlusion and orientation. Beyond reward modeling, progressive capability learning remains largely unex- plored. These gaps motivate us to combine a reward model leveraging vision foundation models for verification with a progressive curriculum, jointly ensuring accuracy and generalizability. Contributions. We propose a curriculum-guided RL paradigm for spatial T2I post-training, com- prising SPATIALCRITIC, an agentic reward model that leverages vision foundation models for robust supervision, and a progressive curriculum from basic spatial capabilities to complex compositions. First, we design an agentic reward model, SPATIALCRITIC, that harnesses specialized vision founda- tion models for grounded spatial verification, optimized via Reinforcement Learning with Verifiable Rewards (RLVR). Its development comprises three components: (1) Agent with a spatial percep- tion harness. To enhance the agentās spatial perception capabilities, we design an agent equipped with a spatial harness that provides reusable verification guidance and executes specialized percep- tual tools. Through this harness, the agent decomposes prompts into atomic claims and selectively queries specialized tools under uncertainty. (2) Spatial preference dataset construction. To enhance the agentic ability on the harness, the verifiable RL environment is necessary for post-training. We therefore construct a spatial preference dataset by counterfactually perturbing targeted spatial rela- tions, such as occlusion, depth, and orientation, while keeping other visual content invariant. (3) Agentāharness alignment via RLVR. Building upon the agent and the preference dataset, we op- timize the agentās harness ability, such as tool-selection and evidence-integration, using verifiable outcome rewards and token-budget penalties to improve spatial judgment accuracy and efficiency. Second, we introduce curriculum learning for spatial T2I post-training, which organizes policy roll- outs from basic spatial capabilities to complex compositions rather than directly optimizing over an unstructured mixture of complex tasks. Our key insight is that higher-order spatial tasks involve dif- ferent combinations of shared foundational capabilities. For example, multi-object layout combines counting, and topology, while occlusion combines depth and topology, sharing topology capabili- ties. Accordingly, we introduce a Compositional Capability Graph (CCG) to capture these depen- dencies. Building on CCG, capability mastery guides progression, while periodic review identifies compositional weaknesses for targeted remediation. Together, these designs enable feedback-driven progression from basic capabilities to complex compositions. Finally, comprehensive experiments validate accurate preference judgments, effective reward-guided image selection, and improved RL alignment. SPATIALCRITIC scores 95.1% on SpaRW-Eval. Cur- riculum RL raises baseline averages to 83.9 on SD3.5-Medium and 86.5 on BAGEL. 2RELATED WORK Reward Models for Visual Generation. Visual reward modeling for generation (Su et al., 2025; Fan et al., 2026; Su et al., 2026) has evolved from similarity heuristics (Radford et al., 2021) to 2 learned human preferences. Early dual encoders (Wu et al., 2023; Zhang et al., 2024) predict scalar preference scores via Bradley-Terry objectives. To overcome single-scalar bottlenecks, subsequent paradigms deployed multimodal VLMs to predict disentangled rewards across aesthetics, alignment, and visual artifacts (Xu et al., 2026; Liu et al., 2025b; Ma et al., 2025; Liu et al., 2026a; Tang et al., 2026). More recently, generative reward models emit dynamic rubrics, critique rationales, preference tokens, or calibrated score distributions (Wu et al., 2025b; Jin et al., 2026; Kan et al., 2026; Wang et al., 2026b;a). Closest to our work, SpatialReward (Zhou et al., 2026b) grounds decomposed spatial constraints with expert detectors and reasons over the resulting observations. Its verification procedure is nonetheless determined prior to observation, following a pre-defined routine per task category (Zhou et al., 2026a). We instead learn evidence acquisition as a per-claim decision over an open tool catalog, aligned to the harness under outcome supervision. Spatial Intelligent Training Paradigm. Recent spatial intelligence studies reveal that learning complex spatial reasoning without grounded perceptual foundations induces severe shortcuts and optimization instability (Li et al., 2025; Liang et al., 2026; Yang et al., 2026). Consequently, struc- tured spatial data and staged learning are explored in multimodal representation (Cai et al., 2026; Li et al., 2025; Liang et al., 2026) and embodied planning (Yang et al., 2026; Qian et al., 2026; Tan et al., 2025; Weng et al., 2026; Tan et al., 2026c;d). However, these works primarily target spa- tial understanding and embodied planning through data scaling, representation adaptation, or staged alignment. In contrast, our work centers on spatial T2I post-training, where a capability dependency graph and an absolute mastery probe guide online prompt scheduling, periodic review, and targeted remediation. 3SPATIALCRITIC: TOOL-MEDIATED AGENTIC REWARD MODEL 3.1AGENT WITH A SPATIAL PERCEPTION HARNESS Problem Formulation. For spatial intelligence in text-to-image generation, the reward model re- quires to assess the spatial consistency of the generated image with the spatial constraints in the prompt, including object existence, attribute binding, counting, comparison, 2D position, back- ground placement, distance, depth, orientation, topology, occlusion, multi-object layout, and 3D pose. Action, physical-plausibility, and image-quality checks provide auxiliary context. Preference judgments typically follow either pairwise comparison or pointwise scoring S(I,c) (Xu et al., 2023; Wang et al., 2025b). Our framework supports both, but primarily uses pairwise comparison. Given x = (I A , I B ,c), SPATIALCRITIC predicts which image better satisfies the spatial constraints in c, yielding I A ā» I B . For score-wise setting, the preference is predicted by S(I A ,c) > S(I B ,c). Motivation: From Complex Spatial Relations to Atomic Verification. Spatial reward assessment benefits from complementary semantic and geometric capabilities. While VLMs excel at open- ended prompt comprehension, they fundamentally struggle with fine-grained spatial relations such as depth ordering and occlusion boundaries (Liu et al., 2026b). Conversely, fixed detector-based evaluators (Ghosh et al., 2023) rely on rigid and predefined heuristics and fail on complex open- world spatial relations. For example, consider the prompt āA palm tree is closer to the camera than a blue beach umbrellaā. The VLM is hard to accurately locate the position of the tree and the um- brella, and estimates their depth ordering, while the detector-based evaluator can provide the precise information. In this case, the complex spatial relation is decomposed into three atomic verification tasks: checking the presence of both objects, the umbrellaās color, and their depth ordering. Key Challenges. As discussed above, spatial reward modeling benefits from combining semantic reasoning with specialized perceptual evidence. Beyond integrating tool outputs within a predefined verification pipeline (Zhou et al., 2026b), we seek an agent that dynamically orchestrate vision foundation models for open-world spatial verification. This raises two complementary challenges: (1) for high-frequency and canonical spatial relations, how to make recurring verification patterns reliable and reusable? (2) for complex and unseen spatial relations, how to adapt these patterns to unfamiliar spatial compositions without being restricted to predefined tools pipeline or relying solely on unverified VLM judgments? Fundamentally, this requires balancing standardized execution on common and frequent relations with compositional flexibility on unseen and novel ones. Spatial Perception Harness. The harness integrates a modular perceptual toolkit, covering ground- ing, counting, segmentation, depth, 3D geometry, orientation, and contextual checks, with structured 3 skill guidance, specifying evidence requirements and output interpretations. For claims flagged as uncertain during decomposition, the harness guides tool planning along two paths. (1) When a claim matches a high-frequency relation, the agent uses a standardized skill with a fixed local tool routine and evidence interpretation, addressing the first challenge. (2) Otherwise, for complex or unfamiliar relations requiring query-specific composition, the harness provides an open tool catalog with ca- pability descriptions, guiding the agent to compose and sequence targeted tool plans. In Fig. 3(b), the plan grounds the instances, checks each dogās orientation relative to the fountain, and aggregates these relations to verify the requested counts. This balances robust execution on high-frequency relations with flexible open-world composition. Toolkit details appear in App. C. A palm tree is closer to the camerathan a blue beach umbrella. Agent Two of the three dogs face the fountain,but one faces away from it. Agent (a) Skill (b) Agent-planned Select & order + Figure 3: Spatial verification. (a) Fixed maskā depth routine; (b) agent-planned tool use. Agent Pipeline.As shown in Fig. 2(c), SPATIALCRITIC verifies spatial consistency through four interactive stages: (1) Claim de- composition: the VLM policy decomposes the prompt into atomic spatial claims and flags per- ceptual uncertainties across candidate images; (2) Tool planning: guided by the spatial percep- tion harness, the agent identifies missing visual evidence and plans targeted tool calls; (3) Tool execution: the harness invokes the selected vi- sion foundation models and collects structured perceptual observations; and (4) Evidence in- tegration: the policy integrates these observa- tions into its reasoning to resolve outstanding claims and produce the final preference verdict. 3.2VERIFIABLE SPATIAL PREFERENCE DATASET Although the proposed spatial perception harness balances standardized execution for frequent tasks with flexibility for out-of-distribution cases, efficiency and accuracy remain limited by redundant or failed tool calls, conflicting tool outputs, and ambiguity among similar tools. Fundamentally, these issues reflect a mismatch between the agentās policy and the spatial perception harness, requiring the agent to adapt its invocation, selection, and judgment policies to the harness. This mismatch is difficult to resolve through harness design alone, as agent and tool capabilities vary across tasks and are difficult to align a priori. We therefore employ RLVR to learn adaptive tool selection and evidence integration through interaction, balancing verification accuracy and tool-use cost without predefined execution trajectories. To this end, we construct a spatial preference dataset of image pairs with explicit preference labels for judgment verification and reward computation. Goal. To make these preferences effective reward signals for RL, the dataset should satisfy two requirements: diversity across spatial tasks and visual contexts, supporting generalizable use of the harness; and accuracy of the preference labels, providing reliable feedback for learning tool selection and evidence integration. We address diversity through diverse task and scene coverage, and accuracy through constraint-targeted preference construction, which combines explicit prompt constraints with text- and image-level validation. Diverse task and scene coverage. Our construction covers a registry of 36 scene settings and a taxonomy of 13 task categories, with 3D pose held out from agent training. We balance target- category coverage and sample compatible scenes, supplying their objects, scales, and reference- frame rules to the LLM. Duplicate checks and multiple image generators further broaden textual and visual variation. App. B.2 lists the task and scene inventories, the complete generation system prompt, and the task-conditioned input fields. Counterfactual preference construction & data pipeline. To obtain reliable spatial preference la- bels, we define unambiguous constraints and verify that the generated images form a valid contrast. We construct image preferences from reference promptsD 0 =c in four stages as shown in Fig. 4: (1) Prompt-pair construction. Targeted perturbations provide compositional negatives and counter- factual image pairs (Ma et al., 2023; Tang et al., 2026). Conditioned on the target category and scene constraints, DeepSeek-V4-Flash (DeepSeek-AI, 2026) jointly generates an unambiguous reference c + and three alternatives c ā , formingD 1 = (c + ,c ā ) (Fig. 4(a)). Each alternative is intended to 4 Training entries by perturbation label Comparison584 Occlusion493 Background446 Distance428 Depth424 Orientation387 Position-2D386 Topological337 Layout328 Attribute195 Count184 Existence173 Role swap24 RetainedText58.7%Image74.5%RL set23.7% (c) Review, selection & statistics Human review Sampled pairs Reference matches Violation is visible ā Keep ĆRemove (a) Preference prompt construction(b) Image generation & verification Reference cāŗ A marble statue stands on the leftside of the image. Perturbed prompts cā» A marble statue stands on the rightside. Twomarble statues stand on the left side. A marble statue stands behind the image. DeepSeek-V4-Flash 1. Generate cāŗ Scenes 36 scene settings UnderwaterForestMuseum Task categories 13 task categories Constraint rules Scene-and task-specific Candidate prompts cāŗ: ... leftside ... cā»: ... rightside ... cā»: Two... left side ... cā»: ... behind the image DeepSeek-V4-Flash Checks Clarity Plausibility Violation Coverage Reviewed Dā Valid prompts Discard Reviewed prompt ... facing the camera ... behind an old man. T2IFLUX.2 ... Qwen3-VL-235B Imageāprompt consistency 0/1010/10 Hard promptselection 2. Generate cā» T2IModels 0/10 PositionDepthCountOcclusionFrameObject setSingle editFormat Input pairs Shuffled order Tool-free judge Keep incorrect / invalid RLVR hard set Deduplicate + stratify DiscardDiscardRetain Figure 4: Spatial preference dataset construction. (a) Conditioned prompt generation, perturba- tion, and review. (b) Multiple image generation and VLM verification. (c) Human review, hard-set selection, and dataset statistics. violate one constraint while preserving the others; at least one targets the designated category. The generation instructions are provided in App. B.2. (2) Text-level validation. A separate review call to DeepSeek-V4-Flash yields D 2 = Filter(D 1 ) by filtering pairs for clarity, physical plausibility, and explicit violations, assigning perturbation categories, and checking target-category coverage. (3) Image-level verification. As illustrated in Fig. 4(b), we render both prompts of each pair inD 2 with FLUX.2-dev (Black Forest Labs, 2025), Hunyuan-Image-3.0 (Tencent Hunyuan Foundation Model Team, 2025), and Qwen-Image (Wu et al., 2025a), using the same generator within a pair, and retain only pairs that pass a Qwen-VL-235B (Bai et al., 2025a) check of reference compliance and the intended violation, so that preferences reflect image content rather than textual changes alone. We apply best-of-n selection over the rendered candidates and keep the highest-scoring reference- compliant image as I + . (4) Human inspection. To mitigate automated verification noise, human an- notators independently audit candidate pairs retained from image-level verification against identical criteria, confirming that I + satisfies c + and I ā exhibits the intended violation; failing pairs are re- moved. The counterfactual prompt c ā is discarded after verification, yieldingD 3 =(c + , I + , I ā ), where I + ā» I ā under c + . The inspection protocol and quality statistics are in App. B.1. 3.3AGENTāHARNESS ALIGNMENT VIA RLVR Building on the spatial perception harness and the spatial preference dataset, we align the agentās tool-selection and evidence-integration policy with the harness through RLVR. We optimize the agent policy while keeping the harness and perceptual tools fixed, using verifiable outcome rewards and token-budget penalties to improve spatial judgment accuracy and token efficiency. Hard set selection. The initial verification capability of the agent can induce an imbalance be- tween positive and negative outcome rewards. To provide balanced feedback across spatial tasks, we identify hard where the tool-free base model fails. Combining this difficulty mining with cat- egory rebalancing prioritizes complex spatial relations over elementary recognition. Deduplication then yields a balanced training set as shown in Fig. 4(c), with complete protocols and counts in App. B.1. Based on this dataset, we define rewards for preference judgment and tool use. Reward formulation. To improve token efficiency while preserving judgment accuracy, we com- bine an outcome reward with a token-budget penalty conditioned on correct judgments: R(Ļ ) =R out (Ļ ) + I[Ėy(Ļ ) = y gt ] clip(R tok (Ļ ),āĪŗ, 0).(1) where Ėy(Ļ ) is the predicted preference, y gt is its ground-truth label, and Īŗ bounds the auxiliary penalty. We set R out to +1, ā1, and ā1.5 for correct, incorrect, and invalid results, respectively. 5 The token penalty isR tok (Ļ ) =āĪ± max(0,n tok ān 0 ), wheren tok counts agent-generated tokens, excluding input prompts and tool observations; n 0 is the free token budget and Ī± penalizes each excess token. This objective encourages accurate judgments, targeted tool use, and concise evidence integration with fewer tokens and lower latency, as illustrated in Figs. 6, 7, and S1. Policy optimization. With the dataset and reward, we optimize Ļ Ļ using GRPO (Shao et al., 2024). For each randomly ordered image pair, we sample G rollouts under Ļ Ļ old following Sec. 3.1, then update the policy using group-relative reward advantages. Details appear in App. D.1. 4CURRICULUM LEARNING FOR SPATIAL T2I POST-TRAINING Building on SPATIALCRITIC, we introduce curriculum learning to address the lack of basic-to- complex progression. This requires (1) acquiring foundational spatial capabilities; and (2) compos- ing them for combination. We use a compositional capability graph to define these capabilities and their dependencies, and a progressive learning strategy to schedule foundational and compositional training accordingly, guided by policy competence, review, and remediation. 4.1COMPOSITIONAL CAPABILITY GRAPH CountPositionTopologyDepth LayoutOcclusion 1.Advance 2.Review Count Position Occlusion Unlocknewskills 3.Revisit Count Figure 5: CCG-guided progressive learning. Solid arrows show capability dependencies. To guide foundational learning and composi- tion, we examine how spatial capabilities re- late. Our key insight is that higher-order spa- tial tasks combine shared foundational capabil- ities in different ways. CCG represents these foundations and dependencies beyond a single easy-to-hard ordering. Specifically, it is a di- rected acyclic graphG = (V,E ) with 13 capabil- ity nodes, where each edge (u,v) requires u to precede v in training. In Fig. 5, Layout and Oc- clusion share Topology; Layout additionally de- pends on Count and Position, and Occlusion on Depth. These prescribed dependencies provide an inductive bias for curriculum design, with the complete mapping in App. D.4. This graph fa- cilitates training scheduling with a progressive learning strategy. 4.2PROGRESSIVE LEARNING WITH REVIEW AND REMEDIATION Our progressive learning framework has two components in Fig. 5: (1) forward progression, which develops foundational capabilities and advances to dependent tasks based on mastery; and (2) review and remediation, which addresses forgetting and joint-constraint failures by reassessing learned capabilities individually and jointly. Forward progression. Let P(v) denote the prerequisites of capability v in CCG. At assessment round t, we obtain the mean evaluation rewardS t v for each assessed capability. To reduce sensitivity to outliers, we update its running score as Ģ S t v = Smooth(S t v , Ģ S tā1 v ), where Smooth is an expo- nential moving average, detailed in App. D.4. Mastery requires Ģ S v ā„ Ļ and at least N min image assessments. To prevent curriculum deadlock, unmastered capabilities permit fallback progression after N max image assessments. We define the mastered and fallback sets as M = v āV | Ģ S v ā„ Ļ, n v ā„ N min | z Mastered capabilities , F = v āV | v /āM, n v ā„ N max |z Unmastered at assessment limit .(2) where n v counts image assessments for capability v. M contains mastered capabilities, while F contains those still unmastered at the assessment limit. Both sets permit downstream progression, but capabilities in F remain candidates for review and remediation. A successor capability v be- comes unlocked, or eligible for training, when all its prerequisites belong to either set: Unlocked(v) āā P(v)āMāŖF.(3) Fig. 5 illustrates progression from foundational to composite capabilities. Once Count, Position, Topology, and Depth meet the mastery criterion, they enterM. Layout requires Count, Position, and 6 Topology, while Occlusion requires Topology and Depth. Both sets therefore lie inM, unlocking Layout and Occlusion for training. The Advance phase shifts training toward these capabilities, with policy updates on the selected prompts using standard diffusion RL (Liu et al., 2025a). Review and remediation. Forward progression alone does not ensure capability retention or com- positional performance. (1) Review periodically evaluates capabilities inMāŖF through individual and compositional tasks, updating Ģ S t v using the same evaluation and smoothing procedure. Capabil- ities failing the mastery criterion in Eq. (2) form the remediation setC and are removed fromM via M ā M . (2) Remediation applies targeted training toC before forward progression resumes. Capabilities satisfying the mastery criterion in either stage enterM and leaveF ; those remaining in F remain eligible for subsequent review and remediation. For example, jointly assessing Count, Po- sition, and Occlusion in Fig. 5 yieldsC =Count when only Count fails the mastery criterion. The Revisit step then applies counting-specific training to improve performance under joint constraints. App. D.4 specifies review schedules and training budgets. 5EXPERIMENTS 5.1EXPERIMENTAL SETUPS Implementation. We fine-tune Qwen3-VL-8B (Bai et al., 2025a) into SPATIALCRITIC with GRPO on 4,389 spatial preference pairs on 16 GPUs. We also evaluate Qwen3-VL-30B with the same spatial perception harness. We set n 0 = 400 free tokens and an excess-token penalty of Ī± = 1.25Ć 10 ā3 ; see App. D.2 for reward coefficients. For T2I post-training, we use Flow-GRPO (Liu et al., 2025a) with SD3.5-Medium (Esser et al., 2024) and BAGEL (Deng et al., 2025). Our curriculum schedules prompts by capability dependencies and mastery; baselines use GenEval (Ghosh et al., 2023) or GenEval-2 (Kamath et al., 2025). Each Flow-GRPO iteration samples 24 images for each of 24 prompts, assessing four per group to track mastery with Ļ = 0.60 and N min = 200. Further details appear in App. D. Benchmarks and Datasets. We evaluate spatial preference judgments on SpaRW-Evalās 5,572 pairs, reporting per-category and category-size-weighted overall accuracy. Construction follows Sec. 3.2, with evaluation details in App. B.3. We evaluate generation on TIIF-Bench (Wei et al., 2025), UniGenBench++ (Wang et al., 2025a), and SpatialGenEval (Wang et al., 2026c), whose 5,000, 600, and 1,230 prompts cover three instruction-following levels, 10 semantic dimensions, and 10 spatial sub-domains, respectively. The alignment table reports seven benchmark scores and their unweighted mean; ImageReward, PickScore, and HPSv2 separately measure preference quality. Following the benchmarkās recommendation to use the Qwen series, we adopt the stronger Qwen3- VL-235B to evaluate generation under the recommended setting. Compared Methods. Reward baselines include Aesthetic Predictor V2.5 (discus0434, 2024), CLIP (Radford et al., 2021), ImageReward (Xu et al., 2023), PickScore (Kirstain et al., 2023), HPSv2 (Wu et al., 2023), HPSv3 (Ma et al., 2025), UnifiedReward (Wang et al., 2025c), and Spa- tialScore (Tang et al., 2026). We also compare zero-shot VLMs: Qwen2.5-VL (Bai et al., 2025b), Qwen3-VL (Bai et al., 2025a), Qwen3-Omni (Xu et al., 2025), DeepSeek (DeepSeek-AI, 2026), GLM (Z.ai, 2026), Doubao-Seed (Bytedance Seed, 2026), Gemini (Google DeepMind, 2026a;b), Qwen3.8 (Qiu et al., 2026), and Grok (xAI, 2026). For RL training, we compare three baselines, Flow-GRPO (Liu et al., 2025a), DiffusionNFT (Zheng et al., 2026), and AWM (Xue et al., 2026), with reward comparisons covering GenEval (Ghosh et al., 2023), GenEval-2 (Kamath et al., 2025), SpatialReward (Zhou et al., 2026b), and SpatialScore (Tang et al., 2026). 5.2MAIN RESULTS Accuracy of Preference Judgments. Tab. 1 shows that 8B SPATIALCRITIC achieves 95.1% ver- ification accuracy, compared with 84.9% for Qwen3-VL-8B and 87.5% for SpatialScore. Agentā harness alignment raises accuracy from 92.9% to 95.1%, surpassing 320B GLM-5.3-Flash at 94.5%. The 30B variant reaches 95.5%, approaching Gemini 3.8 Flash at 96.7%. Quality of Reward Signals. In Tab. 2, SPATIALCRITIC leads Best-of-N selection with 74.4% accuracy, exceeding GLM-5.3-Flash at 71.8% and the selection-free baseline at 70.8%. It leads in 7 Table 1: Spatial preference accuracy (%) on SpaRW-Eval. Per column, bold marks the best open-weight result, underline the second best, and bold italic underlinethe best across all models. Darker shading indicates higher accuracy. Overall subscripts: relative gain over their backbone VLM (Qwen3-VL-8B/-30B). ā : released after July 2026. ModelParams Obj-Attr Obj-Exist Count Bg Comp 3D-Pose Depth Orient Pos-2D Topo Layout Occl DistOverallā Open-weight: reward models Aesthetic predictor0.4B54.351.446.8 46.7 45.047.848.142.451.947.5 46.7 55.4 51.351.6 CLIP0.4B72.785.884.4 52.0 66.776.552.862.050.663.3 66.7 63.1 59.075.3 PickScore1.0B82.386.485.3 49.3 65.079.372.278.350.672.5 73.3 61.5 71.881.1 HPSv21.0B84.394.090.8 40.0 55.075.467.672.846.878.3 76.0 52.3 79.583.1 UnifiedReward7B85.791.489.9 61.3 73.387.163.971.757.084.2 81.3 66.2 89.7 85.7 SpatialScore7B89.198.492.7 98.778.361.776.984.886.195.0 90.7 80.0 92.387.5 Open-weight: large vision-language models Qwen3-VL-8B8B89.582.385.3 58.7 70.091.058.367.448.182.5 81.3 67.7 79.584.9 DeepSeek-V4.1-Flash ā 552B92.294.095.4 70.7 88.391.070.479.374.791.7 86.7 72.3 76.990.9 Qwen3-VL-235B235B95.394.693.6 88.0 81.795.768.576.167.195.0 90.7 70.8 92.3 93.3 Qwen3-Omni-30B35B95.395.595.4 69.3 88.395.174.181.568.491.7 89.3 80.0 89.793.5 Qwen3-VL-30B30B95.395.992.7 73.3 83.394.484.378.369.695.885.3 80.0 94.993.7 GLM-5.3-Flash ā 320B93.996.7 99.177.3 93.397.087.092.474.795.894.7 86.2 94.994.5 Agentic Reward Model SPATIALCRITIC8B94.293.690.890.788.392.688.082.679.793.393.381.589.792.9 +9.4% w/ AHA8B96.096.298.284.088.395.690.785.981.095.094.781.589.795.1 +12.0% SPATIALCRITIC30B96.696.095.489.395.095.988.090.291.197.590.778.592.395.5 +1.9% Proprietary: large vision-language judges Doubao-Seed-2.0-Lite ā86.092.393.6 68.0 80.090.665.776.168.491.7 88.0 61.5 76.987.2 Gemini 3.5 Flashā90.393.192.7 70.7 65.088.356.576.163.390.0 81.3 52.3 76.988.4 Qwen3.8-Flash-Next ā ā97.0 97.9 99.184.0 81.796.676.994.679.796.7 97.376.9 92.395.9 Grok 4.6 ā ā96.698.398.2 88.0 91.795.4 93.591.392.495.8 94.7 84.6 97.496.3 Doubao-Seed-2.0-Pro ā96.998.597.2 93.3 96.796.684.3 95.783.595.8 97.383.1 100.096.6 Gemini 3.8 Flash ā ā96.498.998.2 86.7 95.0 97.491.787.0 97.595.0 94.7 87.794.996.7 1 2 3 1 2 SpatialCriticw/AHA:Thetool confirmsBhasonlytwocandles, ...Therefore,Aisthecorrectverdict. Prompt: Three white candles arranged in a single row on the round table. SpatialCritic:Thetoolconfirms imageBhasonlytwocandles,... Therefore,imageBisthecorrect verdict. AB Stage 4: DecisionāReduced hallucinations SpatialCriticw/AHA:A'ssunflowers areclusteredandoverlapping,...Tool evidenceconfirmsB'srowalignment andA'sclusterarrangement. Prompt: Three sunflowers arranged in a single row on a wooden bench. SpatialCritic:...Bothimagesshow threesunflowersarrangedonabench ...A. 1 2 3 1 2 3 Stage 1:ClaimāAccuratelyfinedecisive factor A B Figure 6: Agentāharness alignment cases. Left: alignment reduces final-judgment hallucinations, correctly selecting A because evidence shows only two candles in B. Right: it identifies the decisive single-row constraint and distinguishes Bās row from Aās cluster using tool evidence. App. Fig. S1 illustrates improved planning. nine of ten categories, including all five geometry and spatial-relation categories. Position accuracy reaches 73.3%, versus 67.6% for the best baseline. &) "' ) &) "' ) + , , , , "$$ $ $"!*#$) ' ) %$ %%"( )' ThetoolconfirmsthatinA,thedonutisrestingon topofthemug,satisfyingtheprompt.InB,themug isontopofthedonut,violatingtheprompt.Visual judgmentwassufficientforStage1,andtools confirmtheverdict. ThetoolevidenceconfirmsthatinImageA,the donutisrestingontopoftheceramicmug,which satisfiestheprompt.InImageB,themugisresting ontopofthedonut,whichviolatestheprompt.Since thetopologicalrelationisthekeycriterionandis onlysatisfiedinA,thefinalverdictisB. 67 tokens 49 tokens Figure 7: Efficiency and decision correction. Alignment re- duces tokens and latency while rectifying judgment errors. Effectiveness of Harness and AgentāHarnessAlignment. The harness provides verifi- cation guidance and grounded evidence,whilealignment improves evidence integration to mitigate hallucinations and support reliable judgments. In Fig. 7, alignment rectifies the unaligned verdict error while reducing tokens from 67 to 49 and normalized latency from 1.000 to 0.653, yielding a more concise and accurate judgment. Figs. 6 and S1 illustrate targeted constraint identification, fewer irrelevant tool calls, and active verification of uncertain claims. 8 Table 2: Reward-guided Best-of-N image generation. Bold and underlined entries indicate the best and second-best results. SelectorAttr.ObjectComp.CausalMotionLayoutOccl.Orient.Pos.Prox.Overallā SD3.5-Medium (base)87.8994.6338.6280.5772.6872.6859.3565.7766.5968.8670.76 Qwen2.5-VL-7B88.1094.4038.5080.6072.2072.5059.7065.3065.6068.5070.54 PickScore89.4394.8039.1180.4972.5273.5057.8065.1265.6969.5170.80 ImageReward88.9495.3738.3781.1473.7472.2058.0566.1065.1269.2770.83 CLIP89.1994.8040.5781.1472.7672.8559.5163.9065.4569.5970.98 HPSv289.1194.8839.8480.4973.3372.2058.7067.4065.5369.5971.11 HPSv388.9494.0740.2480.8174.2373.3358.0565.8566.4271.2271.32 UnifiedReward88.7094.8839.3581.3074.9672.9358.2166.1867.6470.0071.41 GLM-5.3-Flash89.2794.3939.9281.30 75.8574.3157.7266.4267.6470.7371.76 SPATIALCRITIC90.4194.9643.1783.2579.4377.7260.8168.2173.3373.0974.44 Table 3: Reward-guided RL post-training. Bold marks the best result within each backbone; CL denotes curriculum learning. ā This non-curriculum setting uses GenEval-2 prompts without its benchmark-specific questions. MethodReward TIIF-BenchUniGenBench++SpatialGenEvalPreference-model scores Avg BRARRR2D3DBasicSpatialIRPSHPS SD3.5-Medium (base)ā82.277.668.082.981.492.065.00.7400.8510.28378.4 w/ Flow-GRPOGenEval57.853.745.443.638.775.948.1-1.3170.7270.16251.9 w/ DiffusionNFTGenEval68.557.756.866.944.165.953.7-1.3960.7390.14159.1 w/ AWMGenEval72.869.462.969.444.183.256.1-1.1160.7570.16565.4 w/ Flow-GRPOSpatialReward81.380.169.676.277.392.665.00.4510.8230.25977.4 w/ Flow-GRPOSpatialScore83.480.969.492.178.793.767.30.8850.8570.30680.8 w/ Flow-GRPOGenEval-284.881.172.184.284.193.768.40.2410.7950.22681.2 w/ DiffusionNFTGenEval-285.183.773.489.082.894.068.30.5880.8230.27182.3 w/ Flow-GRPO ā SPATIALCRITIC83.380.369.893.385.394.467.00.8570.8550.30481.9 w/ Flow-GRPO (CL)SPATIALCRITIC86.283.974.193.385.395.069.50.9080.8570.30583.9 BAGEL (base)ā86.482.573.891.882.894.270.90.7530.8550.28383.2 w/ DiffusionNFTGenEval86.879.574.592.382.490.867.80.8250.8370.26482.0 w/ DiffusionNFTGenEval-287.680.777.390.687.392.270.00.7640.8260.25783.7 w/ Flow-GRPOGenEval85.078.472.488.077.378.962.30.6660.8320.26877.5 w/ Flow-GRPOGenEval-286.980.974.990.884.080.565.60.6240.8170.23380.5 w/ Flow-GRPO ā SPATIALCRITIC86.182.274.893.282.793.673.10.7750.8900.29483.7 w/ Flow-GRPO (CL)SPATIALCRITIC88.383.977.194.689.394.777.80.8020.9130.29886.5 Effectiveness in RL Post-Training. Tab. 3 shows that SpatialCritic without CL raises averages from 78.4 to 81.9 on SD3.5-Medium and 83.2 to 83.7 on BAGEL, but does not consistently lead. On SD3.5-Medium, its average is below DiffusionNFT + GenEval-2 (82.3), and SpatialGenEval Spatial accuracy (67.0%) trails SpatialScore (67.3%) and Flow-GRPO + GenEval-2 (68.4%). Adding CL raises averages to 83.9 and 86.5 and Spatial accuracy to 69.5% and 77.8%, respectively. The ablation on curriculum dataset in App. A.3 confirms the effectiveness of curriculum scheduling. 6CONCLUSION In this work, we investigate spatial T2I post-training from the perspectives of reward accuracy and rollout organization. We identify two limitations beyond policy optimization: unreliable spatial rewards and rollouts that target complex compositions without progressive learning. First, we in- troduce SPATIALCRITIC, which combines a spatial perception harness with specialized vision tools for grounded preference judgments. Spatial preference data and agentāharness alignment via RLVR improve its tool selection and evidence integration. Second, we introduce curriculum learning based on a Compositional Capability Graph, guiding foundational capability development and subsequent composition through mastery-based progression, review, and remediation. Experiments demonstrate accurate preference judgments on SpaRW-Eval, effective reward signals for Best-of-N selection, and improved RL alignment on SD3.5-Medium and BAGEL. Limitations. Our evaluation uses a fixed tool configuration, leaving transfer to updated or alternative tools unexplored. Additionally, robustness to temporary tool failures or missing outputs also remains untested. Future work will investigate cross-tool transfer and failure-aware fallback strategies. 9 AI USE STATEMENT We used generative models to construct synthetic spatial prompts and images and to review candi- date preference labels, as described in Sec. 3.2 and App. B.1. We used AI assistants for literature search and summarization to support related-work discovery. We also used AI tools to aid and pol- ish paper writing, and for feedback on experimental analysis and inspection of data-processing code and metadata. ETHICS STATEMENT This work studies spatial verification and text-to-image generation using synthetic preference data and existing models and benchmarks. Synthetic data can inherit biases from its generators, and model-based verification can introduce systematic annotation errors. Human inspection provides an additional check but does not guarantee unbiased or error-free labels. Improved spatial fidelity may also make misleading synthetic images more convincing; spatial correctness should not be interpreted as evidence of authenticity or safety. Model and dataset reuse and redistribution remain subject to the applicable licenses and access conditions. REPRODUCIBILITY STATEMENT We describe the perceptual toolkit in App. C, preference-data construction and selection in App. B.1, and generation prompts in App. B.2. App. B.3 details the evaluation data and protocol. Training settings, reward coefficients, policy optimization, and curriculum parameters appear in App. D, D.2, D.1, and D.4. We will open-source our code upon acceptance. REFERENCES Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3- VL technical report. CoRR, abs/2511.21631, 2025a. doi: 10.48550/ARXIV.2511.21631. URL https://doi.org/10.48550/arXiv.2511.21631. Shuai Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, Sibo Song, Kai Dang, Peng Wang, Shijie Wang, Jun Tang, Humen Zhong, Yuanzhi Zhu, Mingkun Yang, Zhaohai Li, Jianqiang Wan, Pengfei Wang, Wei Ding, Zheren Fu, Yiheng Xu, Jiabo Ye, Xi Zhang, Tianbao Xie, Zesen Cheng, Hang Zhang, Zhibo Yang, Haiyang Xu, and Junyang Lin. Qwen2.5-vl technical report. CoRR, abs/2502.13923, 2025b. doi: 10.48550/ARXIV.2502.13923. URL https://doi.org/10. 48550/arXiv.2502.13923. Kevin Black, Michael Janner, Yilun Du, Ilya Kostrikov, and Sergey Levine. Training diffusion models with reinforcement learning.In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https://openreview.net/forum?id=YCWjhGrJFD. Black Forest Labs. FLUX.2: Frontier visual intelligence, 2025. URL https://bfl.ai/blog/ flux-2. Bytedance Seed. Seed2.0 model card: Towards intelligence frontier for real-world complexity. CoRR, abs/2607.00248, 2026. doi: 10.48550/ARXIV.2607.00248. URL https://doi.org/ 10.48550/arXiv.2607.00248. Zhongang Cai, Ruisi Wang, Chenyang Gu, Fanyi Pu, Junxiang Xu, Yubo Wang, Wanqi Yin, Zhitao Yang, Chen Wei, Qingping Sun, Tongxi Zhou, Jiaqi Li, Hui En Pang, Oscar Qian, Yukun Wei, 10 Zhiqian Lin, Xuanke Shi, Kewang Deng, Xiaoyang Han, Zukai Chen, Xiangyu Fan, Hanming Deng, Lewei Lu, Liang Pan, Bo Li, Ziwei Liu, Quan Wang, Dahua Lin, and Lei Yang. Scaling spatial intelligence with multimodal foundation models. In Proceedings of the IEEE/CVF Con- ference on Computer Vision and Pattern Recognition (CVPR 2026), 2026. doi: 10.48550/ARXIV. 2511.13719. URL https://doi.org/10.48550/arXiv.2511.13719. Kevin Clark, Paul Vicol, Kevin Swersky, and David J. Fleet. Directly fine-tuning diffusion mod- els on differentiable rewards. In The Twelfth International Conference on Learning Represen- tations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https: //openreview.net/forum?id=1vmSEVL19f. DeepSeek-AI. DeepSeek-V4: Towards highly efficient million-token context intelligence. arXiv preprint arXiv:2606.19348, 2026. URL https://arxiv.org/abs/2606.19348. Model variant: DeepSeek-V4-Flash. DeepSeek-AI.DeepSeek-V4.1-Flash: Pushing the limits of KV cache compression.CoRR, abs/2609.19969, 2026. doi: 10.48550/ARXIV.2609.19969. URL https://doi.org/10. 48550/arXiv.2609.19969. Chaorui Deng, Deyao Zhu, Kunchang Li, Chenhui Gou, Feng Li, Zeyu Wang, Shu Zhong, Weihao Yu, Xiaonan Nie, Ziang Song, Guang Shi, and Haoqi Fan. Emerging properties in unified multi- modal pretraining, 2025. URL https://arxiv.org/abs/2505.14683. Model: BAGEL. discus0434. Aesthetic Predictor V2.5. GitHub repository, 2024. URL https://github.com/ discus0434/aesthetic-predictor-v2-5. SigLIP-based aesthetic score predictor; last commit 2024-12-18, accessed September 25, 2026. Patrick Esser, Sumith Kulal, Andreas Blattmann, Rahim Entezari, Jonas M Ģ uller, Harry Saini, Yam Levi, Dominik Lorenz, Axel Sauer, Frederic Boesel, Dustin Podell, Tim Dockhorn, Zion English, and Robin Rombach. Scaling rectified flow transformers for high-resolution image synthesis. In Forty-first International Conference on Machine Learning (ICML 2024), volume 235 of Proceed- ings of Machine Learning Research, p. 12606ā12633. PMLR, 2024. doi: 10.48550/ARXIV. 2403.03206. URL https://proceedings.mlr.press/v235/esser24a.html. Yuanting Fan, Jun Liu, Bin-Bin Gao, Xiaochen Chen, Yuhuan Lin, Zhewei Dai, Jiawei Zhan, and Chengjie Wang. UniDG: Universal defect generation via defect-context editing. arXiv preprint arXiv:2604.08915, 2026. Dhruba Ghosh, Hannaneh Hajishirzi, and Ludwig Schmidt. GenEval: An object-focused framework for evaluating text-to-image alignment. In Thirty-seventh Conference on Neural Information Pro- cessing Systems Datasets and Benchmarks Track, 2023. URL https://openreview.net/ forum?id=Wbr51vK331. Google DeepMind. Gemini 3.5 Flash ā model card, 2026a. URL https://deepmind. google/models/model-cards/gemini-3-5-flash. Google DeepMind. Gemini 3.8 Flash ā model card, 2026b. URL https://deepmind. google/models/model-cards/gemini-3-8-flash. Xin Jin, Huanqia Cai, Zhen Li, Zechao Zhan, Dengyang Jiang, Aiming Hao, Yuming Jiang, Xiang- peng Yang, Chunle Guo, Peng Gao, Ming-Ming Cheng, and Steven C. H. Hoi. Z-Reward: Beyond scalar rewards by internalizing reasoning into score distributions. CoRR, abs/2606.09076, 2026. doi: 10.48550/ARXIV.2606.09076. URL https://doi.org/10.48550/arXiv.2606. 09076. Amita Kamath, Kai-Wei Chang, Ranjay Krishna, Luke Zettlemoyer, Yushi Hu, and Marjan Ghazvininejad. GenEval 2: Addressing benchmark drift in text-to-image evaluation. CoRR, abs/2512.16853, 2025. doi: 10.48550/ARXIV.2512.16853. URL https://doi.org/10. 48550/arXiv.2512.16853. Zijian Kan, Wei Wang, Long Luo, Bing Zhao, Xuan Ren, Weixu Qiao, Wenbo Li, Hu Wei, and Lin Qu. Rubricrm: Generative reward modeling via dynamic rubrics for image generation and editing. In Proceedings of the 2026 Conference on Empirical Methods in Natural Language Processing 11 (EMNLP 2026), 2026. doi: 10.48550/ARXIV.2608.26956. URL https://doi.org/10. 48550/arXiv.2608.26956. Yuval Kirstain, Adam Polyak, Uriel Singer, Shahbuland Matiana, Joe Penna, and Omer Levy. Pick-a-pic: An open dataset of user preferences for text-to-image generation. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Ad- vances in Neural Information Processing Systems 36: Annual Conference on Neural Infor- mation Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URL https://papers.nips.c/paper_files/paper/2023/hash/ 73aacd8b3b05b4b503d58310b523553c-Abstract-Conference.html. Hongxing Li, Dingming Li, Zixuan Wang, Yuchen Yan, Hang Wu, Wenqi Zhang, Yongliang Shen, Weiming Lu, Jun Xiao, and Yueting Zhuang. Spatialladder: Progressive training for spatial rea- soning in vision-language models. CoRR, abs/2510.08531, 2025. doi: 10.48550/ARXIV.2510. 08531. URL https://doi.org/10.48550/arXiv.2510.08531. Huizhi Liang, Yichao Shen, Yu Deng, Sicheng Xu, Zhiyuan Feng, Tong Zhang, Yaobo Liang, and Jiaolong Yang. Hispatial: Taming hierarchical 3d spatial understanding in vision-language mod- els. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR 2026), 2026. doi: 10.48550/ARXIV.2603.25411. URL https://doi.org/10. 48550/arXiv.2603.25411. Jie Liu, Gongye Liu, Jiajun Liang, Yangguang Li, Jiaheng Liu, Xintao Wang, Pengfei Wan, Di Zhang, and Wanli Ouyang. Flow-grpo: Training flow matching models via online RL. CoRR, abs/2505.05470, 2025a. doi: 10.48550/ARXIV.2505.05470. URL https://doi.org/10. 48550/arXiv.2505.05470. Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Menghan Xia, Xintao Wang, Xiaohong Liu, Fei Yang, Pengfei Wan, Di Zhang, Kun Gai, Yujiu Yang, and Wanli Ouyang. Improving video generation with human feedback. In Advances in Neural Information Processing Systems 38 (NeurIPS 2025), 2025b. doi: 10.48550/ARXIV. 2501.13918. URL https://papers.nips.c/paper_files/paper/2025/hash/ 76227feb18ea0e40bd15cf02c33e18e-Abstract-Conference.html. Yijun Liu, Jie Huang, Zeyue Xue, Yuming Li, Ruizhe He, Haoran Li, Shijia Ge, and Siming Fu. Hpsv3++: Scaling reward models across the full spectrum of diffusion model capabilities. CoRR, abs/2606.14657, 2026a. doi: 10.48550/ARXIV.2606.14657. URL https://doi.org/10. 48550/arXiv.2606.14657. Yiqian Liu, Iuliia Kotseruba, and John K. Tsotsos. Disentangling pictorial cue understanding from language bias in VLMs via depth ordering task. In Proceedings of the European Conference on Computer Vision (ECCV), 2026b. URL https://arxiv.org/abs/2607.01503. Yuhang Ma, Xiaoshi Wu, Keqiang Sun, and Hongsheng Li. Hpsv3: Towards wide-spectrum human preference score. In IEEE/CVF International Conference on Computer Vision, ICCV 2025, Hon- olulu, HI, USA, October 19-25, 2025, p. 15086ā15095. IEEE, 2025. doi: 10.1109/ICCV51701. 2025.01400. URL https://doi.org/10.1109/ICCV51701.2025.01400. Zixian Ma, Jerry Hong, Mustafa Omer Gul, Mona Gandhi, Irena Gao, and Ranjay Krishna. CREPE: Can vision-language foundation models reason compositionally?In Proceed- ings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 10910ā10921, 2023.URL https://openaccess.thecvf.com/content/ CVPR2023/html/Ma_CREPE_Can_Vision-Language_Foundation_Models_ Reason_Compositionally_CVPR_2023_paper.html. Kangan Qian, Chuchu Xie, Yang Zhong, Jingrui Pang, Siwen Jiao, Sicong Jiang, Zilin Huang, Yunlong Wang, Kun Jiang, Mengmeng Yang, Hao Ye, Guanghao Zhang, Hangjun Ye, Guang Chen, Long Chen, and Diange Yang. Xembodied: A foundation model with enhanced geometric and physical cues for large-scale embodied environments. CoRR, abs/2604.18484, 2026. doi: 10. 48550/ARXIV.2604.18484. URL https://doi.org/10.48550/arXiv.2604.18484. 12 Zihan Qiu, Zekun Wang, Xiao Li, Yanpeng Li, Yang Xu, Yixuan Wang, Huaqing Zhang, Rui Men, Bochao Mao, Chengruidong Zhang, Fan Zhou, Hao Luo, Haofeng Huang, Haoran Lian, Haoyan Huang, Hongqing Chen, Jianwei Zhang, Jing Xu, Junjie Wang, Langshi Chen, Liangyu Wang, Linlang Jiang, Man Yuan, Minmin Sun, Peng Jin, Siqi Zhang, Siyu Wang, Xingzhang Ren, Yakai Wang, Yi Zhang, Yiming Dong, Yizhong Cao, Yubo Ma, Yunfei Mao, Bo Zheng, and Dayiheng Liu. On the design of Qwen3.8-Next architecture: Evaluation, efficiency, and training stability. CoRR, abs/2608.30320, 2026. doi: 10.48550/ARXIV.2608.30320. URL https://doi.org/ 10.48550/arXiv.2608.30320. Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Ma- rina Meila and Tong Zhang (eds.), Proceedings of the 38th International Conference on Ma- chine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Ma- chine Learning Research, p. 8748ā8763. PMLR, 2021. URL http://proceedings.mlr. press/v139/radford21a.html. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo.DeepSeekMath: Pushing the limits of mathematical reasoning in open language models.CoRR, abs/2402.03300, 2024.URL https://arxiv.org/abs/2402.03300. Zihan Su, Xuerui Qiu, Hongbin Xu, Tangyu Jiang, Jun-hao Zhuang, Chun Yuan, Ming Li, Shengfeng He, and Fei Yu. Safe-sora: Safe text-to-video generation via graphical watermarking. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https: //openreview.net/forum?id=pWzcPRIKp8. Zihan Su, Hongyang Wei, Kangrui Cen, Yong Wang, Guanhua Chen, Chun Yuan, and Xiangxiang Chu. Generation enhances understanding in unified multimodal models via multi-representation generation. In Forty-third International Conference on Machine Learning, 2026. URL https: //openreview.net/forum?id=uYaNE2Cys2. Xiaofeng Tan, Hongsong Wang, Xin Geng, and Pan Zhou. SoPo: Text-to-motion generation using semi-online preference optimization. In Advances in Neural Information Processing Systems, 2025. URL https://arxiv.org/abs/2412.05095. Xiaofeng Tan, Jun Liu, Yuanting Fan, Bin-Bin Gao, Xi Jiang, Xiaochen Chen, Jinlong Peng, Chengjie Wang, Hongsong Wang, and Feng Zheng. ConsistentRFT: reducing visual halluci- nations in flow-based reinforcement fine-tuning. CoRR, abs/2602.03425, 2026a. doi: 10.48550/ ARXIV.2602.03425. URL https://doi.org/10.48550/arXiv.2602.03425. Xiaofeng Tan, Jun Liu, Bin-Bin Gao, Yuanting Fan, Xi Jiang, Chengjie Wang, Hongsong Wang, and Feng Zheng. When policy entropy constraint fails: Preserving diversity in flow-based RLHF via perceptual entropy. In Advances in Neural Information Processing Systems, 2026b. URL https://arxiv.org/abs/2605.12112. Xiaofeng Tan, Wanjiang Weng, Haodong Lei, and Hongsong Wang. EasyTune: Efficient step- aware fine-tuning for diffusion-based motion generation. In The Fourteenth International Confer- ence on Learning Representations, 2026c. URL https://openreview.net/forum?id= Fy1EoIaAzQ. Xiaofeng Tan, Wanjiang Weng, Hongsong Wang, Fang Zhao, Xin Geng, and Liang Wang. Mo- tionRFT: Unified reinforcement fine-tuning for text-to-motion generation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2026d. URL https://arxiv.org/abs/2603. 27185. Zhenyu Tang, Chaoran Feng, Yufan Deng, Jie Wu, Xiaojie Li, Rui Wang, Yunpeng Chen, and Daquan Zhou. Enhancing spatial understanding in image generation via reward modeling. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2026), 2026. doi: 10.48550/ARXIV.2602.24233. URL https://doi.org/10.48550/ arXiv.2602.24233. 13 Tencent Hunyuan Foundation Model Team. HunyuanImage 3.0 technical report. arXiv preprint arXiv:2509.23951, 2025. URL https://arxiv.org/abs/2509.23951. Haozhe Wang, Cong Wei, Weiming Ren, Jiaming Liu, Fangzhen Lin, and Wenhu Chen. Ra- tionalrewards: Reasoning rewards scale visual generation both training and test time. CoRR, abs/2604.11626, 2026a. doi: 10.48550/ARXIV.2604.11626. URL https://doi.org/10. 48550/arXiv.2604.11626. Yibin Wang, Zhimin Li, Yuhang Zang, Jiazi Bu, Yujie Zhou, Yi Xin, Junjun He, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. Unigenbench++: A unified semantic evaluation bench- mark for text-to-image generation. CoRR, abs/2510.18701, 2025a. doi: 10.48550/ARXIV.2510. 18701. URL https://doi.org/10.48550/arXiv.2510.18701. Yibin Wang, Zhimin Li, Yuhang Zang, Yujie Zhou, Jiazi Bu, Chunyu Wang, Qinglin Lu, Cheng Jin, and Jiaqi Wang. Pref-grpo: Pairwise preference reward-based GRPO for stable text-to-image reinforcement learning. CoRR, abs/2508.20751, 2025b. doi: 10.48550/ARXIV.2508.20751. URL https://doi.org/10.48550/arXiv.2508.20751. Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for mul- timodal understanding and generation. CoRR, abs/2503.05236, 2025c. doi: 10.48550/ARXIV. 2503.05236. URL https://doi.org/10.48550/arXiv.2503.05236. Yibin Wang, Yuhang Zang, Feng Han, Jiazi Bu, Yujie Zhou, Cheng Jin, and Jiaqi Wang. Unified personalized reward model for vision generation. CoRR, abs/2602.02380, 2026b. doi: 10.48550/ ARXIV.2602.02380. URL https://doi.org/10.48550/arXiv.2602.02380. Zengbin Wang, Xuecai Hu, Yong Wang, Feng Xiong, Man Zhang, and Xiangxiang Chu. Everything in its place: Benchmarking spatial intelligence of text-to-image models. In The Fourteenth In- ternational Conference on Learning Representations, 2026c. URL https://openreview. net/forum?id=ddFN3lWpIr. Xinyu Wei, Jinrui Zhang, Zeqing Wang, Hongyang Wei, Zhen Guo, Bairui Li, and Lei Zhang. Tiif-bench: How does your T2I model follow your instructions? CoRR, abs/2506.02161, 2025. doi: 10.48550/ARXIV.2506.02161. URL https://doi.org/10.48550/arXiv.2506. 02161. Wanjiang Weng, Xiaofeng Tan, Junbo Wang, Guo-Sen Xie, Pan Zhou, and Hongsong Wang. Re- Align: Text-to-motion generation via step-aware reward-guided alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, 2026. URL https://arxiv.org/abs/2511. 19217. Chenfei Wu, Jiahao Li, Jingren Zhou, Junyang Lin, Kaiyuan Gao, Kun Yan, Sheng ming Yin, Shuai Bai, Xiao Xu, Yilei Chen, Yuxiang Chen, Zecheng Tang, Zekai Zhang, Zhengyi Wang, An Yang, Bowen Yu, Chen Cheng, Dayiheng Liu, Deqing Li, Hang Zhang, Hao Meng, Hu Wei, Jingyuan Ni, Kai Chen, Kuan Cao, Liang Peng, Lin Qu, Minggang Wu, Peng Wang, Shuting Yu, Tingkun Wen, Wensen Feng, Xiaoxiao Xu, Yi Wang, Yichang Zhang, Yongqiang Zhu, Yujia Wu, Yuxuan Cai, and Zenan Liu. Qwen-Image technical report. arXiv preprint arXiv:2508.02324, 2025a. URL https://arxiv.org/abs/2508.02324. Jie Wu, Yu Gao, Zilyu Ye, Ming Li, Liang Li, Hanzhong Guo, Jie Liu, Zeyue Xue, Xiaoxia Hou, Wei Liu, Yan Zeng, and Weilin Huang. Rewarddance: Reward scaling in visual generation. CoRR, abs/2509.08826, 2025b. doi: 10.48550/ARXIV.2509.08826. URL https://doi.org/10. 48550/arXiv.2509.08826. Xiaoshi Wu, Yiming Hao, Keqiang Sun, Yixiong Chen, Feng Zhu, Rui Zhao, and Hongsheng Li. Human preference score v2: A solid benchmark for evaluating human preferences of text- to-image synthesis. CoRR, abs/2306.09341, 2023. doi: 10.48550/ARXIV.2306.09341. URL https://doi.org/10.48550/arXiv.2306.09341. xAI. Introducing Grok 4.6, 2026. URL https://x.ai/news/grok-4-6. 14 Jiazheng Xu, Xiao Liu, Yuchen Wu, Yuxuan Tong, Qinkai Li, Ming Ding, Jie Tang, and Yuxiao Dong. Imagereward: Learning and evaluating human preferences for text-to-image generation. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (eds.), Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. URL https://papers.nips.c/paper_files/paper/2023/hash/ 33646ef0ed554145eab65f6250fab0c9-Abstract-Conference.html. Jiazheng Xu, Yu Huang, Jiale Cheng, Yuanming Yang, Jiajun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shurun Li, Jiayan Teng, Zhuoyi Yang, Wendi Zheng, Xiao Liu, Dan Zhang, Ming Ding, Xiaohan Zhang, Shiyu Huang, Xiaotao Gu, Minlie Huang, Jie Tang, and Yuxiao Dong. Visionreward: Fine-grained multi-dimensional human preference learning for image and video generation. In Sven Koenig, Chad Jenkins, and Matthew E. Taylor (eds.), Fortieth AAAI Conference on Artificial Intelligence, Thirty-Eighth Conference on Innovative Applications of Artificial Intelligence, Sixteenth Symposium on Educational Advances in Artificial Intelligence, AAAI 2026, Singapore, January 20-27, 2026, p. 11269ā11277. AAAI Press, 2026. doi: 10. 1609/AAAI.V40I13.38107. URL https://doi.org/10.1609/aaai.v40i13.38107. Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, Baosong Yang, Bin Zhang, Ziyang Ma, Xipin Wei, Shuai Bai, Keqin Chen, Xuejing Liu, Peng Wang, Mingkun Yang, Dayiheng Liu, Xingzhang Ren, Bo Zheng, Rui Men, Fan Zhou, Bowen Yu, Jianxin Yang, Le Yu, Jingren Zhou, and Junyang Lin. Qwen3-Omni technical report. CoRR, abs/2509.17765, 2025. doi: 10.48550/ARXIV.2509.17765. URL https://doi.org/10.48550/arXiv.2509.17765. Shuchen Xue, Chongjian Ge, Shilong Zhang, Yichen Li, and Zhi-Ming Ma. Advantage weighted matching: Aligning RL with pretraining in diffusion models. In Forty-third International Con- ference on Machine Learning (ICML 2026), 2026. doi: 10.48550/ARXIV.2509.25050. URL https://doi.org/10.48550/arXiv.2509.25050. Yuyuan Yang, Junkun Hong, Hongrong Wang, Honghao Cai, Xunpeng Ren, Ge Wang, Mingcong Lei, Shenhao Yan, Jiahao Yang, Chengsi Yao, Xi Li, Yiming Zhao, Yatong Han, and Jinke Ren. SVLL: staged vision-language learning for physically grounded embodied task planning. CoRR, abs/2603.11563, 2026. doi: 10.48550/ARXIV.2603.11563. URL https://doi.org/10. 48550/arXiv.2603.11563. Z.ai. GLM-5.3-Flash: Frontier Intelligence, Flash Cost, 2026. URL https://z.ai/blog/ glm-5.3-flash. Sixian Zhang, Bohan Wang, Junqiang Wu, Yan Li, Tingting Gao, Di Zhang, and Zhongyuan Wang. Learning multi-dimensional human preference for text-to-image generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR 2024), 2024. doi: 10. 48550/ARXIV.2405.14705. URL https://doi.org/10.48550/arXiv.2405.14705. Kaiwen Zheng, Huayu Chen, Haotian Ye, Haoxiang Wang, Qinsheng Zhang, Kai Jiang, Hang Su, Stefano Ermon, Jun Zhu, and Ming-Yu Liu. Diffusionnft: Online diffusion reinforcement with forward process. In The Fourteenth International Conference on Learning Representations (ICLR 2026), 2026. doi: 10.48550/ARXIV.2509.16117. URL https://doi.org/10.48550/ arXiv.2509.16117. Sashuai Zhou, Qiang Zhou, Junpeng Ma, Yue Cao, Ruofan Hu, Ziang Zhang, Xiaoda Yang, Zhibin Wang, Jun Song, Cheng Yu, Bo Zheng, and Zhou Zhao. SpatialReward: Official imple- mentation. Code repository, 2026a. URL https://github.com/LivingFutureLab/ SpatialReward. Implementation of SpatialReward (CVPR 2026, p. 647ā658). Sashuai Zhou, Qiang Zhou, Junpeng Ma, Yue Cao, Ruofan Hu, Ziang Zhang, Xiaoda Yang, Zhibin Wang, Jun Song, Cheng Yu, Bo Zheng, and Zhou Zhao. Spatialreward: Verifiable spatial reward modeling for fine-grained spatial consistency in text-to-image generation. CoRR, abs/2603.22228, 2026b. doi: 10.48550/ARXIV.2603.22228. URL https://doi.org/10.48550/arXiv. 2603.22228. 15