Paper deep dive
RewardVerse: Rubric-Guided Policy Optimization for Video Reward Modeling
Zhenchen Tang, Yang Li, Songlin Yang, Bo Peng, Xiaotong Zhao, Shuai Li, Haotian Fan, Alan Zhao, Jing Dong
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/22/2026, 5:35:55 AM
Summary
The paper introduces RewardVerse, a rubric-guided video reward modeling framework designed to mitigate 'scalar drift' in reinforcement learning for video generation. Unlike existing models that map videos directly to unstable scalar scores, RewardVerse uses a dynamic rubric generator to create explicit evaluation criteria (themes, weights, tips) before scoring. It employs a two-stage training algorithm called Rubric-Guided Policy Optimization (RGPO) to jointly optimize the rubric generator and scorer, achieving state-of-the-art performance on the EvalVerse benchmark with high data efficiency.
Entities (8)
Relation Signals (7)
RewardVerse โ mitigates โ Scalar Drift
confidence 96% ยท RewardVerse mitigates scalar drift... providing a stable semantic anchor that mitigates scalar drift
RewardVerse โ evaluatedon โ EvalVerse
confidence 95% ยท Extensive experiments on the 16-dimensional EvalVerse benchmark
RewardVerse โ uses โ Scorer
confidence 95% ยท Module 2 โ Scorer with Soft-Logits Readout... The scorer then evaluates the video against these rubrics
RewardVerse โ uses โ Dynamic Rubric Generator
confidence 95% ยท RewardVerse... introduces a dynamic rubric as an intermediate representation... Module 1 โ Dynamic Rubric Generator
RGPO โ optimizes โ Scorer
confidence 94% ยท RGPO... continuously aligning the scorer with human ratings
RGPO โ optimizes โ Dynamic Rubric Generator
confidence 94% ยท RGPO... jointly optimizes the rubric generator
RGPO โ basedon โ GRPO
confidence 90% ยท RGPO under Group Relative Policy Optimization (GRPO)
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Reinforcement learning (RL) is vital for optimizing video generation models, with a robust reward model (RM) serving as the cornerstone. However, existing video reward models often produce unstable scalar scores because they directly map complex, subjective video quality into a single score without explicit evaluation criteria. This leads to scalar drift, where the scoring scale collapses or shifts across different prompts, making the reward unreliable for RL. Drawing inspiration from professional human annotation engineering, we address this problem with RewardVerse, a rubric-based video reward framework that introduces a dynamic rubric as an intermediate representation between the evaluation query and the scorer. Instead of unconstrained direct scoring, RewardVerse first generates explicit evaluation criteria and then performs rubric-guided scoring, providing a stable semantic anchor that mitigates scalar drift. To efficiently optimize this collaborative pipeline, we propose Rubric-Guided Policy Optimization (RGPO), a two-stage training algorithm. RGPO first warms up the scorer using self-evolving seed rubrics and then jointly optimizes the rubric generator to produce query-adaptive evaluation criteria while continuously aligning the scorer with human ratings. Extensive experiments on the 16-dimensional EvalVerse benchmark and external datasets demonstrate that RewardVerse mitigates scalar drift, achieves state-of-the-art performance on both pointwise and pairwise evaluation, and provides a robust and interpretable reward signal for RL in video generation.
Tags
Links
- Source: https://arxiv.org/abs/2609.22947v1
- Canonical: https://arxiv.org/abs/2609.22947v1
Trouble viewing inline? Open PDF directly โ
Full Text
82,638 characters extracted from source content.
Expand or collapse full text
REWARDVERSE: RUBRIC-GUIDED POLICY OPTIMIZA- TION FOR VIDEO REWARD MODELING Zhenchen Tang 1,2,4 Yang Li 1,2,4 Songlin Yang 3,4 Bo Peng 1,2 Xiaotong Zhao 4 Shuai Li 4 Haotian Fan 4 Alan Zhao 4 Jing Dong 1,2,โ 1 New Laboratory of Pattern Recognition, Institute of Automation, Chinese Academy of Sciences 2 School of Artificial Intelligence, University of Chinese Academy of Sciences 3 The Hong Kong University of Science and Technology 4 Tencent ABSTRACT Reinforcement learning (RL) is vital for optimizing video generation models, with a robust reward model (RM) serving as the cornerstone. However, existing video reward models often produce unstable scalar scores because they directly map complex, subjective video quality into a single score without explicit evaluation criteria. This leads to scalar drift, where the scoring scale collapses or shifts across different prompts, making the reward unreliable for RL. Drawing inspira- tion from professional human annotation engineering, we address this problem with RewardVerse, a rubric-based video reward framework that introduces a dy- namic rubric as an intermediate representation between the evaluation query and the scorer. Instead of unconstrained direct scoring, RewardVerse first generates explicit evaluation criteria and then performs rubric-guided scoring, providing a stable semantic anchor that mitigates scalar drift. To efficiently optimize this col- laborative pipeline, we propose Rubric-Guided Policy Optimization (RGPO), a two-stage training algorithm. RGPO first warms up the scorer using self-evolving seed rubrics and then jointly optimizes the rubric generator to produce query- adaptive evaluation criteria while continuously aligning the scorer with human ratings. Extensive experiments on the 16-dimensional EvalVerse benchmark and external datasets demonstrate that RewardVerse mitigates scalar drift, achieves state-of-the-art performance on both pointwise and pairwise evaluation, and pro- vides a robust and interpretable reward signal for RL in video generation. 1INTRODUCTION Generative video models have achieved remarkable progress in recent years. As these models con- tinue to improve, reinforcement learning (RL) has become an important paradigm for aligning generated videos with human preferences, with a robust reward model (RM) serving as the cor- nerstone. Despite recent progress, existing video RMs still struggle to produce stable pointwise rewards. Most methods directly map a generated video to a single scalar score, either through discriminative regression or generative reasoning. Since video quality is inherently subjective and multi-dimensional (Wang et al., 2026a), this direct scoring process lacks explicit evaluation criteria. Consequently, the scoring scale often becomes unstable, causing scores to collapse into a narrow range (Jin et al., 2026) or shift under different prompts and contexts. We refer to this phenomenon as scalar drift. To address this challenge, we draw inspiration from professional human annotators, who rarely assign scores directly. Instead, they first decompose an evaluation task into explicit criteria before producing a final judgment, creating a stable semantic anchor that maintains consistency across different samples (Tong et al., 2025; Xu et al., 2026a). Existing video reward models largely omit this explicit criterion-setting stage. By forcing a model to map queries to scores in an unconstrained step, their internal scoring standards shift across different prompts, leading to unstable rewards. 1 arXiv:2609.22947v1 [cs.CV] 19 Sep 2026 Paradigm A โข Unconstrained Direct Scoring Paradigm B โข RewardVerse (Ours) ScalarDriftScalar Drift Mitigated 1 2 3 4 1 2 3 Returnanoverallscalar ํํฆํโ1,5 forthevideo. Task MLLMScorerํด Discriminative 1 2 3 4 5 Generativew/CoT Score-range collapse High variance under context shift scores concentrate in a narrow band scale shifts across instructions Step 1 โ Dynamic Rubric Generator ํ ํฝ ํํํ ํ ํฝ ํํํ Step 2 โ Scorer with Soft-Logits Readout Outcome โ Scalar Drift Mitigated ThemeWeightTips 1Spatial Continuity0.2 Check that eyelines... 2Narrative Continuity0.2 For prompts that... 3Multi-Shot Rationality0.3 Evaluate whether transitions... ............ query ํ no video access Rubric ) ํน ํ Scale Decompressed scores better reflect human rating variation Stable under Context Shift boundaries remain stableundercontextshift Per-Theme Score ํ ํ Generated Video ํ Query ํ=(ํ ,ํ) dimension ํ : โlogicโ prompt ํ: โ[shot_0_prompt] Eye-level close-up: In the center of the frame is an elderly man... [shot_1_prompt] Eye-level close-up: The camera angle shifts but remains focused on the face...โ GT:4.33 + ) ํน ํ 4.45 โ0.20 โ0.20 โ0.15 โ0.15 4.7 4.7 4.8 3.5 โ0.30 4.5 ํบ Aggregation Figure 1: A comparison of different video evaluation paradigms. Compared to unconstrained dis- criminative and generative scoring that suffer from severe scalar drift, RewardVerse inserts a dy- namic rubric as an intermediate representation to anchor the scoring process. Based on this insight, we propose RewardVerse, a highly data-efficient video evaluation framework that natively supports both pointwise and pairwise evaluation. As compared in Figure 1, instead of directly mapping a video to a score, we insert a dynamic rubric as an intermediate representation to decouple evaluation into standard generation and objective execution. Specifically, the rubric gen- erator first decomposes the holistic text query into explicit evaluation themes, weights, and scoring tips. The scorer then evaluates the video against these rubrics independently, rather than relying on an implicit internal standard. This decoupling provides a clear cognitive anchor that mitigates score-range collapse and improves robustness under context shift. To efficiently learn dynamic rubrics and optimize the collaborative pipeline with minimal preference pairs and without prior supervised fine-tuning (SFT), we introduce a two-stage training paradigm called Rubric-Guided Policy Optimization (RGPO) under Group Relative Policy Optimization (GRPO). We first warm up the scorer using self-evolving seed rubrics synthesized offline. Then, we jointly optimize the rubric generator to produce query-adaptive evaluation criteria while contin- uously aligning the scorer with human ratings. Through this optimization, RewardVerse effectively mitigates scalar drift, requiring only 30 preference pairs per dimension to deliver state-of-the-art performance and provide a robust, highly interpretable reward for downstream generation models. Our main contributions are summarized as follows: โข Rubric-as-Reward for Video RMs: To our knowledge, we are the first to introduce the rubric-as-reward paradigm to video reward modeling. We propose RewardVerse, a frame- work that employs a dynamic rubric as an intermediate representation to stabilize pointwise scores and mitigate scalar drift. โข Data-Efficient Joint Policy Optimization: We design RGPO (Rubric-Guided Policy Op- timization), a two-stage training algorithm that learns a dynamic rubric generator from only 30 preference pairs per dimension, while maintaining a human-aligned scorer. โข Systematic Empirical Validation: We conduct extensive evaluations on the multi- dimensional EvalVerse-adapted dataset and external benchmarks. RewardVerse achieves state-of-the-art evaluation performance and serves as a stable, interpretable reward for downstream video generation optimization. 2 2RELATED WORK The Rubric-as-Reward Paradigm. Rubrics decompose quality evaluation into explicit criteria corresponding to concrete aspects of the desired output, providing a natural handle for open-ended, non-verifiable tasks. In LLM and autonomous agent domains, recent works bring rubrics into RL for subjective tasks such as medicine, writing, and research (e.g., RaR (Gunjal et al., 2025), RLCF (Viswanathan et al., 2026), HealthBench (Arora et al., 2025), RLCER (Sheng et al., 2026), OpenRubrics (Liu et al., 2026b), Rubric-ARM (Xu et al., 2026b), and DR Tulu (Shao et al., 2025)). Similarly, in image generation and editing, recent works use explicit rubrics, checklists, or decom- posed criteria to evaluate and optimize visual outputs (e.g., AlphaGRPO (Huang et al., 2026), Edit- R1 (Guo et al., 2026), EditReward-Compass (Bai et al., 2026), ARR-RPO (Tian et al., 2026), and RewardHarness (Zhang et al., 2026)). These works establish explicit criteria as useful reward signals across open-ended generation tasks. In contrast, RewardVerse makes the rubric a dynamic, learned intermediate representation: its themes, weights, and scoring tips are generated from each evaluation query without access to the candidate video, and are jointly optimized with the scorer to stabilize pointwise video rewards. Video Reward Modeling. Existing video reward models generally fall into two categories: dis- criminative and generative. Discriminative models (e.g., VideoScore (He et al., 2024), VideoRe- ward (Liu et al., 2026a)) train regression layers on top of visual encoders to output continuous scalar scores. Generative models (e.g., UnifiedReward (Wang et al., 2025), VideoScore2 (He et al., 2025)) prompt multi-modal LLMs to output scores directly. Because these models score videos directly without explicit anchors, they often suffer from scalar drift. As analyzed in Section 3, scalar drift causes scores to collapse into a narrow range or change wildly across prompts, making them unsta- ble for online RL training. By jointly training rubric generation and scoring under our structured rubric bottleneck, RewardVerse mitigates scalar drift, achieves high data efficiency, and provides a stable reward signal for downstream video generation models. 3THE PHENOMENON OF SCALAR DRIFT IN VIDEO EVALUATION To establish the empirical foundations of our work, we conduct an analytical study of video evalua- tion behaviors in modern Multi-modal Large Language Models (MLLMs). We focus on dissecting scalar drift under unconstrained direct scoring, analyzing how rubric-guided scoring alleviates this problem, and identifying the remaining limitations that motivate joint policy optimization. 3.1DEFINING SCALAR DRIFT We consider an MLLMM tasked with returning a single overall scalar reward S(y | q)โ [1, 5] for a generated video y given a textual query q. Because video quality assessment is multi-dimensional and largely subjective, this direct task is unconstrained: the model has no explicit anchor for what each point on the 1โ5 scale should mean. Under this setting, the internal scoring rubric drifts in two interacting ways, which together we define as scalar drift: โข Score-range collapse. Pointwise scores concentrate in a narrow โsafeโ high band, shrink- ing the score gap between preferred and non-preferred videos and leaving downstream RL with near-flat gradients. โข High variance under context shift. Without explicit anchors, the scoring scale may shift across paraphrased instructions or different input orderings, producing inconsistent scores or pairwise decisions for the same content. A workable pointwise reward must therefore (a) preserve scale resolution across the population of videos, and (b) be robust to context phrasing and ordering. See Appendix A.1 for further analysis and Appendix A.1.3 for score distributions across existing reward models. 3.2EMPIRICAL STUDY: RUBRIC IS NECESSARY BUT NOT SUFFICIENT To isolate which design choice cures which symptom of scalar drift, we construct a 2ร2 ablation de- sign space covering rubric injection (presence vs. absence) and decoding format (natural-language 3 0 50 100 150 200 # videos V1: Unconstrained natural-language (no rubric) prediction (n=240) human label V2: Unconstrained soft-logits (no rubric) prediction (n=240) human label 1.01.52.02.53.03.54.04.55.0 score 0 50 100 150 200 # videos V3: Rubric soft-logits (ours, deployed) prediction (n=240) human label 1.01.52.02.53.03.54.04.55.0 score V4: Rubric natural-language (rubric injected as guide) prediction (n=240) human label Figure 2: Pointwise score distributions of four scoring variants. Blue bars are predictions, orange bars are human labels, and dashed lines show the means. V1 and V2 show score-range collapse, while V4 remains affected by natural-language bias. Only V3 (rubrics with soft-logits) effectively spreads predictions, but it still deviates from human ratings, motivating RGPO in Section 4. generation vs. soft-logits continuous expectation). This yields four variants evaluated on a hold-out pool of 240 pointwise videos with human labels (details in Appendix A.1.1): 1) V1 (No Rubric + Natural Float); 2) V2 (No Rubric + Soft-Logits); 3) V3 (With Rubric + Soft-Logits, Deployed); and 4) V4 (With Rubric + Natural Float). Figure 2 reveals three key insights that drive our down- stream framework design: Cognitive Anchoring via Rubrics. Without rubric guidance, unconstrained direct scoring (V1 and V2) suffers from severe score saturation and narrow variance. Injecting a rubric as a cognitive anchor substantially expands these distributions. For example, holding the soft-logits format fixed, adding the rubric (V 2 โ V 3) expands the scoring standard deviation by +82.6% (ฯ: 0.258 โ 0.471) for Qwen2.5-VL-7B. Under natural text generation, the rubric acts as a vital regularizer: it expands the collapsed standard deviation of Qwen2.5-VL-7B (V 1 โ V 4: 0.203 โ 0.275), while reducing the excessively noisy and unguided variance of Gemini-3.1-Pro (reducing ฯ from 1.500 to 1.158), bringing their score variance closer to that of human judgments (see Appendix A.1.2). Continuous Scale Decompression via Soft-Logits. Crucially, however, the rubric alone is insuffi- cient if paired with natural text generation. In V4, two-thirds of the comparison pairs still tie because natural-language outputs remain anchored to the modelโs high-band linguistic prior. Further miti- gating scalar drift requires coupling the rubric with a continuous, logit-based readout mechanism. Extracting a continuous expected score directly from the logits of rating tokens (soft-logits) success- fully bypasses the modelโs generation bias (formulated in Section 4). Consequently, pairing a rubric with soft-logits (V3) better captures the variation in human ratings and reduces score-range collapse. For our deployment, we choose this pointwise multi-theme soft-logits protocol (V3) as our primary configuration; detailed protocol comparisons are provided in Appendix A.1.5. The Remaining Limitations. Although rubric-guided soft-logits scoring mitigates scalar drift, two limitations remain. First, static rubrics cannot adapt their themes, tips, and weights to different queries. Second, the predicted score distribution may still deviate from human ratings, even for stronger proprietary MLLMs such as Gemini-3.1-Pro (Appendix A.1.2). These results show that rubric prompting alone cannot fully resolve scalar drift, motivating RGPO to jointly optimize rubric generation and scoring. As verified in Appendix A.1.7, the complete RGPO training further im- proves both score alignment and correlation with human ratings. 4 Stage 1 ยท Warm-up the Scorer with Seed Rubrics Stage 2 ยท Jointly Optimize Generator and Scorer 30 pairs / dim ยท 480 total 1 23 4 6 Propose Grow Seed Rubric Revise Verify Seed Rubric ํน ํ โ Dual-Group Rollout Warm-up OptimizationObjective ํ ํ ํ ํ ํน ํ โ ... ... ํฎ ํฎ MLLM (Scorer) ํน ํ =ํ ํํํํ +ํท ํํํ &ํ ํํํ ํ ํํํํโ ํ(ํบ ํ (ํ ํ |ํน ํ โ ) โํบ ํ (ํ ํ |ํน ํ โ )) ํ ํํํโ ํคvalid format ํณ ํฎํนํทํถ ํณ ํํํ โถ(ํบ(ํ ํ )โํบ(ํ ํ ))โ โ 1 ํ ํ,ํ Generator Samples Dynamic Rubrics ํ ... ํฎ Joint OptimizationObjective ํณ ํํํํํํ =ํณ ํฎํนํทํถ ํ (ํฝ)+ํถ(ํ)&ํณ ํํํํํํ (ํฝ) Scorer ํ ํฝ ํํ= Generator ํ ํฝ ํํํ ํน ํ Generator ํณ ํํํ Scorer ํ ํฝ ํํํ ํ ํฝ ํํ= 5 ํน ํ =ํ ํํํํ +ํท ํํํ &ํ ํํํ +ํ&ํ ํํํํํ ํ ํํํํ ํ ํํํ Rubric-Level Reward ํถํํ (ํํํํํํก,ํ ํํ ํกโํํํํ ) Dynamic Rubric K ํน ํ ํ ํ ํ ํ MLLM (Scorer) ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ... ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ ํ Scorerํ ํฝ ํํ/ Generator ํ ํฝ ํํํ MLLM Scorer ํ ํฝ ํํ/ dimensionํ prompt ํ Figure 3: Overview of RewardVerse and its two-stage RGPO training. Stage 1 warms up the scorer with seed rubrics using preference and format rewards plus a calibration loss. Stage 2 learns the dynamic rubric generator while continuing scorer calibration. 4METHODOLOGY Rather than treating the rubric as a fixed prompt scaffold, we make it a learnable intermediate rep- resentation. This allows the model to generate query-adaptive evaluation criteria while keeping the scorer aligned with human ratings. This joint optimization further mitigates the scalar drift that re- mains after rubric-guided scoring. Building on this formulation, we present RewardVerse, which parameterizes adaptive rubric generation and human-aligned scoring within a dual-role policy model ฯ ฮธ . We optimize the model via Rubric-Guided Policy Optimization (RGPO), a two-stage train- ing algorithm. Stage 1 warms up the scorer using self-evolving seed rubrics (ยง4.2), and Stage 2 optimizes the dynamic rubric generator while continuing to calibrate the scorer (ยง4.3). 4.1REWARDVERSE PIPELINE AND FORMULATION Inputs. An evaluation query q = (d,p) pairs a target dimension d โ D with the text prompt p used to synthesize the candidate video y. Each video is scored independently during both training and inference, avoiding the pairwise position bias analyzed in Appendix A.1.6. Module 1 โ Dynamic Rubric Generator ฯ gen ฮธ . The policy first acts as a generator that parses the query q and emits a dynamic rubric under a fixed schema: ห R q โผ ฯ gen ฮธ (ยท| q), ห R q = (t k ,w k ,T k ) K k=1 ,(1) where t k is a theme, w k โ (0, 1) is its weight with P k w k = 1, and T k is a set of execution tips based on queries. Candidate-Independent Rubric Generation: Notably, the generator takes only the query q as input without seeing the video y. This decoupling prevents the generator from producing biased, video- dependent rubrics (e.g., being overly lenient to low-quality videos or overly strict to high-quality ones), safeguarding evaluation objectivity while keeping the criteria query-adaptive. Module 2 โ Scorer ฯ sco ฮธ with Soft-Logits Readout. Given a rubric R, the policy emits one score per theme. To avoid parsing digits under JSON-constrained decoding, at each score slot we read the logits l v of the five rating tokens v โ 1,..., 5 (corresponding to the string representations of integers โ1โ to โ5โ in the vocabulary) and take the expected value: c k (y | R) = 5 X v=1 vยท P ฮธ v | y,t k ,T k , P ฮธ v | y,t k ,T k = exp(l v ) P 5 v โฒ =1 exp(l v โฒ ) .(2) 5 The pointwise reward aggregates theme scores by their weights: S(y | R) = K X k=1 w k ยท c k (y | R).(3) The generator and scorer share the same backbone and are distinguished by role-specific prompts. In Stage 2, the shared policy is trained with separate signals for the two roles. The main prompt templates are provided in Appendix A.4.1. 4.2STAGE 1: SEED-GUIDED SCORER WARM-UP Learning dynamic rubric generation and calibrated scoring at the same time from scratch is highly under-determined. A poor generator produces chaotic rubrics that disrupt scorer learning. Therefore, Stage 1 uses fixed seed rubrics R โ d and updates the shared policy through the scorer role, establishing human-aligned score margins before dynamic rubric generation is introduced. Seed Rubric Self-Evolving. For each dimension d, we build a seed rubric R โ d through an offline loop driven by a frontier MLLMM (Gemini-3.1-Pro) over 30 preference pairs per dimension: โข Propose (M gen ). Given (q,y w ,y l ), M drafts a candidate rubric ฬ R that explains why y w โป y l . โข Verify (M ver ). M scores both videos with ฬ R; ฬ R is accepted only if it recovers y w โป y l . โข Revise (M ref ). On failure,M refines ฬ R using a critique of the misalignment. Verified rubrics are pooled intoV d , deduplicated by a two-level Jaccard filter, and reduced to M =5 representative entries by an MCR 2 sampler (Yu et al., 2020), giving R โ d =r โ d,1 ,...,r โ d,M . Scorer Optimization via Policy Gradient. For each pair (q,y w ,y l ) โผ D train under the seed rubric R โ d , the scorer produces G joint scoring samples. Each sample contains one completion a w,g for the preferred video and one completion a l,g for the non-preferred video, sampled from ฯ sco ฮธ old for g = 1,...,G. Each completion pair g is scored by a pairwise preference-driven reward: r pref (a w,g ,a l,g ) = ฯ S g (y w | R โ d )โ S g (y l | R โ d ) ,(4) where ฯ(ยท) is the sigmoid function, and S g represents the pointwise score (equation 3) decoded from the corresponding completion a x,g (xโw,l). The full warm-up reward is defined as: R 1 (a w,g ,a l,g ) = r pref (a w,g ,a l,g ) + ฮฒ fmt (r fmt (a w,g ) + r fmt (a l,g )),(5) where ฮฒ fmt controls the contribution of the format reward, and r fmt (a x,g ) โ 0, 1 is a binary indicator checking whether the completion conforms to the requested scoring format. Following GRPO (Shao et al., 2024), the joint reward of each pair is standardized across the group to yield the trajectory-level advantage ห A g = R 1 (a w,g ,a l,g ) โ mean(R 1 ) /std(R 1 ), which is broadcast to all tokens in both a w,g and a l,g . Denoting the per-token importance ratio between the current and behavior scorers by ฯ x,g,t (ฮธ) = ฯ sco ฮธ (a x,g,t |y x ,R โ d ) ฯ sco ฮธ old (a x,g,t |y x ,R โ d ) , the Stage-1 GRPO objective optimizes both preferred and non-preferred completions: L (1) GRPO (ฮธ) =โ 1 G G X g=1 1 2 X xโw,l 1 |a x,g | |a x,g | X t=1 h min ฯ x,g,t (ฮธ) ห A g , clip(ฯ x,g,t (ฮธ), 1โฮต, 1+ฮต) ห A g โ ฮฒ KL D KL ฯ sco ฮธ ฯ sco ref x,g,t i . (6) Here, ฮต is the clipping coefficient, ฮฒ KL controls the KL penalty, and D KL [ยทโฅยท] x,g,t is the per-token KL penalty against a frozen reference policy ฯ sco ref . 6 Human-Aligned Margin Calibration. Although the preference reward encourages the scorer to rank the preferred video above the non-preferred one, it does not control the magnitude of the pre- dicted score difference. We therefore add a calibration loss based on the human score margin: L cal (ฮธ | R) = 1 S max โ S min S(y w | R)โ S(y l | R) โ โหs w,l ,(7) where S max = 5, S min = 1, and โหs w,l โ [0, 4] is the target score margin derived from human scores. This loss calibrates the predicted score difference against the human-annotated margin, pre- venting arbitrary compression or inflation of reward gaps. We focus on relative rather than absolute calibration, since reward models mainly require reliable preference ordering and meaningful score differences, while direct absolute score regression can easily overfit in our low-data setting. The overall Stage-1 objective combines the GRPO loss with this calibration term: L stage1 (ฮธ) =L (1) GRPO (ฮธ) + ฮป cal L cal (ฮธ | R โ d ).(8) 4.3STAGE 2: JOINT POLICY OPTIMIZATION VIA RGPO Stage 2 learns the rubric generator to produce evaluation criteria tailored to each query. For each triple (q,y w ,y l ), the generator samples G dynamic rubrics ห R g G g=1 โผ ฯ gen ฮธ old (ยท | q). Each rubric is judged by how well its induced pointwise scores separate the preferred and non-preferred videos: r pref ( ห R g ) = ฯ S g (y w | ห R g )โ S g (y l | ห R g ) .(9) To keep the generated rubric aligned with the target dimension and enforce the required output format, we define the Stage-2 reward as: R 2 ( ห R g ) = r pref ( ห R g ) + ฮฒ r fmt ( ห R g ) + ฮปr align ( ห R g ),(10) where r align ( ห R g ) measures the cosine similarity between the BGE-M3 embeddings (Chen et al., 2024) of the generated and seed themes. Here, ฮป and ฮฒ are fixed coefficients for rubric alignment and format regularization, respectively. r fmt ( ห R g ) checks whether the output conforms to the structured JSON schema and whether the rubric weights sum to one. The composite reward of each generated rubric is standardized across the group to yield the rubric- level advantage ห A g = R 2 ( ห R g )โ mean(R 2 ) /std(R 2 ), which is broadcast to all tokens in the generated rubric ห R g . The generator is optimized using the same clipped GRPO objective as in equa- tion 6, with ห R g as the optimized sequence and ฯ g,t (ฮธ) = ฯ gen ฮธ ( ห R g,t |q) ฯ gen ฮธ old ( ห R g,t |q) as the per-token importance ratio. We denote this generator objective byL (2) GRPO (ฮธ); its full form is provided in Appendix A.4.2. As the rubric distribution changes during Stage 2, we retain scorer calibration to keep the scores aligned with human margins under the sampled rubrics. Asymmetric Optimization Signals. If the scorer were optimized directly by the relative reward R 2 , it could increase the reward simply by enlarging the score difference between the preferred and non-preferred videos, without improving score calibration. We therefore use different optimization signals for the two roles. The generator is optimized by the rubric-level GRPO objective, while the scorer receives no direct policy-gradient update from R 2 and is instead optimized by the margin calibration loss. The generated rubric is treated as fixed text context when computing the scorer loss, so the calibration loss does not backpropagate through the rubric: L scorer (ฮธ) = 1 G G X g=1 L cal ฮธ | ห R g .(11) The overall Stage-2 objective is L stage2 (ฮธ) =L (2) GRPO (ฮธ) + ฮฑ(t)L scorer (ฮธ).(12) Here, ฮฑ(t) is a scheduled coefficient that activates the scorer calibration loss after the initial gener- ator warm-up, improving training stability under the shared parameters. Under this joint objective, the generator learns query-adaptive rubrics beyond the static seed rubrics. The tips T k can adapt to prompt-specific details, while the weights w k vary with the query; meanwhile, the scorer remains guided by the human-aligned margin loss. A qualitative example is provided in Figure 7. 7 Table 1: Per-dimension PLCC (top row) and SRCC (bottom row) on the 16 secondary dimensions of EvalVerse (dimension names defined in Appendix A.2). Avg. denotes the macro-average over the 10 dimensions supported by all compared reward models. Best in bold, second-best underlined . โโโ means the reward model does not natively support that dimension. MethodScene Const. Action Expr. Comp. Lens Pace Visual Chrom. Mater. Light. Ground. Prog. Logic Rhythm Char. Avg. External Reward Models VideoScore-v1.1 PLCC -0.086 0.102 -0.038โ0.119-0.105-0.156 -0.031โ-0.2450.0700.042 -0.033 SRCC 0.002 0.255 -0.042โ0.2420.1170.0120.195โ0.0990.2800.155 0.132 VideoScore2 PLCC 0.449 0.2450.390โ0.5070.3670.6970.534โ0.3720.4270.346 0.433 SRCC 0.443 0.3370.363 โ0.4810.3240.6980.502โ0.3700.3400.409 0.427 UnifiedReward PLCC 0.461 0.3070.175โ0.404-0.0200.3330.463โ0.4650.3590.396 0.334 SRCC 0.4680.3210.183โ0.3320.0630.3170.340โ0.5490.3540.355 0.328 VideoReward PLCC 0.313 0.3810.326โ0.4620.2370.5730.483โ0.3260.2650.474 0.384 SRCC 0.334 0.3600.354โ0.5560.2200.6220.471โ0.3340.3040.430 0.399 VisionReward PLCC 0.457 0.2460.332โ0.5540.2340.5240.514โ0.3990.3870.429 0.408 SRCC 0.400 0.2200.186โ0.5190.1910.5080.479โ0.1710.3700.466 0.351 Q-Scorer PLCC 0.4960.3560.182โ0.5170.2340.6140.623โ0.5930.5170.5340.467 SRCC 0.340 0.2770.139โ0.3700.2300.5050.451โ0.3370.4170.4760.354 RGPO Pipeline Stages Raw-seed Rubric PLCC 0.432 0.2270.144 0.290 0.5470.303 0.0790.2360.1200.5230.1730.4410.2250.5890.1940.471 0.311 SRCC 0.449 0.372 0.141 0.251 0.2750.2780.1590.2660.1480.4040.2130.3410.2510.478-0.0430.326 0.275 Raw-rubric PLCC 0.437 0.1750.277 0.4150.449 0.4840.075 0.5710.1520.5310.3730.0610.183 0.5830.2470.447 0.379 SRCC 0.417 0.1780.293 0.340 0.237 0.166 0.065 0.5570.0550.4570.3090.0340.213 0.2970.1070.263 0.293 Joint (Ours) PLCC 0.520 0.3950.566 0.448 0.575 0.486 0.471 0.6610.4980.5380.5570.4560.510 0.7500.5230.535 0.554 SRCC 0.493 0.3810.403 0.285 0.440 0.333 0.431 0.6100.2430.5140.5150.3910.324 0.6560.1460.500 0.446 5EXPERIMENTAL EVALUATION We evaluate RewardVerse from four aspects: (i) pointwise correlation with human ratings across the 16 EvalVerse dimensions; (i) transfer to pairwise ranking on VGRB; (i) the contribution of each RGPO component; and (iv) downstream use as an RL reward for video generation. 5.1BENCHMARKS AND BASELINES We validate our method on two complementary benchmarks. First, we curate a pointwise test set from EvalVerse (Yang et al., 2026), currently the most comprehensive evaluation suite for generative video models, spanning 16 fine-grained secondary dimensions (details in Appendix A.2). Second, we evaluate on VideoGen-RewardBench (VGRB) (Liu et al., 2026a), a large-scale external pair- wise preference benchmark of over 26.5K video pairs. We report the Visual Quality split as transfer on a seen evaluation dimension and the Text Alignment split as transfer to an unseen dimension. We train our method on Qwen2.5-VL-7B (Team, 2025) and compare against six state-of-the-art video reward models: VideoScore-v1.1 (He et al., 2024), VideoScore2 (He et al., 2025), Uni- fiedReward and its thinking variant (Wang et al., 2025), VideoReward (Liu et al., 2026a), Vision- Reward (Xu et al., 2026a), and Q-Scorer (Tang et al., 2026). To trace the effect of each stage in our pipeline, we further analyze two internal baselines: Raw-seed Rubric (zero-shot with the seed rubric), Raw-rubric (zero-shot with dynamically generated but uncalibrated rubrics). 5.2POINTWISE EVALUATION ON EVALVERSE Table 1 reports the per-dimension pointwise evaluation results across the 16 EvalVerse dimensions. RewardVerse (Joint (Ours)) achieves the strongest overall performance across the 16 dimensions, with the highest PLCC in 14 dimensions. It outperforms other models by a clear margin on chal- lenging cognitive and temporal tasks, such as Logic (PLCC of 0.750 vs. the next-best 0.593) and Action (PLCC of 0.566 vs. 0.390). These results show that RGPO effectively aligns dynamic rubric-guided scores with human judgments. External video scorers can be poorly calibrated on these fine-grained axes (e.g., VideoScore-v1.1 has a PLCC ofโ0.245 on Logic). Interestingly, Q- Scorer remains competitive on some non-quality dimensions such as Lighting and Logic, suggesting that these dimensions are partly correlated with low-level visual quality. Beyond correlation, Re- wardVerse also shows better score alignment with human ratings after RGPO, as further analyzed in Appendix A.1.7. 5.3PAIRWISE TRANSFER ON VGRB Table 2 reports the pairwise agreement on the large-scale external benchmark VGRB. Most notably, RewardVerse achieves the best performance among all non-oracle baselines on both splits. Despite 8 Table 2: Pairwise agreement on VGRB. VQ is seen for RewardVerse; TA is unseen at training. VideoReward is the model whose training set defines VGRB and is shown in gray as an oracle upper bound rather than a comparable baseline. Text Alignment (unseen)Visual Quality (seen) MethodAcc w/ TieAcc w/o TieAcc w/ TieAcc w/o Tie VideoScore-v1.10.3720.5030.4740.477 VideoScore20.3010.4970.3740.630 UnifiedReward0.3030.5640.4120.394 UnifiedReward-Thinking0.4280.582โ VisionReward0.4650.6110.4740.590 RewardVerse (ours)0.4710.6230.5450.660 VideoReward (oracle upper bound)0.5380.7220.5960.756 Table 3: Component-wise ablation on EvalVerse (averaged over 16 dimensions). Best in bold. ConfigurationPLCCโSRCCโ Full RGPO0.5300.416 (a) w/o Stage-1 Scorer Warm-up0.4310.355 (b) w/o Stage-2 Joint Optimization0.4560.364 (c) w/o Hierarchical tips T k (flat rubric)0.4970.398 Rubric-Free Baselines (equal 480-pair budget) (d) V2-Raw (zero-shot, soft-logits)0.3610.234 (e) V2-Training (direct pointwise training)0.4390.351 training on only 30 pairs per dimension (480 pairs in total), our model outperforms the strongest non- oracle baseline (VisionReward) on the seen Visual Quality (VQ) split by 7.1 (Acc w/ Tie) and 7.0 (Acc w/o Tie) percentage points, closing much of the gap to the oracle VideoReward. Furthermore, our model demonstrates strong zero-shot generalization on the unseen Text Alignment (TA) split. It achieves the highest accuracy (0.471 with ties and 0.623 without ties). These results show that mitigating scalar drift in pointwise scoring can also improve pairwise preference alignment, even when transferring to out-of-distribution benchmarks and unseen evaluation dimensions. 5.4ABLATION STUDIES AND DOWNSTREAM EVALUATION Table 3 reports macro-averaged PLCC and SRCC on EvalVerse, together with two rubric-free base- lines under the same 480-pair budget. Three findings emerge. First, using a rubric as an intermediate representation is important for low-data efficiency. Under the same 480-pair budget, direct rubric- free pointwise training (e) improves only slightly over its zero-shot counterpart (d) and remains well below Full RGPO, showing the benefit of introducing the rubric between the query and the scorer when training data is limited. Second, Stage-1 scorer warm-up is important for score calibration. Removing Stage 1 (a) drops PLCC by 0.099 and SRCC by 0.061, showing the benefit of calibrat- ing the scorer before dynamic rubric learning begins. Third, Stage-2 joint optimization improves over static rubrics. Skipping Stage 2 (b) yields a 0.074 PLCC gap compared with the full model, confirming the benefit of adapting rubrics to different queries. Qualitative results are provided in Appendix A.4.3. Downstream Video Generation. We further use RewardVerse as the reward for GRPO fine-tuning of Wan-2.2-A14B (Wan et al., 2025) on Visual Quality. RewardVerse improves Imaging Quality from 0.640 to 0.653, while VBench-Quality remains stable (0.808 to 0.809) and VBench-Text im- proves from 0.428 to 0.446. In comparison, VideoReward improves Imaging Quality to 0.648 but reduces VBench-Quality and VBench-Text to 0.804 and 0.392. These results show that Reward- Verse improves the target quality while better preserving other video capabilities. Full settings and results are provided in Appendix A.3. 6CONCLUSION In this paper, we introduced RewardVerse, a robust pointwise video reward model, and its training algorithm, Rubric-Guided Policy Optimization (RGPO). Addressing scalar drift in unconstrained 9 MLLM scoring, we demonstrated that inserting a dynamic rubric improves score resolution, while RGPO further alleviates the remaining scalar drift through joint optimization. We also showed that both stages of RGPOโseed-guided warm-up and joint optimizationโare essential to this process. With only 30 preference pairs per dimension, RewardVerse achieves high pointwise correlation on EvalVerse, strong pairwise agreement on VGRB, and improves downstream video generation while being less prone to reward hacking. These results show that mitigating scalar drift is crucial for using video reward models as reinforcement learning signals. REFERENCES Rahul K. Arora, Jason Wei, Rebecca Soskin Hicks, Preston Bowman, Joaquin Qui ฬ nonero-Candela, Foivos Tsimpourlas, Michael Sharman, Meghan Shah, Andrea Vallone, Alex Beutel, Johannes Heidecke, and Karan Singhal. Healthbench: Evaluating large language models towards improved human health. arXiv preprint arXiv:2505.08775, 2025. Xuehai Bai, Yang Shi, Yi-Fan Zhang, Xuanyu Zhu, Yuran Wang, Yifan Dai, Xinyu Liu, Yiyan Ji, Xiaoling Gu, and Yuanxing Zhang. Edit-compass & editreward-compass: A unified benchmark for image editing and reward modeling. arXiv preprint arXiv:2605.13062, 2026. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge dis- tillation. arXiv preprint arXiv:2402.03216, 2024. Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean M. Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains. arXiv preprint arXiv:2507.17746, 2025. Hanzhong Guo, Jie Wu, Jie Liu, Yu Gao, Zilyu Ye, Linxiao Yuan, Xionghui Wang, Yizhou Yu, and Weilin Huang. Leveraging verifier-based reinforcement learning in image editing. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 34343โ34352, 2026. Xuan He, Dongfu Jiang, Ge Zhang, Max Ku, Achint Soni, Sherman Siu, Haonan Chen, Abhranil Chandra, Ziyan Jiang, Aaran Arulraj, et al. Videoscore: Building automatic metrics to simulate fine-grained human feedback for video generation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 2105โ2123, 2024. Xuan He, Dongfu Jiang, Ping Nie, Minghao Liu, Zhengxuan Jiang, Mingyi Su, Wentao Ma, Junru Lin, Chun Ye, Yi Lu, et al. Videoscore2: Think before you score in generative video evaluation. arXiv preprint arXiv:2509.22799, 2025. Runhui Huang, Jie Wu, Rui Yang, Zhe Liu, and Hengshuang Zhao. Alphagrpo: Unlocking self- reflective multimodal generation in unified multimodal models via decompositional verifiable reward. In Forty-third International Conference on Machine Learning, 2026. Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianx- ing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. In 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), p. 21807โ21818. IEEE, 2024. Xin Jin, Huanqia Cai, Zhen Li, Zechao Zhan, Dengyang Jiang, Aiming Hao, Yuming Jiang, Xi- angpeng Yang, Chunle Guo, Peng Gao, et al. Z-reward: Beyond scalar rewards by internalizing reasoning into score distributions. arXiv preprint arXiv:2606.09076, 2026. Jie Liu, Gongye Liu, Jiajun Liang, Ziyang Yuan, Xiaokun Liu, Mingwu Zheng, Xiele Wu, Qiulin Wang, Menghan Xia, Xintao Wang, et al. Improving video generation with human feedback. Advances in Neural Information Processing Systems, 38:82155โ82192, 2026a. Tianci Liu, Ran Xu, Tony Yu, Ilgee Hong, Carl Yang, Tuo Zhao, and Haoyu Wang. Openrubrics: Towards scalable synthetic rubric generation for reward modeling and llm alignment. In Pro- ceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 17417โ17437, 2026b. 10 Rulin Shao, Akari Asai, Shannon Zejiang Shen, Hamish Ivison, Varsha Kishore, Jingming Zhuo, Xinran Zhao, Molly Park, Samuel G Finlayson, David Sontag, et al. Dr tulu: Reinforcement learning with evolving rubrics for deep research. arXiv preprint arXiv:2511.19399, 2025. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Mingchuan Zhang, Chengqi Deng, Y. K. Li, Fuli Luo, Ruoyu Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in common language models. arXiv preprint arXiv:2402.03300, 2024. Leheng Sheng, Wenchang Ma, Ruixin Hong, Xiang Wang, An Zhang, and Tat-Seng Chua. Rein- forcing chain-of-thought reasoning with self-evolving rubrics. arXiv preprint arXiv:2602.10885, 2026. Zhenchen Tang, Songlin Yang, Bo Peng, Zichuan Wang, and Jing Dong. Revisiting mllm based im- age quality assessment: Errors and remedy. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, p. 9475โ9483, 2026. Qwen Team. Qwen2.5-vl: Making language models can actually see the world. Alibaba Qwen Blog, 2025. URL https://qwen.ai/blog?id=qwen2.5-vl. Juanxi Tian, Fengyuan Liu, Jiaming Han, Yilei Jiang, Yongliang Wu, Yesheng Liu, Haodong Li, Furong Xu, and Wanhua Li. Auto-rubric as reward: From implicit preferences to explicit multi- modal generative criteria. arXiv preprint arXiv:2605.08354, 2026. Haibo Tong, Zhaoyang Wang, Zhaorun Chen, Haonian Ji, Shi Qiu, Siwei Han, Kexin Geng, Zhongkai Xue, Yiyang Zhou, Peng Xia, et al. Mj-video: Fine-grained benchmarking and re- warding video preferences in video generation. arXiv preprint arXiv:2502.01719, 2025. Vijay Viswanathan, Yanchao Sun, Xiang Kong, Meng Cao, Graham Neubig, and Sherry Wu. Check- lists are better than reward models for aligning language models. Advances in Neural Information Processing Systems, 38:114728โ114754, 2026. Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianxiao Yang, et al. Wan: Open and advanced large-scale video generative models. arXiv preprint arXiv:2503.20314, 2025. Xinran Wang, Songyu Xu, Shan Xiangxuan, Yuxuan Zhang, Muxi Diao, Xueyan Duan, Kongming Liang, Zhanyu Ma, et al. Cinetechbench: A benchmark for cinematographic technique under- standing and generation. Advances in Neural Information Processing Systems, 38, 2026a. Yibin Wang, Yuhang Zang, Hao Li, Cheng Jin, and Jiaqi Wang. Unified reward model for multi- modal understanding and generation. arXiv preprint arXiv:2503.05236, 2025. Yuan Wang, Ouxiang Li, Yulong Xu, Borui Liao, Jiajun Liang, Jinghan Li, Meng Wang, Xintao Wang, Pengfei Wan, Kuien Liu, et al. Think, then score: Decoupled reasoning and scoring for video reward modeling. arXiv preprint arXiv:2605.05922, 2026b. Hou Xia, Zheren Fu, Fangcan Ling, Jiajun Li, Yi Tu, Zhendong Mao, and Yongdong Zhang. Video- levelgauge: Investigating contextual positional bias in video language models. In International Conference on Learning Representations, volume 2026, p. 45877โ45905, 2026. Jiazheng Xu, Yu Huang, Jiale Cheng, Yuanming Yang, Jiajun Xu, Yuan Wang, Wenbo Duan, Shen Yang, Qunlin Jin, Shurun Li, et al. Visionreward: Fine-grained multi-dimensional human prefer- ence learning for image and video generation. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, p. 11269โ11277, 2026a. Ran Xu, Tianci Liu, Zihan Dong, Tony Yu, Ilgee Hong, Carl Yang, Linjun Zhang, Tao Zhao, and Haoyu Wang. Alternating reinforcement learning for rubric-based reward modeling in non- verifiable llm post-training. arXiv preprint arXiv:2602.01511, 2026b. Songlin Yang, Haobin Zhong, Ruilin Zhang, Xiaotong Zhao, Shuai Li, Kai Zheng, Xuyi Yang, Zhe Wang, Zhenchen Tang, Yang Li, et al. Evalverse: Pipeline-aware and expert-calibrated bench- marking for professional cinematic video generation. arXiv preprint arXiv:2605.23271, 2026. 11 Yaodong Yu, Kwan Ho Ryan Chan, Chong You, Chaobing Song, and Yi Ma. Learning diverse and discriminative representations via the principle of maximal coding rate reduction. Advances in neural information processing systems, 33:9422โ9434, 2020. Yuxuan Zhang, Penghui Du, Bo Li, Cong Wei, Junwen Miao, Huaisong Zhang, Songcheng Cai, Yubo Wang, Dongfu Jiang, Yuyu Zhang, et al. Rewardharness: Self-evolving agentic post- training. arXiv preprint arXiv:2605.08703, 2026. Shijie Zhao, Xuanyu Zhang, Weiqi Li, Junlin Li, Li Zhang, Tianfan Xue, and Jian Zhang. Reasoning as representation: Rethinking visual reinforcement learning in image quality assessment. arXiv preprint arXiv:2510.11369, 2025. 12 AAPPENDIX Code, models, and project page: https://github.com/2kxx/RewardVerse. A.1EXTENDED EMPIRICAL DIAGNOSTICS ON SCALAR DRIFT A.1.1FULL 4-VARIANT ABLATION FOR QWEN2.5-VL Table 4 expands upon the core conclusions of Section 3.2. To ensure a controlled and rigorous evaluation, all four variants are evaluated using the same multi-modal LLM backbone (Qwen2.5- VL-7B). They share an identical pointwise hold-out set (n pw = 240) and pairwise comparison pool (n pair = 240), spanning sixteen key dimensions from EvalVerseโs global taxonomy to ensure comprehensive coverage. All inference hyperparameters are held constant across variants, utilizing a frame rate of FPS = 2 and greedy decoding. For evaluation metrics, we let ฯ(หs) denote the standard deviation of predicted scores, bias = หsโs label represent the systematic overestimation error, and |โ| represent the mean prediction gap on decided pairs (i.e., those whose constituent videos receive distinct scores). Table 4: A four-variant ablation study on the white-box Qwen2.5-VL-7B backbone across the shared 16-dimensional hold-out set (n pw = 240) and pair pool (n pair = 240). The deployed config- uration (V3) is highlighted in bold. VariantRubricFormatหs ฯ(หs)Bias Decided Pair Acc.|โ| V1 (NL free-float)โnatural4.747 0.203 +1.09758/2400.931 โ 0.338 V2 (soft-logits)โsoft-log. 4.194 0.258 +0.546239/2400.795 0.168 V3 (rubric + soft-logits)โsoft-log. 4.419 0.471 +0.771240/2400.754 0.216 V4 (rubric + multi-theme NL float)โnatural4.649 0.275 +1.00176/2400.737 โ 0.425 โ V1 and V4 produce only a small portion of distinct pair scores (the rest tie); the reported pair accuracy is evaluated on the decided subset and therefore over-states real-world separability. A.1.2LARGE-SCALE EVALUATION OF GEMINI-3.1-PRO ON 1K TEST SET To evaluate the scalability and generalizability of our rubric-guided paradigm on frontier models, we extend our evaluation to Gemini-3.1-Pro using the complete 1K multi-dimensional pointwise test set. Given that closed-source APIs restrict access to token-level logits, we benchmark the V1 (No Rubric) and V4 (With Rubric) configurations under natural-language float output formats. Table 5 lists the PLCC, SRCC, standard deviation and mean scores on the 1K dataset, while Figure 4 shows the distribution of our predictions against human labels. Table 5: Comprehensive pointwise calibration and correlation metrics of Gemini-3.1-Pro on the large-scale 1K pointwise test set (16 evaluation dimensions) under V1 and V4 configurations. Stan- dard deviation, biases, and discrete rating distributions are included to show scaling behavior and systematic offsets. VariantRubricหs ฯ(หs)BiasPLCC SRCC V1โ3.5581.500-0.398+0.490+0.480 V4โ3.3721.158-0.584+0.593+0.564 The empirical results on the 1K dataset demonstrate that the rubric-as-an-intermediate- representation paradigm generalizes robustly to proprietary, state-of-the-art systems. Integrating the rubric (V 1 โ V 4) unlocks substantial alignment gains: Pearson correlation (PLCC) increases by +0.103 (a 21.0% relative improvement), while Spearman rank correlation (SRCC) increases by +0.084 (17.5% relative), demonstrating that rubric-guided sub-dimensions align much closer with human multi-dimensional preferences. Notably, the calibration behavior of Gemini-3.1-Pro reveals a striking contrast to Qwen2.5-VL. While Qwen2.5-VL suffers from systematic overestimation (positive bias), Gemini-3.1-Pro exhibits a persistent underestimation (negative bias of โ0.398 in V1 and โ0.584 in V4). This divergence demonstrates that different foundational models, regardless of their scale, possess intrinsic score- scale biases. Relying solely on zero-shot static rubrics cannot eliminate these systematic biases, 13 1.01.52.02.53.03.54.04.55.0 score 0 50 100 150 200 250 300 350 # videos V1: Gemini, no rubric, NL-float prediction (n=994) human label (n=994) 1.01.52.02.53.03.54.04.55.0 score V4: Gemini + seed rubric, NL-float prediction (n=992) human label (n=994) Figure 4: Pointwise prediction histograms of Gemini-3.1-Pro (V1 vs. V4) on the complete 1K pointwise test set, overlaid on human gold standard ratings (orange). Rubrics improve correlation but worsen calibration. Decomposing evaluations (V4) aligns the rating variance closer to human baselines, reducing the uncalibrated standard deviation from 1.500 (V1) to 1.158 (V4). 0 20 40 60 80 100 120 # videos Q-Scorer prediction human label UnifiedReward prediction human label VideoReward prediction human label 1.01.52.02.53.03.54.04.55.0 score (predictions remapped to label scale [1, 5]) 0 20 40 60 80 100 120 # videos VisionReward prediction human label 1.01.52.02.53.03.54.04.55.0 score (predictions remapped to label scale [1, 5]) VideoScore-v1.1 prediction human label 1.01.52.02.53.03.54.04.55.0 score (predictions remapped to label scale [1, 5]) VideoScore-v2 prediction human label Figure 5: Score distributions of six reward models on the shared 10-dimensional test subset. Pre- dictions are remapped to the human rating scale [1, 5] for visualization. Blue bars denote model pre- dictions and orange bars denote human labels. Dashed lines show their means. Most reward models exhibit score-range compression or distribution shift. Q-Scorer produces a relatively broader score distribution, but it still does not fully match human ratings. which strongly justifies the necessity of our proposed RGPO training paradigm to explicitly align the reward model with human anchors. A.1.3SCALAR DRIFT ACROSS EXISTING REWARD MODELS We further examine the score distributions of six existing reward models on the shared 10- dimensional test subset. Since these models use different native output ranges, we remap their predictions to [1, 5] for visualization. This remapping is applied only to display the distributions. As shown in Figure 5, most reward models produce scores within a narrow range or show a clear shift from the human rating distribution. VideoReward and VideoScore-v1.1 mainly concentrate in the low-score region, while VisionReward and VideoScore-v2 place many predictions near the upper end. UnifiedReward also produces several narrow score peaks. Q-Scorer is the only model that gives a relatively broad and continuous score distribution, although its mean and overall distribution still differ from human ratings. These results show that score compression or scale shift is common across existing reward models. 14 A.1.4PER-DIM BREAKDOWN The pooled bias of the deployed V3 configuration is +0.771 (bootstrap 95% CI: [+0.64, +0.90], n boot = 2000, n = 240). Variants V1 (+1.10), V2 (+0.55), and V4 (+1.00) similarly overshoot human ratings, ruling out the rubric scaffold as the primary source of overestimation. As shown in Figure 6, 15 out of 16 evaluation dimensions exhibit positive biasโwith rhythm being the sole neg- ative exception (โ0.23). The largest calibration gaps occur in pacing (+1.68), consistency (+1.64), and character (+1.48). This systematic positive offset constitutes the limitation that our proposed RGPO training algorithm (Section 4) aims to mitigate. A dimensional analysis reveals a clear semantic divide: higher-level temporal reasoning and con- sistency dimensions (e.g., pacing, consistency) suffer from severe overestimation, whereas low- level sensory dimensions (e.g., rhythm) show minimal bias. This pattern suggests that uncalibrated MLLMs act as overly optimistic critics: they align well with low-level sensory cues but remain blind to fine-grained temporal artifacts and physical plausibility. An explicit reinforcement learning loop (RGPO) is vital to break through this limitation. rhythm logic lens composition chromaticity expression progression materiality visual_quality lighting action scene grounding character consistency pacing 0.25 0.00 0.25 0.50 0.75 1.00 1.25 1.50 1.75 bias = mean(pred) mean(label) Per-dim systematic overestimation under V3 (rubric + soft-logits) overall bias = +0.77 Figure 6: Per-dimension systematic overestimation bias of the V3 protocol (n = 15 per dimension). Bars represent the difference between predicted means and human ratings (หs d โs label,d ), with the dashed line marking the pooled bias (+0.77). The pervasive positive offset across 15 out of 16 dimensions highlights a limitation that static, zero-shot rubric prompting fails to resolve. A.1.5PROTOCOL SELECTION: HOW TO USE THE RUBRIC Section 3.2 establishes that the deployed evaluator must combine a rubric with a soft-logits readout. A secondary design question remains: within the rubric paradigm, what is the best way to query and format the prompt? We benchmark four candidate protocols on the visual quality hold-out set (n pw = 30, n pair = 30), all utilizing the same rubric: โข PR1 (Pairwise A/B): The model evaluates both videos simultaneously in a single forward pass, selecting video A or B with the rubric injected as an evaluation reference. โข PR2 (Pointwise Overall Integer): One forward pass is conducted per video; the model is prompted to emit a single integer score in the range [1, 5]. โข PR3 (Pointwise Multi-Theme Integer): One forward pass is conducted per video; the model outputs a JSON object containing per-theme integer scores. โข PR4 (Pointwise Multi-Theme Soft-Logits): One forward pass is conducted per video; the continuous soft-logits expectation is computed for each rubric theme, and the overall score is aggregated via a weighted average. This is the deployed configuration (V3). The format-failure rate counts forward passes that either cannot be successfully parsed as valid JSON/integers. Ties at integer granularity are reflected separately by the number of decided pairs. Since PR1 directly outputs a binary preference rather than a scalar value, its correlation metrics (PLCC, SRCC) and score margins are not defined. 15 Table 6: A comparative evaluation of different rubric-querying protocols on the visualquality hold- out set (30 pointwise samples and 30 evaluation pairs). The proposed PR4 is the unique protocol that simultaneously achieves a 0% format-failure rate, generates continuous scalar outputs, and preserves full pairwise separability (100% decided pairs). ProtocolFormat-Fail RateDecidedPair Acc.PLCC|โ| PR1 โ pairwise A/B0.00030/301.000โ PR2 โ pointwise integer (text)0.4679/301.0000.3131.333 PR3 โ multi-theme integer (text)0.5336/300.8330.3741.667 PR4 โ multi-theme soft-logits0.00030/300.8330.2290.269 Although PR1 achieves perfect pairwise accuracy, it is unsuitable as an RL reward signal due to its lack of scalar outputs. Furthermore, PR1 is highly sensitive to input ordering (see Appendix A.1.6), consistent with contextual positional bias observed in video-language models (Xia et al., 2026). While proprietary models (e.g., Gemini-3.1-Pro) handle multi-video comparison robustly, using them as iterative RL reward models is computationally and financially prohibitive. For integer-based configurations (PR2 and PR3), formatting parser failures and scale quantization ties severely hinder evaluation, yielding failure rates of 46.7% and 53.3%, respectively. These high rates demonstrate that free-form text score generation is fragile, as minor JSON variations or tok- enization shifts block automated parsing. Conversely, PR4 bypasses parsing bottlenecks by extract- ing soft logits directly from vocabulary tokens, guaranteeing a 0% failure rate by construction. By avoiding multi-video reasoning bottlenecks and providing a continuous optimization surface, PR4 serves as our final deployment variant. A.1.6STABILITY UNDER PROMPT PERTURBATION AND PAIR SWAP Two diagnostic studies probe the second symptom of scalar drift: high variance under context shift. Diagnostic 1: Prompt perturbation (V2 vs. V3). We evaluate 48 videos (3 per dimension across 16 dimensions) three times under three semantically equivalent overall-quality instructions, report- ing the mean per-video standard deviation. We compare V2 and V3 because both utilize the soft- logits readout, leaving the rubric as the sole independent variable. Variant V1 is excluded as its scores collapse onto a single peak, rendering stability metrics meaningless. To ensure commen- surability between the two distinct scoring scales, each variantโs per-video standard deviation is normalized by its global standard deviation ฯ(หs) from Table 4. Table 7: Per-video score standard deviation acrossK = 3 paraphrased instructions (N = 48 videos). Statistics are computed over per-video prompt-perturbation standard deviationsฯ i N i=1 . โNormal- izedโ reports the Raw Mean scaled by the global standard deviation ฯ(หs). Raw MeanMedianMaxNormalized V2 (soft-logits, no rubric)0.0460.0400.1200.179 V3 (rubric + soft-logits)0.0940.0380.4180.199 After normalization, both variants exhibit comparable relative variance under prompt paraphrase (0.18 vs. 0.20). This demonstrates a crucial structural benefit: adding the rubric (V3) decompresses the scoring scale by 82.6% (ฯ(หs) increases from 0.258 to 0.471) while maintaining similar relative stability under prompt paraphrases. The smaller raw variation of V2 mainly results from its com- pressed score range. Under the soft-logits readout, the rubric substantially expands score resolution without causing a comparable increase in relative sensitivity to prompt paraphrases. Diagnostic 2: Pair A/B order swap (no rubric vs. seed rubric). We evaluate every video pair in both input orderings, (y w , y l ) and (y l , y w ), and report the rate at which preference verdicts disagree across the swap (flip rate). The pairwise A/B protocol is tested on 48 sampled pairs (3 per dimension) under two settings: without the rubric (baseline) and with the seed per-dimension rubric injected as a reference (PR1). Table 8 summarizes the results. Introducing the rubric reduces the flip rate by 23.0 percentage points (from 0.792 to 0.562), confirm- ing that a rubric serves as a grounding reference that aligns model decisions with objective criteria rather than order-sensitive heuristics. However, a flip rate of 56.2% remains very high, confirming 16 Table 8: Pair A/B swap flip rates with and without the rubric (48 pairs). Injecting the rubric sub- stantially reduces the flip rate, yet the residual bias remains a significant bottleneck for the pairwise protocol. SettingDecided pairsFlip rate Pairwise A/B, no rubric48/480.792 Pairwise A/B, seed rubric48/480.562 Table 9: Pointwise scoring before and after RGPO under the same dynamic-rubric protocol. Bias is the difference between the mean prediction and the mean human rating. The 95% confidence interval is obtained by bootstrap sampling. ModelNMean ฯ(หs)Bias95% CIPLCCSRCC Pre-RGPO2404.1980.457+0.610[+0.488, +0.736]0.2400.222 Post-RGPO2403.7521.332+0.164[+0.012, +0.313]0.5050.458 that the binary A/B format is highly vulnerable to position bias when processing two complex video sequences simultaneously. These findings yield two critical conclusions. First, this high residual flip rate disqualifies the pair- wise A/B protocol from serving as an RL reward model, as unstable preference signals would gen- erate highly noisy gradients. This provides an independent justification for adopting our pointwise multi-theme soft-logits protocol (PR4 / V3). Second, it highlights the division of labor in our frame- work: the rubric calibrates the scoring criteria, while the pointwise soft-logits format ensures greater robustness under context shifts by completely bypassing order-dependent comparison bottlenecks. A.1.7SCALAR DRIFT BEFORE AND AFTER RGPO The four-variant study in Appendix A.1.1 analyzes the effect of rubric injection and score readout before training. We further evaluate whether the complete RGPO training further mitigates the remaining scalar drift. We compare the original Qwen2.5-VL-7B model with the jointly trained RewardVerse model. Both models generate a dynamic rubric for each query and use the same multi- theme soft-logits scoring protocol. Score calibration is evaluated on the same pointwise hold-out set (n = 240) in Appendix A.1.1, while prompt stability is evaluated on the 48 videos and three paraphrased instructions used in Appendix A.1.6. As shown in Table 9, the untrained model remains systematically optimistic even when using dy- namic rubrics, with a mean bias of +0.610. Although RGPO is optimized for relative score margins rather than absolute scores, the bias decreases to +0.164, while PLCC and SRCC increase from 0.240 and 0.222 to 0.505 and 0.458, respectively. The prediction standard deviation also increases from 0.457 to 1.332, showing that RGPO further expands the previously compressed score range. At the dimension level, the absolute calibration bias is reduced on 10 out of 16 dimensions, with the largest reductions appearing in pacing, materiality, character, lighting, and scene. We addition- ally repeat the prompt-perturbation diagnostic under the same dynamic-rubric protocol. For each video, we compute the standard deviation of its scores across three semantically equivalent instruc- tions. Since RGPO substantially expands the overall score range, we normalize the mean per-video standard deviation by the global prediction standard deviation. As shown in Table 10, the raw variation slightly increases from 0.094 to 0.113 because RGPO expands the overall score range. After normalization, however, the relative variation decreases from 0.206 to 0.085, corresponding to a 58.7% reduction. This shows that RGPO improves robustness to prompt paraphrases while preserving the expanded score resolution. These results complement the protocol analysis in Appendix A.1. Rubrics with soft-logits reduce score-range collapse before training, while RGPO further improves both score alignment and robustness to prompt paraphrases. A.2EVALVERSE 16-DIMENSIONAL TAXONOMY Throughout the paper, we adopt the sixteen fine-grained secondary dimensions of the EvalVerse suite, which systematically cover the six coarse axes used in generative video evaluation: Visual Concept Design, Acting, Cinematography, Aesthetics, Affectivity, and Multi-Shot Cutting. Table 11 provides the definitions and evaluation focus of each dimension. Rubric generation and scoring in 17 Table 10:Prompt-perturbation stability before and after RGPO under the same dynamic-rubric scoring protocol. Raw statistics are computed over the per-video standard deviations across three paraphrased instructions. Normalized is the Raw Mean divided by the global prediction standard deviation. ModelRaw MeanMedianMaxGlobal ฯ(หs)Normalized Pre-RGPO0.0940.0380.4180.4570.206 Post-RGPO0.1130.0420.7551.3320.085 RewardVerse are conditioned on the target dimension d, enabling the generator to learn dimension- adaptive themes and weights rather than a single generic scoring policy. Table 11: The sixteen secondary evaluation dimensions adapted from the EvalVerse test suite, or- ganized by their high-level axis and specific evaluation focus. All training and testing in Sections 3 and 5 are conducted using this taxonomy. IDDimensionAxisEvaluation Focus 1. Visual Concept Design D 1 CharacterVisual ConceptClarity, recognizability, and adherence of character appear- ance to the designated settings. D 2 SceneVisual ConceptPlausibility and appeal of the scene setup; establishment of a clear, distinct, and aesthetically valuable visual style. 2. Acting D 3 ConsistencyActingCoherence of character identities and attributes across frames without temporal drifting, morphing, or flickering. D 4 ActionActingNaturalness and physical plausibility of character move- ments aligned with their established identity. D 5 ExpressionActingAccuracy and vividness of facial expressions in conveying the intended emotions. 3. Cinematography D 6 CompositionCinematography Clarity, stability, and explicit visual intent of individual shot compositions. D 7 LensCinematography Reasonableness, clarity, and stability of camera parameters (focal length, depth-of-field, etc.). D 8 PacingCinematography Appropriateness of camera movements and blocking; ab- sence of jarring or unmotivated motion. 4. Aesthetics D 9 Visual Quality AestheticsLow-level perceptual quality, temporal consistency, and ab- sence of digital artifacts. D 10 ChromaticityAestheticsColor harmony, saturation control, and emotive power. D 11 MaterialityAestheticsAesthetic appeal and self-consistent realism of textures, sur- face properties, and fine details. D 12 LightingAestheticsPhysical plausibility and artistic execution of illumination, shading, and cinematic lighting logic. 5. Affectivity D 13 GroundingAffectivityStability and recognizability of the overall emotional direc- tion and atmospheric setting. D 14 ProgressionAffectivityNatural and controlled build-up, transition, and resolution of emotion across the temporal dimension. 6. Multi-Shot Cutting D 15 LogicMulti-ShotContinuity of time, space, and action between shot transi- tions; effectiveness of editing in aiding narrative compre- hension. D 16 RhythmMulti-ShotOverall control of shot durations and cutting cadence; effec- tive emotional guidance and clear temporal organization. For each dimension, we manually curate a small seed pool of 30 preference pairs (q,y w ,y l ), totaling 480 pairs across the entire taxonomy. These pairs serve both to self-evolve the seed rubric R โ d (Section 4.2) and to train the RGPO framework. All test-set correlations reported in Table 1 are evaluated on a disjoint pointwise hold-out set containing 1K evaluation samples (n = 994) across the taxonomy. 18 Table 12: Baseline scoring protocols and dimension alignment used in Table 1. MethodScoring protocol and alignment VideoScore-v1.1The model predicts visual quality, temporal consistency, dynamic degree, text-to-video alignment, and fac- tual consistency. Visual quality is used for Visual Quality, Chromaticity, Materiality, and Lighting; temporal consistency for Consistency and Rhythm; dynamic degree for Action; text-to-video alignment for Scene and Character; and factual consistency for Logic. VideoScore2The model generates visual quality, text-to-video alignment, and physical/common-sense consistency scores. Visual quality is used for Visual Quality, Chromaticity, Materiality, and Lighting; text-to-video alignment for Scene and Character; and physical/common-sense consistency for Consistency, Action, Logic, and Rhythm. UnifiedRewardWe use its Style, Physics, and Alignment scores. Style is used for Visual Quality, Chromaticity, Materiality, Lighting, and Character; Physics for Consistency, Action, and Rhythm; and Alignment for Scene and Logic. VideoRewardWe use its visual quality (VQ), motion quality (MQ), and text alignment (TA) scores. VQ is used for Visual Quality, Chromaticity, Materiality, Lighting, and Character; MQ for Consistency, Action, and Rhythm; and TA for Scene and Logic. VisionRewardWe use the score from its fixed video checklist for all reported dimensions. The checklist is not changed across dimensions. Q-ScorerWe sample video frames, score each frame using the official image-quality prompt, and average the frame scores. The same frame-averaged score is used for all reported dimensions. Baseline Evaluation and Dimension Alignment. We briefly introduce the external reward models used in Table 1. VideoScore-v1.1 is a discriminative video evaluator with five regres- sion heads for visual quality, temporal consistency, motion, text alignment, and factual consis- tency. VideoScore2 is a generative video evaluator that scores visual quality, text alignment, and physical/common-sense consistency. UnifiedReward is an MLLM-based reward model that evalu- ates generated content from Style, Physics, and Alignment. VideoReward is a discriminative video reward model trained with human preferences and predicts visual quality, motion quality, and text alignment. VisionReward is a fine-grained reward model that evaluates videos using a fixed weighted checklist. Q-Scorer is a state-of-the-art IQA model that is particularly strong at structural quality assessment. We use the official checkpoints without additional training. Since these models do not cover all 16 EvalVerse dimensions, we report the dimensions supported in Table 1 and mark the others as โโโ. For models with coarse outputs, we use the closest native score following Table 12. A.3DOWNSTREAM GRPO FINE-TUNING A.3.1FINE-TUNING SETUP AND EVALUATION We further evaluate whether RewardVerse can serve as an effective reward signal for downstream video generation optimization. We fine-tune Wan-2.2-A14B with GRPO using either VideoReward or RewardVerse as the reward model, while the original Wan-2.2-A14B is used as the base model. The optimization targets only the Visual Quality dimension. All training settings are kept identical across the two GRPO runs. We train with LoRA rank 64 and a rollout group size of G = 16. For evaluation, we use the VBench prompt suite (Huang et al., 2024) and generate 17-frame videos at 480ร 832 resolution with deterministic sampling. We report VBench-Quality, averaged over six official quality dimensions, and VBench-Text, averaged over three official text-related dimensions, together with several representative VBench dimensions. Since only Visual Quality is optimized, the remaining dimensions are used to examine whether reward optimization degrades other video capabilities. A.3.2DOWNSTREAM RESULTS As shown in Table 13, RewardVerse improves Imaging Quality from 0.640 to 0.653, which is the dimension most directly related to the optimized Visual Quality reward. It also improves Human Ac- tion from 0.900 to 0.950, while Background Consistency, Temporal Flickering, Motion Smoothness, and Subject Consistency remain close to the base model. In contrast, optimization with VideoRe- ward reduces both VBench-Quality from 0.808 to 0.804 and VBench-Text from 0.428 to 0.392. These results show that RewardVerse improves the target visual quality without clear degradation of other video capabilities. In comparison, VideoReward improves Imaging Quality but reduces the overall VBench performance, suggesting that RewardVerse is less prone to reward hacking during downstream optimization. 19 Table 13:Downstream GRPO fine-tuning of Wan-2.2-A14B on the Visual Quality dimension. RewardVerse improves the target quality metric while largely preserving other video capabilities. Best results are shown in bold. MetricBaseVideoRewardRewardVerse Imaging Quality0.6400.6480.653 Human Action0.9000.8000.950 Background Consistency0.9190.9160.920 Temporal Flickering0.9430.9380.944 Motion Smoothness0.9660.9600.966 Subject Consistency0.8980.8810.894 VBench-Quality (6D)0.8080.8040.809 VBench-Text (3D)0.4280.3920.446 A.4PROMPTS AND TRAINING DETAILS A.4.1PROMPT TEMPLATES Seed Rubric Generation. For each training pair, the teacher receives the target dimension, gener- ation prompt, videos, and human scores. It is asked to generate observable and dimension-specific criteria, with all rubric weights summing to one. The generated rubric is then used to score the videos and is revised when its prediction differs from the human label by more than 0.5. We use at most five revision rounds and remove near-duplicate rubrics using a token-level Jaccard threshold of 0.85. Dynamic Rubric Generation. The rubric generator uses the following prompt: You are a cinematic video evaluation expert. Generate exactly five rubric items for scoring videos on the dimension dimension. Task: task_description Caption: caption Each rubric contains: - a clear theme; - a weight in (0, 1), with all weights summing to 1; - at least two observable evaluation tips. Output strict JSON: "rubrics": [ "theme": "...", "weight": 0.N, "tips": ["...", "..."] , ... ] Rubric-Guided Scoring. Given a video and a generated rubric, the scorer uses the following prompt: 20 You are a strict video reward judge for the dimension dimension. Score the single video on each rubric using an integer from 1 to 5. Caption: caption Rubric: rubric_text Respond with exactly one line: theme_1: _ theme_2: _ theme_3: _ theme_4: _ theme_5: _ At each score position, we read the logits of tokens โ1โ to โ5โ and compute their expected value. This fixed output format provides continuous theme scores without parsing free-form text. A.4.2REWARD MODEL TRAINING SETTINGS We train RewardVerse on Qwen2.5-VL-7B using the 480 preference pairs described in Ap- pendix A.2. The training is divided into two separate runs corresponding to the two stages of RGPO. Both the scorer warm-up and joint optimization are trained for one epoch. Each epoch contains ap- proximately 60 optimization steps. For each training pair, we sample a rollout group of size G = 8. For both stages, we use AdamW with a learning rate of 2ร 10 โ6 , 30 learning-rate warm-up steps, a preference temperature of 1.5, and a maximum gradient norm of 1.0. The GRPO KL coefficient is set to 0.1. Training uses bfloat16 precision and DeepSpeed ZeRO-3 on 8 NVIDIA H20 GPUs. Stage 1: Scorer Warm-up. The student and reference policies are initialized from Qwen2.5-VL- 7B. The seed rubric is fixed for all rollouts, so this stage only updates the scorer role. The scorer update coefficient is fixed at 0.4 throughout the epoch. We set the margin calibration weight ฮป cal to 0.2, the score-format gate to 0.55, and use an additional score-side KL coefficient of 0.005 to limit score-scale drift. The model is trained for one epoch. Stage 2: Joint Optimization. We initialize the student from the final Stage-1 checkpoint. The same checkpoint is used as the frozen reference policy for both rubric generation and scoring. The generator produces dynamic rubrics from the query, and the scorer evaluates both videos under each sampled rubric. The rubric-alignment weight and format reward weight are both fixed at 0.5. We increase the margin calibration weight to 0.3 and use a score-side KL coefficient of 0.015. The scorer update coefficient is linearly increased from zero at step 20 to 0.4 at step 30. Joint optimization is also run for one epoch. The Stage-2 checkpoint at step 45 is selected based on the macro-averaged PLCC on the held-out validation set and used for all evaluations. Stage-2 GRPO Objective. For completeness, the generator-side GRPO objective used in Stage 2 is: L (2) GRPO (ฮธ) =โ 1 G G X g=1 1 | ห R g | | ห R g | X t=1 h min ฯ g,t (ฮธ) ห A g , clip(ฯ g,t (ฮธ), 1โฮต, 1+ฮต) ห A g โ ฮฒ KL D KL ฯ gen ฮธ ฯ gen ref g,t i , (13) where ห A g and ฯ g,t (ฮธ) are defined in Section 4.3, and ฯ gen ref is the frozen reference generator. A.4.3QUALITATIVE STUDY: INSIDE THE LEARNED RUBRICS Figure 7 visualizes the learned rubrics generated by RewardVerse across different prompts and eval- uation dimensions. Notably, the generator automatically adjusts the level of abstraction in the tips based on the target dimension. For global, technical, or aesthetic axes such as Visual Quality, the generated tips remain abstract to avoid overfitting to the specific prompt content. In contrast, for 21 Table 14: Main hyperparameters for the two-stage RewardVerse training. Both stages are trained for one epoch. SettingStage 1: Warm-upStage 2: Joint InitializationQwen2.5-VL-7BStage-1 checkpoint Reference policyQwen2.5-VL-7BStage-1 checkpoint Training epochs11 Optimization stepsโผ60โผ60 Rollouts per sample G88 Learning rate2ร 10 โ6 2ร 10 โ6 LR warm-up steps3030 Preference temperature1.51.5 GRPO KL coefficient0.10.1 Score-side KL coefficient0.0050.015 Margin calibration weight0.20.3 Rubric-alignment weightโ0.5 Format reward weightโ0.5 Maximum scorer update weight0.40.4 Scorer update scheduleFixedSteps 20โ30 Score-format gate0.550.55 Maximum gradient norm1.01.0 ...... Prompt ํ [shot_0_prompt]Eye-level close-up: In the center of the frame is an elderly man (Person_01) wearing a light yellow robe with gold patterns. He has a shaved forehead with a Qing Dynasty queue and a sparse beard on his chin. His brows are slightly furrowed, his gaze is firm, and his lips are slightly parted as he speaks toward the leftof the frame. He uses his right index and middle fingers together in an emphatic gesture, pointing toward the man on the right. On the right side of the frame, the right ear and part of the cheek of another young man (Person_02) are visible. The background is an ornately decorated traditional Chinese room with exquisite carved wooden window lattices and hanging classical paintings. The overall lighting is bright and even, with saturated colors, creating a solemn and serious court atmosphere. [shot_1_prompt]Eye-level close-up: The camera angle shifts but remains focused on the face and shoulders of the same elderly man (Person_01), captured from an almost frontal perspective. He maintains a firm expression, with an air of unquestionable authority between his brows... ...... Generated Video ํGenerated Video ํ wThemeTipsScore 10.2 Color Depth and Consistency Evaluate the saturation and depth of colors throughout the frame. Ensure consistent color grading across light sources and objects. 4.21 20.2 Lighting and Shadows Alignment Verify that shadows cast by the character align with the light source. Check for correct gradient of light intensity across the scene. 3.85 30.2 Texture Detail and Material Fidelity Assess the level of detail in textures such as skin, costumes, and background elements. Ensure realistic rendering of materials like sweat or fabric. 3.92 40.2 Visual Style and Genre Compliance Ensure adherence to the specified visual style or genre. Maintain a cohesive visual style from frame to frame. 4.02 50.2 Dynamic Scene Composition Check for proper composition in dynamic scenes with characters in motion. Verify balance and placement of characters within the frame. 4.07 wThemeTipsScore 10.2 Facial Expression Consistency Elderly man's facial expressions should maintain a consistent firmness throughout the shots. Avoid sudden changes in the position or appearance of facial features. 4.74 20.2 Clothing Details Stability The light yellowrobe with gold patterns on Person_01 should remain unchanged in all shots. The texture and pattern of the clothing should stay consistent without alterations. 4.54 30.2 Gaze Direction Accuracy Person_01's gaze direction towards the left should remain consistent across shots. Person_02's focus should stay directed at Person_01's speech. 4.09 40.15 Body Language Continuity Person_01's emphatic gesture using his right index and middle fingers should persist in all shots. Person_02's posture should reflect attentiveness and readiness to respond. 3.88 50.25 Background Elements Consistency The ornately decorated traditional Chinese room with carved wooden window lattices and classical paintings should remain unchanged. The lighting should maintain its brightness and evenness without variation. 4.53 Visual Quality 4.05 4.01 4.39 Dimension Dimension Aggregation Dynamic Rubric Prompt ํ In the center of a smoky and boisterous folk stage, a young man (Person_02) with exaggerated black butterfly face paint and a bright red performance costume adorned with pom-poms is frantically swinging drumsticks. His skin glistens with sweat, and his eyes are filled with the fanaticism and focus of the performance. To his left stands a solemn middle-aged man (Person_01) with rough skin; he frowns slightly, his expression pained as he slowly falls backward, and the young man rushes to support him. The background features layers of hanging pink lanterns and majestic red door god statues. Intense purple and white top lighting pierces through the smoke, creating a distinct Tyndall effect and a mysterious, highly tense folk ritual atmosphere. The color palette is dominated by high- saturation red, purple, and gold, presenting a rich cinematic texture. The camera begins as a static eye-level medium shot, then fluidly pans right following the red-clad man's movements in sync with the drumming rhythm, capturing his intense physical tension and the stage's dynamic energy. Consistency 4.74 Dynamic Rubric Aggregation Overall Score Overall Score Figure 7: Qualitative view of dynamic rubrics generated by RewardVerse. Left (Visual Quality): themes and tips stay abstract. Right (Consistency): tips dynamically ground on specific entities in the prompt, illustrating scene-adaptation. identity-bound or scene-bound axes such as Consistency, the tips dynamically ground on specific entities in the prompt (e.g., monitoring โthe light yellow robe with gold patterns on Person 01โ), as these entities are exactly what must remain stable. This example illustrates how the generator adjusts the tip granularity to different evaluation dimensions. A.5INFERENCE-TIME COST We measure the inference latency of the four pointwise scoring variants on a single H20 under the same inference settings (bf16, greedy decoding, FPS = 2, and a 64-frame budget). All variants use the same visual backbone and frame-sampling pipeline. As shown in Table 15, rubric generation introduces only a small inference overhead. V3 takes approximately 2.0 s per video, compared with 1.7 s for the rubric-free V2 baseline, corresponding 22 to an increase of about 18%. V4 is slightly slower because it additionally generates multiple theme scores in natural text. The rubric depends only on the evaluation query rather than the video. Therefore, when multiple candidate videos are evaluated under the same query, the rubric can be generated once and reused, further reducing the effective per-video overhead. Table 15: Per-video inference latency of the four pointwise scoring variants on a single H20. VariantRubricReadoutLatency V1 (no rubric, NL float)โNL float โ 1.7 s V2 (no rubric, soft-logits)โsoft-logits โ 1.7 s V3 (rubric + soft-logits, deployed)โsoft-logits โ 2.0 s V4 (rubric + multi-theme NL float)โNL float โ 2.2 s A.6LIMITATIONS AND FUTURE WORK Our current results should be read with a few caveats in mind. Backbone Capacity. For a controlled and fair evaluation against existing video reward models, we intentionally selected the standard-sized Qwen2.5-VL-7B as our backbone. However, the effec- tiveness of RGPO is bounded by the capability of the underlying MLLM. RGPO can better align the modelโs existing evaluation ability with human judgments, but it cannot recover visual or temporal details that the backbone fails to recognize. Therefore, stronger backbones are expected to produce more reliable rubrics and scores, leading to a higher performance ceiling for RL optimization. Theme-Level Calibration. Our current calibration objective directly aligns only the aggregated score margin between preferred and non-preferred videos. The score of each individual rubric theme is not directly supervised. As a result, the final reward can be well aligned with human preferences while some theme scores remain biased or inconsistent. Directly calibrating theme-level scores is therefore an important direction for improving both score reliability and interpretability. Scaling. Our current experiments are conducted with a 7B reward backbone, only 30 preference pairs per dimension, and one 14B video generation model for downstream RL. It remains necessary to scale RewardVerse to stronger reward backbones, larger training sets, and more video generation models to verify whether the observed gains remain consistent at scale. Relation to CoT-Based Scoring. Recent CoT-based visual scoring methods treat reasoning as an intermediate representation that summarizes visual evidence before producing a quality score (Zhao et al., 2025; He et al., 2025; Wang et al., 2026b). RewardVerse operates at a different stage. Its rubric is generated from the evaluation dimension and prompt without video access, and specifies what should be checked before scoring. Therefore, CoT summarizes the evidence observed in the video, whereas the rubric defines the evaluation criteria. The two designs are complementary, and the generated rubric may further guide a CoT-based scorer. Future Work. Future work will focus on stronger MLLM backbones, direct theme-level calibra- tion, larger-scale training and downstream RL experiments, and rubric-guided CoT scoring. 23