Paper deep dive
Flow3D-OPD: Multi-Teacher On-Policy Distillation for 3D Geometry Generation with Flow-Matching Diffusion Transformer
Zhiwei Ning, Zhen Zhou, Puhua Jiang, Xintong Han, Gengming Zhang, Jie Yang, Zhonglong Zheng, Yuanjie Zheng, Wei Liu, Chunchao Guo
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 9/13/2026, 4:48:51 AM
Summary
The paper introduces Flow3D-OPD, a two-stage post-training framework for 3D geometry generation using flow-matching diffusion Transformers. It addresses reinforcement learning bottlenecks in 3D generation, such as reward definition and gradient interference, by employing multi-teacher on-policy distillation. Stage 1 involves fine-tuning a base model and training domain-specialized teacher models (for orientation, pose, and consistency) using an agentic verifier and Direct Preference Optimization (DPO). Stage 2 consolidates these teachers into a unified student model via on-policy distillation with hard task-routing and gradient accumulation, achieving superior geometric quality across all dimensions.
Entities (10)
Relation Signals (7)
Flow3D-OPD → uses → On-Policy Distillation
confidence 95% · Flow3D-OPD... introduces multi-teacher distillation into 3D geometry generation... consolidate heterogeneous expertise into a unified student model through on-policy distillation
Flow3D-OPD → uses → Direct Preference Optimization
confidence 95% · cultivate domain-specialized teacher models via direct preference optimization (DPO)
Agentic Verifier → basedon → Gemini 3 Flash
confidence 90% · We design an agentic reward system R_m based on Gemini-3-Flash
Flow3D-OPD → buildson → Hunyuan3D-2.1
confidence 90% · Our model follows the architecture of Hunyuan3D-2.1
Flow3D-OPD → optimizesfor → Orientation
confidence 90% · train three teacher models targeting distinct evaluation dimensions... orient
Flow3D-OPD → optimizesfor → Pose
confidence 90% · train three teacher models targeting distinct evaluation dimensions... pose
Flow3D-OPD → optimizesfor → Consistency
confidence 90% · train three teacher models targeting distinct evaluation dimensions... consist
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recent image-to-3D generation models built on flow-matching diffusion Transformers (DiT) can produce high-fidelity meshes, yet their post-training strategy remains largely unexplored. There exist several critical bottlenecks in reinforcement learning: the inherent difficulty of defining comprehensive rewards for 3D geometric quality, and the gradient interference that arises when jointly optimizing heterogeneous objectives. Inspired by the practicability of on-policy distillation (OPD) in large language models and image generation, we propose \textbf{Flow3D-OPD}, a two-stage post-training framework that introduces multi-teacher distillation into 3D geometry generation. In the first stage, we utilize the semi-policy to enhance the foundational capability of the pretrained model and then design an agentic verifier for 3D geometric quality evaluation. Based on the verifier, we could cultivate domain-specialized teacher models via direct preference optimization (DPO). In the second stage, we consolidate heterogeneous expertise into a unified student model through on-policy distillation with hard task-routing sampling and gradient accumulation, which could mitigate the gradient interference in joint optimization. Without relying on elaborate modifications, our straightforward yet effective design achieves consistent improvements across all geometric quality dimensions and surpasses all teacher models in the average metric. Extensive experiments demonstrate that our approach provides an effective paradigm for reinforcement learning in 3D generation.
Tags
Links
- Source: https://arxiv.org/abs/2609.07137v1
- Canonical: https://arxiv.org/abs/2609.07137v1
Trouble viewing inline? Open PDF directly →
Full Text
61,522 characters extracted from source content.
Expand or collapse full text
Flow3D-OPD: Multi-Teacher On-Policy Distillation for 3D Geometry Generation with Flow-Matching Diffusion Transformer Zhiwei Ning Zhen Zhou Puhua Jiang Xintong Han Gengming Zhang Jie Yang Zhonglong Zheng Yuanjie Zheng Wei Liu Chunchao Guo Abstract Recent image-to-3D generation models built on flow-matching diffusion Transformers (DiT) can produce high-fidelity meshes, yet their post-training strategy remains largely unexplored. There exist several critical bottlenecks in reinforcement learning: the inherent difficulty of defining comprehensive rewards for 3D geometric quality, and the gradient interference that arises when jointly optimizing heterogeneous objectives. Inspired by the practicability of on-policy distillation (OPD) in large language models and image generation, we propose Flow3D-OPD, a two-stage post-training framework that introduces multi-teacher distillation into 3D geometry generation. In the first stage, we utilize the semi-policy to enhance the foundational capability of the pretrained model and then design an agentic verifier for 3D geometric quality evaluation. Based on the verifier, we could cultivate domain-specialized teacher models via direct preference optimization (DPO). In the second stage, we consolidate heterogeneous expertise into a unified student model through on-policy distillation with hard task-routing sampling and gradient accumulation, which could mitigate the gradient interference in joint optimization. Without relying on elaborate modifications, our straightforward yet effective design achieves consistent improvements across all geometric quality dimensions and surpasses all teacher models in the average metric. Extensive experiments demonstrate that our approach provides an effective paradigm for reinforcement learning in 3D generation. 1Shanghai Jiao Tong University 2 Zhejiang Normal University 3 Shandong Normal University 4Tencent Hunyuan3D Introduction Recent 3D geometry generation pipelines (Li et al. 2025; Zhao et al. 2025; Lai et al. 2026; Lai et al. 2025) employ a mixture architecture where a diffusion Transformer (DiT) (Peebles and Xie 2023) generates latent representations, and then the subsequent variational autoencoder (VAE) and isosurface extraction algorithm are utilized to decode the representation into explicit 3D meshes, as shown in Fig. 1. The pipeline produces high-quality 3D assets from the image, enabling a wide range of applications in virtual reality, digital twins, and embodied AI (Mildenhall et al. 2021; Poole et al. 2022). Despite this rapid advancement, post-training approaches through reinforcement learning (RL) remain absent from the 3D generation literature, which has long been employed in large language models (Ouyang et al. 2022; Rafailov et al. 2023) and image generation (Black et al. 2024; Liu et al. 2025). The fundamental obstacle lies in the ambiguity of quality evaluation for 3D geometry. Unlike text-to-image generation where verifiers have been rapidly developed and proven effective (Huang et al. 2023; Ghosh et al. 2023), precise quality assessment for 3D meshes requires multi-view rendering and geometric analysis, which relies on specific multi-modal reasoning. Existing approaches (Lai et al. 2026; Li et al. 2025) utilize ULIP or Uni3D models (Xue et al. 2023; Zhou et al. 2024) to calculate global similarity between 2D images and 3D points, yet such a score cannot reflect fine-grained geometric dimensions such as orientation alignment, pose accuracy, and textural consistency. Moreover, jointly optimizing these heterogeneous objectives via diverse rewards will encounter gradient interference where updates favoring one dimension degrade others, which hinders consistent improvement in various dimensions. Figure 1: Our framework focuses on the on-policy distillation of the DiT module in the 3D generation pipeline. Our Flow3D-OPD enables to achieve a higher average win ratio than three specialized teachers. However, recent image generation works introduce the on-policy distillation (OPD) strategy to improve model performance across multiple evaluation dimensions simultaneously. DiffusionOPD (Li et al. 2026) and Flow-OPD (Fang et al. 2026) prove that multi-teacher OPD can effectively unify various specialized capabilities into a single model by replacing sparse scalar rewards with dense velocity-field supervision along the denoising trajectory in the student model. This distillation paradigm could alleviate gradient interference in optimizing multiple dimensional quality in image generation. However, applying multi-teacher on-policy distillation to 3D geometry generation remains unexplored due to the difficulties in training domain-specialized teachers with 3D-specific rewards and consolidating these heterogeneous capabilities stably. In this paper, we propose Flow3D-OPD, which innovatively introduces multi-teacher on-policy distillation into 3D geometry generation. Our approach follows a two-stage pipeline to overcome the above challenges. In the first stage, we train three teacher models targeting distinct evaluation dimensions. Specifically, we employ a semi-policy preference optimization strategy to enhance the base model’s capability for fine-tuning, then decompose the quality evaluation into three different dimensions. Subsequently, an agentic verifier based on VLMs is designed for pairwise mesh comparison, which conducts multi-round structured reasoning with tool invocation and case memory mechanisms. Based on this verifier, we train the dedicated teacher for each quality dimension via direct preference optimization (DPO). In the second stage, we distill all three frozen teachers into a unified student model through on-policy distillation with hard task-routing and gradient accumulation mechanisms. At each training step, the student model generates its own denoising trajectory, and the corresponding teacher model provides step-level guidance along the hard task-routing. To alleviate gradient interference, we accumulate the gradient in a mini-batch for a single backward pass to ensure balanced multi-dimensional optimization. As shown on the right side of Fig. 1, our student model successfully surpasses all teacher models on the average win-ratio metric. In summary, our contributions are three-fold as follows: • We propose the Flow3D-OPD approach, which is, to the best of our knowledge, the first to introduce on-policy distillation to 3D geometric generation through a two-stage training pipeline. • We design an agentic reward system to enable DPO training of domain-specific teachers, followed by a multi-teacher distillation strategy with hard task-routing and gradient accumulation that consolidates their capabilities into a unified student model. • Extensive experiments demonstrate that our method consistently improves generation quality across all dimensions and achieves competitive or superior performance across multiple metrics. Related Work 3D Generation Models The field of 3D geometry generation has undergone several paradigm shifts. Primitive explorations leverage GANs (Wu et al. 2016), CLIP-guided synthesis (Sanghi et al. 2022), and Transformer-based approaches (Yan et al. 2022; Yin et al. 2025) to produce category-specific shapes. The subsequent works (Ho et al. 2020; Rombach et al. 2022; Poole et al. 2022; Lin et al. 2023) are then introduced to enable text-to-3D and image-to-3D generation by leveraging pretrained 2D diffusion priors, though these methods require expensive per-instance optimization. A more recent line of work trains diffusion models natively on 3D latent spaces, achieving substantial quality improvements. Representative methods such as TripoSG (Li et al. 2025), Hunyuan3D (Zhao et al. 2025; Lai et al. 2025), and LATTICE (Lai et al. 2026) employ flow-matching on structured latent spaces to generate high-fidelity meshes. These systems usually follow a two-stage pipeline where the flow-matching generator produces latent point embeddings conditioned on the input image, followed by a variational autoencoder that decodes into signed distance fields for mesh extraction via differentiable marching cubes. Our work builds upon this architecture, focusing on the post-training alignment strategy that aims to improve generation quality beyond what pretraining alone can achieve. Reinforcement Learning for Generative Models RL-based post-training has become an indispensable paradigm for further enhancing model capabilities (Ouyang et al. 2022), progressing from PPO (Schulman et al. 2017) and DPO (Rafailov et al. 2023) to GRPO (Shao et al. 2024). Its application has also been expanded from large language models to image generation. DDPO (Black et al. 2024) first demonstrates policy gradient optimization for diffusion models by treating each denoising step as an action in a Markov decision process. DPOK (Fan et al. 2023) introduces Kullback-Leibler (KL)-regularized optimization to prevent reward hacking, and DiffusionDPO (Clark et al. 2024) adapts preference optimization to the denoising trajectory. In the flow-matching regime, Flow-GRPO (Liu et al. 2025) leverages the SDE interpretation to define proper action distributions amenable to group-relative advantage estimation, while DiffusionNFT (Zheng et al. 2025) proposes negative-free training that eliminates the need for a reference model. Despite these advances in 2D generation, RL-based optimization for 3D remains largely unexplored, primarily due to the difficulty of designing reliable multi-dimensional quality judgements for 3D meshes during training. Figure 2: Overview of our Flow3D-OPD framework. Stage 1 (left): the pretrained flow-matching DiT is first fine-tuned with ℒFTL_FT, then three teachers are trained through DPO with dimension-specific agentic reward. Stage 2 (right): the student model generates its own denoising trajectory via on-policy sampling, and a hard task-routing cycles through the three frozen teachers with mini-batch gradient accumulation. On-Policy Distillation On-policy distillation (OPD) addresses the distribution shift of offline distillation by requiring the student to generate trajectories from its own current policy under real-time teacher supervision (Agarwal et al. 2024; Gu et al. 2024). In the LLM, GKD (Agarwal et al. 2024) first formalizes on-policy distillation by training on self-generated sequences to mitigate exposure bias. Subsequent work MiniLLM (Gu et al. 2024) adopts reverse KL to encourage the student to concentrate probability mass on high-quality modes, while DistiLLM (Ko et al. 2024) proposes skewed KL interpolation to balance mode-seeking and mode-covering behaviors. In visual generation, consistency distillation (Song et al. 2023; Luo et al. 2023) first explores distilling multi-step diffusion models into fewer-step students. DiffusionOPD (Li et al. 2026) extends OPD to diffusion models by exploiting the Markov structure of the denoising chain, deriving closed-form per-step KL divergence between Gaussian transition kernels. Flow-OPD (Fang et al. 2026) further generalizes this to flow-matching architectures with stochastic differential equation (SDE) formulations and time-adaptive weighting, showing that multi-teacher distillation resolves the reward sparsity and gradient interference of joint multi-objective RL. Our work adapts the OPD mechanism on 3D geometry generation, where the inherently heterogeneous nature of quality assessment makes multi-teacher distillation a particularly natural fit. Preliminary DPO in Diffusion Models. Direct preference optimization (DPO) (Rafailov et al. 2023) enables learning from pairwise preferences without explicit reward modeling. For language models, given winning and losing responses xw,xlx^w,x^l conditioned on prompt c, DPO directly optimizes the policy model pθp_θ via: ℒDPOLLM=−[logσ(βlogpθ(xw|c)pref(xw|c)−βlogpθ(xl|c)pref(xl|c))], _DPO^LLM=-E [ σ (β p_θ(x^w|c)p_ref(x^w|c)-β p_θ(x^l|c)p_ref(x^l|c) ) ], (1) where σ is the sigmoid function, β defines the penalty coefficient, and prefp_ref indicates the reference model. DiffusionDPO (Clark et al. 2024) extends this to diffusion models by deriving an equivalent loss over the denoising trajectory. Under the evidence lower bound (ELBO), the log-likelihood ratio decomposes into per-step noise prediction differences. The final loss for flow-matching models takes the form: ℒDPOFM(θ,xw,xl) _DPO^FM(θ;x^w,x^l) =−t[logσ(−βω(t)⋅Δ(θ,t,xw,xl))], =-E_t [ σ (-βω(t)· (θ,t,x^w,x^l) ) ], Δ(θ,t,xw,xl) (θ,t,x^w,x^l) =(‖vθ(xtw)−vw‖22−‖vref(xtw)−vw‖22) = (\|v_θ(x_t^w)-v^w\|_2^2-\|v_ref(x_t^w)-v^w\|_2^2 ) −(‖vθ(xtl)−vl‖22−‖vref(xtl)−vl‖22), - (\|v_θ(x_t^l)-v^l\|_2^2-\|v_ref(x_t^l)-v^l\|_2^2 ), (2) where xtwx_t^w and xtlx_t^l are noisy intermediates of the winner and loser at timestep t, defined via xtw,l=(1−t)xw,l+tϵx^\w,l\_t=(1-t)x^\w,l\+tε. vw=ϵ−xwv^w=ε-x^w and vl=ϵ−xlv^l=ε-x^l are the target velocities. vθv_θ and vrefv_ref are the policy and reference velocity predictions, and ω(t)ω(t) is a weighting function (Ho et al. 2020; Kingma et al. 2021). On-Policy Distillation. On-policy distillation (OPD) addresses the distribution shift inherent in offline distillation by requiring the student to generate trajectories from its own current policy under real-time teacher supervision (Agarwal et al. 2024) . For autoregressive models, this is formulated as minimizing the reverse Kullback-Leibler (KL) divergence: ℒOPD=−y∼πθ[logπteacher(y|x)πθ(y|x)]=DKL(πθ∥πteacher),L_OPD=-E_y _θ [ _teacher(y|x) _θ(y|x) ]=D_KL( _θ\| _teacher), (3) where πθ _θ denotes the student policy, πteacher _teacher is the frozen teacher, and y are trajectories sampled on-policy from the student. By training on self-generated distributions, OPD suppresses exposure bias and ensures the student receives supervision at the states it actually visits during inference. Our Method Our model follows the architecture of Hunyuan3D-2.1 (Lai et al. 2025), where the flow-matching DiT generates structured latent x∈ℝ4096×64x ^4096× 64 conditioned on the image features. We employ a two-stage post-training strategy to optimize the pretrained DiT model, as illustrated in Fig. 2. Stage 1: Domain-Specialized Teacher Training Considering the defective capability of the basic model in rolling out high-quality samples for reliable preference learning, we first fine-tune the basic model via semi-policy preference optimization, and then train our teacher models on top of this fine-tuned (FT) model in stage 1. Fine-Tuning with Semi-Policy Preference Optimization. In this phase, offline ground-truth 3D meshes serve as the preferred samples, while the current policy generates the rejected samples. Specifically, we replace xwx^w in Δ(θ,t,xw,xl) (θ,t,x^w,x^l) (Eq. 2) with the ground-truth latent xgtx^gt to obtain the semi-policy DPO loss ℒsemiL_semi. This formulation enables the model to optimize its own denoising trajectory along the stochastic differential equation (SDE) or ordinary differential equation (ODE) sampling. The overall fine-tuning loss combines ℒsemiL_semi with an mean squared error (MSE) loss function: ℒFT=λsemiℒsemi+λmset[‖vθ(xt)−vgt‖22],L_FT= _semi\,L_semi+ _mse\,E_t\! [\|v_θ(x_t)-v^gt\|_2^2 ], (4) where λsemi=0.2 _semi=0.2, λmse=0.8 _mse=0.8. Based on the supervision, we enhance the foundational capability of the basic model and obtain the FT model for subsequent teacher model training. Agentic Reward Design and Teacher Training. We design an agentic reward system ℛmR_m based on Gemini-3-Flash (Google DeepMind 2025) for pairwise mesh comparison. Given a 3D mesh pair (Ma,Mb)(M_a,M_b) and the reference image II, our agent performs multi-round structured reasoning: (sa,sb)=ℛm(Ma,Mb,I,m,,),(s_a,s_b)=R_m(M_a,M_b,I;P_m,T,C), (5) where mP_m is the dimension-specific evaluation prompt (m∈ℳ=orient,pose,consistm =\orient,pose,consist\), T denotes the tool set, including zoom-in rendering, viewpoint rotation, and difference-map overlay, which are necessary in geometric quality evaluation. And C is the case memory for judgment consistency. The agent is allowed up to four rounds of tool invocation before outputting structured preference scores (sa,sb)(s_a,s_b). Compared to global scalar metrics like ULIP/Uni3D, our agentic reward provides fine-grained and dimension-specific supervision with multi-view reasoning. Details of prompts and tools are provided in the supplementary material. Building upon the agentic reward ℛmR_m, we train each teacher vϕmv_ _m via DPO. At each step, the sampling model rolls out two candidate latents (xa,xb)(x_a,x_b), which are then decoded to meshes and compared by the agentic reward to produce preference scores (sa,sb)(s_a,s_b). Based on the scores, we filter out pairs with a small margin |sa−sb|<δ|s_a-s_b|<δ and form the winner-loser pair (xw,xl)(x^w,x^l). Following Eq. 2, each teacher is optimized via: ℒteachm=ℒDPOFM(θm,xw,xl),m∈ℳ.L_teach^m=L_DPO^FM( _m;x^w,x^l), m . (6) In practice, we observe that the majority of sampled pairs yield score differences below δ, leading to inefficient training under fully online rollout. To address this, we adopt an iterative “rollout-then-train” strategy: the model first generates a batch of candidate pairs over multiple steps and saves the valid ones filtered by δ, then trains on this collected data before the next rollout stage. Based on these training objectives and strategies, we obtain three domain-specialized teacher models, each excelling at its target quality dimension. Stage 2: Multi-Teacher On-Policy Distillation Given three trained teacher models vϕmm∈ℳ\v_ _m\_m , we then integrate them into a single student vθv_θ with on-policy distillation supervision in this stage. On-Policy Distillation in Flow-Matching. Following the OPD loss function formulated in Eq. 3, we minimize the reverse KL divergence between the student and teacher transition kernels at each step along the student’s on-policy trajectory. Under the SDE discretization (Euler-Maruyama), the one-step transition kernel is Gaussian: πθ(xtj+1|xtj)=(μθ(xtj),σtj2(−Δtj)I), -1.99997pt _θ(x_t_j+1|x_t_j)=N ( _θ(x_t_j),\; _t_j^2(- t_j)I ), (7) where 1=t0>t1>⋯tN=01=t_0>t_1>·s t_N=0 define the denoising timesteps and Δtj=tj+1−tj<0 t_j=t_j+1-t_j<0. σtj _t_j is the per-step diffusion coefficient determined solely by the noise schedule, and the transition mean μθ _θ is deterministically parameterized by the student velocity field. The teacher transition kernel πϕm _ _m uses the teacher velocity to compute its mean μϕm _ _m but shares the same covariance σ¯tj2=σtj2(−Δtj) σ_t_j^2= _t_j^2(- t_j). Since both kernels share the same isotropic covariance, their reverse KL admits a closed form: KL(πθ∥πϕm) -5.0ptKL ( _θ\| _ _m ) =KL((μθ,σ¯tj2I)∥(μϕm,σ¯tj2I)) =KL (N ( _θ,\; σ_t_j^2I )\|N ( _ _m,\; σ_t_j^2I ) ) =‖μθ(xtj)−μϕm(xtj)‖222σ¯tj2. = \| _θ(x_t_j)- _ _m(x_t_j)\|_2^22 σ_t_j^2. (8) The stochastic transition degenerates to a deterministic map under the ODE regime, and distribution matching reduces to direct velocity-field alignment with L2 loss. The reverse KL defines the per-step distillation objective for a single teacher. Hard Task-Routing and Mini-Batch Gradient Accumulation. To consolidate all three teachers while mitigating gradient interference, we introduce a hard task-routing mechanism. At each training step, the sample carries a data-type label m∈orient,pose,consistm∈\orient,pose,consist\ that deterministically routes supervision to the corresponding teacher vϕmv_ _m, so that its gradient stays free from cross-teacher conflict. The routed OPD loss aggregates per-step velocity-matching objectives along the on-policy trajectory via: ℒdistillm(θ)=1N∑j=0N−1w(tj)‖vθ(xtj)−SG(vϕm(xtj))‖22 ^m_distill(θ)= 1N _j=0^N-1w(t_j) \|v_θ(x_t_j)-SG(v_ _m(x_t_j)) \|_2^2 (9) where the stop-gradient operator SG(⋅)SG(·) detaches the teacher output. The weighting function w(tj)w(t_j) controls the distillation strength across different denoising timestep tjt_j, which equals Δtj2(σtj(1−tj)2tj+1σtj)2 t_j2 ( _t_j(1-t_j)2t_j+ 1 _t_j )^2 under SDE sampling and reduces to a constant 11 under ODE sampling. The detailed derivation is provided in the supplementary material. Method Gemini-3-Flash GPT-5.5 User Study ↑ Orient-Eval Pose-Eval Consistency-Eval Average Average A wins ↓ B wins ↑ A wins ↓ B wins ↑ A wins ↓ B wins ↑ A wins ↓ B wins ↑ A wins ↓ B wins ↑ FT (semi-policy) 5.5 11.2 0.5 85.1 12.4 79.4 5.9 54.9 15.7 45.7 – Orient-Teacher 2.7 22.7 0.8 89.0 30.8 53.2 10.3 53.0 13.4 47.3 4.14 Pose-Teacher 10.1 14.3 0.0 93.7 41.7 41.7 16.0 47.9 15.1 43.6 4.06 Consistency-Teacher 6.5 15.9 1.0 83.5 8.0 82.0 5.2 56.9 11.8 50.9 3.40 DPO 13.4 7.2 19.1 71.8 13.0 77.0 15.1 48.3 20.4 41.5 3.56 FT + DPO 5.9 14.4 0.9 91.8 14.0 76.0 6.6 57.2 14.1 44.7 4.08 OPD 10.9 10.9 0.0 93.4 15.0 79.0 8.6 57.3 11.6 48.3 4.36 FT + OPD (Ours) 5.1 18.8 0.0 93.5 10.0 81.0 4.9 61.0 9.9 51.4 4.40 Table 1: Pairwise evaluation across three quality dimensions, with cross-evaluator and human validation. Model A is the pretrained baseline (Hunyuan3D-2.1); Model B is the evaluated model. A/B wins (%) denote the percentage of cases each is judged superior. The left columns use Gemini-3-Flash; GPT-5.5 Avg reports averaged results under the independent GPT-5.5 evaluator; User Study reports the average human ranking score. Best results are in bold, second best are underlined. Method ULIP-T ULIP-I Uni3D-T Uni3D-I Craftsman 1.5 0.074 0.129 0.233 0.299 Michelangelo 0.075 0.115 0.206 0.261 Direct3D-s2 0.074 0.122 0.247 0.314 Hi3DGen 0.076 0.131 0.237 0.299 Trellis2 0.080 0.129 0.241 0.307 HY3D 2.0 0.077 0.130 0.245 0.315 HY3D 2.1 0.081 0.133 0.246 0.314 Flow3D-OPD (Ours) 0.083 0.138 0.252 0.321 Table 2: Comparison with previous works on 3D–text (T) and 3D–image (I) similarity calculated by ULIP/Uni3D models. The best results are in bold. While hard task-routing keeps each mini-batch conflict-free, a single update from one dimension alone is still biased toward that teacher. We therefore organize training data such that every three consecutive mini-batches cycle through all data types in round-robin order: orient → pose → consistency. Gradient accumulation across this full cycle before each optimizer step ensures every parameter update incorporates guidance from all three teachers simultaneously: ℒdistilltotal=1|ℳ|∑m∈ℳℒdistillm(θ).L^total_distill= 1|M| _m L_distill^m(θ). (10) In this way, hard task-routing isolates each teacher’s supervision at the mini-batch level, while gradient accumulation aggregates them into a balanced consensus at the update level, substantially mitigating the gradient interference of multi-objective optimization. Experiments Experimental Setup Training Settings. We build upon Hunyuan3D-2.1 (Lai et al. 2025), a 3B-parameter DiT architecture for 3D latent representation generation. Three domain-specialized teachers are refined from the fine-tuned model via on-policy DPO with learning rate 1×10−61× 10^-6, SDE sampling with noise level a=0.3a=0.3, and a minimum confidence margin of δ=0.167δ=0.167 across 2×82× 8 H20 GPUs. During the second stage, the student is also initialized from the fine-tuned model and distilled by randomly sampling states along the 50-step denoising trajectory. To handle the imbalanced sizes of the three sub-datasets, we employ dataset sampling with replacement across data types. At inference, we apply classifier-free guidance with a default scale of 3.03.0. Evaluation Metrics. We adopt two complementary evaluation protocols. For multi-dimensional quality assessment, we employ pairwise judgement to report the win rates across orientation, pose, and consistency dimensions. For overall generation quality evaluation, we report ULIP (Xue et al. 2023) and Uni3D (Zhou et al. 2024) scores to measure 3D–text and 3D–image cosine similarities in LATTICE-Bench (Lai et al. 2026). We also provide qualitative visualizations of generated 3D meshes for intuitive comparison. Main Results Multi-Dimensional Quality Evaluation Table 1 demonstrates the effectiveness of our method across multiple quality dimensions. We compare the meshes generated by each model (denoted as model B) against those from the pretrained baseline Hunyuan3D-2.1 (Lai et al. 2025) (denoted as model A) with our agentic verifier for pairwise evaluation. “A wins” indicates the ratio that the baseline produces higher quality, while “B wins” indicates that the evaluated model is superior. Compared with the fine-tuned (FT) model, each teacher achieves strong performance on its target dimension but degrades on others. For instance, the orient teacher achieves the lowest A wins (2.7%) on orientation evaluation but causes consistency to drop significantly (30.8% A wins). This phenomenon confirms that the three evaluated dimensions are inherently heterogeneous, and agentic-based DPO can yield substantial improvements on specific target objectives. We further present our final Flow3D-OPD result and compare with different training strategies at the bottom of the table. Directly applying DPO on the initial model without fine-tuning leads to degraded performance due to insufficient basic capability. Applying OPD without the semi-policy stage yields moderate improvements across all dimensions but falls short of the full pipeline. In contrast, our Flow3D-OPD achieves the best overall balance: 18.8%, 93.5%, and 81% B wins ratio on orient, pose, and consistency, respectively. Finally, our method consistently outperforms all teachers on the averaged metrics with 61.0% B wins. To validate the generalization and effectiveness of our model, we additionally conduct two complementary assessments on the right of Table 1. First, we employ GPT-5.5 as an independent evaluator, which is a different model from the Gemini-3-Flash used in the teacher training reward. Our Flow3D-OPD model also achieves the lowest A wins (9.9%) and highest B wins (51.4%) under GPT-5.5, confirming that the observed improvements are not artifacts of evaluator-specific optimization. Second, we conduct a blind user study where human annotators rank all methods by their generation quality from best to worst without knowing model identities. Our method also achieves the highest average ranking score, further validating that the improvements reflect genuine 3D quality gains perceived by human evaluators. Update Strategy A-wins ↓ B-wins↑ Uni3D-I Single-step (sequential) 7.1 59.8 0.319 Single-step (hard task-routing) 6.6 58.5 0.320 Multi-step (hard task-routing) 4.9 61.0 0.321 Table 3: Ablation on different gradient update strategies during OPD. Best results are in bold. Stage Strategy A-wins ↓ B-wins↑ Uni3D-T Uni3D-I FT ODE 6.4 51.7 0.245 0.314 SDE 5.9 54.9 0.248 0.315 OPD ODE 4.9 61.0 0.252 0.321 SDE 7.3 54.8 0.251 0.319 Table 4: Ablation on SDE and ODE sampling in both FT and OPD stages (NFE=50). Best results are in bold. Figure 3: Distillation loss curves under different gradient update strategies. Gradient accumulation in mini-batch exhibits smoother convergence and lower final loss compared to single-step updates. Comparison with Previous Methods Table 2 compares our method with previous 3D generation approaches on previous 3D–text and 3D–image similarity benchmarks (Lai et al. 2026), including Craftsman 1.5 (Li et al. 2024), Michelangelo (Zhao et al. 2023), Trellis2/512 (Xiang et al. 2026), Direct3D-S2 (Wu et al. 2026), Hi3DGen (Ye et al. 2025), and Hunyuan3D 2.0 (Zhao et al. 2025). We evaluate the global similarity with ULIP (Xue et al. 2023) and Uni3D (Zhou et al. 2024) metrics. Results demonstrate that our Flow3D-OPD achieves consistent gains over the baseline Hunyuan3D-2.1 (Lai et al. 2025) across all four metrics and outperforms all compared previous methods. Figure 4: Reward margin and distill loss curves in SDE/ODE Sampling. SDE achieves higher reward margins during FT stage, indicating more discriminative preference pairs. ODE converges faster and reaches a lower final loss than SDE during the OPD stage. Ablation Studies Effect of Gradient Update Strategy We compare three gradient update strategies during the OPD stage in Table 3. The first is single-step backward and uses sequential sampling, where all samples of one dimension are processed before moving to the next. The second is also a single-step backward strategy but with hard task-routing sampling. The third is our gradient accumulation strategy with hard task-routing sampling. The results suggest that our configuration achieves the best performance across all metrics, while hard task-routing sampling is better than sequential sampling. Moreover, we compare the distillation loss curves of single-step and multi-step updates under the same hard task-routing sampling strategy, as shown in Fig. 3. Our gradient accumulation strategy produces more stable distillation loss curves than the single-step backward variant, and also converges to a lower final loss across both total and per-task objectives. This confirms that accumulating multi-task gradients in a mini-batch before each update effectively prevents inter-task interference and achieves better performance. Figure 5: Qualitative comparison on 3D meshes generated with various approaches or training strategies. Our model produces 3D meshes with accurate and consistent structure. Figure 6: Visualization comparison between our model and individual teachers on orientation (top), pose (middle), and consistency (bottom) validation dimensions. Effect of SDE vs. ODE Sampling We compare the effectiveness of SDE and ODE sampling mechanisms across the FT and OPD stages in Table 4. In the FT stage, SDE sampling outperforms ODE across both global and specialized metrics. We attribute this to the additional stochasticity that generates more diverse on-policy losers and produces higher reward margins between winner and loser pairs, as demonstrated by the dashed curves in Fig. 4. Therefore, we initialize the student model with FT model with SDE sampling. However, in the OPD stage, the deterministic ODE formulation proves more effective than SDE. As illustrated in the bottom of Table 4 and Fig. 4, ODE achieves better results with faster loss convergence and lower distillation loss. This is because the student model can align more precisely with the teacher’s velocity field at each step without the interference introduced by the stochastic diffusion term. This asymmetric finding suggests that exploration diversity benefits preference learning during model fine-tuning, while deterministic trajectory is more suitable for distillation in the OPD stage. Qualitative Visualization Fig. 8 presents qualitative comparisons across various approaches or training strategies: Hi3DGen, Trellis2, Hunyuan3D-2,1, OPD-only (distillation without the FT stage), and our full pipeline Flow3D-OPD model. Previous models exhibit limited capability in generating fine-grained details, such as the geometric structures of buildings and the propellers of aircraft models. Our baseline model, Hunyuan3D-2.1, produces geometrically plausible but misaligned or structurally inconsistent meshes. OPD-only model yields moderate improvements yet lacks the stable foundation for consistent high-quality generation. Our full model produces meshes with coherent and precise multi-view structure, demonstrating the complementary benefits of basic capability scaling and multi-teacher on-policy distillation. Fig. 6 further compares our distilled student model against the three individual teacher models on their respective areas of expertise. In the first row of orientation validation, the pose teacher and consistency teacher fail to align the mesh to the canonical front-facing direction, which further introduces asymmetric artifacts in the head region, while our student maintains correct orientation. In the second row of pose validation, the orient teacher and consistency teacher produce visible deformations at the elbows of the people, whereas our student preserves faithful articulated pose though it is unseen in the reference image. In the third row, our student exhibits richer detail in the surface texture compared to the non-specialized teachers. These results confirm that the student model unifies the strengths of all three teachers while avoiding their individual weaknesses in other dimensions. Conclusion We present Flow3D-OPD, a two-stage post-training framework that first introduces multi-teacher on-policy distillation into 3D geometry generation. In the first stage, we design an agentic reward system for fine-grained geometric evaluation and train domain-specialized teachers via DPO across three quality dimensions. In the second stage, multiple teacher distillation with hard task-routing and gradient accumulation enables the student to absorb heterogeneous expertise through velocity-field supervision along its own denoising trajectory. Extensive experiments demonstrate that Flow3D-OPD achieves consistent improvements across all geometric dimensions and surpasses teachers on the averaged metric. References Agarwal et al. (2024) R. Agarwal, N. Vieillard, Y. Zhou, P. Stanczyk, S. Ramos Garea, M. Geist, and O. Bachem On-policy distillation of language models: learning from self-generated mistakes. In International Conference on Learning Representations, Vol. 2024, p. 21246–21263. Cited by: On-Policy Distillation, On-Policy Distillation.. Black et al. (2024) K. Black, M. Janner, Y. Du, I. Kostrikov, and S. Levine Training diffusion models with reinforcement learning. In International Conference on Learning Representations, Vol. 2024, p. 4965–4987. Cited by: Introduction, Reinforcement Learning for Generative Models. Clark et al. (2024) K. Clark, P. Vicol, K. Swersky, and D. Fleet Directly fine-tuning diffusion models on differentiable rewards. In International Conference on Learning Representations, Vol. 2024, p. 4793–4822. Cited by: Reinforcement Learning for Generative Models, DPO in Diffusion Models.. Fan et al. (2023) Y. Fan, O. Watkins, Y. Du, H. Liu, M. Ryu, C. Boutilier, P. Abbeel, M. Ghavamzadeh, K. Lee, and K. Lee Dpok: reinforcement learning for fine-tuning text-to-image diffusion models. Advances in Neural Information Processing Systems 36, p. 79858–79885. Cited by: Reinforcement Learning for Generative Models. Fang et al. (2026) Z. Fang, W. Huang, Y. Zeng, Y. Zhao, S. Chen, K. Feng, Y. Lin, L. Chen, Z. Chen, S. Cao, et al. Flow-opd: on-policy distillation for flow matching models. arXiv preprint arXiv:2605.08063. Cited by: Introduction, On-Policy Distillation. Ghosh et al. (2023) D. Ghosh, H. Hajishirzi, and L. Schmidt Geneval: an object-focused framework for evaluating text-to-image alignment. Advances in Neural Information Processing Systems 36, p. 52132–52152. Cited by: Introduction. Google DeepMind (2025) Google DeepMind Gemini 3 flash. Google AI Blog. Cited by: Agentic Reward Design and Teacher Training.. Gu et al. (2024) Y. Gu, L. Dong, F. Wei, and M. Huang Minillm: knowledge distillation of large language models. In International Conference on Learning Representations, Vol. 2024, p. 32694–32717. Cited by: On-Policy Distillation. Ho et al. (2020) J. Ho, A. Jain, and P. Abbeel Denoising diffusion probabilistic models. Advances in neural information processing systems 33, p. 6840–6851. Cited by: 3D Generation Models, DPO in Diffusion Models.. Huang et al. (2023) K. Huang, K. Sun, E. Xie, Z. Li, and X. Liu T2i-compbench: a comprehensive benchmark for open-world compositional text-to-image generation. Advances in Neural Information Processing Systems 36, p. 78723–78747. Cited by: Introduction. Kingma et al. (2021) D. Kingma, T. Salimans, B. Poole, and J. Ho Variational diffusion models. Advances in neural information processing systems 34, p. 21696–21707. Cited by: DPO in Diffusion Models.. Ko et al. (2024) J. Ko, S. Kim, T. Chen, and S. Yun Distillm: towards streamlined distillation for large language models. arXiv preprint arXiv:2402.03898. Cited by: On-Policy Distillation. Lai et al. (2025) Z. Lai, Y. Zhao, H. Liu, Z. Zhao, Q. Lin, H. Shi, X. Yang, M. Yang, S. Yang, Y. Feng, et al. Hunyuan3d 2.5: towards high-fidelity 3d assets generation with ultimate details. arXiv preprint arXiv:2506.16504. Cited by: Introduction, 3D Generation Models, Our Method, Training Settings., Multi-Dimensional Quality Evaluation, Comparison with Previous Methods. Lai et al. (2026) Z. Lai, Y. Zhao, Z. Zhao, H. Liu, Q. Lin, J. Huang, C. Guo, and X. Yue Lattice: democratize high-fidelity 3d generation at scale. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 19982–19992. Cited by: Introduction, Introduction, 3D Generation Models, Evaluation Metrics., Comparison with Previous Methods. Li et al. (2026) Q. Li, J. Yu, K. Jiang, Y. Wei, Z. Xing, P. Li, R. Chu, S. Zhang, Y. Liu, and Z. Wu DiffusionOPD: a unified perspective of on-policy distillation in diffusion models. arXiv preprint arXiv:2605.15055. Cited by: Introduction, On-Policy Distillation. Li et al. (2024) W. Li, J. Liu, H. Yan, R. Chen, Y. Liang, X. Chen, P. Tan, and X. Long Craftsman3d: high-fidelity mesh generation with 3d native generation and interactive geometry refiner. arXiv preprint arXiv:2405.14979. Cited by: Comparison with Previous Methods. Li et al. (2025) Y. Li, Z. Zou, Z. Liu, D. Wang, Y. Liang, Z. Yu, X. Liu, Y. Guo, D. Liang, W. Ouyang, et al. Triposg: high-fidelity 3d shape synthesis using large-scale rectified flow models. IEEE Transactions on Pattern Analysis and Machine Intelligence. Cited by: Introduction, Introduction, 3D Generation Models. Lin et al. (2023) C. Lin, J. Gao, L. Tang, T. Takikawa, X. Zeng, X. Huang, K. Kreis, S. Fidler, M. Liu, and T. Lin Magic3d: high-resolution text-to-3d content creation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 300–309. Cited by: 3D Generation Models. Liu et al. (2025) J. Liu, G. Liu, J. Liang, Y. Li, J. Liu, X. Wang, P. Wan, D. Zhang, and W. Ouyang Flow-grpo: training flow matching models via online rl. arXiv preprint arXiv:2505.05470. Cited by: Introduction, Reinforcement Learning for Generative Models. Luo et al. (2023) S. Luo, Y. Tan, L. Huang, J. Li, and H. Zhao Latent consistency models: synthesizing high-resolution images with few-step inference. arXiv preprint arXiv:2310.04378. Cited by: On-Policy Distillation. Mildenhall et al. (2021) B. Mildenhall, P. P. Srinivasan, M. Tancik, J. T. Barron, R. Ramamoorthi, and R. Ng Nerf: representing scenes as neural radiance fields for view synthesis. Communications of the ACM 65 (1), p. 99–106. Cited by: Introduction. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems 35, p. 27730–27744. Cited by: Introduction, Reinforcement Learning for Generative Models. Peebles and Xie (2023) W. Peebles and S. Xie Scalable diffusion models with transformers. In Proceedings of the IEEE/CVF international conference on computer vision, p. 4172–4182. Cited by: Introduction. Poole et al. (2022) B. Poole, A. Jain, J. T. Barron, and B. Mildenhall Dreamfusion: text-to-3d using 2d diffusion. arXiv preprint arXiv:2209.14988. Cited by: Introduction, 3D Generation Models. Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn Direct preference optimization: your language model is secretly a reward model. Advances in neural information processing systems 36, p. 53728–53741. Cited by: Introduction, Reinforcement Learning for Generative Models, DPO in Diffusion Models.. Rombach et al. (2022) R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer High-resolution image synthesis with latent diffusion models. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 10684–10695. Cited by: 3D Generation Models. Sanghi et al. (2022) A. Sanghi, H. Chu, J. G. Lambourne, Y. Wang, C. Cheng, M. Fumero, and K. R. Malekshan Clip-forge: towards zero-shot text-to-shape generation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 18582–18592. Cited by: 3D Generation Models. Schulman et al. (2017) J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: Reinforcement Learning for Generative Models. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. Li, Y. Wu, et al. Deepseekmath: pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300. Cited by: Reinforcement Learning for Generative Models. Song et al. (2023) Y. Song, P. Dhariwal, M. Chen, and I. Sutskever Consistency models. arXiv preprint arXiv:2303.01469. Cited by: On-Policy Distillation. Wu et al. (2016) J. Wu, C. Zhang, T. Xue, B. Freeman, and J. Tenenbaum Learning a probabilistic latent space of object shapes via 3d generative-adversarial modeling. Advances in neural information processing systems 29. Cited by: 3D Generation Models. Wu et al. (2026) S. Wu, Y. Lin, F. Zhang, Y. Zeng, Y. Yang, J. Qian, S. Zhu, X. Cao, P. Torr, Y. Yao, et al. Direct3d-s2: gigascale 3d generation made easy with spatial sparse attention. Advances in Neural Information Processing Systems 38, p. 170778–170804. Cited by: Comparison with Previous Methods. Xiang et al. (2026) J. Xiang, X. Chen, S. Xu, R. Wang, Z. Lv, Y. Deng, H. Zhu, Y. Dong, H. Zhao, N. J. Yuan, et al. Native and compact structured latents for 3d generation. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, p. 14419–14429. Cited by: Comparison with Previous Methods. Xue et al. (2023) L. Xue, M. Gao, C. Xing, R. Martín-Martín, J. Wu, C. Xiong, R. Xu, J. C. Niebles, and S. Savarese Ulip: learning a unified representation of language, images, and point clouds for 3d understanding. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 1179–1189. Cited by: Introduction, Evaluation Metrics., Comparison with Previous Methods. Yan et al. (2022) X. Yan, L. Lin, N. J. Mitra, D. Lischinski, D. Cohen-Or, and H. Huang Shapeformer: transformer-based shape completion via sparse representation. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, p. 6229–6239. Cited by: 3D Generation Models. Ye et al. (2025) C. Ye, Y. Wu, Z. Lu, J. Chang, X. Guo, J. Zhou, H. Zhao, and X. Han Hi3dgen: high-fidelity 3d geometry generation from images via normal bridging. In Proceedings of the IEEE/CVF International Conference on Computer Vision, p. 01–12. Cited by: Comparison with Previous Methods. Yin et al. (2025) F. Yin, X. Chen, C. Zhang, B. Jiang, Z. Zhao, W. Liu, G. Yu, and T. Chen Shapegpt: 3d shape generation with a unified multi-modal language model. IEEE Transactions on Multimedia 27, p. 4107–4120. Cited by: 3D Generation Models. Zhao et al. (2025) Z. Zhao, Z. Lai, Q. Lin, Y. Zhao, H. Liu, S. Yang, Y. Feng, M. Yang, S. Zhang, X. Yang, et al. Hunyuan3d 2.0: scaling diffusion models for high resolution textured 3d assets generation. arXiv preprint arXiv:2501.12202. Cited by: Introduction, 3D Generation Models, Comparison with Previous Methods. Zhao et al. (2023) Z. Zhao, W. Liu, X. Chen, X. Zeng, R. Wang, P. Cheng, B. Fu, T. Chen, G. Yu, and S. Gao Michelangelo: conditional 3d shape generation based on shape-image-text aligned latent representation. Advances in neural information processing systems 36, p. 73969–73982. Cited by: Comparison with Previous Methods. Zheng et al. (2025) K. Zheng, H. Chen, H. Ye, H. Wang, Q. Zhang, K. Jiang, H. Su, S. Ermon, J. Zhu, and M. Liu Diffusionnft: online diffusion reinforcement with forward process. arXiv preprint arXiv:2509.16117. Cited by: Reinforcement Learning for Generative Models. Zhou et al. (2024) J. Zhou, J. Wang, B. Ma, Y. Liu, T. Huang, and X. Wang Uni3d: exploring unified 3d representation at scale. In International Conference on Learning Representations, Vol. 2024, p. 46766–46782. Cited by: Introduction, Evaluation Metrics., Comparison with Previous Methods. Supplementary Material Hyperparameter Stage 1a: Semi-Policy FT Stage 1b: On-Policy DPO Stage 2: Multi-Teacher OPD Base / Initialization Hunyuan3D-2.1 (3B DiT) Fine-tuned model (1a) Fine-tuned model (1a) DPO β 1000 1000 – λdpo _dpo / λdistill _distill 0.2 1.0 1.0 DPO type Semi-policy On-policy (fully online) – Rollout sampler SDE (a=0.3) SDE (a=0.3) ODE (deterministic) Rollout NFE 50 50 50 Candidates per step 1 (loser only) 2 (pairs) 1 Confidence margin δ – 0.167 – Run mode End-to-end Iterative rollout/train(400/100) End-to-end Table 5: Hyperparameter configurations across all training stages. Dimension Train Samples Eval Samples Orientation 3,661 133 Pose 4,083 111 Consistency 1,063 100 Table 6: Dataset statistics across the three quality dimensions. Details of the Training Pipeline Hyperparameter Configuration Table 5 summarizes the hyperparameter configurations for all training stages. All stages are trained with 5K steps on 2×82× 8 H20(90G) GPUS. The pipeline begins with semi-policy fine-tuning (stage 1a) on the offline ground-truth winners and on-policy generated losers. The resulting fine-tuned model serves as the initialization for both the domain-specialized teachers (stage 1b) and the final student (stage 2). Each teacher is trained via on-policy DPO with Gemini-3-Flash pairwise reward using an iterative rollout-then-train strategy to handle the low valid-pair ratio. In stage 2, the three frozen teachers are distilled into the student through on-policy trajectory sampling with hard task-routing and gradient accumulation across the three data types. Dataset Statistics Table 6 reports the dataset sizes and the description for each quality dimension. Specifically, in the training stage, we curate three separate preference datasets, one per dimension: an orientation set targeting canonical axis alignment, a pose set covering diverse articulated human configurations, and a consistency set emphasizing multi-view structural and textural coherence. In the evaluation stage, to isolate the model’s generation ability along each aspect, we construct three dedicated test sets that mirror the three training dimensions. Figure 7: The detailed pipeline of our agentic verifier. Agentic Verifier Details In this section, we introduce the details of the agentic verifier used to construct preference pairs during DPO training, depicted in Fig. 7. The verifier is built on the Gemini-3-Flash proprietary model and follows a structured, tool-augmented reasoning process to compare two candidate meshes generated from the same reference image. Below we describe its system prompt, tool call, and case-memory mechanism. System Prompt The reward agent receives a structured system prompt that defines its role, input context, dimension-specific instructions, and metrics. The key components are shown as follows. Role and Input. The agent acts as a strict 3D geometry review agent that compares two candidate meshes (model A vs. model B) generated from the same reference image. It receives: (1) the reference image, (2) multi-view rendered images of two meshes. Dimension-Specific Instructions. Based on the shared system prompt, we design a dedicated instruction for each of the three judge dimensions. In the orientation dimension, the agent enforces canonical axis alignment and treats perspective baking as a fatal defect. In the human-pose dimension, the agent prioritizes correct limb counts and connectivity, then joint angles and overall posture relative to the reference. In the consistency dimension, the agent checks that structures and texture inferred from one viewpoint remain plausible and coherent with the reference image. The three prompts are summarized below. Orientation Prompt You are a review agent focused on canonical orientation. The generated 3D model must be aligned to the standard world axes, while baking in the diagonal tilt, skewed perspective, or dynamic camera angle of the 2D reference is strictly prohibited. For example, an object with a dominant axis (e.g., a trumpet, a standing animal) must have that axis horizontally or vertically aligned in the orthographic views, not slanted. A bilaterally symmetric object (e.g., a forward-facing human) must appear macroscopically symmetric in the front view. Any retained diagonal slant or perspective baking is a fatal defect and must be scored ‘Bad’. Pose Prompt You are a review agent focused on human pose and body structure. Evaluate, in descending priority: Macro limb structure -- exactly two arms and two legs, each a single continuous limb, with no forking, and with limbs/head smoothly connected to the torso (no floating or severed parts). Pose accuracy -- limb angles, bend directions, and global posture must match the reference; penalize mirrored poses, collapsed poses, and residual T-pose. Overall shape and accessories -- clothing, hats, and attachments should be consistent with the reference. Consistency Prompt You are a senior 3D geometry review agent specialized in geometric consistency. Your task is to evaluate how faithfully two 3D models reproduce the geometric or textural structures visible in the reference image (Image-to-3D consistency). You evaluate whether the 3D model accurately reconstructs what is shown in the reference image — silhouette match, structural element count, proportions, and geometric details. You do NOT evaluate orientation alignment or artistic style — only consistency with the reference. Tool Interface Specifications The agent has access to three tools during evaluation: render_extra_views renders additional viewpoints of both models for closer inspection. • Parameters: – angles: List of viewpoints (1–6 items), each specifying yaw (0=Front, 90=Left, 180=Back, 270=Right), pitch (0=eye-level, ± 30=tilted, ± 90=top/bottom), label (descriptive name), and optional render_normal (boolean). – reason: Text explanation of why these views are needed. • Returns: The rendered images at the specified viewpoints, plus corresponding difference maps. render_zoom_views renders zoomed close-up views for fine detail inspection. • Parameters: – zoom_configs: List of zoom viewpoints (1–4 items), each specifying yaw, pitch, label, and optional fov_deg controlling zoom level (∼ 50∘=standard, ∼ 30∘=moderate zoom, ∼ 15∘=strong zoom, ∼ 8∘=extreme close-up). – reason: Text explanation of what detail to inspect. • Returns: Zoomed rendered images at the specified viewpoint and zoom level. different_overlap outputs a heatmap that facilitates the agent to evaluate the difference. • Parameters: – image_index: The index of the rendered image pairs that the agent needs to compare. • Returns: Difference heatmap between the rendered images of the two meshes from the same viewpoint Case Memory Mechanism The agent maintains a case memory of previously judged hard examples to improve judgment accuracy. Before evaluating a new pair, the system retrieves similar past cases (based on object category) and includes them as few-shot examples in the context. This mechanism addresses the variance inherent in VLM-based evaluation by anchoring judgments to established precedents. Derivation of OPD in Flow-Matching We provide the complete derivation of reverse-KL on-policy distillation (OPD) objective in Eq. 9. Let πθ _θ and πϕm _ _m denote the student and teacher policies. The denoising trajectory of the student policy is defined as x0:N=(xt0,…,xtN)x_0:N=(x_t_0,…,x_t_N), where 1=t0>t1>⋯>tN=01=t_0>t_1>·s>t_N=0. Considering the denoising process is Markovian, we decompose the whole OPD objective into stepwise KL along the student rollout: ℒOPD(θ)=x0:N∼πθ[∑j=0N−1KL(πθ(⋅∣xtj)∥πϕm(⋅∣xtj))], _OPD(θ)=E_x_0:N _θ\! [ _j=0^N-1KL\! ( _θ(· x_t_j)\, \|\, _ _m(· x_t_j) ) ], (11) where πθ(⋅∣xtj) _θ(· x_t_j) and πϕm(⋅∣xtj) _ _m(· x_t_j) are the student and teacher one-step transition kernels at the same state xtjx_t_j. For flow-matching denoising with SDE sampling, we discretize the reverse time by Euler–Maruyama with step Δtj:=tj+1−tj<0 t_j:=t_j+1-t_j<0. Let σt=at/(1−t) _t=a t/(1-t) be the SDE diffusion coefficient with global noise level a, and define vθ(xtj)v_θ(x_t_j) and vϕm(xtj)v_ _m(x_t_j) for the student and teacher velocity fields in tjt_j. The student SDE step is: xtj+1=xtj+[vθ+σtj22tj(xtj+(1−tj)vθ)]Δtj+σtj−Δtjεj=(1+σtj22tjΔtj)xtj+(1+σtj2(1−tj)2tj)vθΔtj+σtj−Δtjεj aligned &x_t_j+1=x_t_j+ [\,v_θ+ _t_j^22t_j (x_t_j+(1-t_j)\,v_θ ) ] t_j+ _t_j - t_j\, _j\\ &= (1+ _t_j^22t_j t_j )x_t_j+ (1+ _t_j^2(1-t_j)2t_j )v_θ\, t_j+ _t_j - t_j\, _j aligned (12) where εj∼(0,Id) _j (0,I_d), and vθ:=vθ(xtj)v_θ:=v_θ(x_t_j). Collecting the deterministic drift as the transition mean and the noise term as the covariance, the one-step kernel is the isotropic Gaussian: πθ(⋅∣xtj)=(μθ(xtj),σ¯tj2Id),σ¯tj2:=σtj2(−Δtj), _θ(· x_t_j)=N\! ( _θ(x_t_j),\; σ_t_j^2I_d ), σ_t_j^2:= _t_j^2(- t_j), (13) where the mean is defined as: μθ(xtj)=(1+σtj22tjΔtj)xtj+(1+σtj2(1−tj)2tj)vθΔtj. _θ(x_t_j)= (1+ _t_j^22t_j t_j )x_t_j+ (1+ _t_j^2(1-t_j)2t_j )v_θ\, t_j. (14) The transition kernel in the teacher policy πϕm _ _m is defined in a similar formulation: πϕm(⋅∣xtj)=(μϕm(xtj),σ¯tj2Id),μϕm(xtj)=(1+CLOSEOPENσtj22tjΔtj)xtj+(1+σtj2(1−tj)2tj)vϕmΔtj. aligned _ _m(· x_t_j)&=N\! ( _ _m(x_t_j),\; σ_t_j^2I_d ),\\ _ _m(x_t_j)= (1+& _t_j^22t_j t_j )x_t_j+ (1+ _t_j^2(1-t_j)2t_j )v_ _m\, t_j. aligned (15) Because the covariance σ¯tj2Id σ_t_j^2I_d is identical, πθ _θ and πϕm _ _m differ only in their means. For two d-dimensional Gaussians with common isotropic covariance ∑Σ: KL((μ1,∑)∥(μ2,∑))=12(μ1−μ2)T∑−1(μ1−μ2). \! (N( _1, Σ)\,\|\,N( _2,Σ) )= 12( _1- _2)^T Σ^-1( _1- _2). (16) Applying (16) to the kernels (13) and (15) gives the per-step KL as: KL(πθ(⋅∣xtj)∥πϕm(⋅∣xtj))=‖μθ(xtj)−μϕm(xtj)‖222σ¯tj2. \! ( _θ(· x_t_j)\,\|\, _ _m(· x_t_j) )= \| _θ(x_t_j)- _ _m(x_t_j)\|_2^22\, σ_t_j^2. (17) Subtracting μϕm(xtj) _ _m(x_t_j) from μθ(xtj) _θ(x_t_j), the state-dependent terms cancel, and only the velocity-dependent term remains: μθ _θ (xtj)−μϕm(xtj)= (x_t_j)- _ _m(x_t_j)= (1+σtj2(1−tj)2tj)Δtj(vθ(xtj)−vϕm(xtj)). (1+ _t_j^2(1-t_j)2t_j ) t_j\, (v_θ(x_t_j)-v_ _m(x_t_j) ). (18) Substituting (18) into (17), the per-step KL becomes a weighted squared velocity difference: KL(πθ∥πϕm)=w(tj)∥vθ(xtj)−vϕm(xtj)∥22,w(tj)=(−Δtj)2σtj2(1+σtj2(1−tj)2tj)2=Δtj2(σtj(1−tj)2tj+1σtj)2. aligned &KL\! ( _θ\,\|\, _ _m )=w(t_j) \|v_θ(x_t_j)-v_ _m(x_t_j) \|_2^2,\\ &w(t_j)= (- t_j)2\, _t_j^2 (1+ _t_j^2(1-t_j)2t_j )^2= t_j2 ( _t_j(1-t_j)2t_j+ 1 _t_j )^2. aligned (19) Based on Eq. 19, we apply the stop-gradient operator SG(⋅)SG(·) to the frozen teacher. The final OPD objective is the time-weighted velocity-matching loss ℒdistillm(θ)L^m_distill(θ), defined as: x0:N∼pS,θ[∑j=0N−1w(tj)∥vθ(xtj)−SG(vϕm(xtj))∥22]. _x_0:N p_S,θ\! [ _j=0^N-1w(t_j)\, \|v_θ(x_t_j)-SG (v_ _m(x_t_j) ) \|_2^2 ]. (20) Deterministic (ODE) regime. Setting the noise level a→0a→ 0 makes the transition deterministic: for a given xtjx_t_j, the student and teacher each induce a single transition target. Distribution matching then reduces to pointwise matching of the predicted velocities, and the reverse-KL objective is replaced by a direct squared L2L_2 loss, which is formulated as: x0:N∼pS,θ[1N∑j=0N−1∥vθ(xtj)−SG(vϕm(xtj))∥22], _x_0:N p_S,θ\! [ 1N _j=0^N-1 \|v_θ(x_t_j)-SG (v_ _m(x_t_j) ) \|_2^2 ], (21) which is the ODE instantiation used in our OPD stage. This closed-form, zero-variance objective is the flow-matching analogue of the per-step KL distillation used for autoregressive models, replacing high-variance policy gradients with direct backpropagation through the student velocity field. Additional Experiments Impact of the CFG Weights We further study the effect of the CFG weight w in both the training and the inference stages, as reported in Table 7 and Table 8. During the OPD stage, we find that using the velocity field predicted under the default conditional signal (i.e., w=1.0w=1.0, without classifier-free amplification) as the distillation target yields the best student, indicating that the raw conditional guidance already provides a sufficiently informative supervision signal. At inference time, the model performs best under the default CFG configuration used in Hunyuan3D-2.1 (w=3.0w=3.0), which balances conditional fidelity and generation quality. CFG Scale w A-wins/B-wins Orient-Eval Pose-Eval Consist-Eval w=1.0w=1.0 5.1/18.8 0.0/93.5 10.0/81.0 w=3.0w=3.0 7.1/17.6 1.0/91.9 15.7/77.5 w=6.0w=6.0 7.1/17.9 0.0/88.7 14.0/75.3 Table 7: Per-dimension ablation on the CFG scale w used for the teacher during the OPD stage. Best results are in bold. CFG Scale w A-wins ↓ B-wins ULIP-I Uni3D-I w=1.0w=1.0 32.3 38.5 0.133 0.314 w=3.0w=3.0 4.9 61.0 0.138 0.321 w=6.0w=6.0 5.2 60.6 0.137 0.320 Table 8: Ablation on the CFG scale w during inference after OPD. Best results are in bold. Figure 8: The quality visualization of the refinement in our Flow3D-OPD More Quality Visualization In this section, we provide additional quality visualizations, as shown in Fig. 8. These cases are based on meshes generated by our Flow3D-OPD and further post-processed with LATTICE for high-resolution refinement.