Paper deep dive
DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation
Zhengming Yu, Junkun Yuan, Haotian Yang, Gordon Guocheng Qian, Yizhi Wang, Angtian Wang, Yiding Yang, Bo Liu, Xin Li, Wenping Wang, Chongyang Ma
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 88%
Last extracted: 10/4/2026, 2:35:56 AM
Summary
The paper introduces DMAD (Distribution Matching as Adversarial Distillation), a method for fast visual generation that replaces auxiliary score estimation in Distribution Matching Distillation (DMD) with a classification-based approach. DMAD uses two discriminator heads on a shared backbone to distinguish real data and teacher samples from student samples, training the student via linear losses on logits. It also introduces gap-based reweighting to adapt teacher supervision across noise levels. DMAD achieves state-of-the-art results on ImageNet, COCO, and video generation benchmarks (Wan2.1, MiniMax-H3) with fewer steps.
Entities (11)
Relation Signals (7)
DMAD → evaluatedon → ImageNet-64x64
confidence 95% · DMAD reaches a Fréchet Inception Distance (FID) of 1.04 with one-step generation on ImageNet-64x64
DMAD → evaluatedon → CoCo-10K
confidence 95% · 14.47 with four-step SDXL on COCO-10K
DMAD → evaluatedon → Wan2.1-T2V-14B
confidence 95% · VBench total score of 85.15 with four-step Wan2.1-T2V-14B
DMAD → evaluatedon → MiniMax-H3-33B
confidence 95% · On MiniMax-H3-33B, our four-step student achieves overall human preference rates of 79.1% over DMD2
DMAD → improvesupon → DMD
confidence 95% · DMAD recasts distribution matching as classification... eliminating auxiliary score fitting... which DMD requires.
DMAD → uses → Gap-based reweighting
confidence 95% · We further introduce gap-based reweighting, which adapts teacher supervision across noise levels
DMAD → achievesbetterperformancethan → DMD2
confidence 90% · DMAD reaches a FID of 1.04... best values among the compared few-step methods... 79.1% preference over DMD2
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Distribution Matching Distillation (DMD) trains a few-step student from the difference between separately estimated target and student scores, so it must keep an auxiliary diffusion model fitted to the student's evolving distribution at extra memory and computation cost. We introduce DMAD, Distribution Matching as Adversarial Distillation, which recasts distribution matching as classification and learns the required log-density ratios directly. Two discriminator heads on a shared backbone distinguish real data and teacher samples from the student's, and linear losses on their logits train the student without auxiliary score fitting. We prove that at the discriminator optimum these losses recover the distribution-matching gradient underlying DMD, through the classical identity linking discriminator logits to log-density ratios. We further introduce gap-based reweighting, which adapts teacher supervision across noise levels from the real-data head's empirical logit gap between real and teacher samples. DMAD reaches a Fréchet Inception Distance (FID) of 1.04 with one-step generation on ImageNet-64x64, 14.47 with four-step SDXL on COCO-10K, and a VBench total score of 85.15 with four-step Wan2.1-T2V-14B, the best values among the compared few-step methods and the multi-step teachers. On MiniMax-H3-33B, our four-step student achieves overall human preference rates of 79.1% over DMD2 and 84.6% over rCM for joint audio-video generation, excluding ties. Our code, models and demos are available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2610.02188v1
- Canonical: https://arxiv.org/abs/2610.02188v1
Trouble viewing inline? Open PDF directly →
Full Text
67,286 characters extracted from source content.
Expand or collapse full text
DMAD: Distribution Matching as Adversarial Distillation for Fast Visual Generation Zhengming Yu 1,2 , Junkun Yuan 2 , Haotian Yang 2 , Gordon Guocheng Qian 2 , Yizhi Wang 2 , Angtian Wang 2 , Yiding Yang 2 , Bo Liu 2 , Xin Li 1 , Wenping Wang 1 , Chongyang Ma 2 1 Texas A&M University, 2 Intelligent Creation, ByteDance Abstract Distribution Matching Distillation (DMD) trains a few-step student from the difference between separately estimated target and student scores, so it must keep an auxiliary diffusion model fitted to the student’s evolving distribution at extra memory and computation cost. We introduce DMAD, Distribution Matching as Adversarial Distillation, which recasts distribution matching as classification and learns the required log-density ratios directly. Two discriminator heads on a shared backbone distinguish real data and teacher samples from the student’s, and linear losses on their logits train the student without auxiliary score fitting. We prove that at the discriminator optimum these losses recover the distribution-matching gradient underlying DMD, through the classical identity linking discriminator logits to log-density ratios. We further introduce gap-based reweighting, which adapts teacher supervision across noise levels from the real-data head’s empirical logit gap between real and teacher samples. DMAD reaches a Fréchet Inception Distance (FID) of 1.04 with one-step generation on ImageNet-64×64, 14.47 with four-step SDXL on COCO-10K, and a VBench total score of 85.15 with four-step Wan2.1-T2V-14B, the best values among the compared few-step methods and the multi-step teachers. On MiniMax-H3-33B, our four-step student achieves overall human preference rates of 79.1% over DMD2 and 84.6% over rCM for joint audio-video generation, excluding ties. Our code, models and demos are available at https://yzmblog.github.io/projects/DMAD. 1 Introduction Diffusion models generate high-quality images and videos [11,29,38,40], but sampling requires many sequential denoising steps. Step distillation reduces this cost by training a few-step generator from a pretrained teacher. Existing approaches supervise the student along the teacher’s sampling trajectories [16,31,39] or match distributions [47,48]. Distribution matching lets the student learn its own mapping from noise to samples while preserving the target distribution. Distribution Matching Distillation (DMD) [48] trains the student on the difference between target and student scores. A frozen teacher supplies the target score, while an auxiliary diffusion model, the critic, tracks the evolving student distribution. When the critic lags behind, its score estimate carries error and training destabilizes. DMD2 [47] mitigates this with a two time-scale update rule (TTUR) that runs five critic updates 1 arXiv:2610.02188v1 [cs.CV] 1 Oct 2026 Figure 1 Samples of our four-step student distilled from MiniMax-H3. Zoom in for details. per student update. Stability improves, but the student score is still only approximated, and each student update costs five critic passes. Beyond the critic, DMD2 evaluates the teacher online during training, adding further memory and computation cost. Its denoising task also constrains the architecture of the discriminator. We introduce DMAD, Distribution Matching as Adversarial Distillation, which replaces separate score estimation with two classification tasks, one against teacher samples and one against real data. Two discriminator heads distinguish these targets from student samples, and linear losses on their logits drive 2 the updates. At the discriminator optimum, this loss yields the distribution-matching gradient underlying DMD (Proposition 1). DMAD eliminates auxiliary score fitting. In our experiments, we use one discriminator update per student update. Freed from the denoising task, the discriminator can build on diffusion encoders or pretrained classification features. The teacher now only supplies samples, so we generate them once and cache them, removing online teacher evaluation from training. Errors in separately estimated scores can be large relative to their difference, which drives the student update. We examine this in a controlled experiment on two nearby Gaussian mixtures with known gradients, holding the student distribution fixed and giving both estimators the same teacher pretraining and seed. Figure 2 compares DMD’s score difference with our single-head logit gradient. In this setting, our estimate follows the ground-truth field more closely, with normalized mean squared error of 0.144, compared with 1.539 for DMD. Section B.2 gives the setup and the analytic gradients. DMAD draws on two targets, the teacher’s samples and real data, which play complementary roles. Previous methods, including DMD2 [47], SiDA [52], and SenseFlow [6], add adversarial supervision on top of a score- based objective. In contrast, our discriminator heads carry the distribution-matching gradient themselves. Each head compares student samples with one target, and a linear loss on their logits trains the generator. The real-data head supplies supervision beyond the teacher’s sampled distribution. Teacher samples alone would bound the student by the teacher’s own fidelity; training against real data lifts that bound, and our students outperform their teachers on both ImageNet and COCO. Our ablations in Table 4 demonstrate the complementary roles of the two targets: removing the real-data head raises four-step FID from 14.47 to 21.37 on COCO-10K, while removing teacher supervision reduces semantic fidelity and visual quality, dropping VBench total score from 84.70 to 82.20 and semantic score from 79.55 to 75.36. Target and Student Ground-truth Gradient DMD GradientOurs Gradient NMSE = 1.539NMSE = 0.144 TargetStudent Figure 2 Gradient estimation on two Gaussian mixtures with a fixed student. Our logit gradi- ent more closely matches the ground-truth field than DMD’s score difference. The balance between the two targets can vary across noise levels. During training, the real-data head assigns different mean logits to real and teacher samples (Fig- ure 7(a)). This gap varies across noise levels and depends on the current student and discriminator. We use it as an empirical signal forgap-basedreweighting, assigning larger teacher weights to noise bands with lower gaps. This adaptive allocation improves performance for both image and video generation (Table 4). Our contributions are: • We introduce DMAD, a distribution-matching dis- tillation framework that jointly learns from teacher- generated samples and real data through two binary classification tasks, eliminating auxiliary score fitting and online teacher evaluation. •We develop gap-based reweighting, which uses the real-data head’s mean responses to teacher and real samples within each noise band to adapt the teacher’s supervision as training proceeds. •DMAD achieves 1.04 FID with one-step generation on ImageNet-64×64 and 14.47 with four-step SDXL on COCO-10K, where one step reaches 16.11. Four- step Wan2.1-T2V-1.3B and -14B reach VBench total scores of 84.70 and 85.15. These are the best values among the compared few-step methods and the multi-step teachers. On the 33B MiniMax-H3 [28], human raters prefer our four-step student to DMD2 in 79.1% and to rCM [51] in 84.6%, excluding ties. 3 2 Related Work Trajectory and consistency methods. Early work distills a teacher either by regressing to its outputs [24,27] or by progressively reducing the number of sampling steps [31]. PDD [36] extends this idea to video using multi-interval mean velocities. Consistency and flow-map methods impose consistency along a common trajectory, either by mapping different states to the same endpoint or by relating states at different points on that trajectory [16,37,39]. sCM [22] extends consistency modeling to continuous time, while MeanFlow [7] enables one-step generation from scratch. These methods all constrain how the student follows a particular trajectory. Distribution matching instead constrains the distribution of its outputs. Score-based distribution matching. Diff-Instruct [26], VSD [41], and DMD [48] optimize a generator through score-based distribution matching. DMD uses the difference between a target score and a score fitted to the student’s evolving distribution. DMD2 [47] removes the regression loss and strengthens online critic training, while SenseFlow [6] improves stability for large flow-based models. SiD [53] and SGMD [43] derive alternative objectives from score identities and Fisher-based matching. DP-DMD [42] and PhasedDMD [5] distribute supervision across generation steps or noise intervals. rCM [51] combines score regularization with consistency training, AnyFlow [9] learns flow-map transitions for any-step video generation, and AMD [1] adapts distribution matching using reward proxies. These methods still rely on score estimation. FSF- DMD [17] removes the auxiliary fake-score network but derives a fake-score surrogate from the generator’s own flow map. Our method instead estimates no score and learns the required log-density ratio directly through classification. Adversarial distillation. Adversarial distillation typically trains discriminators against real images or videos [19,20,33], or uses intermediate teacher predictions as adversarial targets [18,34]. D2O-F [49] finetunes a pretrained diffusion model with a projected discriminator. DMD2, SiDA [52], SenseFlow, and ADM [23] add adversarial supervision to a score-based distribution-matching objective. DDO [50] connects likelihood ratios to an implicit discriminator, whilef-distill [44] uses density ratios to reweight score dif- ferences. Unlike these methods, we do not use adversarial training to supplement a score-based objective. The discriminator logits themselves provide the distribution-matching gradient, eliminating student-score estimation. Separate heads match teacher samples and real data, with their balance adapted across noise levels. 3 Method We reformulate DMD’s score-estimation pipeline as joint classification-based matching to teacher samples and real data. Linear losses on two discriminator logits train the few-step student. The same real-data head also scores teacher samples, and the gap it leaves between them and real data adapts teacher supervision across noise levels. Figure 3 illustrates the pipeline for video distillation. Image distillation is the same, with only the discriminator heads adapted to the backbone (Section B). 3.1 Reformulating Distribution Matching Distillation We start from the idealized distribution-matching update underlying DMD [48]. LetG θ be a student with parametersθand the final rollout outputx 0 =G θ (z,c) for initial noisezand conditionc, inducing a distributionp θ . At noise levelt, we formx t =α t x 0 +σ t ε, whereε∼N(0,I) is independent Gaussian noise. The forward schedule specifies signal scale α t and noise standard deviation σ t . Let p t be the noisy marginal of a clean distributionpands p =∇ x t logp t its score. All densities are conditional onc, omitted below. As in DMD, the analysis assumes the same fixed distribution overt,cfor target and student samples, and all expectations below use it. For a fixed target distribution p, the update above is motivated by the reverse-KL objective L DM (θ;p) =E t,c [D KL (p θ,t ∥p t )].(1) 4 teacher video student video real video latents noisy latents forward noising noise z few-step student G θ (⋅,c) F φ (⋅,t,c) teacher head h T T + , G − real head h R R + , G − D = T BCE + R BCE linear generator loss −λ T ω i h T (G) −λ R h R (G) h R ( T ) , h R (R) gap reweighting a b m gap Δ b sg(ω i ) sg(⋅) stop gradient generator gradient backpropagation discriminator gradient backpropagation Figure 3 Overview of DMAD (drawn for video distillation).T,R, andGdenote noisy teacher, real, and student samples, with time and text conditions omitted. A shared backboneF φ feeds a teacher head and a real head, and a linear loss on each logit trains the generator. The real-data head also provides the gap for teacher reweighting. Video frames are decoded only for illustration. Its generator gradient depends on the score difference, with s θ =∇ x t logp θ,t the student score: ∇ θ L DM =−E z,ε,t,c " ∂x t ∂θ T s p (x t ,t,c)−s θ (x t ,t,c) # .(2) DMD estimates these two scores separately using a frozen teacher and a critic trained on student samples. DMAD instead recovers the score difference directly from a discriminator logit. To estimate the required score difference directly, we train a single binary classifier to tell target samples from student samples at each noise level. Leth φ be the scalar logit of a discriminatorDwith parameters φ, andD φ =S(h φ ) its target-class probability, whereS(u) = (1 +e −u ) −1 is the logistic sigmoid function. We minimize the balanced binary cross-entropy (BCE)B(h φ ;p), which weights target and student samples equally, so its optimum is the log-density ratio alone (Section A). Proposition 1 (Classification-based distribution-matching update). At fixedt,c, assume positive differentiable densities on common support and the regularity needed to exchange differentiation and integration. The unrestricted, unregularized BCE optimum is h ⋆ = logp t − logp θ,t , so ∇ x t h ⋆ = s p −s θ .(3) At the current parameters, the generator gradients of −E[h ⋆ ] and L DM are equal with h ⋆ fixed. The proof follows from the optimal discriminator identity [8] and the chain rule, and is given in Section A. We therefore update the generator with the linear loss L lin G (θ;φ) =−E z,ε,t,c [h φ (x t ,t,c)].(4) Withφfixed, gradients through the discriminator’s input train the generator without auxiliary score fitting, and the learned discriminator approximates the ideal logit gradient (Figure 2). Unlike the linear loss, the non-saturating loss− logD φ multiplies each sample’s logit-gradient contribution by 1− D φ . We analyze this weighting in Section A and compare the losses in Section 4.3. One classifier therefore replaces both score estimates, and the linear loss carries its gradient unweighted. 3.2 Distillation with Teacher and Real Targets To provide supervision beyond the teacher’s sampled distribution, we train on both teacher-generated samples and real data, with distributionsp T andp R . A shared backboneF φ feeds two heads with scalar logitsh T and h R , each trained to distinguish its own target samples from student samples. The combined discriminator objective is the sum of the two balanced classification losses: L D =L T BCE +L R BCE ,(5) 5 whereL T BCE =B(h T ;p T ) andL R BCE =B(h R ;p R ). With fixed nonnegative coefficientsλ T andλ R setting the relative strength of the two targets, the generator loss is the linear combination: L G =−E [λ T h T (x t ,t,c) + λ R h R (x t ,t,c)].(6) At their optima, each head supplies its own target’s score difference (Equation (3)), so both targets contribute distribution-matching gradients within one objective. Our ablations in Table 4 show that real supervision improves image FID, while teacher supervision preserves detail and text alignment. 3.3 Gap-Based Reweighting In Equation (6),λ T weights the teacher equally at every noise level. The real-data head’s mean-logit gap between real and teacher samples varies across noise levels and over training (Figure 7(a)). It depends on the current student and discriminator. We use this empirical signal to assign larger teacher weights to noise bands with lower gaps. To measure the gap at a usable resolution, we group noise levels into bands and writeb(t) for the band containing t. For each band b, we maintain an exponential moving average (EMA): ∆ b ← EMA β h h R (x R t ,t,c) b −h R (x T t ,t,c) b i .(7) The bars average the same real head’s logits over noisy real samplesx R t and teacher samplesx T t within a band. We use EMA decay β = 0.99 and update only bands observed in both sources. The gap then sets a raw teacher weight a b , centered at the median gap over the measured bands R: m = median b∈R ∆ b , a b = S m− ∆ b τ .(8) A band with a lower mean-logit gap receives a larger weight. The temperatureτ >0 sets how sharply the weight varies around the median, and centering at the median gap keeps the weights relative as the gaps drift over the course of training. We normalize these weights to control their overall scale. For sampleiin a minibatch of sizeB, letω i =a b(t i ) /C. For ImageNet and SDXL,C=max(B −1 P j a b(t j ) ,B −1 ); for Wan,C=max(|R| −1 P b∈R a b ,10 −3 ). MiniMax- H3 uses the same band-based normalization with a floor of 10 −4 . Collecting these terms, the final generator loss is b L G =− 1 B B X i=1 [λ T ω i h T (x t i ,i ,t i ,c i ) + λ R h R (x t i ,i ,t i ,c i )].(9) wherex t i ,i is the noisy student output for samplei. We hold the gap statistics and weights fixed during generator backpropagation, so each ω i scales its term without adding a gradient. 4 Experiments 4.1 Experimental Setup We evaluate DMAD on class-conditional image generation, text-to-image synthesis, and text-to-video synthesis. Following DMD2 [47], we distill EDM [14] for ImageNet-64×64 [4] and SDXL [29] for zero-shot generation on 10K COCO 2014 prompts [21]. We use Fréchet Inception Distance (FID) [10] to measure image quality and diversity, and CLIP score [30] to evaluate text–image alignment. For SDXL, we additionally report patch FID [3,18] to assess high-resolution details, following DMD2. For video, we distill Wan2.1-T2V 1.3B and 14B [40] to four steps and evaluate them with VBench [12] on the same augmented prompt set as rCM [51]. We further distill MiniMax-H3 [28] to four steps and evaluate them in a user study and AVGen-Bench [54]. Training and evaluation details are provided in Section B. More results are shown in our supplementary demo video. 6 Teacher DMD2 sCM rCM Ours Teacher DMD2 sCM rCM Ours ... a woman riding a bus in the rain, cool blue tones, sad mood ...... a single red rose bud ... slowly and gracefully unfurls ... ... a tiger ... a small boy sits on its back ...... model in beige-toned clothing ... harsh sunlight ... shot slowly upward... Figure 4 Qualitative comparisons of our DMAD and baseline models on Wan2.1-T2V-1.3B. Rows show the teacher, DMD2, sCM, rCM, and ours. Within each example, columns show frames 1, 28, 55, and 81 in order. All compared videos are generated using identical initial noise and text prompts. Prompts are abbreviated for presentation. 4.2 Main Results Class-conditional image generation. Table 1 compares DMAD with recent baselines. Our one-step model achieves an FID of 1.24, outperforming DMD2, including its longer-training variant, and SiDA trained directly from the teacher. It also outperforms the EDM teacher with both ODE and SDE samplers. With a projected discriminator following D2O-F [49], DMAD achieves an FID of1.04, surpassing D2O-F and SiD 2 A, which is initialized from a SiD-distilled generator. The advantage comes from our formulation: the discriminator itself provides the distillation gradient, so discriminative representations directly guide generator training. DMD2, in contrast, requires a critic that must be refitted continually to the student’s evolving distribution, five times per student update. Text-to-image synthesis. We follow DMD2 [47] for both training data and evaluation. Our generators are trained by distilling SDXL [29] using 500K real images from LAION-Aesthetics [35] and corresponding teacher-generated samples, and evaluate on zero-shot COCO 2014 [21]. Our four-step generator achieves an FID of 14.47 and a patch FID of 19.88 (Table 2), outperforming DMD2, SenseFlow [6], and the SDXL teacher on both metrics. With a single forward pass, DMAD achieves an FID of 16.11 and a CLIP score of 0.338, the best one-step FID and highest CLIP score among the compared models. Qualitative comparisons appear in Figure 8. Text-to-video synthesis. We distill Wan2.1-T2V 1.3B, 14B [40] and MiniMax-H3 [28] into four-step generators using real videos from UltraVideo [45] and corresponding videos generated by the teacher. For VBench [12] 7 ... a crocodile swimming in water while a bird flies above ... ... a dolphin paints on a submerged canvas ... a shark glides alongside ...... a golfer ... a precise, smooth stroke ... the ball drops cleanly into the hole ... ... a woman practicing yoga in a peaceful park ... Teacher DMD2 rCM Ours Teacher DMD2 rCM Ours Figure 5 Qualitative comparisons of our DMAD and baseline models on MiniMax-H3. Rows show the teacher, DMD2, rCM, and ours, in that order. Within each example, columns show frames 1, 42, 83, and 124 using the same indices for all methods. All compared videos are generated using identical initial noise and text prompts. Prompts are abbreviated for presentation. Table 1 Comparisons with the EDM [14] on ImageNet- 64×64. Method # Fwd Pass (↓) FID (↓) BigGAN-deep [2]1 4.06 RIN [13]1000 1.23 StyleGAN-XL [32]1 1.52 Consistency Model [39]1 6.20 iCT-deep [37]1 3.25 CTM [16]1 1.92 DMD [48]1 2.62 D2O-F [49]1 1.16 DMD2 [47]1 1.51 DMD2+longer training [47]1 1.28 SiD [53]1 1.52 SiDA [52]1 1.35 SiD 2 A [52]1 1.11 DMAD (Ours)1 1.24 + projected discriminator (Ours)1 1.04 EDM (Teacher, ODE) [14]511 2.32 EDM (Teacher, SDE) [14]511 1.36 Table 2 Comparisons with the SDXL [29] on 10K prompts from COCO 2014. Method # Fwd Pass (↓) FID (↓) Patch FID (↓) CLIP (↑) LCM-SDXL [25] 1 81.62 154.40 0.275 4 22.16 33.92 0.317 SDXL-Turbo [33] 1 24.57 23.94 0.337 4 23.19 23.27 0.334 SDXL Lightning [18] 1 23.92 31.65 0.316 4 24.46 24.56 0.323 DMD2 [47] 1 19.01 26.98 0.336 4 19.32 20.86 0.332 SenseFlow [6]4 22.83 26.05 0.335 DMAD (Ours) 1 16.11 32.42 0.338 4 14.47 19.88 0.328 SDXL Teacher, cfg=6100 19.36 21.38 0.332 SDXL Teacher, cfg=8100 20.39 23.21 0.335 evaluation, we generate videos using the same augmented prompt set as rCM [51]. Our 1.3B model achieves a VBench total score of 84.70, outperforming DMD2, sCM [22], rCM, and the 50-step teacher (Table 3). At 14B, our model reaches 85.15, also surpassing rCM and the teacher. Both models achieve the highest quality scores among the compared methods. Additional samples from our four-step Wan2.1-T2V-14B student appear in Figure 9. Qualitative comparisons in Figure 4 show more natural colors and better preservation of prompted details, including petal texture and tiger anatomy. Our models achieve the highest VBench total and quality scores among the compared methods, while rCM scores higher on semantics. 8 020406080100 Preference Rate (%) Overall Preference Visual Quality Audio Quality Motion Quality Prompt Alignment DMAD vs DMD2 79.1% 20.9% 84.2% 15.8% 87.9% 12.1% 60.3% 39.7% 54.3% 45.7% 020406080100 Preference Rate (%) DMAD vs rCM 84.6% 15.4% 91.3% 8.7% 86.0% 14.0% 73.8% 26.2% 61.1% 38.9% DMAD winsBaseline wins Figure 6 User study comparing DMAD with DMD2 and rCM on MiniMax-H3. All models use 4 sampling steps. Preference rates use prompt-level majority votes, excluding ties. Dashed lines indicate 50%. Table 3 VBench results for Wan2.1 T2V (480p). † Retested with Diffusers and augmented prompts. ModelResolutionNFE Total (↑)Quality (↑) Semantic (↑) Wan2.1 T2V - 1.3B Wan2.1 T2V Teacher [40] † 832× 480× 81 50× 283.0283.9579.26 DMD2 [47]832× 480× 81484.5685.5880.50 sCM [22]832× 480× 81482.0684.0074.30 rCM [51]832× 480× 81484.4385.3880.63 DMAD (Ours)832× 480× 814 84.7085.9879.55 Wan2.1 T2V - 14B Wan2.1 T2V Teacher [40] † 832× 480× 81 50× 283.5984.2680.92 rCM [51]832× 480× 81484.9285.4382.88 DMAD (Ours)832× 480× 81485.1585.9681.90 For MiniMax-H3, Figure 5 shows improved visual quality and preservation of prompted elements, such as the bird in the crocodile scene. A user study on VideoGenEval [46] prompts evaluates five dimensions (Figure 6). DMAD receives overall preference rates of 79.1% against DMD2 and 84.6% against rCM, excluding ties, with the strongest preferences in visual and audio quality. Additional quantitative audio-video results are provided in Section C. 4.3 Ablation Studies We study four ablations in Table 4: removing the teacher head, removing the real head, disabling gap reweighting with fixed loss weights, and replacing the linear generator loss with the non-saturating GAN loss. Removing a head also removes its discriminator and generator losses. For removing the real head, we also disable gap-based reweighting, which depends on the real-data head. Removing the real head causes the largest FID increase on ImageNet and SDXL in Table 4. The teacher-only curve in Figure 7(b) also stays above the full model and rises again after about 400K iterations. Removing the teacher instead slows the decrease in ImageNet FID. Although it gives the lowest SDXL FID, it also gives the worst patch FID and CLIP, and reduces the VBench semantic score by 4.19 points. Our combined adversarial objective improves ImageNet convergence and SDXL detail and text alignment over either target alone. Replacing the linear generator loss with the non-saturating loss slows the reduction in ImageNet FID. As shown in Figure 7(b), our model first records FID below 1.5 at 209K iterations, compared with 354K for the non-saturating variant. The linear loss also gives better image FID and VBench total score in Table 4. 9 Table 4 Ablation experiments of our DMAD. One-step image generation on ImageNet, four-step image generation on COCO-10K (SDXL), and four-step video generation on VBench (Wan2.1-1.3B). ImageNetCOCO-10K (SDXL)VBench (Wan2.1-1.3B) SettingFID (↓) FID (↓) Patch FID (↓) CLIP (↑)Total (↑)Quality (↑) Semantic (↑) w/o Teacher1.3013.8526.030.31282.2083.9175.36 w/o Real1.6721.3723.700.32584.3885.5779.59 w/o Gap-reweight1.33 16.6220.570.32783.5684.5379.69 w/o Linear loss1.3114.7821.040.32684.1285.1979.86 DMAD (Ours)1.24 14.4719.880.32884.7085.9879.55 0100200300400500 Training iterations (K) 0 1 2 3 4 Gap EMA Δ b (a) Teacher-real gap 10100200300400500600 Training iterations (K) 1.0 1.25 1.5 2.0 2.5 3.0 FID (b) ImageNet ablations Noise level σ 0.003-0.016 0.017-0.084 0.085-0.316 0.319-0.959 0.969-2.504 2.527 Method w/o Teacher w/o Real w/o Gap reweighting w/o Linear loss DMAD (Ours) Figure 7 ImageNet training dynamics. (a) EMA of the real-head gap between real and teacher samples, the signal that drives gap-based reweighting (Section 3.3). (b) FID of DMAD and four ablations without smoothing on a 1K-iteration grid. The no-real curve ends at 504K iterations. Our linear objective uses the discriminator’s logit gradient directly, without the additional sample weighting introduced by the non-saturating loss. Disabling gap reweighting worsens ImageNet FID and all SDXL metrics in Table 4, and lowers VBench total by 1.14 points. Adapting the teacher weight across noise levels lowers FID at matched iterations in Figure 7(b), while the real-data coefficient stays fixed. 5 Conclusion We presented DMAD, a reformulation of distribution matching distillation that jointly learns from teacher- generated samples and real data without auxiliary score fitting. Starting from DMD’s score-difference update, DMAD trains the student through linear losses on two discriminator logits, which at the discriminator optimum recover the same distribution-matching gradient. Training requires no online teacher evaluation and uses one discriminator update per student update. We further use the real-data head’s gap between real and teacher samples to adapt teacher supervision across noise levels. Experiments show improvements over the compared few-step methods and multi-step teachers in image FID and VBench total score. On MiniMax-H3, users prefer our four-step student over DMD2 and rCM, particularly in visual and audio quality. These results support discriminative learning as an effective approach to distribution matching distillation. 10 AI use statement We used large language models to assist with polishing the writing, retrieving and discovering related literature, and drafting portions of the manuscript. The authors take responsibility for the final content of this work, including its claims, citations, and any text produced with the aid of these tools. Ethics statement This work develops efficient distillation methods for image and video generation. While faster generation can benefit creative applications and research, it can also lower the cost of producing misleading or harmful synthetic media. Distilled models may inherit biases and inappropriate generation behaviors from their teachers and training data. We encourage responsible use of these methods, with appropriate safeguards against impersonation, misinformation, and non-consensual content. Reproducibility statement Section 3 describes our method, and Appendix A provides the theoretical derivation. The experimental setup and implementation details, including model architectures, training hyperparameters, and evaluation protocols, are provided in Section 4 and Appendix B. We will publicly release our training and evaluation code, dataset preparation and preprocessing scripts, and experiment configurations to facilitate independent reproduction and further research. References [1]Lichen Bai, Zikai Zhou, Shitong Shao, Wenliang Zhong, Shuo Yang, Shuo Chen, Bojun Chen, and Zeke Xie. Optimizing few-step generation with adaptive matching distillation.arXivpreprintarXiv:2602.07345, 2026. [2]Andrew Brock, Jeff Donahue, and Karen Simonyan. Large scale gan training for high fidelity natural image synthesis. InICLR, 2019. [3]Lucy Chai, Michael Gharbi, Eli Shechtman, Phillip Isola, and Richard Zhang. Any-resolution training for high-resolution image synthesis. InECCV, 2022. [4]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. Imagenet: A large-scale hierarchical image database. InCVPR, 2009. [5]Xiangyu Fan, Zesong Qiu, Zhuguanyu Wu, Fanzhou Wang, Zhiqian Lin, Tianxiang Ren, Dahua Lin, Ruihao Gong, and Lei Yang. Phased dmd: Few-step distribution matching distillation via score matching within subintervals. InProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecognition, pages 41667–41676, 2026. [6]Xingtong Ge, Xin Zhang, Tongda Xu, Yi Zhang, Xinjie Zhang, Yan Wang, and Jun Zhang. Senseflow: Scaling distribution matching for flow-based text-to-image distillation.arXivpreprintarXiv:2506.00523, 2025. [7] Zhengyang Geng, Mingyang Deng, Xingjian Bai, J Zico Kolter, and Kaiming He. Mean flows for one-step generative modeling.arXivpreprintarXiv:2505.13447, 2025. [8]Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron C. Courville, and Yoshua Bengio. Generative adversarial nets. InAdvancesinNeuralInformationProcessing Systems, volume 27, pages 2672–2680, 2014. [9]Yuchao Gu, Guian Fang, Yuxin Jiang, Weijia Mao, Song Han, Han Cai, and Mike Zheng Shou. Anyflow: Any-step video diffusion model with on-policy flow map distillation.arXivpreprintarXiv:2605.13724, 2026. [10] Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. InNeurIPS, 2017. [11]Jonathan Ho, Ajay Jain, and Pieter Abbeel. Denoising diffusion probabilistic models.Advancesinneural informationprocessingsystems, 33:6840–6851, 2020. [12]Ziqi Huang, Yinan He, Jiashuo Yu, Fan Zhang, Chenyang Si, Yuming Jiang, Yuanhan Zhang, Tianxing Wu, Qingyang Jin, Nattapol Chanpaisit, et al. Vbench: Comprehensive benchmark suite for video generative models. 11 InProceedingsoftheIEEE/CVFConferenceonComputerVisionandPatternRecognition, pages 21807–21818, 2024. [13] Allan Jabri, David Fleet, and Ting Chen. Scalable adaptive computation for iterative generation. InICML, 2023. [14]Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine. Elucidating the design space of diffusion-based generative models. InNeurIPS, 2022. [15] Tero Karras, Miika Aittala, Jaakko Lehtinen, Janne Hellsten, Timo Aila, and Samuli Laine. Analyzing and improving the training dynamics of diffusion models. InProceedingsoftheIEEE/CVFConferenceonComputer VisionandPatternRecognition, pages 24174–24184, 2024. [16]Dongjun Kim, Chieh-Hsin Lai, Wei-Hsiang Liao, Naoki Murata, Yuhta Takida, Toshimitsu Uesaka, Yutong He, Yuki Mitsufuji, and Stefano Ermon. Consistency trajectory models: Learning probability flow ode trajectory of diffusion. InICLR, 2024. [17]Youngjoong Kim, Deokyeong Lee, and Jaesik Park. Distribution matching distillation without fake score network. arXivpreprintarXiv:2605.19256, 2026. [18] Shanchuan Lin, Anran Wang, and Xiao Yang. Sdxl-lightning: Progressive adversarial diffusion distillation.arXiv preprintarXiv:2402.13929, 2024. [19]Shanchuan Lin, Xin Xia, Yuxi Ren, Ceyuan Yang, Xuefeng Xiao, and Lu Jiang. Diffusion adversarial post-training for one-step video generation.arXivpreprintarXiv:2501.08316, 2025. [20]Shanchuan Lin, Ceyuan Yang, Hao He, Jianwen Jiang, Yuxi Ren, Xin Xia, Yang Zhao, Xuefeng Xiao, and Lu Jiang. Autoregressive adversarial post-training for real-time interactive video generation.arXivpreprint arXiv:2506.09350, 2025. [21]Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C Lawrence Zitnick. Microsoft coco: Common objects in context. InECCV, 2014. [22]Cheng Lu and Yang Song. Simplifying, stabilizing and scaling continuous-time consistency models. InInternational ConferenceonLearningRepresentations, volume 2025, pages 50611–50649, 2025. [23]Yanzuo Lu, Yuxi Ren, Xin Xia, Shanchuan Lin, Xing Wang, Xuefeng Xiao, Andy J Ma, Xiaohua Xie, and Jian-Huang Lai. Adversarial distribution matching for diffusion distillation towards efficient image and video synthesis. In2025IEEE/CVFInternationalConferenceonComputerVision(ICCV), pages 16818–16829. IEEE, 2025. [24]Eric Luhman and Troy Luhman. Knowledge distillation in iterative generative models for improved sampling speed.arXivpreprintarXiv:2101.02388, 2021. [25]Simian Luo, Yiqin Tan, Suraj Patil, Daniel Gu, Patrick von Platen, Apolinário Passos, Longbo Huang, Jian Li, and Hang Zhao. Lcm-lora: A universal stable-diffusion acceleration module.CoRR, abs/2311.05556, 2023. doi: 10.48550/ARXIV.2311.05556. URL https://doi.org/10.48550/arXiv.2311.05556. [26]Weijian Luo, Tianyang Hu, Shifeng Zhang, Jiacheng Sun, Zhenguo Li, and Zhihua Zhang. Diff-instruct: A universal approach for transferring knowledge from pre-trained diffusion models. InNeurIPS, 2023. [27]Chenlin Meng, Robin Rombach, Ruiqi Gao, Diederik Kingma, Stefano Ermon, Jonathan Ho, and Tim Salimans. On distillation of guided diffusion models. InProceedingsoftheIEEE/CVFconferenceoncomputervisionand patternrecognition, pages 14297–14306, 2023. [28]MiniMax. Minimax h3: An open model breaking the boundaries between tasks and modalities, July 2026. URL https://w.minimax.io/blog/minimax-h3. [29]Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. Sdxl: Improving latent diffusion models for high-resolution image synthesis.arXivpreprint arXiv:2307.01952, 2023. [30]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al. Learning transferable visual models from natural language supervision. InICML, 2021. [31] Tim Salimans and Jonathan Ho. Progressive distillation for fast sampling of diffusion models. InICLR, 2022. 12 [32]Axel Sauer, Katja Schwarz, and Andreas Geiger. Stylegan-xl: Scaling stylegan to large diverse datasets. In SIGGRAPH, 2022. [33] Axel Sauer, Dominik Lorenz, Andreas Blattmann, and Robin Rombach. Adversarial diffusion distillation.arXiv preprintarXiv:2311.17042, 2023. [34]Axel Sauer, Frederic Boesel, Tim Dockhorn, Andreas Blattmann, Patrick Esser, and Robin Rombach. Fast high-resolution image synthesis with latent adversarial diffusion distillation. InSIGGRAPHAsia2024Conference Papers, pages 1–11, 2024. [35] Christoph Schuhmann, Romain Beaumont, Richard Vencu, Cade Gordon, Ross Wightman, Mehdi Cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Mitchell Wortsman, et al. Laion-5b: An open large-scale dataset for training next generation image-text models. InNeurIPS, 2022. [36]Neta Shaul, Chao Liu, Arash Vahdat, and Julius Berner. Parallel decoding distillation for fast image and video generation.arXivpreprintarXiv:2607.26004, 2026. [37] Yang Song and Prafulla Dhariwal. Improved techniques for training consistency models. InICLR, 2024. [38] Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. Score-based generative modeling through stochastic differential equations.arXivpreprintarXiv:2011.13456, 2020. [39] Yang Song, Prafulla Dhariwal, Mark Chen, and Ilya Sutskever. Consistency models. InICML, 2023. [40]Team Wan, Ang Wang, Baole Ai, Bin Wen, Chaojie Mao, Chen-Wei Xie, Di Chen, Feiwu Yu, Haiming Zhao, Jianx- iao Yang, et al. Wan: Open and advanced large-scale video generative models.arXivpreprintarXiv:2503.20314, 2025. [41]Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu. Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation.Advancesinneuralinformation processingsystems, 36:8406–8441, 2023. [42]Tianhe Wu, Ruibin Li, Lei Zhang, and Kede Ma. Diversity-preserved distribution matching distillation for fast visual synthesis.arXivpreprintarXiv:2602.03139, 2026. [43]Zhuguanyu Wu, Ruihao Gong, Yang Yong, Yushi Huang, Xiangyu Fan, Lei Yang, Dahua Lin, and Xianglong Liu. Sgmd: Score gradient matching distillation for few-step video diffusion distillation.arXivpreprintarXiv:2605.30116, 2026. [44] Yilun Xu, Weili Nie, and Arash Vahdat. One-step diffusion models withf-divergence distribution matching. arXivpreprintarXiv:2502.15681, 2025. [45]Zhucun Xue, Jiangning Zhang, Teng Hu, Haoyang He, Yinan Chen, Yuxuan Cai, Yabiao Wang, Chengjie Wang, Yong Liu, Xiangtai Li, et al. Ultravideo: High-quality uhd video dataset with comprehensive captions.arXiv preprintarXiv:2506.13691, 2025. [46]Yuhang Yang, Ke Fan, Shangkun Sun, Hongxiang Li, Ailing Zeng, FeiLin Han, Wei Zhai, Wei Liu, Yang Cao, and Zheng-Jun Zha. Videogen-eval: Agent-based system for video generation evaluation.arXivpreprint arXiv:2503.23452, 2025. [47]Tianwei Yin, Michaël Gharbi, Taesung Park, Richard Zhang, Eli Shechtman, Fredo Durand, and Bill Freeman. Improved distribution matching distillation for fast image synthesis.Advancesinneuralinformationprocessing systems, 37:47455–47487, 2024. [48]Tianwei Yin, Michaël Gharbi, Richard Zhang, Eli Shechtman, Frédo Durand, William T Freeman, and Taesung Park. One-step diffusion with distribution matching distillation. InCVPR, 2024. [49]Bowen Zheng and Tianming Yang. Revisiting diffusion models: From generative pre-training to one-step generation. arXivpreprintarXiv:2506.09376, 2025. [50]Kaiwen Zheng, Yongxin Chen, Huayu Chen, Guande He, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Direct discriminative optimization: Your likelihood-based visual generative model is secretly a gan discriminator. In InternationalConferenceonMachineLearning, pages 78067–78094. PMLR, 2025. [51]Kaiwen Zheng, Yuji Wang, Qianli Ma, Huayu Chen, Jintao Zhang, Yogesh Balaji, Jianfei Chen, Ming-Yu Liu, Jun Zhu, and Qinsheng Zhang. Large scale diffusion distillation via score-regularized continuous-time consistency. InInternationalConferenceonLearningRepresentations, volume 2026, pages 2582–2603, 2026. 13 [52]Mingyuan Zhou, Huangjie Zheng, Yi Gu, Zhendong Wang, and Hai Huang. Adversarial score identity distillation: Rapidly surpassing the teacher in one step.arXivpreprintarXiv:2410.14919, 2024. [53] Mingyuan Zhou, Huangjie Zheng, Zhendong Wang, Mingzhang Yin, and Hai Huang. Score identity distillation: Exponentially fast distillation of pretrained diffusion models for one-step generation. InForty-firstInternational ConferenceonMachineLearning, 2024. [54]Ziwei Zhou, Zeyuan Lai, Rui Wang, Yifan Yang, Zhen Xing, Yuqing Yang, Qi Dai, Lili Qiu, and Chong Luo. AVGen-Bench: A task-driven benchmark for multi-granular evaluation of text-to-audio-video generation, 2026. URL https://arxiv.org/abs/2604.08540. A Proof of Proposition 1 Optimal discriminator. Fixtandcand writeP=p t ,Q=p θ,t for the target and student noisy marginals. As in the main text, densities are conditional onc, which we omit from their arguments. Assume positive differentiable densities on common support and an unrestricted, unregularized discriminator with equal positive and negative class weights. The balanced BCE objective is B(h φ ;p) =−E t,c E x t ∼p t logD φ (x t ,t,c) +E x t ∼p θ,t log 1− D φ (x t ,t,c) ,(10) where D φ = S(h φ ) and S(u) = (1 + e −u ) −1 . Pointwise, this is −P (x) logS(h(x))− Q(x) log 1− S(h(x)) . Using ∂ ∂h logS(h) = 1− S(h) and ∂ ∂h log(1− S(h)) =−S(h), the stationarity condition is−P(1− S(h)) + QS(h) = 0, i.e.S(h ⋆ ) =P/(P+Q), which is the familiar optimal discriminator of Goodfellow et al.[8]. Hence, h ⋆ (x) = log S(h ⋆ ) 1− S(h ⋆ ) = log P (x) Q(x) = logp t (x)− logp θ,t (x). Since the integrand is strictly convex inhforP,Q >0, this stationary point is the unique minimizer. Differentiating in x yields Equation (3). If positives and negatives instead use different fixed sampling priorsπ + (t,c) andπ − (t,c), the optimal joint- classification logit additionally containslog[π + (t,c)/π − (t,c)]. On their common support, this term is constant inx, so its input gradient vanishes. The conditional log-ratio expression above assumes matching priors, not samplewise pairing. Gradient of the distribution-matching objective. Writex t =α t G θ (z,c) +σ t εand letp θ,t be its density. The targetpdoes not depend onθ. Assume the densities and reparameterization are sufficiently regular to interchange differentiation and integration, with support independent of θ. Then ∇ θ D KL (p θ,t ∥p t ) =∇ θ E z,ε h logp θ,t (x t )− logp t (x t ) i =E z,ε " ∂x t ∂θ T s θ (x t ,t,c)−s p (x t ,t,c) # +E x t ∼p θ,t h ∇ θ logp θ,t (x)| x=x t i , where the second term collects the dependence of the density onθat fixed argument. That term vanishes, sinceE p θ,t [∇ θ logp θ,t ] = R ∇ θ p θ,t =∇ θ R p θ,t = 0. Averaging overtandcgives Equation (2). Their sampling distributions are held fixed as θ varies. Equivalence. Replaceh φ in Equation (4) by the optimal logit at the current student distribution. Hold this function fixed during the generator update, while retaining its derivative with respect tox t . The chain rule 14 gives ∇ θ L lin G =−E " ∂x t ∂θ T ∇ x t h ⋆ (x t ,t,c) # =−E " ∂x t ∂θ T s p −s θ # =∇ θ L DM (θ;p). This is a local equality of generator gradients under an optimal, fixed critic. It does not differentiate through the dependence of the optimal critic on θ, or assert equality of the two loss values. Extension to weighted objectives. The same equivalence holds when each noise level is assigned a nonnegative weight w t that is independent of θ. In particular, E t,c [w t D KL (p θ,t ∥p t )] has the same generator gradient as −E [w t h ⋆ (x t ,t,c)], withh ⋆ held fixed. This extension does not cover DMD’s sample-dependent score normalization. It also treats our gap-based weights as fixed during each generator update rather than differentiating through their evolution over training. Choice of generator loss. For any differentiable ℓ, a fixed critic gives ∇ θ E w t ℓ(h φ ) =E " w t ℓ ′ (h φ ) ∂x t ∂θ T ∇ x t h φ # .(11) Relative to the linear loss, the per-sample multiplier is−ℓ ′ (h φ ), which is nonnegative for a non-increasing loss. The linear lossℓ(h) =−hhas multiplier one. The non-saturating lossℓ(h) =− logS(h) instead gives 1− D φ , reducing the contribution of samples the discriminator already accepts. At an optimal critic, this reweights the ideal distribution-matching gradient for each sample and can change the direction of the aggregate update. We compare the two losses in Section 4.3. Two fixed targets. Applying the result separately to teacher samples and real data gives h ⋆ T = log p T,t p θ,t , h ⋆ R = log p R,t p θ,t .(12) These are the unrestricted optima of the two balanced classification tasks. Assume both logits are optimal and the coefficientsλ T ,λ R are constant and nonnegative. With gap reweighting disabled, the generator update is the gradient of E t,c [λ T D KL (p θ,t ∥p T,t ) + λ R D KL (p θ,t ∥p R,t )]. Let Λ = λ T + λ R > 0 and ρ = λ T /Λ. At each fixed t,c, define M t (x) = Z −1 t p T,t (x) ρ p R,t (x) 1−ρ , Z t = Z p T,t (x) ρ p R,t (x) 1−ρ dx. For 0< Z t <∞, the weighted KL sum equals Λ[D KL (p θ,t ∥M t )− logZ t ]. SinceZ t is independent ofθ, the corresponding gradient is that of ΛE t,c [D KL (p θ,t ∥M t )]. The geometric mixture is defined separately at each noise level. Forward noising does not generally commute with geometric mixing, and the marginalsp θ,t all arise from a single clean student distribution. This identity does not identify a clean geometric mixture as the global optimum, or apply as a fixed objective when gap reweighting changes the coefficients. 15 B Implementation Details Training and evaluation configuration. We use AdamW with (β 1 ,β 2 ) = (0,0.99) and one discriminator update per generator update. All reported models use generator EMA weights. ImageNet and SDXL use an EMA half-life of 10 6 generated images with ramp-up factor 0.05. Wan and MiniMax-H3 uses power-function EMA [15] withσ rel = 0.1. For ImageNet-64×64, we train on 8 H100 GPUs with global batch size 384, learning ratesη G =η D = 2×10 −6 ,λ T =λ R = 0.003, and 570K iterations. For SDXL, we train on 8 H100 GPUs with global batch size 64,η G =η D = 5×10 −7 ,λ T =λ R = 0.005, and 17K iterations. For Wan2.1-T2V-1.3B, we train on 8 H100 GPUs with global batch size 8, η G = η D = 2× 10 −6 , λ T = λ R = 1, and 17K iterations. For Wan2.1-T2V-14B, we train on 64 H100 GPUs with global batch size 8,η G =η D = 10 −6 ,λ T =λ R = 1, and 19K iterations, using FSDP and context parallelism with groups of 8 GPUs. For evaluation, we follow the protocols of DMD2 [47] for ImageNet and SDXL, and rCM [51] for Wan2.1-T2V. For MiniMax-H3, we train on 8 H200 GPUs with global batch size 8 at a resolution of 1344×768 for 800 iterations, usingη G =η D = 4×10 −5 andλ T =λ R = 1. Both the generator and the discriminator backbone use LoRA with rank and alpha set to 128. For evaluation, we follow the protocols of DMD2 [47] for ImageNet and SDXL, rCM [51] for Wan2.1-T2V and AVGen-Bench [54] for MiniMax-H3. For user study in MiniMax-H3, we conduct a pairwise user study using 387 prompts from VideoGenEval [46]. For each prompt, we compare the audio-video output of DMAD with those of DMD2 and rCM, yielding 774 pairs. All models use four sampling steps. Pairs are presented as A/B comparisons with the left/right placement of DMAD balanced within each baseline. Each pair is evaluated by three annotators on five dimensions: prompt alignment, visual quality, motion quality, audio quality, and overall preference. For each dimension, annotators select A, B, or a tie. This yields 2,322 pairwise evaluations. For each prompt and dimension, we aggregate the three judgments by majority vote, recording a tie if all three judgments differ. Each baseline therefore has 387 prompt-level outcomes per dimension. We report the preference rate for DMAD asN win /(N win +N loss ) over these outcomes, excluding ties. For overall preference, the win/tie/loss counts are 204/129/54 against DMD2 and 247/95/45 against rCM, with tie rates of 33.3% and 24.5%, respectively. Discriminator architecture. The real and teacher heads share a feature backbone. For ImageNet and SDXL, the diffusion discriminator uses a pretrained U-Net encoder and middle block with two convolutional heads. Each head downsamples the features using 4×4 convolutions, group normalization, and SiLU, then predicts a scalar logit. For the projected-discriminator variant on ImageNet, we follow D2O-F [49] and use frozen ImageNet-pretrained VGG16-BN and EfficientNet-Lite0 backbones with two trainable sets of multi-scale discriminator heads. For Wan, two MLP heads with a SiLU activation operate on final-block tokens, and their logits are averaged over the video. For MiniMax-H3, final-block features undergo non-affine RMS normalization before entering two MLP heads, each consisting of two linear layers with a SiLU activation between them. For noise-band reweighting, ImageNet uses six bands over Karras indices 20–500, grouped by 100 indices, with the last containing only index 500 (σ ≈2.527), SDXL uses 100-timestep bands, Wan uses 10 equal-probability bands under its discriminator-time sampling distribution, and MiniMax-H3 uses ten equal-width bands in the base time variable over [0,1]. We useτ= 0.5 for ImageNet and SDXL training and τ = 2 for Wan and MiniMax-H3. B.1 Training Speed and Memory Table 5 compares the online training cost of DMD2 and DMAD on NVIDIA H200 GPUs, with matched batch sizes and input shapes within each setting. ImageNet and Wan-14B use full-parameter training, while MiniMax-H3 uses rank-128 LoRA. We measure CUDA-synchronized update times after warm-up and peak allocated GPU memory. Time includes one generator update and its associated critic updates. Following the official DMD2 configuration, we use a 5:1 critic-to-generator update ratio for DMD2, compared with 1:1 for DMAD. Evaluation and checkpointing are excluded. B.2 Toy Gradient Estimation Distributions. We compare two estimates of the same correction field using two-dimensional Gaussian mixtures with known parameters. The targetprepresents real data, andqis a fixed student distribution. For 16 Table 5 Online training time per generator update, including its critic updates, and peak allocated memory per GPU. Lower is better. Time (s)Memory (GiB/GPU) SettingH200 GPUs DMD2 Ours DMD2Ours ImageNet-64× 6412.321 0.496 40.3433.99 Wan2.1-T2V-14B4162.9 38.50 131.04124.56 MiniMax-H3-33B4818.1 233.3 127.23114.72 k = 0,..., 7, define μ k = 2 cos(2πk/8), sin(2πk/8) T , δ = (0.08,−0.04) T . The clean distributions are p(x) = 7 X k=0 1 8 N (x| μ k , 0.35 2 I), q(x) = 7 X k=0 π k N (x| μ k +δ, 0.35 2 I), where (π 0 ,...,π 7 ) = (0.16,0.14,0.12,0.10,0.10,0.12,0.13,0.13). Adding independent noiseσε, withε ∼ N(0,I), gives marginalsp σ andq σ . Their component means and weights are unchanged, and their component covariance is v σ I, where v σ = 0.35 2 + σ 2 . Analytic reference. For a mixturef σ with weightsa k and meansm k , its posterior component weights and score are r k (x,σ) = a k exp −∥x−m k ∥ 2 /(2v σ ) P j a j exp −∥x−m j ∥ 2 /(2v σ ) , s(x,σ) =∇ x logf σ (x) = P k r k (x,σ)m k −x v σ . Applying this expression separately to the target and student gives the ground-truth field g ⋆ = s p −s q =∇ x log p σ q σ . Balanced binary classification has the ideal logith ⋆ =log(p σ /q σ ), motivating the comparison with∇ x h φ . Analytic scores and density ratios are used only for evaluation. Neither estimator receives them as training supervision. Training. We first train a teacher noise predictor on samples drawn directly frompfor 20,000 updates. It has two width-64 tanh hidden layers and a linear two-dimensional output. Its input is (x 1 /3,x 2 /3, log(σ/0.5)), and its denoising objective is 1 2 E∥ε ω (x 0 +σε,σ)−ε∥ 2 . Noise levels are sampled uniformly from0.2,0.5,1.0 for each example during pretraining and adaptation. Both estimators inherit this pretrained teacher. The fake noise predictor copies all its parameters. The discriminator copies both hidden layers and uses a new scalar linear head, whose weights have independent Gaussian initialization with standard deviation 0.02 and whose bias is zero. The teacher then stays frozen. All parameters of the fake predictor and discriminator, including their hidden layers, are adapted for 1,000 updates. The fake predictor uses denoising onq. The discriminator uses balanced binary cross-entropy on noisy target positives and student negatives. Pretraining and adaptation use Adam with learning rate 0.001, (β 1 ,β 2 ) = (0.9,0.999), and numerical constant 10 −8 . Each adaptation optimizer starts with fresh state. The denoising batch contains 512 student examples. The classification batch contains 256 target and 256 student examples, sharing those student inputs with the first half of the denoising batch. Pretraining also uses batch size 512. Both branches therefore receive 512 training inputs per update. Computation uses float64. 17 Evaluation. The experiment in Figure 2 uses seed 0 andσ= 0.5. The labelsDMDgradientandOurs gradient refer respectively to b g DMD =− ε teacher σ + ε fake σ , b g ours =∇ x h φ . Both are evaluated on the same N = 2,048 independent held-out samples x i ∼ q 0.5 using NMSE( b g) = N −1 P i ∥ b g(x i )−g ⋆ (x i )∥ 2 N −1 P i ∥g ⋆ (x i )∥ 2 . Their NMSE values are 1.539462 and 0.144375. This experiment isolates single-target gradient estimation with a fixed student distribution. Target samples for both teacher pretraining and discriminator training are drawn directly from p. C Additional Audio-Video Evaluation We follow the evaluation metrics and score aggregation of AVGen-Bench [54], using Qwen3-Omni-Thinking as the VLM judge. The same H3-enhanced prompts are used for generation and judging across all methods. Table 6 reports all eleven metrics for MiniMax-H3. DMAD achieves a total score of 65.4, compared with 62.6 for DMD2 and 59.8 for rCM, while the teacher scores 67.2. Table 6 AVGen-Bench results on MiniMax-H3. All scores are normalized to 0–100 (↑). Bold marks the best total among distilled models. QualitySynchronizationFine-grained evaluation MethodTotal Visual Audio AVLip Face Lo-Phy Hi-Phy Holistic Text Speech Music DMD262.6 90.1 67.4 56.722.7 46.180.295.087.1 64.681.00.0 rCM59.8 84.0 65.8 48.715.9 49.079.795.387.3 48.381.36.2 DMAD (Ours) 65.4 93.3 70.4 49.130.6 53.182.393.287.4 70.980.6 11.8 Teacher67.2 92.6 70.6 58.339.0 59.279.794.587.0 74.684.01.6 D Additional Qualitative Results D.1 SDXL Comparisons Qualitative comparisons in Figure 8 show that DMAD produces more natural colors and textures and follows the prompts more closely, while some DMD2 and SenseFlow samples appear oversaturated. The tennis example highlights natural skin texture, while the bird example shows distinct wing structure and fine feather detail. In the bench example, only our model satisfies the instruction to exclude cats. It also generates exactly four people in the family portrait, whereas the teacher, DMD2, and SenseFlow include extra people. D.2 MiniMax-H3 Teaser Prompts We list the prompts for the six samples in Figure 1, ordered from left to right and top to bottom. 1. A medium shot opens on a seasoned, grey-bearded man in sunglasses and a paisley shirt, Beside him, a younger man in a tank top. In the background, a vibrant mural splashes across a wall, hinting at an urban setting. Faint city murmurs and distant chatter drift in, accompanied by a mellow, soulful hip-hop beat that adds a contemplative yet grounded atmosphere. The older man speaks while the younger man turns toward him, laughs softly, shakes his head in amusement, and then looks back at the camera. The camera slowly pushes in. "The city always got a story," the older man murmurs with a slight nod. The younger man chuckles and replies, "Yeah, you always say that." 2. A breathtaking, painterly 2D animated continuous visual narrative, rendered with the lush, vibrant, and slightly surreal, almost dreamlike, infused with the intricate, delicate detail of traditional Japanese 18 A tennis player in a red dress taking a swing. a bench without any cats on it a family of four posing at the Grand Canyon A small white bird flying over the water. TeacherDMD2SenseFlowOurs Figure 8 Quality comparison of DMAD with the SDXL teacher, DMD2 [47], and SenseFlow [6]. All distilled models use 4 sampling steps, while the teacher uses 50 sampling steps. Images in each row share the same text prompt and initial Gaussian noise. Our model produces more natural colors and realistic textures with better text alignment. (Zoom in for details.) woodblock prints (Ukiyo-e), follows a young, adventurous, and kind-hearted girl (perhaps with bright, curious eyes and wearing simple, practical, beautifully patterned traditional Japanese farm attire) as she befriends a colossal, gentle, ancient Forest Spirit. The Spirit is a magnificent, awe-inspiring creature, its form a harmonious blend of animal and plant – perhaps with moss-covered, antler-like branches, fur like shimmering leaves that change color with its mood, and eyes like deep, tranquil forest pools. They meet in a sun-dappled, sacred grove deep within an ancient, primeval forest, where impossibly tall, gnarled trees form a living cathedral and tiny, glowing, friendly forest sprites (Kodama-like) peek from behind 19 mossy rocks and giant, fantastical mushrooms. The girl, initially awestruck, offers the massive Spirit a small, carefully cultivated offering – perhaps a perfectly ripe persimmon or a handful of wild berries – her gesture one of pure, innocent respect and affection. The Forest Spirit responds with a slow, gentle inclination of its massive head, its leafy fur rustling like a thousand whispers, and perhaps causes a shower of magical, luminous flower petals to drift down from the canopy, or a tiny, new sapling to sprout at the girl’s feet. The animation captures the incredible, detailed textures of the forest, the Spirit’s majestic yet gentle presence, and the profound, unspoken emotional connection forming between the child and this ancient guardian of nature. The color palette is a rich symphony of deep forest greens, earthy browns, vibrant floral hues, and the soft, magical glow of the sprites and the Spirit’s own subtle luminescence. This continuous, sweeping visual journey is a celebration of the profound, often mystical, bond between humanity and nature, the innocence and courage of childhood, and the power of kindness and respect to bridge even the most fantastical of divides, an affectionate, visually intoxicating ode to ecological harmony and interspecies understanding. The only implied sounds are the gentle rustling of leaves, the distant calls of unseen forest birds, the girl’s soft, respectful breathing, the Spirit’s deep, resonant, almost inaudible hum, and a soaring, emotionally resonant, orchestral score. 3. The video opens with a medium, eye-level shot of a beautiful woman with dark hair and warm brown eyes. She wears a magnificent, high-fashion flamingo dress with layers of pink and fuchsia feathers, complemented by whimsical pink, heart-shaped sunglasses. She walks with serene confidence through the crystal-clear, shallow turquoise water of a sun-drenched lagoon. The camera slowly pulls back to a medium-wide shot, revealing the breathtaking scene as the dress’s long train glides and floats gracefully on the water’s surface behind her. The cinematic, dreamlike atmosphere is enhanced by the vibrant colors of the dress against the serene, minimalist landscape, capturing a moment of pure elegance and high-fashion fantasy. 4.An adorable happy otter confidently stands on a surfboard wearing a yellow lifejacket, riding along turquoise tropical waters near lush tropical islands, 3D digital render art style. 5.Static overhead shot of a printed photo of a tree trunk lying on a wooden table.Action Sequence:1, A real human hand enters, places a single pinecone on the paper next to the tree hole, and exits immediately.2, A realistic 3D squirrel emerges from the 2D hole in the photo. The squirrel comes out empty-handed.3, The squirrel sniffs the pinecone sitting on the paper, looks curious, blinks, and tilts its head.4, The squirrel reaches out, grabs that specific pinecone from the table.Key details: Seamless interaction between real world and photo, surreal VFX, squirrel paws are empty initially, heavy weight perception on the pinecone. 6.An origami dragon unfolds from paper in stop-motion style, then flies over folded mountains. The camera pans with paper dust fluttering midair. Every flap and fold is sharply geometric. 20 D.3 Wan2.1-T2V-14B Samples Figure 9 Video samples from our four-step Wan2.1-T2V-14B student. Top: representative frames from nine generated videos. Bottom: three frames in chronological order from each of six videos. Full prompts are provided in Section D.3. We list the prompts for the nine samples in the upper grid of Figure 9, ordered from left to right and top to bottom. 1.A close up view of a glass sphere that has a zen garden within it. There is a small dwarf in the sphere who is raking the zen garden and creating patterns in the sand. 2. This close-up shot of a chameleon showcases its striking color changing capabilities. The background is blurred, drawing attention to the animal’s striking appearance. 3. A drone camera circles around a beautiful historic church built on a rocky outcropping along the Amalfi Coast, the view showcases historic and magnificent architectural details and tiered pathways and patios, waves are seen crashing against the rocks below as the view overlooks the horizon of the coastal waters and hilly landscapes of the Amalfi Coast Italy, several distant people are seen walking and enjoying vistas on patios of the dramatic ocean views, the warm glow of the afternoon sun creates a magical and romantic feeling to the scene, the view is stunning captured with beautiful photography. 21 4. An extreme close-up of an gray-haired man with a beard in his 60s, he is deep in thought pondering the history of the universe as he sits at a cafe in Paris, his eyes focus on people offscreen as they walk as he sits mostly motionless, he is dressed in a wool coat suit coat with a button-down shirt , he wears a brown beret and glasses and has a very professorial appearance, and the end he offers a subtle closed-mouth smile as if he found the answer to the mystery of life, the lighting is very cinematic with the golden light and the Parisian streets and city in the background, depth of field, cinematic 35m film. 5.A female traveler traverses a desert alone. The camera first shows a distant view of her slightly unsteady footsteps and flowing clothes, then switches to a medium side shot of her scarf billowing in the wind, and finally focuses on her determined gaze. 6.3D animation of a small, round, fluffy creature with big, expressive eyes explores a vibrant, enchanted forest. The creature, a whimsical blend of a rabbit and a squirrel, has soft blue fur and a bushy, striped tail. It hops along a sparkling stream, its eyes wide with wonder. The forest is alive with magical elements: flowers that glow and change colors, trees with leaves in shades of purple and silver, and small floating lights that resemble fireflies. The creature stops to interact playfully with a group of tiny, fairy-like beings dancing around a mushroom ring. The creature looks up in awe at a large, glowing tree that seems to be the heart of the forest. 7.The camera slowly pushes forward into a breathtaking ice cave, its jagged walls sculpted by nature into intricate patterns of blues and whites, reflecting the ethereal light from an opening ahead. The crunch of ice underfoot and the drip-drip of melting water create a serene, echoing soundscape. As the camera moves closer, a gentle, ambient melody begins, swelling with the light from the cave’s exit. The camera emerges from the narrow opening into a vast, sun-drenched valley, revealing a group of polar bears playfully sliding down an ice slope, their roars echoing with joy. 8.In the style of watercolor illustration, a little white rabbit wearing a bow is picking mushrooms in the forest. 9. A polar bear is playing the violin in the snow. 22 D.4 ImageNet-64×64 Samples Figures 10 and 11 show one-step samples from our FID of 1.24 and FID of 1.04 results on ImageNet. Each figure shows a 32× 32 grid of generated images at their native 64× 64 resolution. Figure 10 One-step samples from DMAD on ImageNet-64×64 (FID 1.24). Zoom in for details. 23 Figure 11 One-step samples from DMAD on ImageNet-64×64 (FID 1.04). Zoom in for details. 24 D.5 More results on MiniMax-H3 Prompt: Birthday cake with exactly nine lit candles; a child blows out three candles on the left, leaving six burning. Smoke rises only from the extinguished three. Frame 1Frame 42Frame 83Frame 124 Teacher DMD2 Ours Prompt: On the rooftop of a Korean high school, distant city lights glimmer in the background with a soft wind rustling, and stars twinkle in the night sky. The girl leans against the railing, lost in thought. The boy walks over with two cans of cola, hands one to her, and she takes it and pops the tab open. Boy (casual tone, Korean): "숙제 다 했어? 왜 여기 있어?" Girl (sighing, Korean): "시험이 너무 무서워". Boy (gentle tone, Korean): “걱정 마, 넌 잘할 거야.” Frame 1Frame 42Frame 83Frame 124 Teacher DMD2 Ours Figure 12 Qualitative comparisons of our DMAD and baseline models on MiniMax-H3. Within each case, rows show Teacher, DMD2, and Ours, from top to bottom. Columns show frames 1, 42, 83, and 124, using the same indices for all methods. All compared videos are generated using identical initial noise and text prompts. 25 Prompt: An empty white-walled room containing only a single wooden chair in the far corner: no windows, no pictures, no lamps, no other furniture. The camera slowly dollies forward toward the chair. Frame 1Frame 42Frame 83Frame 124 Teacher DMD2 Ours Prompt: A beautiful woman wearing red clothes. A round of glances and a laugh. Frame 1Frame 42Frame 83Frame 124 Teacher DMD2 Ours Figure 13 Qualitative comparisons of our DMAD and baseline models on MiniMax-H3. Within each case, rows show Teacher, DMD2, and Ours, from top to bottom. Columns show frames 1, 42, 83, and 124, using the same indices for all methods. All compared videos are generated using identical initial noise and text prompts. 26 Prompt: Extreme close-up of a an eye with city reflected in it. Frame 1Frame 42Frame 83Frame 124 Teacher DMD2 Ours Prompt: Talking to the barista from across the counter, a woman in a coffee shop places an order for a cup of coffee with cream and sugar, and a chocolate croissant. The barista listens to the order, responds sure :) and then turns to the commercial espresso machine that is behind him. The woman patiently waits across the counter as her order is being prepared by the barista. Frame 1Frame 42Frame 83Frame 124 Teacher DMD2 Ours Figure 14 Qualitative comparisons of our DMAD and baseline models on MiniMax-H3. Within each case, rows show Teacher, DMD2, and Ours, from top to bottom. Columns show frames 1, 42, 83, and 124, using the same indices for all methods. All compared videos are generated using identical initial noise and text prompts. 27 Prompt: A bee with a puppy's head Frame 1Frame 42Frame 83Frame 124 Teacher DMD2 Ours Prompt: First-person perspective, high-speed flight, symmetrical composition, rotation, countless lightning bolts amidst dark clouds, motion blur. Frame 1Frame 42Frame 83Frame 124 Teacher DMD2 Ours Figure 15 Qualitative comparisons of our DMAD and baseline models on MiniMax-H3. Within each case, rows show Teacher, DMD2, and Ours, from top to bottom. Columns show frames 1, 42, 83, and 124, using the same indices for all methods. All compared videos are generated using identical initial noise and text prompts. 28