Paper deep dive
FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation--Full Version
Dat Nguyen-Cong, Tung Kieu, Hoang Thanh-Tung
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 4/10/2026, 3:16:39 AM
Summary
FastDiSS is a novel training framework for diffusion language models that addresses two primary bottlenecks in few-step sequence-to-sequence generation: training-inference mismatch in self-conditioning and late-stage training saturation. It introduces Self-conditioning Perturbation (SCP) to regularize self-conditioning by simulating inference-time noise, and Model-aware Noise Scaling (MANS) to dynamically adjust noise levels based on token-level denoising confidence, significantly improving inference speed and generation quality.
Entities (4)
Relation Signals (4)
FastDiSS → includes → Self-conditioning Perturbation
confidence 100% · FastDiSS introduces two complementary components... First, we propose Self-conditioning Perturbation (SCP)
FastDiSS → includes → Model-aware Noise Scaling
confidence 100% · FastDiSS introduces... Second, we introduce Model-aware Noise Scaling (MANS)
Model-aware Noise Scaling → mitigates → Late-stage loss saturation
confidence 95% · We introduce MANS to make the diffusion objective token-aware, mitigating late-stage loss saturation.
Self-conditioning Perturbation → reduces → Training-Inference Mismatch
confidence 95% · We propose SCP to reduce the training-inference mismatch of self-conditioning.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Self-conditioning has been central to the success of continuous diffusion language models, as it allows models to correct previous errors. Yet its ability degrades precisely in the regime where diffusion is most attractive for deployment: few-step sampling for fast inference. In this study, we show that when models only have a few denoising steps, inaccurate self-conditioning induces a substantial approximation gap; this mistake compounds across denoising steps and ultimately dominate the sample quality. To address this, we propose a novel training framework that handles these errors during learning by perturbing the self-conditioning signal to match inference noise, improving robustness to prior estimation errors. In addition, we introduce a token-level noise-awareness mechanism that prevents training from saturation, hence improving optimization. Extensive experiments across conditional generation benchmarks demonstrate that our framework surpasses standard continuous diffusion models while providing up to 400x faster inference speed, and remains competitive against other one-step diffusion frameworks.
Tags
Links
- Source: https://arxiv.org/abs/2604.05551v1
- Canonical: https://arxiv.org/abs/2604.05551v1
Trouble viewing inline? Open PDF directly →
Full Text
89,096 characters extracted from source content.
Expand or collapse full text
FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation–Full Version Dat Nguyen-Cong1, Tung Kieu2, Hoang Thanh-Tung3 1FPT Software AI Center, FPT Corporation, 2Department of Computer Science, Aalborg University, Denmark, 3Quantum AI and Cyber Security Institute, FPT Corporation dat27072002@gmail.com, tungkvt@cs.aau.dk, htt210@gmail.com Abstract Self-conditioning has been central to the success of continuous diffusion language models, as it allows models to correct previous errors. Yet its ability degrades precisely in the regime where diffusion is most attractive for deployment: few-step sampling for fast inference. In this study, we show that when models only have a few denoising steps, inaccurate self-conditioning induces a substantial approximation gap; this mistake compounds across denoising steps and ultimately dominate the sample quality. To address this, we propose a novel training framework that handles these errors during learning by perturbing the self-conditioning signal to match inference noise, improving robustness to prior estimation errors. In addition, we introduce a token-level noise-awareness mechanism that prevents training from saturation, hence improving optimization. Extensive experiments across conditional generation benchmarks demonstrate that our framework surpasses standard continuous diffusion models while providing up to 400x faster inference speed, and remains competitive against other one-step diffusion frameworks. FastDiSS: Few-step Match Many-step Diffusion Language Model on Sequence-to-Sequence Generation–Full Version Dat Nguyen-Cong1, Tung Kieu2, Hoang Thanh-Tung3 1FPT Software AI Center, FPT Corporation, 2Department of Computer Science, Aalborg University, Denmark, 3Quantum AI and Cyber Security Institute, FPT Corporation dat27072002@gmail.com, tungkvt@cs.aau.dk, htt210@gmail.com 1 Introduction Diffusion models have recently emerged as a compelling alternative to autoregressive text generation, matching and in some settings surpassing autoregressive models in quality and controllability DBLP:journals/corr/abs-2502-09992; DBLP:journals/corr/abs-2507-15857. A key advantage of diffusion models lies in their ability to generate all tokens in parallel, offering a linear-time decoding rather than a quadratic complexity as in autoregressive decoding radford2018improving; radford2019language; DBLP:conf/nips/BrownMRSKDNSSAA20. This property makes diffusion models attractive for a broad range of natural language processing tasks beyond unconditional generation, such as conditional and structured sequence modeling DBLP:conf/iclr/GongLF0K23; DBLP:conf/nips/YeGCZGSWJLBK24; DBLP:journals/corr/abs-2508-15487. Despite these advantages, diffusion language models face a central efficiency bottleneck: high-quality generation typically requires a long reverse process with many denoising steps DBLP:conf/iclr/GongLF0K23. The resulting iterative sampling cost eventually offset the latency gains from parallel token generation. To mitigate this issue, a widely adopted trick is to use self-conditioning DBLP:conf/iclr/ChenZH23; DBLP:conf/nips/GulrajaniH23, which reuses previous predictions as an additional conditioning signal to improve prediction under fewer steps. While self-conditioning indeed strengthens few-step sampling, we find that it introduces underappreciated failure modes that become critical precisely in the fast-inference application. Mismatching between training-inference self-condition under few-step denoising. Self-conditioning introduces an intrinsic mismatch between training and inference DBLP:conf/emnlp/Schmidt19; DBLP:conf/iclr/NingLSSE24; DBLP:conf/naacl/GaoG0Z0X24. During training, a model can be conditioned on ground-truth targets, whereas at inference, it must be conditioned on its own imperfect previous predictions. This distribution shift induces error accumulation along the reverse trajectory, leading to sampling drift DBLP:conf/nips/DarasDDD23 and degraded generation quality. Our analysis shows that the problem is amplified in few-step settings: predictions made at high noise levels differ significantly from those made later at low noise, turning the reused self-conditioning signal into a biased condition. Consequently, self-conditioning can become unstable, and in the worst case, the reused estimates can steer subsequent denoising updates in the wrong direction. Loss saturation in the late-stage training. Diffusion language models often fit the denoising objective quickly early in training, but subsequently exhibit a pronounced loss plateau. This slow improvement suggests that the sampled noise levels become insufficiently informative: the training signal is dominated by “easy” cases where tokens are already predicted with high confidence, leading to inefficient learning. Prior works attribute this behavior in part to applying a uniform noise schedule across tokens, which ignores token-wise heterogeneity in denoising difficulty DBLP:conf/naacl/YuanYTHH24. Consistent with this view, our analysis shows that uniform noise sampling is suboptimal. In particular, increasing noise for well-predicted tokens yields a more effective learning signal and improves optimization efficiency, thus achieving lower evaluation loss than models trained with the uniform schedule. To address these challenges, we propose Fast Diffusion Sequence-to-Sequence (FastDiSS), a novel training framework designed to improve the robustness and efficiency of the diffusion model in the few-step setting. Building on self-conditioning, FastDiSS introduces two complementary components that directly target the above failure modes. First, we propose Self-conditioning Perturbation (SCP), a simple regularization strategy for self-conditioning. Initially, during training, we obtain this condition by running the denoising network on a more-noised forward process, producing a weaker and noisier estimation. We then train the network conditioned on this corrupted signal, better matching inference-time errors and reducing sampling drift. Second, we introduce Model-aware Noise Scaling (MANS), a token-level noise allocation strategy that dynamically adjusts noise based on denoising confidence. MANS applies higher noise to high-confidence tokens, prevents trivial training, and further reduces self-conditioning errors at high noise levels. We evaluate FastDiSS on six benchmarks covering diverse sequence-to-sequence tasks and few-step generation settings. Across settings, FastDiSS consistently narrows the gap between few-step and many-step sampling, outperforming prior text diffusion baselines in both quality and efficiency. In particular, FastDiSS improves BLEU score DBLP:conf/acl/PapineniRWZ02 while achieving substantial speedups ranging from 4× to 400×, and remains competitive with other few-step diffusion approaches. In summary, our contributions are threefold: (1) we identify and analyze two bottlenecks that limit self-conditioned diffusion language models under few-step samplings, highlighting the roles of discretization-induced mismatch and late-stage training saturation; (2) we introduce FastDiSS, combining SCP to regularize self-conditioning under realistic inference noise and MANS to avoid trivial denoising via confidence-driven token-wise noise allocation; and (3) we demonstrate consistent gains on six benchmarks, showing that FastDiSS improves generation quality while substantially accelerating inference. 2 Background 2.1 Denoising Diffusion Probabilistic Models We revisit Gaussian diffusion process in its continuous-time formulation DBLP:conf/iclr/0011SKKEP21; DBLP:conf/nips/ChuangHLGLCL24, which defines a trajectory tt=01,t∈ℝ\ z_t\_t=0^1,t of increasing noise starting from the clean data 0∼p(0) z_0 p( z_0) and ending with 1∼(0,) z_1 (0,I). For any t, the noise schedule comprises the decay factor αt _t and the diffusion rate σt _t, which are strictly positive and monotonic over time. Given that q(t|0)q( z_t| z_0) satisfies the Markovian property, the forward process is formulated as follows. q(t|0) q( z_t| z_0) =(t;αt0,σt2), =N( z_t; _t z_0, _t^2I), (1) q(t|s) q( z_t| z_s) =(t;(αt/αs)s,σt|s2) =N( z_t;( _t/ _s) z_s, _t|s^2I) (2) Here, 0≤s<t≤10≤ s<t≤ 1 and σt|s2=σt2−(αt2/αs2)σs2 _t|s^2= _t^2-( _t^2/ _s^2) _s^2. As t→1t→ 1, αt→0 _t→ 0 and σt→1 _t→ 1, the endpoint follows a Gaussian distribution. The goal of the diffusion model is to denoise 0∼p(0|t) z_0 p( z_0| z_t) through a neural network Dθ(t)D_θ( z_t), which is trained using a mean-squared error loss: ℒdiffusion=0,t∼[0,1][‖Dθ(t)−0‖2]L_diffusion=E_ z_0,t [0,1][||D_θ( z_t)- z_0||^2] (3) Here, [0,1]U[0,1] denotes the continuous uniform distribution. Recursive sampling from the distribution p(s|t) p( z_s| z_t) =p(0|t)[q(s|t,0)] =E_p( z_0| z_t)[q( z_s| z_t, z_0)] ≈p(^θt|t)[q(s|t,^θt)], _p( z_θ^t| z_t)[q( z_s| z_t, z_θ^t)], (4) where ^θt=Dθ(t) z_θ^t=D_θ( z_t), starting at 1∼(0,) z_1 (0,I), enables generating data from p(0)p( z_0). Full expression of q(s|t,0)q( z_s| z_t, z_0) is demonstrated in Appx. A.1. Figure 1: Overview of FastDiSS. The tokenized sequence is first encoded to 0 z_0, while concurrently sampling the initial timestep t. Both 0 z_0 and t are passed into MANS to obtain the new timestep tθt_θ. Subsequently, noise level at tθt_θ is added to 0 z_0 using SCP to obtain t′ z_t . The rest is the same as in the training objective in Eq. 5. 2.2 Conditional Sequence Modeling With Diffusion Models Diffusion models rely on a continuous space where Gaussian noise can be smoothly added and subtracted. However, texts are composed of discrete tokens with no inherent notion of “small changes” between them. To address this, DiffusionLM DBLP:conf/nips/LiTGLH22 maps the text sequence ∈0,1L×V x∈\0,1\^L× V, where each token is represented as a one-hot vector, into a continuous latent space 0∈ℝL×H z_0 ^L× H, with sequence length L, hidden dimension H, and vocabulary size V. Hence, 0∼q(0|) z_0 q( z_0| x) is the embedding codebook of x. The reverse process aims to generate 0 z_0, which is then mapped back to x. The diffusion model is trained on the latent space with the objective ℒtotal=ℒdiffusion +ℒround _total=L_diffusion +L_round =0,t[‖Dθ(t)−0‖2]+0[−logpθ(|0)], =E_ z_0,t[||D_θ( z_t)- z_0||^2]+E_ z_0 [- p_θ( x| z_0) ], (5) where ℒroundL_round is the 0→ z_0→ x reconstruction loss. To extend the model for conditional generation, a simple yet effective approach is to incorporate the conditioning sequence c as an additional input to the denoising network, i.e., Dθ(t,)D_θ( z_t, c). The target sequence length L can be inferred from the conditioning context via a learned prior L∼p(L|)L p(L| c). Aside from this conditioning, the diffusion process remains unchanged as in Eq. 4. 2.3 Self-conditioning Diffusion Models During training, the initial forward prediction ^θt z_θ^t is fed back into the denoising model as an auxiliary condition. The 0 z_0-prediction then becomes ¯θt=Dθ(t,sg(^θt)) z_θ^t=D_θ( z_t,sg( z_θ^t)), where sg(⋅)sg(·) denotes the stop-gradient operation, preventing gradient propagation through ^θt z_θ^t. For clarity, we omit the source condition c, as the discussion here focuses solely on the self-conditioning mechanism. The self-conditioning training objective becomes: ℒsc=0,t[‖Dθ(t,sg(^θt))−0‖2]. _sc=E_ z_0,t[||D_θ( z_t,sg( z_θ^t))- z_0||^2]. (6) The training process alternates between optimizing ℒdiffusionL_diffusion and ℒscL_sc. At each step, estimating ^θt z_θ^t followed by ¯θt z_θ^t double the inference time. Instead, the prediction from the previous step u is reused as the conditioning to avoid additional inference overhead. We denote this estimation as ¯θtu z_θ^tu, for any 0<s<t<u≤10<s<t<u≤ 1. 3 Methodology In this section, we describe the design of FastDiSS for efficient sequence-to-sequence language generation. Fig. 1 provides an overview. Building upon the standard text diffusion architecture, FastDiSS augments training with two tightly coupled components: Self-conditioning Perturbation (SCP) and Model-aware Noise Scaling (MANS). Together, they target the two bottlenecks identified in Sec. 1: (i) mismatch between training-time and inference-time self-conditioning under few-step discretization, and (i) late-stage training saturation caused by uninformative, token-unaware noise. 3.1 Self-conditioning Limitations As described in Sec. 2.3, using the reused estimation z¯θtu z_θ^tu in place of the step-matched prediction z¯θt z_θ^t introduces a training-inference mismatch. Intuitively, estimation from a coarser step u carries larger uncertainty than that from step t DBLP:conf/icml/BaoLSZZ22; DBLP:conf/iclr/NingLSSE24, so ¯θtu z_θ^tu is more likely to deviate from the target embedding 0 z_0. To measure the effect of this estimation gap, we report the BLEU score on the generation of IWSLT14 De-En dataset, using the correct ¯θ z_θ and the original ¯θtu z_θ^tu self-condition sampling. In Tab. 1, we conduct experiments with different numbers of denoising steps (NFEs). Model Number of denoising steps (NFEs) 5 20 50 100 1,000 Original 27.85 29.83 29.97 30.10 30.12 Correct 29.70 30.21 30.34 30.20 30.23 Table 1: BLEU scores. The table shows that the original self-conditioning signifies the approximation error when NFE is small, revealing a training and inference empirical gap. In contrast, the performance of the corrected self-condition slightly drops when NFE is reduced from 2020 to 55, while it remains consistent for the rest NFEs. This observation highlights the importance of a training design that aligns with the inference process, which not only boosts performance but also enables more efficient inference. We conduct a more thorough theoretical analysis of the estimation gap in Appx. A.2. 3.2 Self-conditioning Perturbation We propose SCP to reduce the training-inference mismatch of self-conditioning. Intuitively, SCP simulates inference self-condition behaviors by regularizing the correct self-condition in training, thereby improving stability and reducing drift without changing the sampling procedure. The perturbed self-conditioning is obtained from the denoising output of a modified forward process, in which a perturbed version t′ z _t of t z_t is introduced: t′=αtλt0+σt1+γt2ϵt, z _t= _t _t z_0+ _t 1+ _t^2\, ε_t, (7) where ϵt∼(0,) ε_t (0,I). The factors λt _t and γt _t corrupt the signal term and inflate the effective noise level, yielding a more aggressively noised input and thus a less reliable self-conditioning estimate. We parameterize them as linear schedules, λt=(λmin−λmax)t+λmax _t=( _ - _ )t+ _ and γt=(γmin−γmax)t+γmax _t=( _ - _ )t+ _ , with monotonic variation over t. The hyperparameter choice is guided by the forward ratios αu/αt _u/ _t and σu/σt _u/ _t, so that t′ z _t matches the scale of a later state along the trajectory and approximates the conditioning statistics induced by prior estimation. Empirically, we observe that the reused estimate behaves as an approximately Gaussian perturbation around the current prediction ^θt z_θ^t, supporting the view that SCP captures the noise pattern introduced by inference-time self-conditioning. A simplified derivation and detailed distributional analysis are provided in Appx. B. We summarize the training procedure in Appendix (see Alg. 1). Conventionally, we randomly apply self-conditioning with probability 50%50\%, alternating between conditioned and unconditioned updates to keep the initial prediction meaningful. 3.3 Model-aware Noise Scaling overpic[width=424.94574pt]figure/loss_scale.pdf overpic Figure 2: Validation loss and BLEU during training under fixed, double, and linear step noise scaling. Dashed lines denote BLEU scores, color-matched to the corresponding loss curves. Method MBR NFE IWSLT14 WMT14 WMT16 DE→ EN→ DE→ EN→ RO→ EN→ Transformer 5 - 33.61 28.30 30.55 26.85 33.08 32.86 CMLM 5 - 29.41 24.34 28.71 23.22 31.13 31.26 DiffusionLM‡ 50 20 29.11 22.91 19.69 17.41 30.17 29.39 Diffomer‡ 20 20 28.01 23.31 25.30 23.80 29.37 29.20 DINOISER 50 20 31.61 25.70 29.05 24.26 31.22 31.08 DiffuSeq 10 2000 29.43 - 22.72 - - - SeqDiffuSeq 10 2000 30.45 22.12 23.93 19.76 - - AR-Diffusion 10 20 31.80 26.01 - - - - FastDiSS 10 5 32.46 25.73 29.54 24.33 31.55 30.88 FastDiSS 20 5 32.70 26.02 29.75 24.69 31.81 30.90 FastDiSS 10 20 32.81 26.29 29.47 24.50 31.89 31.37 FastDiSS 20 20 32.88 26.39 29.83 24.57 31.99 31.44 Table 2: Main results on IWSLT14, WMT14, and WMT16. The best NAR results are bold and the second-best results are underlined. ‡ indicates results reported by DBLP:journals/corr/abs-2302-10025. Other results are from their original papers. We introduce MANS to make the diffusion objective token-aware, mitigating late-stage loss saturation. The key idea is to adaptively increase noise for tokens the model already denoises confidently, while leaving uncertain tokens unchanged. This strengthens supervision at higher noise levels where few-step discretization is most brittle, and improves self-conditioning even under heavier corruption. To quantify denoising confidence, we evaluate reconstructed embedding quality ^θt z_θ^t by mapping it to the nearest token embeddings m e_m: i i =argminm=1:V‖0−m‖2, = m=1:V \ || z_0- e_m||_2, (8) j j =argminm=1:V‖^θt−m‖2. = m=1:V \ || z_θ^t- e_m||_2. (9) where i and j denote the ground-truth and reconstructed tokens, respectively. We treat i=ji=j as a high-confidence token, indicating that the denoiser can already recover it reliably at noise level t. We then define a noise scaling schedule that increases denoising difficulty for these high-confidence tokens. Specifically, at training iteration n, we rescale the effective timestep used to corrupt the token by a model-aware factor β(n)β(n): tθ=β(n)⋅tifi=j,totherwise.t_θ= casesβ(n)· t&if i=j,\\ t&otherwise. cases (10) Intuitively, correctly reconstructed tokens are pushed to a higher-noise regime so the model continues to receive informative gradients, while incorrectly reconstructed tokens remain at the original difficulty to avoid destabilizing learning. While β(⋅)β(·) can in principle take many forms, our ablations indicate that progressively increasing the noise is crucial to overcoming loss saturation. As shown in Fig. 2, a constant doubling of noise (β=2.0β=2.0) yields limited gains and does not consistently break saturation, whereas a Linear schedule continues to improve both validation loss and BLEU even in late training. Accordingly, we instantiate β(⋅)β(·) as a linear stepping schedule, where the scaling factor increases after predefined iteration milestones. This ensures that once the model has sufficiently optimized the current noise regime, it is presented with a systematically harder denoising target, keeping the training signal non-trivial throughout optimization. Qualitative examples illustrating high and low confidence tokens are provided in Appx. G. 4 Experiments 4.1 Experimental Settings Datasets. Following prior works DBLP:conf/iclr/GongLF0K23, we evaluate Machine Translation on IWSLT14 (En–De/De–En) DBLP:conf/iwslt/CettoloNSBF14, WMT14 (En-De/De-En), and WMT16 (En-Ro/Ro-En) DBLP:conf/wmt/BojarBFHKLMPPSS14; Summarization on Gigaword DBLP:conf/emnlp/NarayanCL18; Question Paraphrase on QQP; and Text Simplification on Wiki-Auto. Evaluation Metrics. We report SacreBLEU for Machine Translation DBLP:journals/corr/abs-2302-10025; DBLP:conf/iclr/GongLF0K23; ROUGE-1/2/L for Summarization DBLP:conf/emnlp/QiYGLDCZ020; and for Question Paraphrasing and Text Simplification, we follow the evaluation setup of DBLP:conf/iclr/GongLF0K23, using sentence-level BLEU, ROUGE-L, and BERTScore DBLP:conf/iclr/ZhangKWWA20 to assess quality, along with sentence-level self-BLEU DBLP:conf/sigir/ZhuLZGZWY18 to measure diversity. Method MBR NFE QQP Wiki-Auto BLEU↑ Rouge-L↑ BertScore↑ BLEU↑ Rouge-L↑ BertScore↑ Transformer‡ - - 27.22 57.48 83.81 26.93 49.07 73.81 GPT2-large FT‡ - - 20.59 54.15 83.63 26.93 51.11 78.82 CMLM× - 10 21.78 56.12 - 35.26 58.46 81.83 LevT‡ - - 22.68 57.95 83.44 20.52 44.02 72.54 Difformer∗ 10 20 27.95 59.24 82.97 34.78 54.55 78.86 DINOISER† 10 20 19.49 53.16 80.36 23.88 48.21 67.87 DiffuSeq 10 2000 24.13 58.80 83.65 36.22 58.49 81.26 SeqDiffuSeq 10 2000 24.34 - 84.00 37.12 - 82.14 DiffuSeq-v2∗DiffuSeq-v2^* 10 10 23.07 58.35 82.36 26.60 51.33 77.04 FlowSeq∗FlowSeq^* 10 1 14.30 46.10 66.90 29.02 53.74 72.46 DLM-One 10 1 22.13 57.41 82.97 36.30 58.39 80.84 FastDiSS 10 1 27.16 58.10 81.16 38.20 57.66 80.35 FastDiSS 10 2 27.94 58.47 81.81 40.23 59.10 81.60 FastDiSS 10 5 28.88 59.34 82.58 40.90 59.64 82.16 FastDiSS 10 20 28.32 58.88 82.62 40.81 59.64 82.17 Table 3: Main results on QQP and Wiki-Auto. The best NAR results are bold and the second-best results are underlined. ‡ , ×, and † indicate results reported by DBLP:conf/iclr/GongLF0K23, DBLP:conf/acl/TangWZLCZ23, and DBLP:conf/nips/ChuangHLGLCL24, respectively. ∗* indicates reproduced results. Other results are from their original papers. Baselines. We consider three baseline groups: (1) Autoregressive models: Transformer DBLP:conf/nips/VaswaniSPUJGKP17, GRU with attention, and fine-tuned GPT2-Large; (2) Non-autoregressive model: CMLM DBLP:conf/emnlp/GhazvininejadLL19 and LevT DBLP:conf/nips/GuWZ19; and (3) Diffusion language models: DiffusionLM DBLP:conf/nips/LiTGLH22, Difformer DBLP:conf/naacl/GaoG0Z0X24, DINOISER DBLP:journals/corr/abs-2302-10025, DiffuSeq DBLP:conf/iclr/GongLF0K23, SeqDiffuSeq DBLP:conf/naacl/YuanYTHH24, and AR-Diffusion DBLP:conf/nips/WuFLZGS0LWGDC23. For Summarization, we include LSTM DBLP:journals/tnn/GreffSKSS17 and NAG-BERT DBLP:conf/eacl/SuCWVBLC21. For Question Paraphrase, we include Discrete Diffusion with RDM DBLP:journals/corr/abs-2302-05737. We also include few-step generation benchmarks, DiffuSeq-v2 DBLP:conf/emnlp/GongLF0K23, FlowSeq DBLP:conf/eacl/HuWAMFOS24 and DLM-One DBLP:journals/corr/abs-2506-00290. Training and Inference. During training, we adopt sqrt noise schedule DBLP:conf/nips/LiTGLH22 with T=2000T=2000 diffusion steps. The anchor points (λmin,λmax)( _ , _ ) and (γmin,γmin)( _ , _ ) are (0.9,0.95)(0.9,0.95) and (0.15,0.35)(0.15,0.35), respectively. MANS is randomly applied with 50%50\% probability to fasten training. Our implementation is based on Difformer, with the same sampling configurations at NFE∈2,5,20NFE∈\2,5,20\. For every task, we construct the vocabulary with Byte Pair Encoding DBLP:conf/emnlp/KudoR18. We also apply Minimum Bayes Risk (MBR) decoding DBLP:conf/naacl/KumarB04 following previous works DBLP:conf/nips/LiTGLH22; DBLP:conf/iclr/GongLF0K23. Further details are described in Appx. E and F. 4.2 Main Results Overall Performance. Tabs. 2 and 3 summarize results across tasks and datasets. Overall, FastDiSS improves few-step diffusion generation, outperforming both Diffusion and Non-autoregressive baselines on most settings (varying MBR and NFEs), while approaching Autoregressive performance. On WMT14, the 55-step model even surpasses 2020-step sampling, indicating that SCP can effectively close the gap between few and many inference. In terms of efficiency, FastDiSS is 4×4× faster than DINOISER, Difformer, and up to 400×400× faster than long-trajectory methods such as DiffuSeq and SeqDiffuSeq. Compared to one-step baselines, FastDiSS remains competitive without relying on distillation (DLM-One) or flow-matching (FlowSeq) objectives (Tab. 3). Additional results on Text Summarization are reported in Appx. D. Sampling Speed. We analyze the quality–speed trade-off on QQP by comparing FastDiSS with DiffuSeq-v2 using its original self-conditioning mechanism DBLP:conf/emnlp/GongLF0K23. Fig. 3 shows that FastDiSS consistently achieves higher quality at low NFEs, highlighting its advantage in the few-step regime. The margin narrows at medium NFEs, where DiffuSeq-v2 slightly overtakes FastDiSS, suggesting that the benefit of SCP diminishes once discretization error becomes small. Sampling Diversity. We evaluate diversity on QQP using BLEU and self-BLEU. Fig. 4 shows the trade-off between diversity and quality. Fig. 4 shows that FastDiSS-22NFE matches the strongest baseline, Difformer-2020NFE, with 10× smaller bugdet. Increasing NFEs improves quality at the expected cost of latency. Notably, FastDiSS-55NFE achieves substantially higher BLEU while maintaining similar self-BLEU, indicating that quality gains do not come at the expense of diversity. Figure 3: Generation speed and quality with different NFE. The speed is averaged over 3 runs. Figure 4: Diversity and quality comparison. 4.3 Ablation Studies Effect of Each Component. We quantify the contribution of SCP and MANS in Tab. 4. The evaluation is conducted on IWSLT14 De-En using BLEU. The results show that both components provide consistent gains over the base model. MANS is particularly effective at small NFEs, suggesting it improves prediction at large steps, reducing self-conditioning errors. In contrast, SCP mainly improves inference by reducing the training-inference gap, which is most visible in the few-step regime. Combining SCP and MANS yields the strongest performance and significantly narrows the gap between 5-step and 20-step sampling. SCP MANS NFE=5NFE=5 NFE=20NFE=20 Δ × × 27.98 29.78 1.80 ✓ × 29.64 30.36 0.72 × ✓ 30.77 31.49 0.72 ✓ ✓ 31.17 31.66 0.49 Table 4: Ablation study. Effect of Varying NFEs. We vary NFEs on IWSLT14 De-En and compare FastDiSS against the original codebase, Difformer. Fig. 5 shows that FastDiSS is markedly more effective under few-step inference: it surpasses the 2020-step baseline within 33 sampling steps, corresponding to approximately 7×7× speedup. Moreover, performance converges after roughly 77 steps, reaching a higher BLEU than the 2020-step baseline. Figure 5: Effect of Number of Denoising Steps. Effect of γt _t and λt _t. We study sensitivity to γt _t and λt _t in SCP (Eq. 7) on QQP. Tab. 5 compares linear-time schedules against fixed variants where λt _t and γt _t are held constant. The best results are obtained with settings derived from the 20-step ratios, where γt _t ranges from 0.900.90 to 0.950.95 and λt _t ranges from 0.150.15 to 0.350.35. We use this configuration in the remaining experiments. Steps 5 20 100 Fixed λt _t 0.60 - 0.85 0.90 - 0.95 0.98 - 0.99 0.85 γt _t 0.25 - 0.90 0.15 - 0.35 0.05 - 0.15 0.50 BLEU 25.48 26.35 25.70 25.84 ROUGE-L 57.29 57.47 56.82 57.18 Table 5: Effect of γt _t and λt _t. Effect of Noise Schedulers. Finally, we compare FastDiSS under standard diffusion noise schedules, including linear DBLP:conf/nips/HoJA20 and cosine DBLP:conf/icml/NicholD21, on QQP. Tab. 6 shows that the linear schedule is generally less sensitive to the choice of NFE. FastDiSS consistently yields improvements in the few-step regime across schedulers, confirming that our gains are not tied to a specific base schedule. Noise Schedule NFE=2 NFE=5 NFE=20 linear Orig. 26.50 27.17 27.54 Ours 26.84 27.52 27.54 cosine Orig. 25.43 27.05 27.57 Ours 27.32 27.77 27.94 Table 6: Effect of Noise Schedulers. 4.4 Extension to Large-scale Benchmark We evaluate SCP adaptability to reasoning benchmark GSM8K DBLP:journals/corr/abs-2110-14168 and discrete setting in Tabs. 7 and 8. Reasoning benchmark. We adopt DoT DBLP:conf/nips/YeGCZGSWJLBK24, using Plaid-1B DBLP:conf/nips/GulrajaniH23 as the backbone. We introduce SCP as a lightweight modification following Plaid training recipe. Tab. 7 shows that SCP consistently improves accuracy under both standard diffusion inference and Chain-of-Thought (CoT) inference. More importantly, when reducing NFEs, SCP substantially mitigates the drop in accuracy, indicating improved robustness in the low-compute regime. Qualitative examples comparing Plaid with and without SCP are provided in Appx. H. NFE DoT (Plaid) DoTmp DoT^mp (CoT) Normal SCP Normal SCP 8 35.4 ± 0.5 38.2±0.738.2± 0.7 30.9 ± 0.6 35.1±1.035.1± 1.0 16 39.0 ± 0.5 41.1±0.741.1± 0.7 36.1 ± 0.7 39.8±0.839.8± 0.8 32 39.9 ± 0.3 43.0±1.143.0± 1.1 37.2 ± 0.7 40.4±0.940.4± 0.9 64 41.3 ± 0.2 43.7±0.243.7± 0.2 36.6 ± 0.6 40.8±0.640.8± 0.6 Table 7: Comparison of Accuracy on GSM8K benchmark. We experimented with two settings, DoT with Plaid inference and DoTmp DoT^mp with CoT reasoning. The Accuracy is averaged over 5 runs. Discrete Diffusion Model Extension. MDLM, the masking diffusion framework behind LLaDA DBLP:journals/corr/abs-2502-09992, operates via iterative masking and unmasking. In this setting, SCP corresponds to increasing the masking rate during training, producing a more corrupted conditioning context that better matches few-step inference. As shown in Tab. 8, SCP improves robustness at small NFEs compared to self-conditioning, and yields consistent gains over the original model as NFEs increase. NFE Gen Perplexity (↓ ) None SC SCP 2 3178.74 3250.22 2873.67 5 1144.42 1433.79 1215.41 20 243.81 254.72 244.40 50 143.82 133.79 130.52 100 113.34 97.88 95.48 1000 55.23 45.95 45.21 5000 32.12 27.78 28.14 Table 8: Main results on MDLM. The best Generation Perplexity results are bold. 5 Related Works Non-autoregressive Language Generation. Non-autoregressive models were early introduced to reduce generation latency by predicting tokens in parallel DBLP:journals/corr/abs-1711-02281. To improve accuracy, later work incorporated iterative refinement and editing-style decoding DBLP:conf/nips/GuWZ19; DBLP:conf/emnlp/GhazvininejadLL19. Despite these advances, the conditional independence assumption in many NAR formulations leads to a single mode selection issue. Prior efforts mitigate this issue via structured prediction DBLP:conf/naacl/ZhangW0G0QL22; DBLP:conf/acl/RanLLZ20; DBLP:conf/icml/HuangTZLH22, data augmentation and selection (e.g., reference rephrasing) DBLP:conf/naacl/ShaoW022; DBLP:conf/aaai/ShaoZ0023, and distillation to transfer knowledge from autoregressive teachers DBLP:journals/corr/abs-2112-11640; DBLP:journals/corr/abs-2205-11162; DBLP:conf/aaai/LiuBZH23. Diffusion Models for Language Generation. Text diffusion models can be broadly categorized into discrete and continuous formulations. Discrete diffusion defines Markov transitions directly over token space. Early approaches such as D3PM DBLP:conf/nips/AustinJHTB21 and Analog-bits DBLP:conf/iclr/ChenZH23 rely on absorbing or uniform transitions, while more recent methods adopt masking-style transitions that scale more naturally to language, including MDLM DBLP:conf/nips/SahooASGMCRK24 and RDM DBLP:journals/corr/abs-2302-05737. These advances have enabled large-scale diffusion LLMs such as LLaDA DBLP:journals/corr/abs-2502-09992, Dream-7B DBLP:journals/corr/abs-2508-15487, and Block Diffusion DBLP:conf/iclr/ArriolaGCYQHSK25. In contrast, continuous diffusion maps tokens into a continuous embedding space and applies Gaussian processes, enabling denoising in latent space. Representative models include DiffusionLM DBLP:conf/nips/LiTGLH22, Difformer DBLP:conf/naacl/GaoG0Z0X24, DINOISER DBLP:journals/corr/abs-2302-10025, SeqDiffuSeq DBLP:conf/naacl/YuanYTHH24, AR-Diffusion DBLP:conf/nips/WuFLZGS0LWGDC23, and DiffuSeq DBLP:conf/iclr/GongLF0K23, spanning encoder-decoder and decoder-only designs. Several works further tailor diffusion to language by modifying the noise schedule or corruption process, including Masked-Diffuse DBLP:conf/emnlp/ChenZ0SY23 and Meta-DiffuBB DBLP:conf/nips/ChuangHLGLCL24. Our method is developed in the continuous latent setting to explore the full potential of continuous modeling. Accelerating the Diffusion Language Model. Widely adopted methods, such as DDIM DBLP:conf/iclr/SongME21, have demonstrated success in prior text generation works. Advances in ODE solvers DBLP:conf/nips/0011ZB0L022; DBLP:journals/ijautcomp/LuZBCLZ25 have made great success in enhancing the efficiency of the reverse process in DiffuSeq-v2 DBLP:conf/emnlp/GongLF0K23. DBLP:conf/acl/TangWZLCZ23 also addresses training-inference discrepancies to enhance generation speed. Recently, FlowSeq DBLP:conf/eacl/HuWAMFOS24 adapts flow matching for sequence modeling, while DLM-One DBLP:journals/corr/abs-2506-00290 distills the teacher model, DiffuSeq, to learn a one-step generator. Our work advances this line by promoting fast, adaptable, and few-step generation, aiming to make diffusion-based language models more viable for real-world applications. 6 Conclusion We introduced FastDiSS, a novel training framework for diffusion language models, combining Self-conditioning Perturbation (SCP) to align self-conditioning under few-step discretization and Model-aware Noise Scaling (MANS) to increase noise on high-confidence tokens for more informative training. Experiments on six benchmarks demonstrate that FastDiSS enables more effective training, narrows the gap between few-step and many-step decoding, and achieves significant improvements in both efficiency and generation quality. While our method advances the practicality of fast inference with diffusion models, future work will focus on further refining self-conditioning and closing the gap with autoregressive approaches. Limitation Since we aim to align training to inference, the proposed techniques are constrained by the goodness of the previous self-conditioning prediction. It would be more promising if we approached the problem from the opposite direction, as refining the prediction to be more accurate would significantly boost the performance. Next, our current noise scaling strategy relies on predefined values at each training phase, which may be suboptimal when earlier scaling stages are insufficiently trained. A more adaptive scaling function could further enhance performance. Broader Impact This work advances diffusion-based language modeling by proposing mechanisms that improve training efficiency and training-inference alignment. By reducing exposure bias and accelerating convergence, our approach contributes to the development of faster, more robust text generation systems. These improvements can benefit a wide range of natural language applications, including machine translation, summarization, and question answering, where efficiency and quality are critical. Despite these potential benefits, diffusion-based models, like other large generative models, may still propagate social biases or produce harmful or misleading content when trained on unfiltered data. We emphasize the importance of responsible deployment, including careful dataset selection, bias evaluation, and human oversight. References M. Arriola, A. Gokaslan, J. T. Chiu, Z. Yang, Z. Qi, J. Han, S. S. Sahoo, and V. Kuleshov (2025) Block diffusion: interpolating between autoregressive and diffusion language models. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §5. J. Austin, D. D. Johnson, J. Ho, D. Tarlow, and R. van den Berg (2021) Structured denoising diffusion models in discrete state-spaces. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), p. 17981–17993. Cited by: §5. F. Bao, C. Li, J. Sun, J. Zhu, and B. Zhang (2022) Estimating the optimal covariance with imperfect mean in diffusion probabilistic models. In Proceedings of the International Conference on Machine Learning (ICML), p. 1555–1584. Cited by: §3.1. O. Bojar, C. Buck, C. Federmann, B. Haddow, P. Koehn, J. Leveling, C. Monz, P. Pecina, M. Post, H. Saint-Amand, R. Soricut, L. Specia, and A. Tamchyna (2014) Findings of the 2014 workshop on statistical machine translation. In Proceedings of the Workshop on Statistical Machine Translation (WMT@ACL), p. 12–58. Cited by: §4.1. T. B. Brown, B. Mann, N. Ryder, M. Subbiah, J. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. M. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner, S. McCandlish, A. Radford, I. Sutskever, and D. Amodei (2020) Language models are few-shot learners. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: §1. M. Cettolo, J. Niehues, S. Stüker, L. Bentivogli, and M. Federico (2014) Report on the 11th IWSLT evaluation campaign. In Proceedings of the International Workshop on Spoken Language Translation: Evaluation Campaign (IWSLT), p. 2–17. Cited by: §4.1. J. Chen, A. Zhang, M. Li, A. Smola, and D. Yang (2023a) A cheaper and better diffusion language model with soft-masked noise. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), H. Bouamor, J. Pino, and K. Bali (Eds.), p. 4765–4775. Cited by: §5. T. Chen, S. Zhang, and M. Zhou (2025) DLM-one: diffusion language models for one-step sequence generation. CoRR abs/2506.00290. Cited by: §4.1, §5. T. Chen, R. Zhang, and G. E. Hinton (2023b) Analog bits: generating discrete data using diffusion models with self-conditioning. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §1, §5. Y. Chuang, H. Hsu, K. Lin, C. Gu, L. Z. Li, R. Chang, and H. Lee (2024) Meta-DiffuB: A contextualized sequence-to-sequence text diffusion model with meta-exploration. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: §C.2, §2.1, Table 3, §5. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman (2021) Training verifiers to solve math word problems. CoRR abs/2110.14168. Cited by: §4.4. G. Daras, Y. Dagan, A. Dimakis, and C. Daskalakis (2023) Consistent diffusion models: mitigating sampling drift by learning to be consistent. In Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, Cited by: §1. Z. Gao, J. Guo, X. Tan, Y. Zhu, F. Zhang, J. Bian, and L. Xu (2024) Empowering diffusion models on the embedding space for text generation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), p. 4664–4683. Cited by: Table 10, §1, §4.1, §5. M. Ghazvininejad, O. Levy, Y. Liu, and L. Zettlemoyer (2019) Mask-predict: parallel decoding of conditional masked language models. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), p. 6111–6120. Cited by: §4.1, §5. S. Gong, M. Li, J. Feng, Z. Wu, and L. Kong (2023a) DiffuSeq-v2: bridging discrete and continuous text spaces for accelerated seq2seq diffusion models. In Findings of the EMNLP, p. 9868–9875. Cited by: §4.1, §4.2, §5. S. Gong, M. Li, J. Feng, Z. Wu, and L. Kong (2023b) DiffuSeq: sequence to sequence text generation with diffusion models. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: Appendix F, §1, §1, §4.1, §4.1, §4.1, §4.1, Table 3, §5. K. Greff, R. K. Srivastava, J. Koutník, B. R. Steunebrink, and J. Schmidhuber (2017) LSTM: A search space odyssey. IEEE Trans. Neural Networks Learn. Syst. 28 (10), p. 2222–2232. Cited by: §4.1. J. Gu, J. Bradbury, C. Xiong, V. O. K. Li, and R. Socher (2017) Non-autoregressive neural machine translation. CoRR abs/1711.02281. Cited by: §5. J. Gu and X. Kong (2021) Fully non-autoregressive neural machine translation: tricks of the trade. In Findings of ACL, p. 120–133. Cited by: Appendix E. J. Gu, C. Wang, and J. Zhao (2019) Levenshtein transformer. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), p. 11179–11189. Cited by: §4.1, §5. I. Gulrajani and T. B. Hashimoto (2023) Likelihood-based diffusion language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Cited by: §1, §4.4. J. Guo, M. Wang, D. Wei, H. Shang, Y. Wang, Z. Li, Z. Yu, Z. Wu, Y. Chen, C. Su, M. Zhang, L. Lei, S. Tao, and H. Yang (2021) Self-distillation mixup training for non-autoregressive neural machine translation. CoRR abs/2112.11640. Cited by: §5. J. Ho, A. Jain, and P. Abbeel (2020) Denoising diffusion probabilistic models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: §4.3. V. T. Hu, D. Wu, Y. M. Asano, P. Mettes, B. Fernando, B. Ommer, and C. Snoek (2024) Flow matching for conditional text generation in a few sampling steps. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2024 - Volume 2: Short Papers, St. Julian’s, Malta, March 17-22, 2024, p. 380–392. Cited by: §4.1, §5. F. Huang, T. Tao, H. Zhou, L. Li, and M. Huang (2022) On the learning of non-autoregressive transformers. In Proceedings of the International Conference on Machine Learning (ICML), p. 9356–9376. Cited by: §5. T. Kudo and J. Richardson (2018) SentencePiece: A simple and language independent subword tokenizer and detokenizer for neural text processing. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 66–71. Cited by: §4.1. S. Kumar and W. J. Byrne (2004) Minimum bayes-risk decoding for statistical machine translation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), p. 169–176. Cited by: Appendix E, §4.1. X. L. Li, J. Thickstun, I. Gulrajani, P. Liang, and T. B. Hashimoto (2022) Diffusion-LM improves controllable text generation. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: §2.2, §4.1, §4.1, §5. M. Liu, Y. Bao, C. Zhao, and S. Huang (2023) Selective knowledge distillation for non-autoregressive neural machine translation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), p. 13246–13254. Cited by: §5. C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu (2022) DPM-solver: A fast ODE solver for diffusion probabilistic model sampling in around 10 steps. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: §5. C. Lu, Y. Zhou, F. Bao, J. Chen, C. Li, and J. Zhu (2025) DPM-solver++: fast solver for guided sampling of diffusion probabilistic models. Mach. Intell. Res. 22 (4), p. 730–751. Cited by: §5. S. Narayan, S. B. Cohen, and M. Lapata (2018) Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the Conference on Empirical Methods in Natural Language Processing EMNLP, p. 1797–1807. Cited by: §4.1. A. Q. Nichol and P. Dhariwal (2021) Improved denoising diffusion probabilistic models. In Proceedings of the International Conference on Machine Learning (ICML), p. 8162–8171. Cited by: §4.3. S. Nie, F. Zhu, Z. You, X. Zhang, J. Ou, J. Hu, J. Zhou, Y. Lin, J. Wen, and C. Li (2025) Large language diffusion models. CoRR abs/2502.09992. Cited by: §1, §4.4, §5. M. Ning, M. Li, J. Su, A. A. Salah, and I. Ö. Ertugrul (2024) Elucidating the exposure bias in diffusion models. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §1, §3.1. M. Ning, E. Sangineto, A. Porrello, S. Calderara, and R. Cucchiara (2023) Input perturbation reduces exposure bias in diffusion models. In Proceedings of the International Conference on Machine Learning (ICML), p. 26245–26265. Cited by: §A.2. K. Papineni, S. Roukos, T. Ward, and W. Zhu (2002) Bleu: a method for automatic evaluation of machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), p. 311–318. Cited by: §1. M. Prabhudesai, M. Wu, A. Zadeh, K. Fragkiadaki, and D. Pathak (2025) Diffusion beats autoregressive in data-constrained settings. CoRR abs/2507.15857. Cited by: §1. W. Qi, Y. Gong, Y. Shen, J. Jiao, Y. Yan, H. Li, R. Zhang, W. Chen, and N. Duan (2022) A self-paced mixed distillation method for non-autoregressive generation. CoRR abs/2205.11162. Cited by: §5. W. Qi, Y. Yan, Y. Gong, D. Liu, N. Duan, J. Chen, R. Zhang, and M. Zhou (2020) ProphetNet: predicting future N-gram for sequence-to-sequence pre-training. In Findings of EMNLP, p. 2401–2410. Cited by: §4.1. A. Radford, K. Narasimhan, T. Salimans, I. Sutskever, et al. (2018) Improving language understanding by generative pre-training. Cited by: §1. A. Radford, J. Wu, R. Child, D. Luan, D. Amodei, I. Sutskever, et al. (2019) Language models are unsupervised multitask learners. OpenAI Blog 1 (8), p. 9. Cited by: §1. Q. Ran, Y. Lin, P. Li, and J. Zhou (2020) Learning to recover from multi-modality errors for non-autoregressive neural machine translation. In Proceedings of the Annual Meeting of the Association for Computational Linguistics (ACL), p. 3059–3069. Cited by: §5. S. S. Sahoo, M. Arriola, Y. Schiff, A. Gokaslan, E. Marroquin, J. T. Chiu, A. Rush, and V. Kuleshov (2024) Simple and effective masked diffusion language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: §5. F. Schmidt (2019) Generalization in generation: A closer look at exposure bias. In Proceedings of the Workshop on Neural Generation and Translation (GNT@EMNLP-IJCNLP), p. 157–167. Cited by: §1. C. Shao, X. Wu, and Y. Feng (2022) One reference is not enough: diverse distillation with reference selection for non-autoregressive translation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), p. 3779–3791. Cited by: §5. C. Shao, J. Zhang, J. Zhou, and Y. Feng (2023) Rephrasing the reference for non-autoregressive machine translation. In Proceedings of the AAAI Conference on Artificial Intelligence (AAAI), p. 13538–13546. Cited by: §5. S. S. Shapiro and M. B. Wilk (1965) An analysis of variance test for normality (complete samples). Biometrika 52 (3-4), p. 591–611. Cited by: §B.2. J. Song, C. Meng, and S. Ermon (2021a) Denoising diffusion implicit models. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §5. Y. Song, P. Dhariwal, M. Chen, and I. Sutskever (2023) Consistency models. In Proceedings of the International Conference on Machine Learning (ICML), p. 32211–32252. Cited by: §A.2. Y. Song, J. Sohl-Dickstein, D. P. Kingma, A. Kumar, S. Ermon, and B. Poole (2021b) Score-based generative modeling through stochastic differential equations. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §2.1. Y. Su, D. Cai, Y. Wang, D. Vandyke, S. Baker, P. Li, and N. Collier (2021) Non-autoregressive text generation with pre-trained language models. In Proceedings of the Conference of the European Chapter of the Association for Computational Linguistics: Main Volume (EACL), p. 234–243. Cited by: §4.1. Z. Tang, P. Wang, K. Zhou, J. Li, Z. Cao, and M. Zhang (2023) Can diffusion model achieve better performance in text generation? bridging the gap between training and inference!. In Findings of the ACL, p. 11359–11386. Cited by: §C.1, Table 3, §5. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, L. Kaiser, and I. Polosukhin (2017) Attention is all you need. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), p. 5998–6008. Cited by: Appendix E, §4.1. T. Wu, Z. Fan, X. Liu, H. Zheng, Y. Gong, Y. Shen, J. Jiao, J. Li, Z. Wei, J. Guo, N. Duan, and W. Chen (2023) AR-Diffusion: auto-regressive diffusion model for text generation. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: Appendix F, §4.1, §5. J. Ye, S. Gong, L. Chen, L. Zheng, J. Gao, H. Shi, C. Wu, X. Jiang, Z. Li, W. Bi, and L. Kong (2024) Diffusion of thought: chain-of-thought reasoning in diffusion language models. In Proceedings of the Annual Conference on Neural Information Processing Systems (NeurIPS), Cited by: §1, §4.4. J. Ye, Z. Xie, L. Zheng, J. Gao, Z. Wu, X. Jiang, Z. Li, and L. Kong (2025) Dream 7b: diffusion large language models. CoRR abs/2508.15487. Cited by: §1, §5. J. Ye, Z. Zheng, Y. Bao, L. Qian, and M. Wang (2023) DINOISER: diffused conditional sequence learning by manipulating noises. CoRR abs/2302.10025. Cited by: §C.2, Table 2, §4.1, §4.1, §5. H. Yuan, Z. Yuan, C. Tan, F. Huang, and S. Huang (2024) Text diffusion model with encoder-decoder transformers for sequence-to-sequence generation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), p. 22–39. Cited by: §C.2, §1, §4.1, §5. K. Zhang, R. Wang, X. Tan, J. Guo, Y. Ren, T. Qin, and T. Liu (2022) A study of syntactic multi-modality in non-autoregressive machine translation. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (NAACL), p. 1747–1757. Cited by: §5. T. Zhang, V. Kishore, F. Wu, K. Q. Weinberger, and Y. Artzi (2020) BERTScore: evaluating text generation with BERT. In Proceedings of the International Conference on Learning Representations (ICLR), Cited by: §4.1. L. Zheng, J. Yuan, L. Yu, and L. Kong (2023) A reparameterized discrete diffusion model for text generation. CoRR abs/2302.05737. Cited by: §4.1, §5. Y. Zhu, S. Lu, L. Zheng, J. Guo, W. Zhang, J. Wang, and Y. Yu (2018) Texygen: A benchmarking platform for text generation models. In Proceedings of the ACM SIGIR International Conference on Research & Development in Information Retrieval (SIGIR), p. 1097–1100. Cited by: §4.1. Algorithm 1 Training with SCP Input: Text sequence x, denoising network DθD_θ 1:while not converged do 2: 0∼qθ(0|) z_0 q_θ( z_0| x) 3: t∼(ϵ,1)\ t (ε,1), ϵ∼(0,) ε (0,I) 4: t′=αtλt0+σt1+γt2ϵ z_t = _t _t z_0+ _t 1+ _t^2 ε 5: ^θt←Dθ(t′,0) z_θ^t← D_θ( z_t ,0) 6: r∼(0,1)r (0,1) 7: θt←Dθ(t′,^θt) z_θ^t← D_θ( z_t , z_θ^t) if r<0.5r<0.5 else ^θt z_θ^t 8: ℒtotal=‖θt−0‖2−logpθ(|0)L_total=|| z_θ^t- z_0||^2- p_θ( x| z_0) 9: θ←θ−∇θℒtotalθ←θ- _θL_total 10:end while Appendix A Theoretical Details A.1 Derivation Of The Posterior Distribution Since the forward process is a Markov chain, for t>st>s, we have q(s,t|0)=q(s|0)q(t|s)q( z_s, z_t| z_0)=q( z_s| z_0)q( z_t| z_s). Following the Bayes rule, the posterior equals to q(t|s)q(s|0)/q(t|0)q( z_t| z_s)q( z_s| z_0)/q( z_t| z_0). Given that every term is a Gaussian likelihood (Eq. 1 and 2), we plug this into the posterior, which yields: q(s|t,0) q( z_s| z_t, z_0) =(s;~(t,0),σ~t2) =N( z_s; μ( z_t, z_0), σ_t^2 I) (11) where~(t,0) \ \ μ( z_t, z_0) =αtαsσs2σt2t+αsσt|s2σt20 = _t _s _s^2 _t^2 z_t+ _s _t|s^2 _t^2 z_0 (12) andσ~t \ \ \ σ_t =σsσtσt|s. = _s _t _t|s. (13) A.2 Analysis On The Estimation Gap In contrast to prior studies, which suggested that error could accumulate across steps DBLP:conf/icml/NingSPCC23, we showed that, given a sufficiently large number of denoising steps, the discretization errors between consecutive steps become small enough for the model to estimate the correct self-condition. Specifically, we first denote ti+1t_i+1 and tit_i as two selected consecutive steps used during generation, and ¯θtiti+1 z_θ^t_it_i+1 as the estimation with the condition is the denoising output from the previous step ti+1t_i+1. Then, we state the following theorem: Theorem 1 Let t0,t1,…,tn∈[ϵ,1]t_0,t_1,...,t_n∈[ε,1] such that t0<t1<…<tn=1t_0<t_1<...<t_n=1; Δt:=maxi∈[1,n−1]|ti+1−ti| t:= _i∈[1,n-1]\|t_i+1-t_i|\. Assume DθD_θ satisfies the Lipschitz condition: there exists K>0K>0 such that for all t∈[ϵ,1]t∈[ε,1], x, and y, we have ‖Dθ(t,)−Dθ(t,)‖2≤K‖−‖2||D_θ( z_t, x)-D_θ( z_t, y)||_2≤ K|| x- y||_2. Assume further that for all i∈[0,n−1]i∈[0,n-1], the denoising estimation at ti+1t_i+1 has local error uniformly bounded by ((ti+1−ti)p+1)O((t_i+1-t_i)^p+1) with p≥1p≥ 1. Then, the supremum of local error expectation: sup i∼[0,n−1][‖Dθ(ti,¯θti+1ti+2)−Dθ(ti,^θti)‖] i [0,n-1]E [\|D_θ( z_t_i, z_θ^t_i+1t_i+2)-D_θ( z_t_i, z_θ^t_i)\| ] =((Δt)p). =O(( t)^p). (14) Proof. Because Dθ(ti,⋅)D_θ( z_t_i,·) is K-Lipschitz, we have i∼[0,n−1]‖Dθ(ti,¯θti+1ti+2)−Dθ(ti,^θti)‖ i [0,n-1]E\|D_θ( z_t_i, z_θ^t_i+1t_i+2)-D_θ( z_t_i, z_θ^t_i)\| ≤Ki∼[0,n−1]‖¯θti+1ti+2−^θti‖ ≤ K i [0,n-1]E\| z_θ^t_i+1t_i+2- z_θ^t_i\| Furthermore, from our assumption that the local error is bounded by ((ti+1−ti)p+1)O((t_i+1-t_i)^p+1), we have the upper bound of the total local error is Ki∼[0,n−1]‖¯θti+1ti+2−^θti‖ K i [0,n-1]E\| z_θ^t_i+1t_i+2- z_θ^t_i\| ≤(i)Kn⋅∑i=0n−1((ti+1−ti)p+1) (i)≤ Kn· _i=0^n-1O((t_i+1-t_i)^p+1) ≤∑i=0n−1((ti+1−ti)p+1) ≤ _i=0^n-1O((t_i+1-t_i)^p+1) =∑i=0n−1(ti+1−ti)((ti+1−ti)p) = _i=0^n-1(t_i+1-t_i)O((t_i+1-t_i)^p) ≤((Δt)p)∑i=0n−1(ti+1−ti) (( t)^p) _i=0^n-1(t_i+1-t_i) =((Δt)p)(tn−t0) =O(( t)^p)(t_n-t_0) ≤((Δt)p)(1−ϵ) (( t)^p)(1-ε) ≤((Δt)p) (( t)^p) where (i)(i) holds due to the uniform sampling distribution of t. Our proof builds on the error bounds for the ODE Solver in Consistency Models DBLP:conf/icml/SongD0S23, but it is different since we provide the total local error bounds rather than targeting the empirical approximation bounds. This theorem suggests that the self-condition estimation can become arbitrarily accurate, as long as the number of sampling steps is large enough. In Tab. 9, we practically demonstrate this estimation error through different numbers of denoising steps. NFEs 5 20 50 100 Sup E 0.047 0.008 0.009 0.010 Table 9: Estimation Error. Appendix B Self-conditioning Error Formulation This section provides empirical and analytical support for SCP. We show that the perturbed forward sample t′ z _t used in training induces self-conditioning statistics that closely match those observed at inference, and we motivate the linear parameterization of λt _t and γt _t. overpic[width=433.62pt]figure/var_sched.pdf (39.0,23.0) [width=130.08731pt]figure/var_zoom_in.pdf to0pt to0pt 0.0pt to0.0pt @beginscope @invoke pgfstrokecolorrgb0,0,0 @color@rgb@stroke000 @invoke @color@rgb@fill000 @invoke @setlinewidth @invoke to0.0pt @beginscope @invoke @beginscope @invoke @setlinewidth @invoke @moveto100.29579pt21.39644pt @lineto68.85562pt40.11848pt @stroke @invoke @invoke @endscope @beginscope @invoke @setlinewidth @invoke @moveto140.27188pt21.39644pt @lineto120.55432pt40.17535pt @stroke @invoke @invoke @endscope @invoke @endscope to0.0pt @discardpath @invoke @endscope (29.0,25.0)1.006 (29.0,39.0)1.031 overpic Figure 6: Mean (top) and variance (bottom) ratio differences measured at different denoising steps. We manually select hyperparameters of the scaling terms to fit 20-step sampling. B.1 Approximate Gaussian Distribution Between Consecutive Estimates Following Sec. 3.1, we train a standard continuous diffusion language model on IWSLT14 De-En and run the original 2020-step denoising procedure on the validation set. At each step, we store the previous-step reused estimate ¯θtu z_θ^tu and compute the corrected self-conditioning estimate for the current step, denoted ^θs z_θ^s. Empirically, we observe that ¯θtu z_θ^tu is well-approximated by a Gaussian perturbation around ^θs z_θ^s: ¯θtu∼(^st^θs,^st2), z_θ^tu ( μ_st z_θ^s, σ_st^2 I), (15) where ^st μ_st and ^st σ_st are dimension-wise mean and standard deviation vectors (details in Appx. B.2). This relation makes the inference-time self-conditioning mismatch explicit and allows us to connect it to the perturbed forward construction in Eq. 7. Assuming that ^θs z_θ^s perfectly denoises ¯s z_s at step s, we start from the standard forward parameterization, ¯s=αs^θs+σsϵs, z_s= _s z_θ^s+ _s ε_s, (16) and substitute ^θs z_θ^s using Eq. 15. Rearranging terms yields ¯s z_s =αs¯θtu−^stϵ^st+σsϵs = _s z_θ^tu- σ_st _u μ_st+ _s ε_s =αs1^st¯θtu+σs1+(αsσs^st^st)2ϵ = _s 1 μ_st z_θ^tu+ _s 1+ ( _s _s σ_st μ_st )^2 ε =αs^st¯θtu+σs1+^st2ϵ, = _s λ_st z_θ^tu+ _s 1+ γ_st^2 ε, (17) where ^st=1/^st λ_st=1/ μ_st, ^st=(αs/σs)(^st/^st) γ_st=( _s/ _s)( σ_st/ μ_st), and ϵ∼(0,) ε (0,I) is obtained via reparameterization of the combined noise terms. Since ^st μ_st and ^st σ_st vary independently across dimensions, all operations are element-wise. Eq. 17 mirrors the structure of our perturbed forward process (Eq. 7): both corrupt the signal term and inflate the effective noise through a multiplicative factor. Consistent with this connection, the empirically observed scaling patterns in Fig. 7 match the behavior induced by SCP in Fig. 6, supporting the view that SCP exposes the model during training to the noise pattern encountered when reusing self-conditioning at inference. Figure 7: Inference mean λ^t λ_t (top) and variance γ^t γ_t (bottom) scaling factors of prediction mismatch in a pre-trained network, plotted across timestep t for randomly selected embedding dimensions. Figure 8: The empirical distribution of ϵti _t^i with different random values of t and i. In principle, the empirical schedules ^st,^stt=1T\ λ_st, γ_st\_t=1^T could be estimated directly from Eq. 17. However, these quantities depend on the discretization size and vary across dimensions, making them difficult to express with a single global function. To avoid high-dimensional hyperparameter search, we instead use feature-independent anchor points (λmin,λmax)( _ , _ ) and (γmin,γmax)( _ , _ ) to define monotone linear schedules. This yields a simple end-to-end procedure that preserves the intended dynamics without introducing additional preprocessing overhead. B.2 Estimate The Self-condition Gaussian Distribution Error We now describe how we estimate the statistics in Eq. 15 on IWSLT14 De-En. For each diffusion time t and each embedding dimension i∈1,…,Hi∈\1,…,H\, we consider the dimension-wise residual ϵti=z¯θtu,i−μ^tiz^θs,i, _t^i= z_θ^tu,i- μ_t^i z_θ^s,i, (18) and test whether ϵti _t^i is approximately Gaussian with variance (σ^ti)2( σ_t^i)^2. We uniformly select 20 timesteps in [ϵ,1][ε,1], run denoising at these timesteps, and record paired values of ¯θtu z_θ^tu and ^θs z_θ^s. For each selected t, we sample 1,000 sentences z∈z , flatten all tokens, and estimate μ^ti μ_t^i and σ^ti σ_t^i for each dimension. Estimating μ^ti μ_t^i reduces to a one-dimensional linear regression (OLS) per dimension: μ^ti=∑z∈z¯uiz^ti∑z∈(z^ti)2, μ_t^i= _z z_u^i z_t^i _z ( z_t^i)^2, (19) and σ^ti σ_t^i is computed as the standard deviation of the residuals: σ^ti=1||∑z∈(z¯ui−μ^tiz^ti)2. σ_t^i= 1|D| _z ( z_u^i- μ_t^i z_t^i)^2. (20) We then standardize residuals and apply the Shapiro-Wilk test shapiro1965analysis to 50 randomly selected standardized residuals per dimension. Using a 95% confidence level, we reject normality if p<0.05p<0.05. The null hypothesis is rejected only in a small minority of cases, supporting our Gaussian approximation. Fig. 8 shows histograms of ϵti _t^i across different dimensions. Appendix C Comparison with Other Methods Method MBR NFE Rouge-1 Rouge-2 Rouge-L (↑)( ) (↑)( ) (↑)( ) LSTM 5 - 34.2 16.0 31.8 CMLM - - 34.4 15.6 32.2 NAG-BERT - - 35.1 16.5 33.3 Difformer* 10 20 34.9 17.0 32.4 DiffuSeq 10 2000 31.2 12.2 29.2 SeqDiffuSeq 0 2000 31.9 12.4 29.2 FastDiSS 10 5 34.9 16.9 32.5 FastDiSS 10 20 35.3 17.3 32.8 Table 10: Main results on Gigaword. The best NAR results are bold and the second-best results are underlined. Baseline results are from Difformer DBLP:conf/naacl/GaoG0Z0X24. ∗* indicates reproduced results. C.1 Training And Inference Mismatch Distance Penalty DBLP:conf/acl/TangWZLCZ23 is proposed to perturb the forward process for train-test discrepancy reduction. While conceptually related, it does not target the self-conditioning mismatch that is central in our analysis. In addition, this strategy applies a fixed penalty across steps, which mirrors the fixed variants in Tab. 5 and is empirically less effective than our time-varying perturbation. TREC also discusses collapse phenomena under self-conditioning, however their explanation emphasizes shortcut behavior induced by a predicted prior rather than the discretization-amplified self-conditioning mismatch that we studied. Our approach directly regularizes the self-condition so that training better matches inference-time reuse errors under few-step updates. C.2 Modified Noise Scheduler SeqDiffuSeq DBLP:conf/naacl/YuanYTHH24 assigns token difficulty by position, yielding position-specific training trajectories. However, the far later position schedules become similar (e.g., their Fig. 2), suggesting diminishing improvement as sequence length grows. In contrast, MANS is length-agnostic: it adjusts noise based on model confidence at the token level, and therefore independent of the sequence length. DINOISER DBLP:journals/corr/abs-2302-10025 emphasizes high-intensity noise to strengthen learning, but high-noise training can introduce large variance and bias learning toward marginal distributions, eventually reducing sample diversity. Our MANS instead increases noise selectively for high-confidence tokens while leaving uncertain tokens unchanged, maintaining meaningful supervision. Meta-DiffuBB DBLP:conf/nips/ChuangHLGLCL24 learns a planning schedule using an auxiliary controller (LSTM) optimized with reinforcement learning. This adds extra modules and increases training complexity, which can be unstable in practice, and its generalization to unseen contexts or long sequences depend largely on the planner choice. In contrast, FastDiSS avoids training an auxiliary planner, and are simple yet effective. Appendix D Additional Results Tab. 10 shows the experimental results on Text Summarization benchmark. Consistent with previous results in Tabs. 2 and 3, we observe that our model outperforms most of the diffusion and non-autoregressive baselines on all metrics. D.1 SCP Sensitivity Analysis. We analyze SCP over a broad range of anchor points (λmin,λmax,γmin,γmax)( _ , _ , _ , _ ), as shown in Table 11. Empirically, SCP consistently improves BLEU across all tested settings, indicating that the method is robust and not overly sensitive to the exact choice of anchor values. Param Metric Default 1 2 3 4 5 6 7 8 9 λmin _ Value – 0.85 0.86 0.87 0.88 0.89 0.90 0.91 0.92 0.93 BLEU 26.94 28.67 28.58 28.73 28.65 28.68 28.67 28.34 28.11 28.44 λmax _ Value – 0.91 0.92 0.93 0.94 0.95 0.96 0.97 0.98 0.99 BLEU 26.94 28.54 28.45 28.68 28.53 28.60 28.59 28.45 28.66 28.59 γmin _ Value – 0.11 0.12 0.13 0.14 0.15 0.16 0.17 0.18 0.19 BLEU 26.94 28.60 28.37 28.00 28.46 28.53 28.48 28.40 28.79 28.52 γmax _ Value – 0.31 0.32 0.33 0.34 0.35 0.36 0.37 0.38 0.39 BLEU 26.94 28.22 28.14 28.41 28.82 28.79 28.66 28.64 28.52 28.43 Table 11: BLEU on IWSLT14 baseline under different SCP schedule parameters. Default denotes λt=1 _t=1 and γt=0 _t=0. Finally, the MDLM experiment is a direct application of the SCP parameter selection strategy. The results in Tab. 8 further confirm that SCP outperforms standard self-conditioning. Appendix E Experimental Settings Configurations WMT14 WMT16 IWSLT14 Gigaword QQP Wiki-Auto Split Training 4,500,966 608,319 160,215 3,803,957 144,715 677,751 Validation 3,000 1,999 7,282 189,651 2,048 2,048 Test 3,003 1,999 6,750 1,951 2,500 5,000 Preprocess BPE 40,000 30,000 10,000 60,000 15,000 40,000 Vocab 40,624 34,976 15,480 56,392 15,136 45,376 Architecture dmodeld_model 512 512 512 512 768 768 dffnd_ffn 2048 2048 1024 2048 3072 3072 Heads 8 8 4 8 12 12 Training GPUs 2 2 2 2 2 4 Steps 600K 150K 300K 300K 50K 30K Tokens/GPU 32K 32K 4K 32K 4K 8K Phase [100K,200K,600K] [50K,100K,150K] [100K,200K,300K] [100K,200K,300K] [10K,20K,30K] [5K,10K,30K] Scaling [2.0,3.0,4.0] [2.0,3.0,4.0] [2.0,3.0,4.0] [2.0,3.0,4.0] [2.0,3.0,4.0] [2.0,4.0,8.0] Table 12: The dataset details, model architectures, and hyperparameters used in our experiments. Data. For preprocessing, we use fairseq library for IWSLT14, and use the preprocessed data released by Fully-NAT DBLP:conf/acl/GuK21 for WMT14 and WMT16111https://github.com/shawnkx/Fully-NAT. For Wiki and QQP, we use the ones from DiffuSeq222https://github.com/Shark-NLP/DiffuSeq, and for Gigaword we use the HuggingFace version333https://huggingface.co/datasets/Harvard/gigaword. All datasets are tokenized with byte-pair encoding (BPE) and processed with fairseq-preprocess. BPE settings and vocabulary sizes are reported in Tab. 12. Model. We use a Transformer-base backbone DBLP:conf/nips/VaswaniSPUJGKP17 with 6 encoder and 6 decoder layers for all datasets. The number of attention heads, hidden size, and related hyperparameters are listed in Tab. 12. The diffusion embedding dimension is 128. For SCP, we set anchor points tuned to 2020-step sampling, setting (γmin,γmax)=(0.90,0.95)( _ , _ )=(0.90,0.95) and (λmin,λmax)=(0.15,0.35)( _ , _ )=(0.15,0.35). Training. All models are trained with fp16 on 4 NVIDIA H100 GPUs. We use an inverse-sqrt learning-rate schedule with 10,000 warmup steps and lrmax=5×10−4lr_ =5× 10^-4 for all benchmarks. We set gradient norm clipping to 1.0, dropout to 0.1, and label smoothing to 0.1. Runtime is approximately 8.5 hours for WMT and Gigaword, and around 4 hours on average for the other datasets. Inference. At inference, the reverse process follows Eq. 4. Self-conditioning reuses the previous-step estimate, as in prior work. We apply Minimum Bayes-Risk (MBR) decoding DBLP:conf/naacl/KumarB04 following DiffusionLM and DiffuSeq. MANS. We implement MANS with three training phases and increase the scaling factor β(n)β(n) over time. The phase interval and scaling factor are given in Tab. 12. For example, on WMT14 we use β(n)=2.0β(n)=2.0 for n<100Kn<100K, β(n)=3.0β(n)=3.0 for 100K≤n<200K100K≤ n<200K, and β(n)=4.0β(n)=4.0 thereafter. This modification increases total training time by less than 5%5\% in our experiments. Appendix F On The Effectiveness Of Minimum Bayesian Risk Decoding Since FastDiSS builds on Difformer, we evaluate MBR decoding with both length beam search and noise-candidate search to improve output quality while maintaining diversity. We note that this search overhead can often be reduced by alternative stochastic length strategies DBLP:conf/iclr/GongLF0K23; DBLP:conf/nips/WuFLZGS0LWGDC23. Here we focus on characterizing the search-space trade-offs under the standard MBR setup. As shown in Tab. 2, increasing either the length beam or the noise beam improves BLEU. Fig. 10 further determine which axis scales better. The results reveal that scaling the length beam (vertical axis) yields faster gains than scaling the noise beam (horizontal axis). Intuitively, a larger length beam mitigates length prediction errors and expands the set of plausible candidates, which in turn improves the effectiveness of MBR selection. Noise beam, otherwise is not as effective, since our method already ensures convergence for each length. Figure 9: Word cloud of the easy and hard tokens during training. Figure 10: SacreBLEU score on IWSLT14 De-En with various length beams and noise beams. Appendix G High And Low Confidence Tokens Fig. 9 visualizes tokens identified by MANS as high or low confidence during training. High-confidence tokens are predominantly common words, indicating that the model can already reconstruct them reliably. MANS increases their effective noise level, forcing the model to denoise these “easy” tokens even under high noise, strengthening the self-conditioning signal. In contrast, low-confidence tokens tend to be rarer or domain-specific words that appear less often in the training data. Their noise level is left unchanged, allowing the model to learn them under a less aggressive corruption regime. This behavior aligns with a coarse-to-fine denoising process: common words are forced to generate faster, while rare words use reliable generated words to refine prediction. Appendix H Qualitative Results We present qualitative case studies in Tabs. 13 and 14 to illustrate how FastDiSS changes generation dynamics at the instance level. In Tab. 13, the first example shows that the baseline fails to effectively use the reused estimate, leading to persistent artifacts across steps (e.g., the repetition “Summer Summer” is not corrected in the next update). In contrast, FastDiSS repairs the error in subsequent steps, consistent with SCP improving the robustness of the model to erroneous self-conditioning under few-step discretization. The second example highlights that FastDiSS produces a sharper, more accurate reconstruction target, aligning with the role of MANS in strengthening supervision on high-confidence tokens. Tabs. 15, 16, and 17 further evaluate SCP adaptability to Plaid. Across these settings, SCP yields more coherent and consistent generations, reducing drift and improving paragraph-level continuity under limited denoising steps. Step Example 1 Source - He also twice participated in the Summer Olympics, starting in 1996. Target - He was also in the 1996 Summer Olympics. Baseline 2 He also twice in the Summer Summer Olympics 1996 Summer 1996. 1 He also twice in the Summer Summer in the in 1996. FastDiSS 2 He also twice twice in in Summer Summer Olympics, starting in 1996. 1 He also twice twice in the Summer Olympics, starting in 1996. Step Example 2 Source - Whedon served as an executive producer, along with Tim Minear. Target - Whedon was the executive producer, along with Tim Minear. Baseline 2 Whedon was an an executive producer with Tim Minear. 1 He was an an executive producer with Tim Minear. FastDiSS 2 Whedon was an executive producer with Tim Minear. 1 He was an executive producer with Tim Minear. Table 13: Examples of generation throughout the denoising process, with NFE=2NFE=2. Example 1 Source How do you think Federer, Nadal, Djokovic, Murray, Wawrinka, Delpo & Cilic rank in terms of potential when it comes to greatest spot in the tennis? Target How would you rank Federer, Nadal, Djokovic, Murray, Wawrinka, Delpo and Cilic in terms of greatness achieved and potential spot in the history of sport? Baseline How do you think about the Federer, rank in a good of Mury Wawrinka, Delgreatest rank and Cilic rank in terms of for tennis? FastDiSS What is the review of vic, MurAur, Murray, Dray, Wawrinka, DelPO & Cilic rank in tennis? Example 2 Source How will the ban on 500 and 1000 rupee notes bring out the black money of the big shots who have lots of it in the Swiss bank in a different currency? Target How will the decision to illegalize the 500 and 1000 Rs notes help to get rid of black money in the Swiss bank or maybe in other foreign banks and currencies? Baseline What will be the effect of banning 500 and 1000 notes on people having money in Swiss banks? FastDiSS How is demonetizing the 500 and 1000 rupees notes affect Indian economy? How it affect the black money in Swiss bank accounts? Table 14: Examples of generation on Wiki-Auto dataset. The resulting outputs demonstrate higher coherence in the long-term context. Example 1 Source This easy chicken curry recipe is made with just a handful of ingredients Plaid This easy chicken curry recipe is made with just a handful of ingredients make provide add that helps easy add help give add that helps welcome give help offer help for how you learn give help help help for act give help help help and perform give help help help for for how pay for help help help and even add help help help for give help that add give help up for the add provide help provide for help help help and add for how pay for help give help provide help and add give help help help and add help provide help to create help give help help help for for how doit add this give help help help that add help help give help help for and that give help add help give help give help for the run in the middle of the day add help help provide help help that gives simply give help help give help help provide help help provide help help for pay help help give help help pay give help give help give help help pay help the member on your family make help shop it … Plaid + SCP This easy chicken curry recipe is made with just a handful of ingredients at your disposal including a bow for glowing red ropes for strutter. It alone will be the trick since making anything else would normally be a lengthy method. Once that completes the dungeon is start laid out again towards the end where you learn the way to make naughty but once you find this little hidden dungeon point you will also need the basics of the tools to help you find your way to the recipe for cider seeds.↩ many would say these simple caves and dungeons are like paster glue in a Oddband environment. It is not just the recipe that is perhaps also the highlight in that the have side rooms for dungeon for completion in small spells or just locked in a spooky cave somewhere. The lack of questline support will prevent really good and fun faciettes. The formula the ingredients are a lot worth then anything else but for an hour of stats capable of requiring three seemed some how effective when you have the basics … Example 2 Source Generative models of text are very versatile: they can be used Plaid Negative models of text are very versatile: they can be used in the workplace and in the secondary market provide availability provides published versions varies and competitive paid provided available for tender available as the selection of availability choices were very common on a high … fair to read the initial sales framework very often … provide selection for the except immediately in commonly based on how pricing control edited ask that everyone creates as a matter of how much published provide is actually apply control on that immediately on offered provide on exchange for them provide paid write line of commissioned provide access made applied to more creative offered works cover different parts of the market agree that paid sharing is much more what use sharing work on the ability to in understanding in support of a contribution of a meaningful piece on say you can provide featured provide paid paid assignment paid submissions management provided applied arrangements with mixed paid paid shared fit easily perform create commissioned work in private … why is not offered applied availability on the subject or backing and reuse … to … more successful provide writing for working provide featured work submission creative exclusively created collection of icons is a symbol medium always learn how to make use of owns offering portfolio offering total free commissioned formats provided published licensing paid Plaid+SCP Generative models of text are very versatile: they can be used to extract depth and status information cheaply. Others would argue that static models of text are a powerful aid for explersion; however, it was in the core caricatures that it didn’t offer much benefit because in the instance they were not comprehend, the translation to say showed a very high degree of complexity. If you are looking for help overcoming the CPP constraints, you will want to consider means with which the manual models have been effectively replaced. How much structure really can be leveraged here is left as an observation regarding numeric computation and the need to adapt.↩ if the topic had not made its true contribution in the Enchanted Mongoose core system, the switch to primary structures would be significant. When I first saw the stream, I was told that the intent was to take the existing core system and replace it entirely with a trim and circular setup. That was described very well and perhaps would have been an option but as inconsistent with the extra-mainruction requirement as new core support for and directly related to static features. The details the stream provided suggest that they are not based on current text models, which probably means that the team is hoping for a design fix as you no longer get to the point of accurate input even in corramures. Primary layers, on the other hand, still provide a significant degree of input. You will have to navigate over simulated character emulation with replacement and recreate the basic static dynamic of the characters at a much condensed level. However, we are told that it will still be possible to use a simple filter in the secondary system to generate specific surface plans. Table 15: Examples of 32-step long sequence generation on Prefix Completion with Plaid-1B. The output results indicate that SCP helps maintain the coherence and fluency of the paragraph, while ensuring consistency across sentences. Example Source Lexically Constraints: Donald Plaid Sample 1: … focused in comments on newly uncovered FBI collusion with President Donald Trump, Bill Nunes revelation that almost no trump prosecute matters, despite widespread evidence, making him the primary target of these very crimes … Sample 2: … It also includes some mud slinging with Obama FGA Chairman John Masters attacking Donald Trump for loose familiarity with the banking cartel. It’s no secret that Trump was beaten to the drapes by President Barack Obama on the actual exam. Sample 3: … March in London Members of Parliament looking at paintings of Prime Minister Donald trump during EU festival day Getty Images Brexit protest … Plaid+SCP Sample 1: … It is also an equal, far more debasive sellout than the academia; some may have reasonably attributed the whole spread to Donald Nhutbout not clicking into the comments I have written this time; it seems to be unfortunate because both the coverage and commentary involved that continued focus could just seem to call out bias and denial … Sample 2: … But at least on campuses, the so called feminist war on radical conservatives may actually be having a negative effect on Loomer attendance. When it comes to that Donald Trump presidency, we might actually find men who don’t have the viciousness, sorry love, of the feminist left … Sample 3: … The overproduction chells fields and destroys towns, creates great energy exhaustion and destabilization and allot in the free residents and the general approach to produce any other good or industry fails to expand operations and flows.↩ Stone: Within one of the smaller areas, on the street of Donald Stone’s home, lies the Exit Man, now the best live erifot and Pisa _example courtesy of writer B. JonKing … Table 16: Examples of 32-step long sequence generation on Lexically Constrained Generation with Plaid-1B. The Plaid-1B samples lack diversity while ours cover more word combinations. Example Source Negation: Donald but not Trump Plaid … [profile] Frederic Blanchfield Britain’s earliest basalt monument in Little Choyside, Chepole, Dorset. Source: MORI.mauipomwork-8x17-1.jpg… [profile] Marco Bori Special Projects Marco Bastoni Participants in discussion with Russian crionic residents in Venice. Artist/21-ponoco-8x17-1.jpg… [profile] Marco Baba.alexnet-8x17-1.jpg… [profile] Giananna Boone Interior Photography Histature from Rome Client – Social Architecture – Vanguard.photo/46-photograph-164x72-r.jpg… [profile] Donald Buckingham Industrial Art Izomage ProKnakon Call to exercise Tradarpones in Real Materials-stone-piping-8x17-608px.jpg… [slide] artmagazine-20x26x24.jpg… [slide] … Plaid+SCP … It was dress for contempt. The dishonesty of this whole development was clear, an astonishing and obvious, necessary to humanity, but seen in the remark Il Quartilver made which showed an almost impervolent fit of irony, necessary to discern truth from record. ‘ Our exile and our exile have been secured. The ocean has risen. Hell, Donald haath been a fool, before he came to save me, Taracharvi, and finally … not only in these troubled times it was taking a giant it-own for their charisma, but sol gallur phir bariet die Pope Donald. If they couldn’t say ‘wrong’, where did they get it? The slanders at Timon knew to just make radios, they could have solved … Table 17: Examples of 32-step long sequence generation on Negation with Plaid-1B. The results demonstrate that the Plaid-1B sample is likely memorized from the dataset when against an unusual pattern, while our method produces a normal paragraph that satisfies the constraint.