Paper deep dive
LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization
Zexun Lin, Yuan Feng, Junlin Lv, Kevin S. Zhou, Xike Xie
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 89%
Last extracted: 8/12/2026, 2:27:20 AM
Summary
The paper introduces LibraSpec, a training-free, plug-and-play algorithm for dynamic speculative decoding in large language models. It addresses the inefficiencies of existing methods by reformulating speculative length selection as an expected-speedup optimization problem, specifically tailored for diffusion-based drafters that generate tokens in parallel. LibraSpec uses a marginal criterion based on drafter confidence scores to iteratively adjust the speculative length, ensuring that the acceptance gain outweighs verification costs. Theoretical proofs demonstrate monotonic convergence to the optimal speculative length. Experiments show consistent improvements over baselines, achieving up to 8.49x speedup over autoregressive decoding.
Entities (8)
Relation Signals (8)
LibraSpec → uses → Marginal Criterion
confidence 95% · Building on this criterion, we develop LibraSpec... iteratively determines the speculative length using drafter confidence scores.
LibraSpec → optimizes → Expected Speedup
confidence 93% · We therefore reformulate dynamic speculative-length selection as expected-speedup optimization... LibraSpec... iteratively determines the speculative length
LibraSpec → achieves → 8.49x Speedup
confidence 90% · achieving a further 0.5∼1.5× improvement over baselines and up to 8.49× speedup over autoregressive decoding.
LibraSpec → integrateswith → Diffusion-Based Drafters
confidence 90% · LibraSpec, a training-free and plug-and-play algorithm that iteratively determines the speculative length using drafter confidence scores... seamlessly integrated with these diffusion-based speculative drafters.
Diffusion-Based Drafters → enables → Parallel Generation
confidence 88% · diffusion-based drafters, however, generates candidate blocks in parallel at substantially lower drafting cost
LibraSpec → improves → FastdLLM
confidence 85% · we demonstrate its effectiveness when combined with representative methods, including FastdLLM... consistently achieving additional decoding speedups
LibraSpec → improves → DFlash
confidence 85% · we demonstrate its effectiveness when combined with representative methods, including... DFlash... consistently achieving additional decoding speedups
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically determined by the speculative length selected at each decoding round. Existing dynamic speculation methods select the speculation length by estimating how many tokens will be accepted, which is reasonable for autoregressive drafters that generates tokens sequentially. The recent wave of diffusion-based drafters, however, generates candidate blocks in parallel at substantially lower drafting cost, shifting the key question from how many tokens to generate to how many generated tokens are worth verifying. We therefore reformulate dynamic speculative-length selection as expected-speedup optimization and derive a marginal criterion that extends the speculative sequence only when its acceptance gain outweighs the additional verification cost. Building on this criterion, we develop \textit{LibraSpec}, a training-free and plug-and-play algorithm that iteratively determines the speculative length using drafter confidence scores. Theoretically, we prove that LibraSpec monotonically converges toward the optimal speculative length. Experiments across six target models, three diffusion-based speculative decoding methods, and math, coding, and chat benchmarks show consistent improvements under both greedy and sampling settings, achieving a further $0.5\sim1.5\times$ improvement over baselines and up to $8.49\times$ speedup over autoregressive decoding.
Tags
Links
- Source: https://arxiv.org/abs/2608.08721v1
- Canonical: https://arxiv.org/abs/2608.08721v1
Trouble viewing inline? Open PDF directly →
Full Text
65,902 characters extracted from source content.
Expand or collapse full text
LibraSpec: Dynamic Diffusion-Based Speculative Decoding via Marginal-Gain-Driven Optimization Zexun Lin Suzhou Institute for Advanced Research, University of Science and Technology of ChinaSuzhouJiangsuChina zexunlin@mail.ustc.edu.cn , Yuan Feng Suzhou Institute for Advanced Research, University of Science and Technology of ChinaSuzhouJiangsuChina yfung@mail.ustc.edu.cn , Junlin Lv Suzhou Institute for Advanced Research, University of Science and Technology of ChinaSuzhouJiangsuChina junlinlv@mail.ustc.edu.cn , Kevin S. Zhou Suzhou Institute for Advanced Research, University of Science and Technology of ChinaSuzhouJiangsuChina skevinzhou@ustc.edu.cn and Xike Xie Suzhou Institute for Advanced Research, University of Science and Technology of ChinaSuzhouJiangsuChina xkxie@ustc.edu.cn Abstract. Speculative decoding accelerates large language model inference by drafting multiple tokens for parallel verification, with efficiency critically determined by the speculative length selected at each decoding round. Existing dynamic speculation methods select the speculation length by estimating how many tokens will be accepted, which is reasonable for autoregressive drafters that generates tokens sequentially. The recent wave of diffusion-based drafters, however, generates candidate blocks in parallel at substantially lower drafting cost, shifting the key question from how many tokens to generate to how many generated tokens are worth verifying. We therefore reformulate dynamic speculative-length selection as expected-speedup optimization and derive a marginal criterion that extends the speculative sequence only when its acceptance gain outweighs the additional verification cost. Building on this criterion, we develop LibraSpec, a training-free and plug-and-play algorithm that iteratively determines the speculative length using drafter confidence scores. Theoretically, we prove that LibraSpec monotonically converges toward the optimal speculative length. Experiments across six target models, three diffusion-based speculative decoding methods, and math, coding, and chat benchmarks show consistent improvements under both greedy and sampling settings, achieving a further 0.5∼1.5×0.5 1.5× improvement over baselines and up to 8.49×8.49× speedup over autoregressive decoding. †ccs: Computing methodologies Natural language processing 1. Introduction Autoregressive large language models (LLMs) generate tokens sequentially, making decoding a major bottleneck in long-context reasoning, including code generation and interactive applications (Singh et al., 2026; Tay et al., 2022). Speculative Decoding (Leviathan et al., 2023; Cai et al., 2024; Li et al., 2025c, 2024, b; Chen et al., 2026; Zhang et al., 2024; Yang et al., 2026) alleviates this bottleneck with lossless acceleration with formal guarantees. At each decoding round, a lightweight draft model first proposes a sequence of candidate tokens, which are then verified in parallel by the target model; multiple valid tokens can consequently be accepted at once. The resulting speedup depends critically on the speculative length, i.e., the number of candidate tokens submitted for verification. A short length underutilizes parallel verification, whereas an excessively long length wastes target-model computation when an early rejection invalidates the remaining suffix (Mamou et al., 2024; Zhang et al., 2026b, a). Because token predictability varies across decoding rounds, the speedup-maximizing length also changes throughout generation. This motivates dynamic speculative-length selection: at each round, the system determines the length that best balances the acceptance gain of additional draft tokens against their computational cost. Figure 1. Speedup gains achieved by LibraSpec when integrated to different methods (see experiments for other benchmarks). Existing dynamic-length methods select the speculative length by predicting the accepted length, which is well-suited to autoregressive drafters because each additional candidate incurs sequential drafting cost. The recent wave of diffusion-based drafters, however, is reshaping speculative decoding by generating candidate blocks in parallel at substantially lower drafting cost (Liu et al., 2026; Chen et al., 2026). This changes the optimization target: the key question shifts from how many tokens to generate to how many generated tokens are worth verifying. Consequently, estimating the expected accepted length alone is insufficient to determine whether extending the speculative prefix will improve speedup, as even a likely accepted extension may provide insufficient gain to justify its additional verification cost (Marzollo et al., 2026). Figure 2. Static speculative length vs. LibraSpec vs. optimal speculative length, using the FastdLLM drafter with Qwen2.5-7B as the target model on MATH-500. To address this mismatch, we reformulate dynamic speculative-length selection from an accepted-length prediction problem into an expected-speedup optimization problem. Concretely, extending the speculative sequence improves expected speedup only when the added segment provides a higher acceptance-gain-to-verification-cost ratio than the current sequence. Although the globally optimal speculative length is not directly available in closed form, whether a local length adjustment improves expected speedup can be evaluated under our model. We formalize such a speedup-improving update as a beneficial adjustment and derive necessary and sufficient conditions for determining its maximum admissible length. We further prove that expected speedup is unimodal with respect to the speculative length. This structure recasts dynamic-length selection as iterative optimization: under our speedup model, successive beneficial adjustments monotonically improve expected speedup and converge in finitely many steps to a globally optimal speculative-length interval. Based on this analysis, we propose LibraSpec, a training-free and plug-and-play algorithm for diffusion-based speculative decoding. At each decoding round, LibraSpec uses confidence scores produced by the diffusion drafter for the probabilities unavailable before verification. It then iteratively applies the marginal criterion to determine how many candidate tokens should be submitted for verification. This design requires neither an auxiliary predictor nor additional training and can be directly integrated with existing diffusion-based speculative decoding methods. As illustrated in Figure 2, LibraSpec closely tracks the oracle optimal length obtained by exhaustive search, while adapting to changes across decoding rounds. We evaluate LibraSpec with six LLMs and three state-of-the-art (SOTA) diffusion-based speculative decoding methods. Extensive experiments on mathematical reasoning, code generation, and chat benchmarks demonstrate consistent improvements brought in by LibraSpec under both greedy and sampling settings. It further improves the end-to-end speedup by 0.5∼1.5×0.5 1.5× over diffusion-based methods and achieves up to 8.49×8.49× speedup over vanilla autoregression-based methods. The main contributions are summarized as follows. • We identify that the emerging shift toward diffusion-based drafters changes the objective of dynamic speculative-length selection from accepted-length estimation to direct optimization of the acceptance-gain–verification-cost trade-off. • We formulate the problem as expected-speedup optimization, derive a marginal criterion for beneficial length adjustments, and derive necessary and sufficient conditions identifying the maximum admissible adjustment that guarantees improvement. • We propose LibraSpec, a training-free and plug-and-play dynamic-length algorithm that consistently accelerates multiple diffusion-based speculative decoding methods across models, tasks, and decoding settings. 2. Related work 2.1. Diffusion-Based Speculative Decoding Since Speculative Diffusion Decoding (Christopher et al., 2025) first introduced discrete diffusion models as speculative drafters, several follow-up studies have been proposed. Methods (Pan et al., 2025; Sandler et al., 2025; Samragh et al., 2025) represented by DiffuSpec (Li et al., 2025a) employ large-scale pretrained diffusion language models as training-free speculative drafters, leveraging inference-time search strategies or train–test alignment techniques to improve draft quality and acceptance rates. However, these methods depend on large-scale draft models, which incur significant memory overhead and inference latency. In contrast, DFlash (Chen et al., 2026) trains a lightweight block-wise diffusion drafter that exploits KV Injection to extract rich contextual features from the target model, allowing an entire draft block to be generated in a single forward pass. Likewise, methods (Liu et al., 2026; Wang et al., 2026; Hu et al., 2026) represented by DDTree (Ringel and Romano, 2026), build draft trees based on the token distributions obtained from a single diffusion forward pass, thereby further improving the accepted draft length. Our method can be seamlessly integrated with these diffusion-based speculative drafters. In this work, we demonstrate its effectiveness when combined with representative methods, including FastdLLM, DFlash and DDTree, consistently achieving additional decoding speedups across diverse benchmarks. 2.2. Dynamic Speculative Decoding Since the speculative block length plays a critical role in determining the overall decoding efficiency, a growing body of work has focused on dynamically adjusting the draft length during inference. Existing methods are mainly divided into two categories. Methods (Huang et al., 2025; Zhang et al., 2026b, a; Kim et al., 2026) represented by DISCO (Mamou et al., 2024) train a predictor to forecast the length of draft generation, which achieves relatively accurate prediction of the expected received length. However, they require additional training and rely heavily on the generalization ability of the predictor. In contrast, methods (Zarch et al., 2025; Wei et al., 2025; Hou et al., 2025) such as Gemma 4 MTP (Lacombe and Grootendorst, 2026) and FailFast (Pan et al., 2026b) leverage heuristic signals, including historical acceptance rates, confidence scores, and entropy, as indirect indicators to guide the expected future acceptance length. While these approaches incur no additional training cost, they fail to account for the computational overhead introduced by the verification stage. More importantly, these methods inherit the optimization paradigm of autoregressive speculative decoding, where the objective is to estimate the expected accepted prefix length. In contrast, LibraSpec explicitly models the trade-off between the expected benefit of successful speculation and the verification overhead, providing a novel perspective for dynamic speculative decoding. 3. Methodology 3.1. Preliminaries Speculative decoding accelerates autoregressive generation by combining a lightweight draft model ℳdraftM_draft with a target model ℳtargetM_target (Leviathan et al., 2023). Given a prefix x≤tx_≤ t, the draft model generates a speculative continuation (xt+1,…,xt+d)(x_t+1,…,x_t+d), producing a probability distribution qv=Pℳdraft(xt+v∣x<t+v)q_v=P_M_draft(x_t+v x_<t+v) for each drafted position v∈1,…,dv∈\1,…,d\. The target model then evaluates all drafted positions in parallel via a single forward pass, yielding the target distribution pv=Pℳtarget(xt+v∣x<t+v)p_v=P_M_target(x_t+v x_<t+v). A subsequent verification step compares qvq_v and pvp_v to determine whether each drafted token should be accepted or corrected. Consequently, this speculative mechanism effectively bypasses the sequential bottleneck of standard autoregressive decoding without altering target output distribution of ℳtargetM_target. 3.2. How Speculative Length Affects Diffusion-Based Speculative Speedup To quantify the speedup of diffusion-based speculative decoding, consider a prefix x≤tx_≤ t and a length-d draft block 1:d=(xt+1,…,xt+d)D_1:d=(x_t+1,…,x_t+d). Let TddraftT_d^draft and TdverifyT_d^verify denote the time required to generate and verify this block. Assuming the target model accepts τd _d consecutive tokens and has a standard per-token latency of LdtargetL_d^target, the overall speculative decoding speedup, ηd _d, can present as: (1) ηd=LdtargetLdspec=τdLdtargetTddraft+Tdverify _d= L_d^targetL_d^spec= _d\,L_d^targetT_d^draft+T_d^verify where Ldspec=Tddraft+TdverifyτdL_d^spec= T_d^draft+T_d^verify _d is the average per-token latency of speculative decoding. In practice, as shown in Fig. 3, compared with autoregressive drafters, diffusion-based drafters reduce the drafting overhead of speculative decoding by several folds, rendering the drafting cost negligible relative to the target-model inference cost. Figure 3. Latency breakdown at a speculative length of 16 tokens, measured on an A100 GPU with 1K context length. Therefore, to analyze how varying the speculative length affects overall efficiency, we consider a simplified setting where an infinitely long draft sequence is already available, allowing us to freely adjust the speculative length. (2) ηd≈τdLdtargetTdverify. _d≈ _dL_d^targetT_d^verify. In this setting, the drafting cost is fixed and independent of the speculative length, leaving the verification stage as the only component affected by the choice of speculative length. Consequently, the key question becomes how the speculative length should be adjusted to boost the decoding speedup. Directly optimizing this objective is intractable. Therefore, we instead iteratively adjust the speculative length, allowing the optimization process to progressively converge toward the optimal speculative length. We require every speculative-length adjustment to satisfy the conditions of a single beneficial adjustment, as formalized in Definition 3.1. Definition 3.1 (A Single Beneficial Adjustment). We say that a speculative length adjustment from d to d′d is a single beneficial adjustment if it satisfies the following conditions. (3) [ηd′]>[ηd]E[ _d ]>E[ _d] 3.3. Achieving a Single Beneficial Adjustment of Speculative Length Definition 3.1 defines the speedup criterion that a single beneficial adjustment from d to d′d must satisfy. As shown in Eq. 2, decoding speedup is jointly determined by the accepted draft length and the verification latency. Therefore, a single beneficial adjustment fundamentally corresponds to achieving a more favorable trade-off between the expected acceptance gain and the additional verification cost. This trade-off can be characterized by quantifying the marginal gain and the marginal cost introduced by the adjustment, as formalized in Theorem 3.2. Theorem 3.2. A speculative length adjustment achieves a single beneficial adjustment if and only if the marginal benefit of the adjusted draft segment is more favorable than the current average benefit. (4) [τd:d′]Td:d′verify>[τd]Tdverify,d<d′,[τd′:d]Td′:dverify<[τd]Tdverify,d>d′. cases E[ _d:d ]T^verify_d:d > E[ _d]T^verify_d,&d<d ,\\[15.00002pt] E[ _d :d]T^verify_d :d< E[ _d]T^verify_d,&d>d . cases where the first case (d<d′d<d ) corresponds to extending the draft sequence, requiring the marginal benefit of the appended draft segment to exceed the current average benefit, while the second case (d>d′d>d ) corresponds to truncating the draft sequence, requiring the marginal benefit of the removed suffix to be lower than the current average benefit. Proof. See Appendix A.1 for details. ∎ Theorem 3.2 provides a criterion for determining whether a speculative length adjustment is beneficial. Under prefix acceptance, however, the contribution of later speculative tokens depends on whether all preceding tokens are accepted. As a result, different draft positions contribute differently to the overall gain-cost trade-off. In particular, we strengthen the criterion of a single beneficial adjustment by requiring that the marginal benefit of the speculative suffix starting from position i exceeds the average benefit accumulated up to that position. By expanding the expressions for the marginal gain, [τi:d′]=∑j=id′∏k=ijpkE[ _i:d ]= _j=i^d _k=i^jp_k, and the marginal verification cost Ti:d′verify=c(d′−i)T_i:d ^verify=c\,(d -i), where c denotes the average verification cost per speculative position, we obtain the corresponding necessary condition for a speculative length adjustment to achieve a single beneficial adjustment, as shown in Theorem 3.3. Theorem 3.3. Under the strengthened position-wise criterion, A Single Beneficial Adjustment from d to d′d only if it satisfies: (5) d′ d ≤ϵi,∀i∈[1,d], ≤ _i, ∀\,i∈[1,d], ϵi _i =maxd′∈ℤ∣d′<Tiverifyτic∑j=id′∏k=ijpk+i. = \d d < T_i^verify _i\,c\, _j=i^d _k=i^jp_k+i \. Proof. See Appendix A.2 for details. ∎ The preceding theorem establishes the position-wise bounds that every beneficial adjustment must satisfy. Necessity alone, however, does not establish that an adjusted length satisfying these bounds actually improves the decoding speedup. We therefore prove sufficiency by exploiting the monotonic structure induced by prefix acceptance. As the candidate length increases, each newly added token provides a non-increasing conditional acceptance contribution, and hence the average marginal gain associated with each draft position is also non-increasing. Consequently, the lengths satisfying each position-wise constraint form a contiguous feasible interval. Taking the intersection of these intervals shows that satisfying all position-wise bounds jointly is sufficient for a single beneficial adjustment. A detailed proof is provided in Appendix A.3. Theorem 3.4. A Single Beneficial Adjustment from d to d′d if it satisfies: (6) d′ d =miniϵi,∀i∈[1,d], = _i _i, ∀\,i∈[1,d], ϵi _i =maxd′∈ℤ∣d′<Tiverifyτic∑j=id′∏k=ijpk+i. = \d d < T_i^verify _i\,c\, _j=i^d _k=i^jp_k+i \. Theorem 3.4 provides a sufficient condition for achieving a single beneficial adjustment. Any speculative length exceeding this bound inevitably violates the necessary condition for achieving a single beneficial adjustment, while choosing a smaller length leads to conservative updates that unnecessarily sacrifice expected acceptance gains without guaranteeing additional speedup improvements. Algorithm 1 LibraSpec 1:Block size n, Maximum Speculation Length dmaxd_max 2:Draft tokens 3:Generate one draft block of size n. 4:d←nd← n; B←0B← 0 5:while d<dmaxd<d_max do 6: ϵ∗←miniϵiε^*← _i _i where ϵi=⌊α∑j=id∏k=ijqk+i⌋ _i= α\, _j=i^d _k=i^jq_k+i , ∀i=1,…,d∀ i=1,…,d 7: if ϵ∗≤dε^*≤ d then 8: d←ϵ∗d←ε^*; break 9: end if 10: B←B−(ϵ∗−d)B← B-(ε^*-d); d←ϵ∗d←ε^* 11: if B<0B<0 then 12: Generate ⌈−Bn⌉ -Bn draft blocks; B←B+⌈−Bn⌉nB← B+ -Bn n. 13: end if 14:end while 15:Submit d tokens for verification. 3.4. Determining the Final Speculative Length via Iterative Adjustments Theorem 3.4 provides a theoretically grounded criterion for obtaining a single beneficial adjustment. Repeatedly applying this criterion progressively refines the speculative length toward the optimal decoding speedup. An immediate question is whether repeatedly applying beneficial adjustments indeed converges to an optimal speculative length. To answer this question, we begin by analyzing the marginal contribution of a single draft token. Under prefix acceptance, the conditional marginal benefit of each additional token is non-increasing, whereas the marginal verification cost remains approximately constant. Therefore, the marginal efficiency of extending the speculative block decreases as the speculative length grows. Once this marginal efficiency no longer exceeds the current average efficiency, subsequent extensions cannot restore an improvement. We therefore establish the unimodality of the expected decoding speedup with respect to the speculative length, as shown in 3.5. Theorem 3.5. The expected speedup is a unimodal function of the speculative length. (7) [τd+1]Td+1verify>[τd]Tdverify⟺∏i=1d+1pi>c[τd]Tdverify E[ _d+1]T^verify_d+1> E[ _d]T^verify_d _i=1^d+1p_i>c\, E[ _d]T^verify_d Proof. See Appendix A.4 for details. ∎ The unimodal structure implies that every beneficial adjustment moves the speculative length toward the unique optimum. We therefore establish the convergence of the iterative adjustment procedure, as shown in 3.6. Theorem 3.6. By iteratively applying single beneficial adjustments, the speculative length converges in finitely many steps to d′d , which lies in the globally optimal interval of speculative lengths. (8) [ηd′]=maxd∈ℤ>0[ηd].E[ _d ]= _d _>0E[ _d]. Proof. See Appendix A.5 for details. ∎ To apply the theoretical result in practice, two practical challenges must be addressed. First, the target-model acceptance probabilities pip_i are unavailable before verification. Second, the idealized assumption of an infinitely long draft sequence does not hold in practice. Since draft tokens are generated on demand, the confidence scores of future draft tokens are unavailable before they are generated. Assumption 3.7. The draft-model confidence scores qiq_i are well calibrated with the corresponding target-model acceptance probabilities pip_i (Li et al., 2026; Pan et al., 2026a). To address the first challenge, under this assumption, the draft-model confidence scores can be used as a proxy for the unknown target-model acceptance probabilities. However, this calibration generally deteriorates as the speculative horizon increases, and the rate of deterioration varies across draft models. We therefore constrain online speculation with a model-dependent maximum speculative length, dmaxd_max. The influence of dmaxd_max is further analyzed in Section 4.6. To address the second challenge, as a practical compromise, we compute the adjustment criterion using only the confidence scores of the draft tokens generated so far in the current draft block. Since TiverifyT_i^verify, τi _i, and c are runtime-dependent, we absorb them into a single trade-off hyperparameter α=Tiverifyτi−1c−1α=T_i^verify _i^-1c^-1, which controls the aggressiveness of speculative length adjustment. Building upon the above theoretical derivations, we propose a plug-and-play dynamic speculative decoding strategy, LibraSpec (Algorithm 1), which dynamically adjusts the speculative block length through iterative refinements. At each iteration, LibraSpec seeks a single beneficial adjustment, thereby progressively improving the theoretical decoding speedup. To avoid frequent calls to the draft model caused by continual adjustments of the speculative length, LibraSpec maintains a draft budget B. At the beginning of each speculative round, an initial draft block is generated, and subsequent length extensions first consume the remaining budget before requesting additional draft blocks. New draft blocks are generated only when the budget is exhausted, reducing unnecessary draft-model calls and the associated I/O overhead. LibraSpec initializes the iterative refinement from the default block size of each draft model, using it as the initial speculative length d. Inspired by the design philosophy of damping techniques in iterative optimization, LibraSpec performs at most one rollback once the estimated speculative length falls below the current length. Since repeated updates may introduce oscillatory behavior due to the interaction between speculative length adjustment and probability estimation, LibraSpec terminates the adjustment after a single correction. In practice, this design improves stability while avoiding additional computation that often provides limited benefit. Table 1. Comparison with Dynamic Speculative Length Methods on Qwen2.5-Instruct. FailFast, G4-style, and Ours are all built upon FastdLLM. (FdLLM: FastdLLM; η: speedup; τ: avg. acceptance length.) Model Method Temperature = 0 Temperature = 1 MATH-500 GSM8K HumanEval MT-Bench Avg MATH-500 GSM8K HumanEval MT-Bench Avg η τ η τ η τ η τ η τ η τ η τ η τ η τ η τ Q2.5-7B EAGLE-3 2.23× 3.71 2.12× 3.52 2.25× 3.21 1.79× 2.91 2.10× 3.34 2.12× 3.58 2.03× 3.42 2.16× 3.06 1.66× 2.77 1.99× 3.21 FdLLM 1.92× 3.29 1.53× 2.25 1.61× 2.43 1.40× 2.03 1.62× 2.50 1.81× 3.16 1.48× 2.16 1.47× 2.27 1.29× 1.87 1.51× 2.37 FdLLM+FailFast 2.98× 4.48 2.25× 3.79 2.52× 4.83 1.93× 3.16 2.42× 4.07 2.79× 3.87 2.07× 3.65 2.33× 4.66 1.79× 3.02 2.25× 3.80 FdLLM+G4-style 2.29× 3.80 1.76× 2.64 2.02× 3.80 1.40× 2.05 1.87× 3.07 2.10× 3.51 1.78× 2.86 1.75× 3.48 1.26× 1.82 1.72× 2.92 FdLLM+Ours 3.54× 5.11 3.09× 4.23 3.12× 5.29 2.39× 3.43 3.04× 4.52 3.31× 4.94 2.88× 4.05 3.00× 5.09 2.14× 3.29 2.83× 4.34 Q2.5-14B EAGLE-3 2.36× 3.80 2.19× 3.59 2.61× 3.83 1.81× 2.99 2.24× 3.55 2.25× 3.69 2.06× 3.48 2.48× 3.73 1.66× 2.82 2.11× 3.43 FdLLM 2.47× 4.19 2.21× 3.68 2.23× 2.94 1.93× 2.76 2.21× 3.39 2.40× 4.08 2.09× 3.50 2.11× 2.83 1.82× 2.61 2.11× 3.26 FdLLM+FailFast 3.73× 6.38 2.87× 4.37 3.41× 5.12 2.25× 3.37 3.07× 4.81 3.56× 6.11 2.65× 4.22 3.23× 4.92 1.99× 3.17 2.86× 4.61 FdLLM+G4-style 2.59× 4.37 1.82× 3.27 2.76× 3.89 2.04× 2.95 2.30× 3.62 2.66× 4.24 2.18× 3.73 2.41× 3.48 1.87× 2.70 2.28× 3.54 FdLLM+Ours 4.71× 7.31 3.68× 5.23 4.08× 5.97 2.64× 3.72 3.78× 5.56 4.51× 6.96 3.49× 4.98 3.92× 5.84 2.42× 3.61 3.59× 5.35 Q2.5-32B EAGLE-3 2.50× 3.95 2.27× 3.69 2.68× 3.86 1.90× 3.14 2.34× 3.66 2.39× 3.76 2.14× 3.50 2.56× 3.72 1.74× 2.95 2.21× 3.48 FdLLM 3.57× 4.75 3.13× 4.38 3.16× 3.97 2.18× 2.98 3.01× 4.02 3.31× 4.59 2.84× 4.03 2.83× 3.82 1.99× 2.84 2.74× 3.82 FdLLM+FailFast 4.90× 6.45 3.71× 4.59 4.06× 5.15 2.41× 3.49 3.77× 4.87 4.67× 6.08 3.52× 4.16 3.82× 4.97 2.20× 3.32 3.55× 4.63 FdLLM+G4-style 3.89× 4.98 3.20× 4.39 3.31× 4.18 2.23× 3.03 3.16× 4.15 3.48× 4.86 3.18× 4.10 2.88× 3.90 1.99× 2.82 2.88× 3.92 FdLLM+Ours 5.45× 7.32 4.11× 4.78 4.70× 5.94 2.93× 4.25 4.30× 5.57 5.11× 6.99 3.87× 4.32 4.41× 5.63 2.58× 3.98 3.99× 5.23 4. Experiments 4.1. Experimental Setup Models and Evaluations. We conduct experiments on Qwen2.5-7, 14, 32B-Instruct (Yang et al., 2025b) and Qwen3-4B, 8B, Coder-30B-A3B-Instruct (Yang et al., 2025a) pre-trained models. We evaluate our method on benchmarks spanning three categories: Math, including MATH-500 (Lightman et al., 2023) and GSM8K (Cobbe et al., 2021); Code, including HumanEval (Chen et al., 2021); and Chat, including MT-Bench (Zheng et al., 2023). For each benchmark, we assess decoding efficiency using end-to-end speedup (η) over the autoregressive baseline and average acceptance length (τ). Table 2. Performance of LibraSpec Integrated with Diffusion-Based Speculative Decoding Methods (DFlash and DDTree). Model Method Temperature = 0 Temperature = 1 MATH-500 GSM8K HumanEval MT-Bench Avg MATH-500 GSM8K HumanEval MT-Bench Avg η τ η τ η τ η τ η τ η τ η τ η τ η τ η τ Q3-4B DFlash 5.54× 7.73 5.10× 6.50 4.81× 6.52 2.64× 4.33 4.52× 6.27 4.50× 6.61 4.31× 5.97 4.36× 5.99 2.54× 4.08 3.93× 5.66 DFlash + Ours 5.98× 8.04 5.54× 7.74 5.58× 7.83 2.94× 4.93 5.01× 7.14 4.95× 7.03 4.72× 6.61 5.11× 7.12 2.82× 4.58 4.40× 6.34 DDTree 7.26× 10.21 6.37× 9.31 6.81× 9.22 4.16× 6.59 6.15× 8.83 6.52× 9.56 5.88× 8.84 5.94× 9.15 3.79× 6.24 5.53× 8.45 DDTree + Ours 8.03× 10.97 7.18× 9.98 7.36× 10.27 4.77× 6.87 6.84× 9.52 6.96× 9.87 6.49× 9.50 6.53× 9.92 4.27× 6.91 6.06× 9.05 Q3-8B DFlash 5.56× 7.80 4.84× 6.55 4.73× 6.58 2.58× 4.24 4.43× 6.29 4.52× 6.46 3.93× 5.91 4.17× 5.49 2.28× 3.78 3.73× 5.41 DFlash+Ours 6.17× 8.27 5.66× 7.91 5.72× 7.99 2.91× 4.61 5.12× 7.20 5.14× 7.38 4.47× 6.74 5.08× 6.44 2.65× 4.29 4.34× 6.21 DDTree 7.52× 10.53 6.45× 9.45 6.80× 9.68 4.15× 6.59 6.23× 9.06 6.50× 9.52 5.96× 8.90 5.96× 8.53 3.51× 5.97 5.48× 8.23 DDTree+Ours 8.03× 11.08 7.24× 10.08 7.28× 10.15 4.69× 6.78 6.81× 9.52 7.07× 10.10 6.51× 9.43 6.50× 9.18 4.09× 6.51 6.04× 8.81 Q3-Coder 30B-A3B DFlash 4.29× 5.56 4.01× 5.18 6.09× 7.99 2.04× 3.52 4.11× 5.56 4.02× 5.31 3.89× 5.06 5.69× 7.58 1.88× 3.39 3.87× 5.34 DFlash+Ours 4.68× 6.22 4.45× 5.77 6.51× 8.56 2.40× 4.27 4.51× 6.21 4.51× 6.03 4.26× 5.50 6.17× 8.24 2.10× 3.93 4.26× 5.93 DDTree 6.10× 8.04 5.83× 7.48 8.12× 10.33 3.07× 5.32 5.78× 7.79 5.82× 7.68 5.63× 7.29 7.71× 10.01 2.96× 5.12 5.53× 7.53 DDTree+Ours 6.58× 8.58 6.25× 8.09 8.49× 10.67 3.38× 5.81 6.18× 8.29 6.24× 8.17 6.15× 7.83 8.21× 10.47 3.33× 5.76 5.98× 8.06 Baselines. We use vanilla autoregressive decoding as the baseline, which serves as the benchmark for speedup ratios (1.00×). We compare against representative speculative decoding approaches, including the autoregressive method EAGLE-3 (Li et al., 2025b) and diffusion-based approaches Fast-dLLM-v2-1.5B (Wu et al., 2025) (hereinafter referred to as FastdLLM), FailFast (Pan et al., 2026b), DFlash (Chen et al., 2026), and DDTree (Ringel and Romano, 2026). We additionally adapt the dynamic speculative length strategy used in Gemma 4 MTP (Lacombe and Grootendorst, 2026) to the FastdLLM for comparison with representative dynamic speculative length methods (hereinafter referred to as G4-style). Since our method is a plug-and-play acceleration technique, we further evaluate its integration with existing speculative decoding methods to assess its compatibility and complementary benefits. Implementation Details. Unless otherwise stated, all experiments are conducted on NVIDIA A100 GPUs. Details of the configurations for each benchmark and draft models are provided in Appendix B. 4.2. Comparison with Dynamic Speculative Length Methods In this section, we evaluate LibraSpec against existing dynamic speculative length methods. We adopt FastdLLM, a widely used diffusion-based drafter, and follow its officially released Qwen2.5 configuration as our experimental setup. The dynamic speculative length baselines include FailFast, a dynamic speculation strategy specifically designed for FastdLLM, and G4-style, the speculative length strategy proposed in Gemma 4, which is adapted to FastdLLM for a fair comparison. We further include EAGLE-3 as a representative autoregressive speculative decoding baseline. As shown in Table 1, LibraSpec consistently outperforms all existing dynamic speculative-length strategies across all Qwen2.5 models, benchmark datasets, and decoding settings. Under greedy decoding (temperature=0temperature=0), integrating LibraSpec with FastdLLM improves the average end-to-end speedup by 1.42×, 1.57×, and 1.29× on Qwen2.5-7B, 14B, and 32B, respectively, while consistently surpassing FailFast and G4-style. The performance gains remain robust under non-greedy sampling (temperature=1temperature=1), where LibraSpec further improves the average speedup of FastdLLM by 1.32×, 1.48×, and 1.25×, respectively. These results demonstrate that, compared with existing methods, LibraSpec enables substantially more effective speculative length adjustment, and that its performance gains consistently generalize across both greedy decoding and stochastic sampling. Table 3. Performance Comparison of Reasoning Models on Different Benchmarks. Model Method Temperature = 0 Temperature = 1 MATH-500 GSM8K HumanEval MT-Bench Avg MATH-500 GSM8K HumanEval MT-Bench Avg η τ η τ η τ η τ η τ η τ η τ η τ η τ η τ Q3-4B DFlash 3.92× 5.72 3.63× 5.01 3.38× 4.71 2.18× 3.11 3.28× 4.64 3.39× 4.84 3.34× 4.68 3.09× 4.28 2.09× 3.01 2.98× 4.20 DFlash+Ours 4.31× 6.09 3.96× 5.57 3.66× 5.03 2.39× 3.49 3.58× 5.05 3.71× 5.29 3.73× 5.25 3.34× 4.74 2.31× 3.38 3.27× 4.67 Q3-8B DFlash 4.01× 5.82 3.62× 5.15 3.32× 4.70 2.31× 3.37 3.32× 4.76 3.53× 5.06 3.31× 4.69 3.03× 4.29 2.15× 3.13 3.01× 4.29 DFlash+Ours 4.39× 6.18 3.91× 5.60 3.61× 5.07 2.51× 3.62 3.61× 5.12 3.87× 5.42 3.60× 5.09 3.30× 4.67 2.33× 3.42 3.28× 4.65 Table 4. Sensitivity of LibraSpec to the Hyperparameter α, Compared with Static Speculative Length Baselines. Target: Q2.5-7B Draft: FastdLLM Target: Q3-8B Draft: DFlash w/ Ours α MATH-500 GSM8K HumanEval MT-Bench Avg w/ Ours α MATH-500 GSM8K HumanEval MT-Bench Avg η τ η τ η τ η τ η τ η τ η τ η τ η τ η τ × / 1.92× 3.29 1.53× 2.25 1.61× 2.43 1.40× 2.03 1.62× 2.50 × / 5.56× 7.80 4.84× 6.55 4.73× 6.58 2.58× 4.24 4.43× 6.29 ✓ 1.8 3.48× 5.01 2.94× 4.05 3.01× 5.18 2.36× 3.40 2.95× 4.41 ✓ 1.8 5.88× 7.98 5.36× 7.48 5.47× 7.56 2.72× 4.33 4.86× 6.84 1.9 3.51× 5.05 3.03× 4.15 3.11× 5.27 2.39× 3.43 3.01× 4.48 1.9 5.99× 8.09 5.42× 7.63 5.58× 7.76 2.80× 4.47 4.95× 6.99 2.0 3.54× 5.11 3.09× 4.23 3.12× 5.29 2.39× 3.43 3.04× 4.52 2.0 6.05× 8.16 5.50× 7.69 5.63× 7.83 2.91× 4.61 5.02× 7.07 2.1 3.49× 5.01 3.02× 4.14 3.05× 5.21 2.29× 3.38 2.96× 4.44 2.1 6.15× 8.24 5.58× 7.84 5.71× 7.96 2.91× 4.61 5.09× 7.16 2.2 3.43× 4.89 2.92× 4.03 2.96× 5.13 2.17× 3.32 2.87× 4.34 2.2 6.17× 8.27 5.66× 7.91 5.72× 7.99 2.89× 4.57 5.11× 7.19 4.3. Integration with State-of-the-Art Diffusion-Based Drafters We further integrate LibraSpec into state-of-the-art diffusion-based speculative decoding methods, including DFlash and DDTree, and evaluate them on the Qwen3 models with thinking mode disabled. Following their official configurations, we evaluate the decoding speedup based on their publicly released Qwen3 draft models. As shown in Table 2, LibraSpec consistently improves the performance of both DFlash and DDTree across all Qwen3 models, benchmarks, and decoding settings. Compared with the original DFlash, LibraSpec provides an additional average decoding speedup of 0.39∼0.69×0.39 0.69× under greedy decoding and 0.39∼0.67×0.39 0.67× under non-greedy sampling. Similarly, LibraSpec brings an additional average decoding speedup of 0.39∼0.69×0.39 0.69× and 0.45∼0.56×0.45 0.56× over DDTree under greedy decoding and non-greedy sampling, respectively. These results demonstrate that LibraSpec is complementary to existing diffusion-based speculative decoding methods and consistently enhances their decoding efficiency across different model architectures and sampling strategies. 4.4. Robustness under Thinking-Mode Decoding In this section, we evaluate LibraSpec on the Qwen3 models with thinking mode enabled. As shown in Table 3, LibraSpec consistently improves the performance of DFlash across all reasoning models, benchmark datasets, and decoding settings. Compared with the original DFlash, LibraSpec provides an additional average decoding speedup of approximately 0.29∼0.30×0.29 0.30× under greedy decoding (temperature=0temperature=0) and 0.27∼0.29×0.27 0.29× under non-greedy sampling (temperature=1temperature=1), corresponding to a relative improvement of approximately 9%∼11%9\% 11\% over the original methods. Notably, the improvements remain evident even under reasoning-intensive scenarios, where the generated outputs typically exhibit higher entropy and greater uncertainty, making speculative decoding more challenging. Nevertheless, LibraSpec consistently improves decoding speed across all evaluated settings, demonstrating that the proposed dynamic speculative-length adjustment generalizes well to reasoning workloads. 4.5. Hyperparameter (α) Sensitivity Analysis We study the sensitivity of LibraSpec to the trade-off hyperparameter α under greedy decoding (temperature=0temperature=0), which balances the expected acceptance gain against the verification overhead. Specifically, a larger α encourages more aggressive draft expansion by placing greater weight on the acceptance gain, while a smaller α favors a more conservative strategy by assigning a higher penalty to failed expansions. As shown in Table 4, LibraSpec is robust across a broad range of α values, requiring little hyperparameter tuning in practice. Moreover, compared with the corresponding baselines without LibraSpec, all evaluated α values consistently improve both decoding speedup and accepted speculative length, demonstrating that LibraSpec remains effective across a wide range of hyperparameter settings. For Qwen2.5-7B with FastdLLM, the best average speedup is achieved at α=2.0α=2.0, while larger values slightly reduce both speedup and τ. In contrast, for Qwen3-8B with DFlash, performance consistently improves as α increases, with the best results obtained at α=2.2α=2.2. This suggests that stronger drafters can benefit from more aggressive speculative expansion because the cost of invalidated draft tokens is lower. Based on these observations, we use α=2.0α=2.0 for FastdLLM, and α=2.2α=2.2 for both DFlash and DDTree in all experiments. (a) FastdLLM (b) DFlash Figure 4. Effect of Maximum Speculative Length on the Speedup of LibraSpec, Compared with Static Speculative Length. 4.6. Maximum speculative Length (dmaxd_max) Sensitivity Analysis In this section, we investigate how the maximum speculative length dmaxd_max affects decoding efficiency for different draft models. As shown in Figure 4, regardless of the choice of dmaxd_max, LibraSpec consistently outperforms the original static length decoding strategy (denoted as w/o Ours in the figure) across all evaluated draft models and datasets. The performance gap becomes even more pronounced as the maximum speculative length increases, highlighting the advantage of dynamic speculative length adjustment over static length decoding under more aggressive speculation. This demonstrates that LibraSpec is robust to the choice of dmaxd_max, consistently delivering speedup improvements under different maximum speculative length configurations. Moreover, as draft confidence calibration deteriorates over longer speculative horizons, different draft models exhibit different effective ranges of dmaxd_max. For FastdLLM, speedup increases before plateauing at approximately dmax=50∼70d_max=50 70, suggesting reliable calibration over relatively long horizons. Based on this phenomenon, we therefore set dmax=60d_max=60 for all FastdLLM experiments. In contrast, DFlash has a shorter effective range because its reliance on target model hidden states inherently limits the parallel decoding block size. Beyond approximately 22∼2422 24 tokens, the calibration of the draft-model distribution q against the target-model distribution p gradually deteriorates, leading to larger prediction errors and lower acceptance rates; further extending the speculative length therefore yields little additional benefit, while Assumption 3.7 becomes less valid. We therefore set dmax=24d_max=24 for all DFlash and DDTree experiments. (a) MATH-500 (b) GSM8K (c) HumanEval (d) MT-Bench Figure 5. Case Study of Dynamic Speculative Length Adjustment with LibraSpec across Four Benchmarks: a FastdLLM Draft Model and a Qwen2.5-7B Target Model. 4.7. Case study of dynamic speculative length adjustment In this section, to examine how LibraSpec adapts the speculative length during decoding, Figure 5 visualizes decoding trajectories from four benchmarks, comparing three schedules: a static speculative length, the oracle optimal length obtained by exhaustive search, and the length selected by our LibraSpec. The oracle speculative length varies substantially across decoding steps and depends strongly on the local generation context. This confirms the premise of Section 3.2 — the optimal gain–cost trade-off drifts with local content predictability, so any static length inevitably alternates between under-speculation and over-speculation. Despite having no access to the oracle schedule, LibraSpec closely tracks it using only the confidence and cost signals available online. Across all four examples, LibraSpec follows the oracle schedule with a mean absolute deviation of only 2.78 tokens — far below the 9.48 deviation tokens of the static length schedule — correctly anticipating both expansions and contractions. This is a direct consequence of Theorem 3.5: since the expected speedup is unimodal in the speculative length, each beneficial adjustment provably moves toward the current optimum. As the decoding context changes, repeatedly applying such adjustments allows LibraSpec to track the evolving oracle length without explicitly predicting the accepted length. 5. Conclusion In this paper, we show that the recent shift toward diffusion-based drafters changes the optimization objective of dynamic speculative-length selection. Because diffusion drafters generate candidate blocks in parallel at substantially lower marginal drafting cost, estimating the accepted length is no longer sufficient for maximizing end-to-end speedup. We therefore formulate speculative-length selection as expected-speedup optimization and derive a marginal criterion that balances acceptance gain against verification cost. Based on this criterion, we develop LibraSpec, a training-free and plug-and-play algorithm that dynamically determines how many draft tokens should be verified. We show that beneficial adjustments monotonically improve expected speedup and converge to a globally optimal speculative-length interval, while practical LibraSpec approximates these adjustments using drafter confidence scores. Extensive experiments across mathematical reasoning, code generation, and general chat benchmarks demonstrate that LibraSpec consistently improves verification efficiency under both greedy and sampling settings, yielding a consistent and substantial end-to-end improvement over SOTA methods. References T. Cai, Y. Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, and T. Dao (2024) Medusa: simple llm inference acceleration framework with multiple decoding heads. External Links: 2401.10774, Link Cited by: §1. J. Chen, Y. Liang, and Z. Liu (2026) DFlash: block diffusion for flash speculative decoding. External Links: 2602.06036, Link Cited by: §B.1, §1, §1, §2.1, §4.1. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, et al. (2021) Evaluating large language models trained on code. External Links: 2107.03374, Link Cited by: Table 5, §4.1. J. K. Christopher, B. R. Bartoldson, T. Ben-Nun, M. Cardei, B. Kailkhura, and F. Fioretto (2025) Speculative diffusion decoding: accelerating language generation through diffusion. External Links: 2408.05636, Link Cited by: §2.1. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman (2021) Training verifiers to solve math word problems. External Links: 2110.14168, Link Cited by: Table 5, §4.1. Y. Hou, F. Zhang, C. Du, X. Zhang, J. Pan, T. Pang, C. Du, V. Y. F. Tan, and Z. Yang (2025) BanditSpec: adaptive speculative decoding via bandit algorithms. External Links: 2505.15141, Link Cited by: §2.2. L. Hu, Z. Feng, Y. Wu, H. Yuan, Y. Zhao, Y. Qian, B. Wang, P. Zhao, D. Jiang, Y. Zhu, T. Rosing, and H. Zhang (2026) JetSpec: breaking the scaling ceiling of speculative decoding with parallel tree drafting. External Links: 2606.18394, Link Cited by: §2.1. K. Huang, X. Guo, and M. Wang (2025) SpecDec++: boosting speculative decoding via adaptive candidate lengths. External Links: 2405.19715, Link Cited by: §2.2. S. Kim, J. Kim, D. Yoon, J. Shin, J. Lee, and J. Seo (2026) Speculative verification: exploiting information gain to refine speculative decoding. External Links: 2509.24328, Link Cited by: §2.2. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica (2023) Efficient memory management for large language model serving with pagedattention. External Links: 2309.06180, Link Cited by: §B.2. O. Lacombe and M. Grootendorst (2026) Accelerating gemma 4: faster inference with multi-token prediction drafters. Note: https://blog.google/innovation-and-ai/technology/developers-tools/multi-token-prediction-gemma-4/Google AI Blog Cited by: §2.2, §4.1. Y. Leviathan, M. Kalman, and Y. Matias (2023) Fast inference from transformers via speculative decoding. External Links: 2211.17192, Link Cited by: §1, §3.1. G. Li, Z. Fu, M. Fang, Q. Zhao, M. Tang, C. Yuan, and J. Wang (2025a) DiffuSpec: unlocking diffusion language models for speculative decoding. External Links: 2510.02358, Link Cited by: §2.1. S. Li, C. Wang, Y. Zhu, Y. Wang, F. Yin, S. Shi, Y. Chen, X. Dong, Q. Chen, J. Pan, J. Li, L. Xie, Y. Zhang, L. Yu, Y. Wen, I. Tsang, and T. Zhang (2026) SpecForge: a flexible and efficient open-source training framework for speculative decoding. External Links: 2603.18567, Link Cited by: Assumption 3.7. Y. Li, F. Wei, C. Zhang, and H. Zhang (2024) EAGLE-2: faster inference of language models with dynamic draft trees. External Links: 2406.16858, Link Cited by: §1. Y. Li, F. Wei, C. Zhang, and H. Zhang (2025b) EAGLE-3: scaling up inference acceleration of large language models via training-time test. External Links: 2503.01840, Link Cited by: §1, §4.1. Y. Li, F. Wei, C. Zhang, and H. Zhang (2025c) EAGLE: speculative sampling requires rethinking feature uncertainty. External Links: 2401.15077, Link Cited by: §1. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe (2023) Let’s verify step by step. External Links: 2305.20050, Link Cited by: Table 5, §4.1. F. Liu, X. Li, K. Zhao, Y. Gao, Z. Zhou, Z. Zhang, Z. Wang, W. Dou, S. Zhong, and C. Tian (2026) DART: diffusion-inspired speculative decoding for fast llm inference. External Links: 2601.19278, Link Cited by: §1, §2.1. J. Mamou, O. Pereg, D. Korat, M. Berchansky, N. Timor, M. Wasserblat, and R. Schwartz (2024) Dynamic speculation lookahead accelerates speculative decoding of large language models. External Links: 2405.04304, Link Cited by: §1, §2.2. M. Marzollo, J. Zhuang, N. Roemer, N. Zwingenberger, L. K. Müller, and L. Cavigelli (2026) SSSD: simply-scalable speculative decoding. External Links: 2411.05894, Link Cited by: §1. J. Pan, J. Xu, Y. Zhou, and G. Dai (2025) SpecDiff: accelerating diffusion model inference with self-speculation. External Links: 2509.13848, Link Cited by: §2.1. L. Pan, Z. Tao, R. Pang, X. Wang, J. Zhao, and Y. Zhang (2026a) Making every verified token count: adaptive verification for moe speculative decoding. External Links: 2605.00342, Link Cited by: Assumption 3.7. R. Pan, Z. Chen, H. Liu, A. Krishnamurthy, and R. Netravali (2026b) Fail fast, win big: rethinking the drafting strategy in speculative decoding via diffusion llms. External Links: 2512.20573, Link Cited by: §B.2, §2.2, §4.1. L. Ringel and Y. Romano (2026) Accelerating speculative decoding with block diffusion draft trees. External Links: 2604.12989, Link Cited by: §2.1, §4.1. M. Samragh, A. Kundu, D. Harrison, K. Nishu, D. Naik, M. Cho, and M. Farajtabar (2025) Your llm knows the future: uncovering its multi-token prediction potential. External Links: 2507.11851, Link Cited by: §2.1. J. Sandler, J. K. Christopher, T. Hartvigsen, and F. Fioretto (2025) SpecDiff-2: scaling diffusion drafter alignment for faster speculative decoding. External Links: 2511.00606, Link Cited by: §2.1. A. Singh, A. Fry, A. Perelman, A. Tart, A. Ganesh, A. El-Kishky, et al. (2026) OpenAI gpt-5 system card. External Links: 2601.03267, Link Cited by: §1. Y. Tay, M. Dehghani, D. Bahri, and D. Metzler (2022) Efficient transformers: a survey. External Links: 2009.06732, Link Cited by: §1. Z. Wang, J. Huang, and X. Chen (2026) TAPS: target-aware prefix tree selection for diffusion-drafted speculative decoding. External Links: 2606.00487, Link Cited by: §2.1. Z. Wei, W. Chen, X. Zhu, and Y. Meng (2025) AdaDecode: accelerating llm decoding with adaptive layer parallelism. External Links: 2506.03700, Link Cited by: §2.2. C. Wu, H. Zhang, S. Xue, S. Diao, Y. Fu, Z. Liu, P. Molchanov, P. Luo, S. Han, and E. Xie (2025) Fast-dllm v2: efficient block-diffusion llm. External Links: 2509.26328, Link Cited by: §B.2, §4.1. A. Yang, A. Li, B. Yang, B. Zhang, B. Hui, B. Zheng, et al. (2025a) Qwen3 technical report. External Links: 2505.09388, Link Cited by: §4.1. A. Yang, B. Yang, B. Zhang, B. Hui, B. Zheng, B. Yu, et al. (2025b) Qwen2.5 technical report. External Links: 2412.15115, Link Cited by: §4.1. P. Yang, C. Du, F. Zhang, H. Wang, T. Pang, C. Du, and B. An (2026) LongSpec: long-context lossless speculative decoding with efficient drafting and verification. External Links: 2502.17421, Link Cited by: §1. H. E. Zarch, L. Gao, C. Jiang, and M. Annavaram (2025) DEL: context-aware dynamic exit layer for efficient self-speculative decoding. External Links: 2504.05598, Link Cited by: §2.2. H. Zhang, Y. Hu, Y. Wang, M. Mo, X. Xiao, and X. Chu (2026a) BlockPilot: instance-adaptive policy learning for diffusion-based speculative decoding. External Links: 2606.31315, Link Cited by: §1, §2.2. J. Zhang, Z. Yu, L. Wang, N. Yang, E. J. Yu, Z. Li, Y. Song, D. Zhu, X. Zhang, F. Wei, and S. Li (2026b) Learning to draft: adaptive speculative decoding with reinforcement learning. External Links: 2603.01639, Link Cited by: §1, §2.2. J. Zhang, J. Wang, H. Li, L. Shou, K. Chen, G. Chen, and S. Mehrotra (2024) Draft & verify: lossless large language model acceleration via self-speculative decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 11263–11282. External Links: Link, Document Cited by: §1. L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica (2023) Judging llm-as-a-judge with mt-bench and chatbot arena. External Links: 2306.05685, Link Cited by: Table 5, §4.1. Appendix A Theoretical Proofs A.1. Proof for Theorem 3.2 Theorem 3.2. A speculative length adjustment achieves a single beneficial adjustment if and only if the marginal benefit of the adjusted draft segment is more favorable than the current average benefit. (9) [τd:d′]Td:d′verify>[τd]Tdverify,d<d′,[τd′:d]Td′:dverify<[τd]Tdverify,d>d′. cases E[ _d:d ]T^verify_d:d > E[ _d]T^verify_d,&d<d ,\\[15.00002pt] E[ _d :d]T^verify_d :d< E[ _d]T^verify_d,&d>d . cases where the first case (d<d′d<d ) corresponds to extending the draft sequence, requiring the marginal benefit of the appended draft segment to exceed the current average benefit, while the second case (d>d′d>d ) corresponds to truncating the draft sequence, requiring the marginal benefit of the removed suffix to be lower than the current average benefit. Proof. Substituting the definition of speedup from Definition 3.1, we obtain: (10) [ηd′]>[ηd] [ _d ]>E[ _d] ([τd]+[τd:d′])LtargetTdverify+Td:d′verify>[τd]LtargetTdverify,d<d′,([τd]−[τd′:d])LtargetTdverify−Td′:dverify<[τd]LtargetTdverify,d>d′. [τd]+[τd:d′]Tdverify+Td:d′verify>[τd]Tdverify,d<d′,[τd]−[τd′:d]Tdverify−Td′:dverify<[τd]Tdverify,d>d′. Using the ratio comparison identity, we obtain: (11) [τd:d′]Td:d′verify>[τd]Tdverify,d<d′,[τd′:d]Td′:dverify<[τd]Tdverify,d>d′. cases E[ _d:d ]T^verify_d:d > E[ _d]T^verify_d,&d<d ,\\[15.00002pt] E[ _d :d]T^verify_d :d< E[ _d]T^verify_d,&d>d . cases ∎ A.2. Proof for Theorem 3.3 Theorem 3.3. Under the strengthened position-wise criterion, A Single Beneficial Adjustment from d to d′d only if it satisfies: (12) d′ d ≤ϵi,∀i∈[1,d], ≤ _i, ∀\,i∈[1,d], ϵi _i =maxd′∈ℤ∣d′<Tiverifyτic∑j=id′∏k=ijpk+i. = \d d < T_i^verify _i\,c\, _j=i^d _k=i^jp_k+i \. Proof. Let Ai:j=xi,…,xj are all accepted,i≤j.A_i:j=\x_i,…,x_j are all accepted\,\ i≤ j. Since acceptance must occur consecutively, (13) Pr(Ai:j)=∏k=ijpk. (A_i:j)= _k=i^jp_k. The expected marginal increase in accepted draft tokens is given by (14) [τi:j]=∑k=ijPr(Ai:k)=∑k=ij∏l=ikpl,E[ _i:j]= _k=i^j (A_i:k)= _k=i^j _l=i^kp_l, where the expectation follows from the linearity of expectation. Retaining the draft token at position i incurs a wasted verification cost of (15) Ti:j=c(j−i).T_i:j=c\,(j-i). where c denotes the average verification cost per speculative position. We strengthen the criterion of a single beneficial adjustment by requiring that the marginal benefit of the speculative suffix starting from position i exceeds the average benefit accumulated up to that position. (16) [τi:d′]Ti:d′verify>τiTiverify,∀i∈[1,d]. E[ _i:d ]T^verify_i:d > _iT^verify_i,\,∀\,i∈[1,d]. Expanding the marginal gain and the marginal cost yields (17) ∑j=id′∏k=ijpkc(d′−i) _j=i^d _k=i^jp_kc\,(d -i) >τiTiverify > _iT^verify_i Tiverify∑j=id′∏k=ijpk T^verify_i\, _j=i^d _k=i^jp_k >τic(d′−i) > _i\,c\,(d -i) Tiverifyτic∑j=id′∏k=ijpk+i T_i^verify _i\,c\, _j=i^d _k=i^jp_k+i >d′. >d . For a single beneficial adjustment, the condition must hold for every draft position i. Therefore, (18) d′ d ≤ϵi,∀i∈[1,d], ≤ _i, ∀\,i∈[1,d], ϵi _i =maxd′∈ℤ∣d′<Tiverifyτic∑j=id′∏k=ijpk+i. = \d d < T_i^verify _i\,c\, _j=i^d _k=i^jp_k+i \. ∎ A.3. Proof for Theorem 3.4 Theorem 3.4. A Single Beneficial Adjustment from d to d′d if it satisfies: (19) d′ d =miniϵi,∀i∈[1,d], = _i _i, ∀\,i∈[1,d], ϵi _i =maxd′∈ℤ∣d′<Tiverifyτic∑j=id′∏k=ijpk+i. = \d d < T_i^verify _i\,c\, _j=i^d _k=i^jp_k+i \. Proof. (20) d′=miniϵid = _i _i According to Theorem 3.3, since d′d must satisfy d′≤ϵid ≤ _i for all i, it follows that (21) d′ d ≤ϵi,∀i∈[1,d], ≤ _i, ∀\,i∈[1,d], ϵi _i =maxd′∈ℤ∣d′<Tiverifyτic∑j=id′∏k=ijpk+i. = \d d < T_i^verify _i\,c\, _j=i^d _k=i^jp_k+i \. For ∀i∈[1,d]∀ i∈[1,d], (22) d′ d <Tiverifyτic∑j=id′∏k=ijpk+i < T_i^verify _i\,c\, _j=i^d _k=i^jp_k+i Tiverifyτi T_i^verify _i >c(d′−i)∑j=id′∏k=ijpk > c\,(d -i) _j=i^d _k=i^jp_k [τi:d′]Ti:d′verify E[ _i:d ]T^verify_i:d >τiTiverify. > _iT^verify_i. In particular, substituting i=di=d, (23) [τd:d′]Td:d′verify>[τd]Tdverify,d<d′,[τd′:d]Td′:dverify<[τd]Tdverify,d>d′. cases E[ _d:d ]T^verify_d:d > E[ _d]T^verify_d,&d<d ,\\[15.00002pt] E[ _d :d]T^verify_d :d< E[ _d]T^verify_d,&d>d . cases In this case, a single beneficial adjustment can be achieved. ∎ A.4. Proof for Theorem 3.5 Theorem 3.5. The expected speedup is a unimodal function of the speculative length. (24) [τd+1]Td+1verify>[τd]Tdverify⟺∏i=1d+1pi>c[τd]Tdverify E[ _d+1]T^verify_d+1> E[ _d]T^verify_d _i=1^d+1p_i>c\, E[ _d]T^verify_d Proof. Since 0≤pi≤10≤ p_i≤ 1, the product ∏i=1dpi _i=1^dp_i is monotonically non-increasing with respect to d. If the marginal verification cost incurred by adding each speculative position is a constant c, then there exists a threshold d such that: (25) ∏i=1d+1pic>[τd]Tdverify _i=1^d+1p_ic> E[ _d]T^verify_d By Theorem 3.2, it follows that (26) [τd+1]Td+1verify>[τd]Tdverify⟺∏i=1d+1pi>c[τd]Tdverify. E[ _d+1]T^verify_d+1> E[ _d]T^verify_d _i=1^d+1p_i>c\, E[ _d]T^verify_d. Moreover, its contrapositive also holds. (27) [τd+1]Td+1verify≤[τd]Tdverify⟺∏i=1d+1pi≤c[τd]Tdverify E[ _d+1]T^verify_d+1≤ E[ _d]T^verify_d _i=1^d+1p_i≤ c\, E[ _d]T^verify_d This immediately implies that the expected decoding speedup is a unimodal function of the speculative length. ∎ A.5. Proof for Theorem 3.6 Theorem 3.6. By iteratively applying single beneficial adjustments, the speculative length converges in finitely many steps to d′d , which lies in the globally optimal interval of speculative lengths. (28) [ηd′]=maxd∈ℤ>0[ηd].E[ _d ]= _d _>0E[ _d]. Proof. Choose (29) d⋆∈argmaxd∈ℤ>0[ηd],d ∈ *arg\,max_d _>0E[ _d], Let d(k)k≥0\d^(k)\_k≥ 0 denote the sequence of beneficial adjustments. By Definition 3.1, (30) [ηdk+1]>[ηdk]⟹[ηdℓ]>[ηdk],∀ℓ>k.E[ _d_k+1]>E[ _d_k] [ _d_ ]>E[ _d_k], ∀\, >k. By Theorem 3.5, (31) k<ℓ,dk,dℓ<d⋆ k< , d_k,d_ <d ⟹dk<dℓ, d_k<d_ , k<ℓ,dk,dℓ>d⋆ k< , d_k,d_ >d ⟹dk>dℓ. d_k>d_ . Let, (32) I−=k:dk<d⋆,I+=k:dk>d⋆.I_-=\k:d_k<d \, I_+=\k:d_k>d \. Then (33) |I−|≤d⋆−1,I+≠∅⟹|I+|≤dminI+−d⋆<∞.|I_-|≤ d -1, I_+≠ |I_+|≤ d_ I_+-d <∞. Hence, the sequence reaches the globally optimal interval after finitely many beneficial adjustments. At the terminal length d′d , (34) ∄d∈ℤ>0:[ηd]>[ηd′]. \,d _>0:E[ _d]>E[ _d ]. Therefore, (35) [ηd′]=maxd∈ℤ>0[ηd].E[ _d ]= _d _>0E[ _d]. ∎ Appendix B Baseline and Benchmark Details B.1. Benchmark Details Table 5 lists the number of evaluated examples for each dataset. We follow the DFlash (Chen et al., 2026) benchmark setup for these sample counts. Table 5. Number of evaluated examples per dataset in the benchmark suite. Dataset Examples MATH-500 (Lightman et al., 2023) 128 GSM8K (Cobbe et al., 2021) 128 HumanEval (Chen et al., 2021) 164 MT-Bench (Zheng et al., 2023) 80 B.2. Baseline Details EAGLE-3. All EAGLE-3 baselines are run using vLLM v0.13.0 (Kwon et al., 2023). We run an extensive parameter sweep of speculative length from 3 to 20, and pick a different speculation length that yielded the best speedup for each model-dataset pair. For comparisons with EAGLE-3 on Qwen2.5 models, we use the checkpoints released by FailFast (Pan et al., 2026b). FastdLLM. All FastdLLM baselines are run using Fast-dLLM-v2-1.5B (Wu et al., 2025). we also conduct an extensive parameter sweep of speculative length from 3 to 20, and pick a different speculative length that achieves the best speedup for each model-dataset pair. When combined with LibraSpec, the initial speculative length is set to 10, while the maximum speculative length is set to 60. FailFast. For the hyperparameters in FailFast, we adopt the optimal configurations reported in the original paper. Specifically, we set τ=0.5,0.45,0.4τ=\0.5,0.45,0.4\ for the Qwen2.5-7,14,32B-Instruct target models, respectively. The maximum speculative length is set to 80, following the original paper, while the initial speculative length is set to 10. G4-style. We adapt the dynamic speculative-length control strategy used in Gemma 4 MTP to the FastdLLM. Following the original heuristic, the speculative length for the next decoding round is increased by 2 if all drafted tokens are accepted in the current round, and decreased by 1 whenever at least one drafted token is rejected. DFlash. All DFlash experiments use a default initial speculative length (block size) of 16. When integrated with LibraSpec, the maximum speculative length is set to 24. DDTree. All DDTree experiments also use an initial block size of 16. We search the node budget over 64,128,256,512\64,128,256,512\ and report the configuration that achieves the highest decoding speedup. When integrated with LibraSpec, the maximum speculative length is likewise set to 24.