Paper deep dive
AdaMTP: An Adaptive Training Paradigm for Multi-Token Prediction
Ziqiang Cui, Han Shi, Bowei He, Yu Pan, Peiyang Liu, Shengyin Sun, Yankai Chen, Haoli Bai, Yichun Yin, Xue Liu, Chen Ma
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/4/2026, 4:25:38 AM
Summary
The paper introduces AdaMTP, an adaptive training paradigm for Multi-Token Prediction (MTP) in Large Language Models. It addresses the limitation of fixed-length prediction horizons in standard MTP by using an entropy-based segmentation algorithm to dynamically align the prediction horizon with the intrinsic predictability of the sequence. AdaMTP partitions sequences into variable-length groups based on semantic boundaries detected via entropy surges, assigns adaptive prediction depths, and uses a dynamically masked MTP objective to suppress noisy gradients across high-entropy boundaries. Evaluated on Llama-3.1-8B, Qwen-2.5-7B, and Gemma-3-12B, AdaMTP outperforms standard MTP and Next-Token Prediction (NTP) in task performance and inference speedup across mathematical reasoning, code generation, and general benchmarks.
Entities (14)
Relation Signals (13)
AdaMTP → evaluatedon → LLaMA-3.1-8B
confidence 95% · Across mathematical reasoning, code generation, and general benchmarks on three backbones (Llama-3.1-8B, Qwen-2.5-7B, Gemma-3-12B)
AdaMTP → evaluatedon → Qwen-2.5-7B
confidence 95% · Across mathematical reasoning, code generation, and general benchmarks on three backbones (Llama-3.1-8B, Qwen-2.5-7B, Gemma-3-12B)
AdaMTP → evaluatedon → Gemma 3 12B
confidence 95% · Across mathematical reasoning, code generation, and general benchmarks on three backbones (Llama-3.1-8B, Qwen-2.5-7B, Gemma-3-12B)
AdaMTP → uses → Entropy-Based Segmentation
confidence 95% · At its core, an entropy-based segmentation algorithm leverages the base model to detect sudden surges in uncertainty as semantic boundaries
AdaMTP → improves → Multi-Token Prediction
confidence 90% · AdaMTP consistently outperforms standard MTP in both task performance and inference speedup.
AdaMTP → outperforms → Next-Token Prediction
confidence 90% · AdaMTP consistently outperforms both NTP and standard MTP in task performance while decoding faster
IFeval → usedfor → General Proficiency
confidence 90% · alongside MMLU Hendrycks et al. (2020) and IFEval Zhou et al. (2023a) to measure general proficiency.
MATH500 → →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-Token Prediction (MTP) has emerged as an effective paradigm that augments a shared Large Language Model backbone with auxiliary heads, training the model to predict several future tokens in parallel to enrich its supervision signal and accelerate inference. However, existing training frameworks adopt a rigid, fixed-length prediction horizon, disregarding the highly non-uniform information density of natural language and code. Forcing the auxiliary heads to predict across high-entropy semantic boundaries injects noisy, conflicting training signals; because these heads share the backbone's latent representations, the resulting gradients backpropagate and interfere with the model's core capabilities. We propose AdaMTP, an adaptive training paradigm that dynamically aligns the prediction horizon with the intrinsic predictability of the sequence. At its core, an entropy-based segmentation algorithm leverages the base model to detect sudden surges in uncertainty as semantic boundaries, partitioning sequences into variable-length groups. Each token is assigned an adaptive prediction depth, and a dynamically masked MTP objective suppresses the loss for predictions that cross these boundaries, attenuating the noisy gradients that degrade the backbone. Across mathematical reasoning, code generation, and general benchmarks on three backbones (Llama-3.1-8B, Qwen-2.5-7B, Gemma-3-12B), AdaMTP consistently outperforms standard MTP in both task performance and inference speedup.
Tags
Links
- Source: https://arxiv.org/abs/2608.00434v1
- Canonical: https://arxiv.org/abs/2608.00434v1
Trouble viewing inline? Open PDF directly →
Full Text
48,572 characters extracted from source content.
Expand or collapse full text
AdaMTP: An Adaptive Training Paradigm for Multi-Token Prediction Ziqiang Cui1, Han Shi2, Bowei He3,4, Yu Pan2, Peiyang Liu5, Shengyin Sun1, Yankai Chen3,4, Haoli Bai2, Yichun Yin2, Xue Liu3,4, Chen Ma122footnotemark: 2 1City University of Hong Kong, 2Huawei Technologies 3Mohamed bin Zayed University of Artificial Intelligence 4McGill University, 5Peking University Email: ziqiang.cui@my.cityu.edu.hkCorresponding authors. Abstract Multi-Token Prediction (MTP) has emerged as an effective paradigm that augments a shared Large Language Model backbone with auxiliary heads, training the model to predict several future tokens in parallel to enrich its supervision signal and accelerate inference. However, existing training frameworks adopt a rigid, fixed-length prediction horizon, disregarding the highly non-uniform information density of natural language and code. Forcing the auxiliary heads to predict across high-entropy semantic boundaries injects noisy, conflicting training signals; because these heads share the backbone’s latent representations, the resulting gradients backpropagate and interfere with the model’s core capabilities. We propose AdaMTP, an adaptive training paradigm that dynamically aligns the prediction horizon with the intrinsic predictability of the sequence. At its core, an entropy-based segmentation algorithm leverages the base model to detect sudden surges in uncertainty as semantic boundaries, partitioning sequences into variable-length groups. Each token is assigned an adaptive prediction depth, and a dynamically masked MTP objective suppresses the loss for predictions that cross these boundaries, attenuating the noisy gradients that degrade the backbone. Across mathematical reasoning, code generation, and general benchmarks on three backbones (Llama-3.1-8B, Qwen-2.5-7B, Gemma-3-12B), AdaMTP consistently outperforms standard MTP in both task performance and inference speedup. AdaMTP: An Adaptive Training Paradigm for Multi-Token Prediction Ziqiang Cui1†thanks: Email: ziqiang.cui@my.cityu.edu.hk, Han Shi2†thanks: Corresponding authors., Bowei He3,4, Yu Pan2, Peiyang Liu5, Shengyin Sun1, Yankai Chen3,4, Haoli Bai2, Yichun Yin2, Xue Liu3,4, Chen Ma122footnotemark: 2 1City University of Hong Kong, 2Huawei Technologies 3Mohamed bin Zayed University of Artificial Intelligence 4McGill University, 5Peking University 1 Introduction Large Language Models (LLMs) have achieved remarkable success across a wide range of natural language processing, mathematical reasoning, and code generation tasks. Currently, these models are predominantly trained using the standard Next-Token Prediction (NTP) objective, where models are trained to predict the immediate next token given the preceding context. Despite their robust generative capabilities, the strictly autoregressive nature of NTP imposes a fundamental bottleneck during inference. Generating tokens one by one leads to high latency and computational inefficiency, especially for long-form generation and interactive applications. To mitigate this inference bottleneck, Multi-Token Prediction (MTP) has recently emerged as a highly effective paradigm. By augmenting a shared LLM backbone with multiple auxiliary output heads, MTP generalizes the standard training objective to predict several future tokens simultaneously. During training, this multi-token objective provides richer supervision signals, equipping the model with enhanced long-term planning capabilities. During inference, MTP circumvents the strict autoregressive bottleneck by generating multiple tokens per step, thereby accelerating decoding speed. Figure 1: Token-level predictive entropy of the LLM over an example sequence. Within a cohesive semantic group, the entropy follows an approximately monotonically decreasing trend as the local context becomes increasingly constrained; at a semantic boundary, the entropy exhibits a sudden surge. AdaMTP exploits these entropy surges to partition the sequence into variable-length groups. However, existing MTP frameworks enforce a rigid, fixed-length prediction horizon for every token, overlooking the varying predictability of the underlying context. Natural language and code exhibit highly non-uniform information density. Within contiguous chunks such as common phrases or local code blocks, the sequence is highly predictable; notably, we observe that predictive uncertainty (entropy) approximately follows a monotonically decreasing trend in these regions (as illustrated in Figure 1). In contrast, transitions between different conceptual ideas or syntactic structures are characterized by sudden spikes in uncertainty. Forcing the auxiliary heads to predict a fixed number of tokens across these high-entropy boundaries inherently injects noisy, conflicting training signals. Because the auxiliary heads and the main language modeling head share the same latent representations, these noisy gradients backpropagate and cause severe representation interference, ultimately degrading the base model’s core capabilities. Motivated by these limitations, we propose AdaMTP (Adaptive Multi-Token Prediction), a novel framework that dynamically aligns the multi-token prediction horizon with the intrinsic predictability of the sequence. At the core of AdaMTP is an entropy-based data segmentation algorithm. By using the base LLM to estimate token-level predictive entropy, we identify sudden surges in uncertainty—which disrupt the aforementioned approximately monotonic decrease—as semantic boundaries, partitioning the sequence into cohesive, variable-length groups. Based on these partitions, we assign each token an adaptive prediction depth, defined as the distance to the end of its current group (or the subsequent group for boundary tokens). To incorporate this into training, we introduce a dynamically masked MTP loss: for any given token, the loss for future predictions that exceed its adaptive depth is masked out. This crucial design prevents the auxiliary heads from forcibly predicting across unpredictable boundaries, thereby attenuating the noisy gradients that degrade the model’s core capabilities. At inference time, AdaMTP provides two decoding modes. By default, it adopts the same fixed-horizon scheme as standard MTP, yet achieves faster inference; alternatively, an adaptive-horizon mode prunes low-confidence branches via the real-time entropy signal to cut verification cost—an efficiency edge that becomes pronounced under large-batch serving. We comprehensively evaluate AdaMTP on diverse benchmarks spanning mathematical reasoning, code generation, and general language proficiency. Using three representative base models—Llama-3.1 (8B), Qwen-2.5 (7B), and Gemma-3 (12B)—our extensive experiments demonstrate that AdaMTP improves on both task performance and inference efficiency. In terms of quality, it mitigates the representation interference inherent in standard MTP and consistently surpasses both the NTP and fixed-horizon MTP baselines in average score. In terms of efficiency, it retains and further strengthens the self-speculative acceleration of MTP, delivering substantial speedups over NTP while also decoding faster than standard MTP. Ultimately, these results indicate that adapting the MTP objective to local predictability is a more reliable way to retrofit pretrained LLMs with efficient multi-token generation. In summary, our main contributions are as follows: • We identify and empirically corroborate a key limitation of existing MTP training: uniformly predicting a fixed number of future tokens can adversely affect the pretrained backbone, which we attribute to the noisy supervision forced across high-entropy linguistic boundaries. • We propose AdaMTP, an adaptive training paradigm that uses entropy-based data segmentation to assign each token an adaptive prediction depth, together with a dynamically masked MTP objective that suppresses training signals beyond this depth to curb the resulting noisy gradients. • Extensive experiments across three backbones and eight benchmarks show that AdaMTP consistently outperforms both NTP and standard MTP in task performance while decoding faster—delivering substantial speedups over NTP and surpassing the inference speed of standard MTP. 2 Preliminaries Figure 2: Overview of AdaMTP training paradigm. (Left) Entropy-Based Data Segmentation: using the base model’s next-token entropy, the sequence is split into variable-length groups at points of sudden entropy surge, and each token is assigned an adaptive prediction depth. (Right) Dynamically Masked MTP Training: the shared LLM backbone feeds n parallel heads that predict future tokens; losses for predictions falling within each token’s adaptive depth are retained as valid signal, while those crossing group boundaries are masked out and ignored. 2.1 Next-Token Prediction (NTP) The standard training paradigm for LLMs relies on NTP task. Given a sequence of tokens x1:T=(x1,x2,…,xT)x_1:T=(x_1,x_2,…,x_T), the primary objective is to minimize the cross-entropy loss: ℒNTP=−∑t=1T−1logPθ(xt+1∣x1:t)L_NTP=- _t=1^T-1 P_θ(x_t+1 x_1:t) (1) where x1:tx_1:t denotes the context up to step t, and Pθ(xt+1∣x1:t)P_θ(x_t+1 x_1:t) represents the model’s predicted probability distribution over the vocabulary for the next token. During inference, NTP operates in a strictly autoregressive manner. The model generates a single token at a time by sampling from the predicted distribution. 2.2 Multi-Token Prediction (MTP) MTP extends the standard NTP paradigm by training the model to predict n future tokens xt+1:t+nx_t+1:t+n at each position, using a shared backbone together with one main head and n−1n-1 auxiliary heads. ℒMTP=−∑t=1T−nlogPθ(xt+1:t+n∣x1:t).L_MTP=- _t=1^T-n P_θ(x_t+1:t+n x_1:t). (2) In recent studies, the overall architecture θ is decoupled into a shared backbone with parameters θ′θ , a main language modeling head θ0 _0, and n−1n-1 auxiliary output heads with parameters θjj=1n−1\ _j\_j=1^n-1. The shared backbone maps the context x1:tx_1:t to a hidden representation z1:t=fθ′(x1:t)z_1:t=f_θ (x_1:t). The main head reuses z1:tz_1:t to predict the next token xt+1x_t+1, while the j-th auxiliary head predicts the token at offset j+1j+1, i.e. xt+j+1x_t+j+1. All n heads operate in parallel on the same representation, so that n future tokens are produced within a single forward pass. Assuming conditional independence of the future tokens given z1:tz_1:t, the joint distribution of the next n tokens is factorized as Pθ(xt+1:t+n∣x1:t)=Pθ0(xt+1∣z1:t)∏j=1n−1Pθj(xt+j+1∣z1:t), P_θ(x_t+1:t+n x_1:t)=P_ _0(x_t+1 z_1:t) _j=1^n-1P_ _j(x_t+j+1 z_1:t), (3) where z1:tz_1:t is a deterministic function of the context and Pθj(⋅∣z1:t)P_ _j(· z_1:t) denotes the distribution predicted by the j-th auxiliary head. In practice, rather than pretraining from scratch, these auxiliary heads are commonly attached to an already pretrained LLM and trained via fine-tuning, retrofitting multi-token prediction onto an existing NTP model at a fraction of the pretraining cost. During inference, the MTP model leverages its multiple heads to propose subsequent n candidate tokens simultaneously. To guarantee generation quality, this parallel drafting mechanism is paired with a verification step, effectively forming a self-speculative decoding framework. The model evaluates the proposed candidates in a single forward pass, accepts the valid tokens, and advances the context window accordingly. This approach accelerates inference while strictly preserving the original output distribution of the LLM. 3 Methodology In this section, we detail AdaMTP (Adaptive Multi-Token Prediction), our framework that dynamically adjusts the multi-token prediction horizon based on the intrinsic predictability of the sequence. AdaMTP consists of an entropy-based data segmentation algorithm, a two-stage adaptive training pipeline, and a dual-mode accelerated decoding mechanism for inference. 3.1 Entropy-Based Data Segmentation During training, standard MTP forces the model to predict a fixed number of future tokens at every time step, regardless of the difficulty or predictability of the context. This rigid approach can be highly detrimental: forcing the model to predict across high-entropy boundaries—where future tokens are conceptually unrelated to the current context—injects noisy, conflicting training signals. Since the auxiliary MTP heads and the main language modeling head share the same underlying hidden representations, these noisy gradients can severely interfere with the main head’s optimization, ultimately degrading the base model’s core capabilities. To alleviate this optimization burden and prevent representation interference, we propose an entropy-based segmentation algorithm that groups tokens according to their predictive uncertainty. Given a sequence of tokens X=(x1,x2,…,xN)X=(x_1,x_2,…,x_N), we use the exact same pre-trained base model designated for supervised fine-tuning (SFT) as our reference model to compute the entropy of the next-token distribution at each position t: Et=ℋ(P(⋅∣x<t))=−∑v∈P(v∣x<t)logP(v∣x<t), 385.92152pt$ E_t=H(P(· x_<t))=- _v P(v x_<t) P(v x_<t)$\,, (4) where V is the vocabulary. Using the SFT base model as the reference ensures that the computed uncertainty perfectly aligns with the target model’s intrinsic probability distribution. We then calculate the delta entropy between consecutive tokens as ΔEt=Et+1−Et E_t=E_t+1-E_t. Conceptually, as the model generates tokens within a cohesive semantic chunk, the local context becomes increasingly constrained, causing the predictive entropy to exhibit a roughly monotonically decreasing trend (i.e., uncertainty tends to diminish as the phrase nears completion). Conversely, a large positive ΔEt E_t signifies a sudden spike in uncertainty, typically corresponding to crossing a linguistic boundary or transitioning to a new semantic unit. This entropy dynamic, illustrated in Figure 1, implies that detecting these surges alone suffices to cleanly recover coherent semantic groups. Therefore, we segment the sequence into contiguous groups by splitting at indices where the surge in entropy exceeds a predefined threshold (ΔEt>τ E_t>τ). The threshold τ is calibrated via a dataset-level search so that the resulting average group size matches the total number of heads n (i.e., the maximum prediction depth), thereby aligning the segmentation granularity with the model’s multi-token prediction capacity. Let Gk=[sk,ek)G_k=[s_k,e_k) denote the k-th token group spanning from index sks_k to ek−1e_k-1. For a token xtx_t located within GkG_k, we define its adaptive prediction depth dtd_t as follows: dt=ek−t−1,if t<ek−1|Gk+1|,if t=ek−1d_t= casese_k-t-1,&if t<e_k-1\\ |G_k+1|,&if t=e_k-1 cases (5) where |Gk+1|=ek+1−sk+1|G_k+1|=e_k+1-s_k+1 denotes the length of the subsequent group. Intuitively, tokens strictly inside a predictable group only predict the remainder of their current group, while the boundary token (which precedes a high-entropy transition) is tasked with predicting the entirety of the next group. This design ensures that the training prediction depth flexibly adapts to the natural chunking of language. Throughout, we use horizon to denote the number of future tokens the model is asked to predict at a step, and (adaptive) depth dtd_t to denote the per-token training target assigned by our method. Since the model is equipped with only n−1n-1 auxiliary heads, the effective supervised depth is capped at the horizon n: for any token, whenever dt>nd_t>n, supervision is applied only up to offset n, i.e., min(dt,n) (d_t,n). 3.2 AdaMTP Training At the heart of AdaMTP is its adaptive training objective: instead of forcing every token to predict a fixed number of future tokens, we supervise each token only within its adaptive prediction depth dtd_t, masking out any prediction that crosses a semantic boundary. To instill this objective into a pretrained LLM while minimizing disruption to its foundational language modeling ability, we adopt a two-stage pipeline—an auxiliary-head warm-up followed by joint fine-tuning—following the standard training practice for retrofitting MTP heads onto pretrained models (Cai et al., 2024; Liu et al., 2025). Auxiliary-Head Warm-Up. Our training procedure builds upon a pretrained base language model. To equip the model with multi-token prediction capabilities, we augment the base LLM with n−1n-1 auxiliary output heads, each responsible for predicting a token at a specific future offset. Architecturally, each auxiliary head is implemented as a multi-layer perceptron (MLP) followed by a linear projection layer that maps the hidden states directly into the vocabulary space. The primary objective of this initial stage is to align these newly introduced heads with the base LLM. To achieve this, we freeze the LLM backbone and its original language modeling head, restricting parameter optimization entirely to the auxiliary heads. During this stage, we utilize self-distilled data for training, which is generated by feeding prompts into the base LLM and collecting its outputs. The training objective is formulated as: ℒwarm-up=∑j=1n−1∑t=1T−j−1ℒCE(Headj(t),xt+j+1),L_warm-up= _j=1^n-1 _t=1^T-j-1L_CE (Head_j(h_t),x_t+j+1 ), (6) where th_t is the frozen hidden state of the backbone at time step t, and ℒCEL_CE denotes the standard cross-entropy loss. Adaptive Joint Training. In the second stage, we conduct joint training by applying Low-Rank Adaptation (LoRA) across all model components, encompassing both the LLM backbone and the output heads. During this joint training process, we utilize the pre-computed adaptive depths dtd_t to selectively mask the MTP loss. Let th_t denote the hidden state at time step t. The base model predicts the next token xt+1x_t+1, while the j-th MTP head (1≤j<n1≤ j<n) predicts the future token xt+j+1x_t+j+1. Unlike traditional MTP training, which uniformly penalizes predictions up to a fixed depth regardless of context difficulty, our approach dynamically masks the loss for tokens beyond the adaptive depth dtd_t. Consequently, the adaptive MTP loss is defined as: ℒMTP=∑j=1n−1∑t=1T−j−1(j+1≤dt)⋅ℒCE(Headj(t),xt+j+1), splitL_MTP= _j=1^n-1 _t=1^T-j-1&I(j+1≤ d_t)\\ &·L_CE (Head_j(h_t),x_t+j+1 ), split (7) where (⋅)I(·) is the indicator function and ℒCEL_CE is the cross-entropy loss. To enable joint optimization, the total loss is formulated as a weighted sum of the standard language modeling loss ℒLML_LM (for the base model) and the adaptive MTP loss ℒMTPL_MTP: ℒtotal=ℒLM+λℒMTP,L_total=L_LM+ _MTP, (8) where λ controls the contribution of the auxiliary heads. This adaptive joint training prevents the model from attempting to predict excessively far into unpredictable futures, thereby concentrating its capacity on highly certain and structured predictions. 3.3 Inference Procedure During inference, the trained MTP heads are used for self-speculative decoding: at each step they draft several candidate future tokens, and the base model verifies these candidates in a single forward pass, accepting only the prefix consistent with its own predictions. Because every accepted token is validated by the base model, this draft-then-verify procedure is lossless—it yields exactly the same output as standard autoregressive decoding while reducing the number of sequential steps and thus accelerating generation. Within this framework, we provide two generation settings: Fixed-Horizon Generation. In this default setting, the MTP heads always draft the full horizon of n candidate tokens at every step. Following Medusa (Cai et al., 2024), these candidates are organized into a token tree and verified by the base model in a single forward pass. Since this verification pass is dominated by the base model’s forward computation and, at small batch sizes, is only weakly sensitive to the number of candidates it checks, drafting the maximum number of candidates maximizes the expected number of tokens accepted per step. Adaptive-Horizon Generation. Alternatively, we offer an adaptive strategy that dynamically determines how many candidates to generate from the real-time entropy delta: the MTP heads stop drafting further into the future once the entropy increase between consecutive predicted tokens exceeds a threshold. This prunes the candidate tree and reduces the number of tokens the base model must verify. Since it only forgoes low-confidence tail candidates that would largely have been rejected anyway, under an appropriate threshold it incurs no loss in per-step acceptance length relative to fixed-horizon generation. The resulting saving is marginal for single-sample decoding—where verifying a few extra candidates adds negligible latency—but becomes substantial under large-batch, compute-bound serving, whose latency scales with the total number of candidates verified. Math500 GSM8K MBPP MBPP+ HumanEval HumanEval+ MMLU IFEval Avg. Base 3.20 9.02 62.43 52.12 37.80 30.49 63.45 16.91 34.43 NTP 5.60 11.30 61.34 50.95 42.26 35.37 63.64 20.08 36.32 MTP 5.00 11.30 60.38 50.00 41.46 35.98 63.33 19.74 35.90 Llama3.1-8B AdaMTP 7.20 13.12 61.64 50.26 42.68 35.98 63.67 20.26 36.85 Base 62.80 54.85 75.20 64.06 78.05 71.20 71.76 41.65 64.95 NTP 49.20 52.75 76.50 64.29 77.44 69.26 71.58 42.35 62.92 MTP 48.60 47.69 74.34 63.76 76.22 70.12 71.77 40.29 61.60 Qwen2.5-7B AdaMTP 49.40 50.72 76.19 64.81 78.05 71.95 71.79 42.57 63.19 Base 0.00 9.76 72.22 58.99 45.12 35.37 24.39 29.52 34.42 NTP 9.00 9.33 70.63 58.99 61.02 56.10 70.73 29.94 45.72 MTP 12.60 13.86 65.34 54.50 59.15 51.22 71.82 31.41 44.99 Gemma3-12B AdaMTP 16.00 14.81 66.08 54.50 60.37 53.05 72.04 32.13 46.12 Table 1: Performance comparison among different prediction paradigms (NTP, MTP, and our AdaMTP) across diverse tasks and benchmarks. For each backbone, the best average result (Avg.) among NTP, MTP, and AdaMTP is highlighted in bold. 4 Experiments 4.1 Experimental Settings Datasets. In alignment with the previous work Liu et al. (2025), our training corpus is assembled from the Math Hendrycks et al. (2021), Evol-Instruct-Code Luo et al. (2023); Chaudhary (2023), and Alpaca-GPT4 Peng et al. (2023) datasets. The training procedure is divided into two phases. Initially, the entire dataset is leveraged to perform self-distillation. Subsequently, the second phase utilizes a randomly sampled subset of 10,000 instances, distributed across mathematical, programming, and general domains in a 4:4:2 ratio. To rigorously assess the proposed methodologies, we employ a diverse suite of benchmarks: Math500 Lightman et al. (2023) and GSM8K Cobbe et al. (2021) (both evaluated in a 4-shot setting) for mathematical reasoning; MBPP, MBPP+ Austin et al. (2021); Liu et al. (2023), HumanEval, and HumanEval+ Chen et al. (2021); Liu et al. (2023) for code generation capabilities; alongside MMLU Hendrycks et al. (2020) and IFEval Zhou et al. (2023a) to measure general proficiency. Evaluation Metrics. To evaluate task performance, we report accuracy for mathematical and general-domain benchmarks, while employing the pass@1 metric for code generation tasks. Furthermore, we assess efficiency via a speedup ratio relative to standard autoregressive NTP decoding on the same model. Base LLMs. Our experimental framework employs three base large language models: Llama-3.1 (8B), Qwen-2.5 (7B), and Gemma-3 (12B). This selection was made to ensure a comprehensive evaluation across a diverse spectrum of model architectures and parameter capacities. Baselines. To evaluate generation efficacy, we benchmark our approach against two established paradigms: standard NTP and MTP. These baselines serve to measure the models’ capacity to produce accurate and contextually appropriate text. For the efficiency analysis, standard autoregressive NTP decoding provides the 1×1× reference against which speedups are measured, while we compare AdaMTP with standard MTP, both of which employ self-speculative decoding for lossless acceleration. Implementation Details. During the initial head warm-up stage, we freeze the LLM backbone and exclusively train the prediction heads for 1 epoch with a learning rate of 1×10−31× 10^-3. In the subsequent stage, we utilize LoRA (rank r=32r=32, α=16α=16) to fine-tune the model for 3 epochs with a learning rate of 1×10−51× 10^-5. We set the prediction depth to n=4n=4 and the auxiliary MTP loss weight to λ=0.1λ=0.1 by default. To ensure a fair comparison, identical training configurations are applied to the standard MTP baseline. All experiments are conducted on four NVIDIA H800 GPUs with a total batch size of 256. 4.2 Overall Performance Table 1 compares AdaMTP against NTP and standard MTP across three base models and eight benchmarks; we highlight the key findings below. AdaMTP consistently achieves the best overall performance. Across all three backbones, AdaMTP attains the highest average score—36.85 on Llama-3.1-8B, 63.19 on Qwen-2.5-7B, and 46.12 on Gemma-3-12B—surpassing both NTP and standard MTP. This consistency across model families and scales shows that adaptively aligning the prediction horizon with sequence predictability is a robust, architecture-agnostic strategy. Intriguingly, on Qwen-2.5-7B all fine-tuning paradigms (including NTP) fall below the Base model, an observation consistent with prior findings (Liu et al., 2025); we attribute this to the distribution shift of our fine-tuning corpus rather than the MTP training paradigm, as plain NTP is affected identically. Even in this regime AdaMTP still outperforms both NTP and standard MTP, indicating that the adaptive objective remains beneficial regardless of data quality. As our focus is isolating the effect of the adaptive horizon relative to standard MTP, we leave higher-quality data curation to future work. GSM8K HumanEval IFEval Llama3.1 NTP 1.00× 1.00× 1.00× MTP 1.65× 1.86× 1.48× AdaMTP 2.12× 2.01× 1.52× Qwen2.5 NTP 1.00× 1.00× 1.00× MTP 1.84× 1.55× 1.40× AdaMTP 1.87× 1.56× 1.43× Gemma3 NTP 1.00× 1.00× 1.00× MTP 2.38× 1.59× 1.57× AdaMTP 2.75× 1.62× 1.61× Table 2: Inference speedup comparison of NTP, MTP, and AdaMTP across three backbones. Standard MTP suffers from representation interference. Standard MTP consistently underperforms NTP on average across all three models. This corroborates our hypothesis: forcing the auxiliary heads to predict a rigid number of tokens across high-entropy boundaries injects noisy, conflicting gradients into the shared backbone, corrupting its core capabilities. AdaMTP instead masks these unpredictable predictions, suppressing the interfering signals and exceeding the performance of NTP. Gains span math reasoning, code, and general tasks. AdaMTP improves consistently across task categories, most notably on mathematical reasoning: since these tasks demand long-range planning, concentrating multi-token supervision within cohesive, predictable chunks yields cleaner training signals. On code generation, it recovers the losses incurred by standard MTP and matches or exceeds NTP. It also preserves general-domain ability, achieving the best IFEval and MMLU scores among the trained paradigms. 4.3 Inference Acceleration Beyond task performance, a central promise of the MTP paradigm is its ability to accelerate decoding via self-speculative generation. Table 2 reports the speedup ratio of NTP, standard MTP, and AdaMTP on GSM8K, HumanEval, and IFEval across all three backbones, using autoregressive NTP decoding as the reference (1.00×1.00×); here AdaMTP adopts its default Fixed-Horizon decoding, the same scheme used by standard MTP. Across every backbone and benchmark, AdaMTP delivers substantial acceleration over NTP, ranging from 1.43×1.43× up to 2.75×2.75×, with the largest gains on the highly structured GSM8K task (e.g., 2.75×2.75× on Gemma-3-12B). More importantly, AdaMTP consistently surpasses standard MTP—for instance, improving the GSM8K speedup from 1.65×1.65× to 2.12×2.12× on Llama-3.1-8B—while simultaneously achieving superior task performance. Because the two share an identical inference procedure, this acceleration gain over MTP arises purely from our adaptive training objective, which yields auxiliary heads whose drafts are more frequently accepted by the verifier, allowing AdaMTP to enhance the inference speedups that make MTP attractive in practice. Figure 3: Impact of the number of prediction heads n on Llama-3.1-8B (n=1n=1 corresponds to NTP). AdaMTP stays consistently above standard MTP. 4.4 Impact of the Number of Prediction Heads We vary the total number of heads from n=1n=1 (i.e., NTP) to n=6n=6 on GSM8K with Llama-3.1-8B, holding all other settings fixed. As shown in Figure 3(a), the accuracy of standard MTP declines almost monotonically as n grows, falling from 11.6011.60 at n=2n=2 to 9.689.68 at n=6n=6—well below the 11.3011.30 NTP reference. This is a direct consequence of representation interference: each additional head forces the shared backbone to predict one token further ahead, crossing more high-entropy semantic boundaries and injecting proportionally more noisy gradients. AdaMTP instead masks precisely these cross-boundary predictions, so it stays above NTP throughout and peaks at n=4n=4 with 13.1213.12; accordingly, its gap over MTP widens steadily with n (from +0.53+0.53 at n=2n=2 to +1.90+1.90 at n=6n=6). In terms of inference speedup (Figure 3(b)), AdaMTP likewise dominates standard MTP at every operating point and degrades more gracefully as n increases, since concentrating supervision within predictable chunks keeps even its deepest drafts frequently acceptable to the verifier. Overall, across every head budget n, AdaMTP consistently surpasses standard MTP in both accuracy and inference speedup, confirming that its advantage is robust to the number of prediction heads rather than tied to a particular setting. 4.5 Discussion on Adaptive-Horizon Decoding We now compare AdaMTP’s two inference modes—Fixed-Horizon and Adaptive-Horizon Generation—to quantify the benefit of entropy-based candidate pruning. To this end, we evaluate Adaptive-Horizon against the Fixed-Horizon baseline on GSM8K and HumanEval with Llama3.1-8B and Qwen2.5-7B, measuring the average number of tokens verified per step. As shown in Figure 4, the adaptive strategy consistently reduces the verification burden while keeping accuracy statistically indistinguishable from the baseline, confirming that the pruned tail candidates were largely redundant. This reduction, however, yields little speedup for single-sample inference, which is memory-bandwidth bound: latency is dominated by streaming the model weights from DRAM, so trimming a few candidates neither reduces weight loading nor shortens the already-parallel verification pass. Under large-batch, compute-bound inference, by contrast, latency grows with the total number of candidate tokens, so pruning directly cuts the workload and yields clear throughput gains as batch size increases. Figure 4: Average number of candidate tokens verified per decoding step under the fixed-horizon and adaptive-horizon generation. The adaptive strategy reduces the per-step verification cost without degrading task accuracy. 5 Related Work 5.1 Multi-Token Prediction Qi et al. (2020) introduce n-step-ahead prediction in sequence-to-sequence pretraining to encourage future planning and reduce overfitting to local correlations. Gloeckle et al. (2024) formalize multi-token prediction by adding parallel heads during pretraining, improving reasoning over next-token prediction, and Basharin et al. (2024) generalize these independent heads to a rank-r canonical tensor decomposition to better capture dependencies among future tokens. MTP has also been adopted at scale during the pretraining of industrial LLMs (Liu et al., 2024; Xiaomi et al., 2025) for better data efficiency and long-horizon planning. Beyond pretraining from scratch, a more efficient line retrofits an existing pretrained NTP model into an MTP architecture (Cai et al., 2025): Medusa (Cai et al., 2024) attaches multiple lightweight heads that each forecast a token at a distinct future offset at a fraction of the pretraining cost; Samragh et al. (2025) equip the model with gated LoRA modules and a learnable sampler for simultaneous multi-token prediction; and L-MTP (Liu et al., 2025) adds a leap-based mechanism that predicts non-sequential positions in a single forward pass to capture longer-range dependencies. These methods collectively show that retrofitting a pretrained NTP model with extra prediction heads is a practical, cost-effective route to multi-token prediction, yielding substantial inference speedups while preserving generation quality. In addition, some approaches use MTP purely as an auxiliary training objective to improve the generation quality of NTP models, and thus provide no inference acceleration, since the model still decodes one token at a time at inference. For instance, MuToR (Gerontopoulos et al., 2025) interleaves learnable register tokens into the training sequence to predict future targets, while TOP (Zuhri et al., 2025) replaces exact future-token prediction with a learning-to-rank loss that orders upcoming tokens by proximity. 5.2 LLM Inference Acceleration The growing cost of LLM inference has motivated acceleration methods that target different bottlenecks. One line reduces the per-step cost via model compression—quantization (Hubara et al., 2018; Kim et al., 2023; Lin et al., 2024), pruning (Frantar and Alistarh, 2023; Sun et al., 2023; Ma et al., 2023; Gao et al., 2024), and knowledge distillation (Gu et al., 2023; Hinton et al., 2015; Hsieh et al., 2023; Ho et al., 2023)—and efficient attention, whether linear (Katharopoulos et al., 2020; Yang et al., 2024), sparse (Child et al., 2019; Lu et al., 2025), or low-rank (Liu et al., 2024). A second line improves system-level throughput through operator fusion (Dao et al., 2022; Dao, 2023), KV-cache management (Kwon et al., 2023; Zheng et al., 2024), and parallelism (NVIDIA, 2023). Orthogonally, a third line reduces the number of sequential decoding steps: speculative decoding (Leviathan et al., 2023; Chen et al., 2023; Miao et al., 2024) losslessly amortizes autoregressive generation by cheaply drafting candidate tokens and validating them in a single target-model forward pass, with candidates proposed either by a separate lightweight model (Leviathan et al., 2023; Yang et al., 2025; Zhou et al., 2023b) or by the target model itself via auxiliary prediction heads (Stern et al., 2018; Cai et al., 2024; Li et al., 2024). 5.3 Adaptive Modeling Motivated by the non-uniform information density of natural language, recent work dynamically adjusts processing granularity rather than treating all tokens uniformly (Barrault et al., 2024). The two most relevant to ours both adapt granularity on the input or representation side—the Byte Latent Transformer (Pagnoni et al., 2025) patches bytes for the encoder, and Dynamic Large Concept Models (Qu et al., 2025) compress tokens into concepts—yet still emit a single token (or byte) per step and therefore cannot accelerate decoding. AdaMTP instead applies entropy-based, variable-length segmentation to the multi-token prediction itself: it turns each segment into an adaptive prediction depth that masks cross-boundary supervision, simultaneously protecting the pretrained backbone and enabling accelerated multi-token generation. 6 Conclusion In this paper, we identified a key limitation of existing MTP: its fixed-length horizon ignores the non-uniform information density of language, forcing auxiliary heads to predict across high-entropy boundaries and injecting noisy gradients that degrade the shared backbone. To address this, we proposed AdaMTP, which uses entropy-based segmentation to assign each token an adaptive prediction depth and applies a dynamically masked MTP objective that suppresses loss for cross-boundary predictions. Across mathematical reasoning, code, and general benchmarks on three backbones, AdaMTP consistently surpasses NTP and standard MTP in both task performance and inference speedup, showing that aligning the multi-token objective with local predictability is a robust, architecture-agnostic way to retrofit pretrained LLMs with efficient multi-token generation. Promising future directions include higher-quality data curation and stronger adaptive-horizon pruning for high-throughput batched inference. References J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, et al. (2021) Program synthesis with large language models. arXiv preprint arXiv:2108.07732. Cited by: 1st item. L. Barrault, P. Duquenne, M. Elbayad, A. Kozhevnikov, B. Alastruey, P. Andrews, M. Coria, G. Couairon, M. R. Costa-jussà, D. Dale, et al. (2024) Large concept models: language modeling in a sentence representation space. arXiv preprint arXiv:2412.08821. Cited by: §5.3. A. Basharin, A. Chertkov, and I. Oseledets (2024) Faster language models with better multi-token prediction using tensor decomposition. arXiv preprint arXiv:2410.17765. Cited by: §5.1. T. Cai, Y. Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, and T. Dao (2024) Medusa: simple llm inference acceleration framework with multiple decoding heads. arXiv preprint arXiv:2401.10774. Cited by: §3.2, §3.3, §5.1, §5.2. Y. Cai, X. Liang, X. Wang, J. Ma, H. Liang, J. Luo, X. Zuo, L. Duan, Y. Yin, and X. Chen (2025) Fastmtp: accelerating llm inference with enhanced multi-token prediction. arXiv preprint arXiv:2509.18362. Cited by: §5.1. S. Chaudhary (2023) Code alpaca: an instruction-following llama model for code generation. Cited by: 1st item. C. Chen, S. Borgeaud, G. Irving, J. Lespiau, L. Sifre, and J. Jumper (2023) Accelerating large language model decoding with speculative sampling. arXiv preprint arXiv:2302.01318. Cited by: §5.2. M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. D. O. Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, et al. (2021) Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. Cited by: 1st item. R. Child, S. Gray, A. Radford, and I. Sutskever (2019) Generating long sequences with sparse transformers. arXiv preprint arXiv:1904.10509. Cited by: §5.2. K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, et al. (2021) Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. Cited by: 1st item. T. Dao, D. Fu, S. Ermon, A. Rudra, and C. Ré (2022) Flashattention: fast and memory-efficient exact attention with io-awareness. Advances in neural information processing systems 35, p. 16344–16359. Cited by: §5.2. T. Dao (2023) Flashattention-2: faster attention with better parallelism and work partitioning. arXiv preprint arXiv:2307.08691. Cited by: §5.2. E. Frantar and D. Alistarh (2023) Sparsegpt: massive language models can be accurately pruned in one-shot. In International conference on machine learning, p. 10323–10337. Cited by: §5.2. S. Gao, C. Lin, T. Hua, T. Zheng, Y. Shen, H. Jin, and Y. Hsu (2024) Disp-llm: dimension-independent structural pruning for large language models. Advances in Neural Information Processing Systems 37, p. 72219–72244. Cited by: §5.2. A. Gerontopoulos, S. Gidaris, and N. Komodakis (2025) Multi-token prediction needs registers. arXiv preprint arXiv:2505.10518. Cited by: §5.1. F. Gloeckle, B. Y. Idrissi, B. Rozière, D. Lopez-Paz, and G. Synnaeve (2024) Better & faster large language models via multi-token prediction. arXiv preprint arXiv:2404.19737. Cited by: §5.1. Y. Gu, L. Dong, F. Wei, and M. Huang (2023) Minillm: knowledge distillation of large language models. arXiv preprint arXiv:2306.08543. Cited by: §5.2. D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt (2020) Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300. Cited by: 1st item. D. Hendrycks, C. Burns, S. Kadavath, A. Arora, S. Basart, E. Tang, D. Song, and J. Steinhardt (2021) Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874. Cited by: 1st item. G. Hinton, O. Vinyals, and J. Dean (2015) Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. Cited by: §5.2. N. Ho, L. Schmid, and S. Yun (2023) Large language models are reasoning teachers. In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: long papers), p. 14852–14882. Cited by: §5.2. C. Hsieh, C. Li, C. Yeh, H. Nakhost, Y. Fujii, A. Ratner, R. Krishna, C. Lee, and T. Pfister (2023) Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes. In Findings of the Association for Computational Linguistics: ACL 2023, p. 8003–8017. Cited by: §5.2. I. Hubara, M. Courbariaux, D. Soudry, R. El-Yaniv, and Y. Bengio (2018) Quantized neural networks: training neural networks with low precision weights and activations. journal of machine learning research 18 (187), p. 1–30. Cited by: §5.2. A. Katharopoulos, A. Vyas, N. Pappas, and F. Fleuret (2020) Transformers are rnns: fast autoregressive transformers with linear attention. In International conference on machine learning, p. 5156–5165. Cited by: §5.2. S. Kim, C. Hooper, A. Gholami, Z. Dong, X. Li, S. Shen, M. W. Mahoney, and K. Keutzer (2023) Squeezellm: dense-and-sparse quantization. arXiv preprint arXiv:2306.07629. Cited by: §5.2. W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica (2023) Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th symposium on operating systems principles, p. 611–626. Cited by: §5.2. Y. Leviathan, M. Kalman, and Y. Matias (2023) Fast inference from transformers via speculative decoding. In International Conference on Machine Learning, p. 19274–19286. Cited by: §5.2. Y. Li, F. Wei, C. Zhang, and H. Zhang (2024) Eagle: speculative sampling requires rethinking feature uncertainty. arXiv preprint arXiv:2401.15077. Cited by: §5.2. H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe (2023) Let’s verify step by step. In The twelfth international conference on learning representations, Cited by: 1st item. J. Lin, J. Tang, H. Tang, S. Yang, W. Chen, W. Wang, G. Xiao, X. Dang, C. Gan, and S. Han (2024) Awq: activation-aware weight quantization for on-device llm compression and acceleration. Proceedings of machine learning and systems 6, p. 87–100. Cited by: §5.2. A. Liu, B. Feng, B. Xue, B. Wang, B. Wu, C. Lu, C. Zhao, C. Deng, C. Zhang, C. Ruan, et al. (2024) Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437. Cited by: §5.1, §5.2. J. Liu, C. S. Xia, Y. Wang, and L. Zhang (2023) Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. Advances in neural information processing systems 36, p. 21558–21572. Cited by: 1st item. X. Liu, X. Xia, W. Zhao, M. Zhang, X. Yu, X. Su, S. Yang, S. Ng, and T. Chua (2025) L-mtp: leap multi-token prediction beyond adjacent context for large language models. arXiv preprint arXiv:2505.17505. Cited by: §3.2, 1st item, §4.2, §5.1. E. Lu, Z. Jiang, J. Liu, Y. Du, T. Jiang, C. Hong, S. Liu, W. He, E. Yuan, Y. Wang, et al. (2025) Moba: mixture of block attention for long-context llms. arXiv preprint arXiv:2502.13189. Cited by: §5.2. Z. Luo, C. Xu, P. Zhao, Q. Sun, X. Geng, W. Hu, C. Tao, J. Ma, Q. Lin, and D. Jiang (2023) Wizardcoder: empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568. Cited by: 1st item. X. Ma, G. Fang, and X. Wang (2023) Llm-pruner: on the structural pruning of large language models. Advances in neural information processing systems 36, p. 21702–21720. Cited by: §5.2. X. Miao, G. Oliaro, Z. Zhang, X. Cheng, Z. Wang, Z. Zhang, R. Y. Y. Wong, A. Zhu, L. Yang, X. Shi, et al. (2024) Specinfer: accelerating large language model serving with tree-based speculative inference and verification. In Proceedings of the 29th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 3, p. 932–949. Cited by: §5.2. NVIDIA (2023) TensorRT-LLM. External Links: Link Cited by: §5.2. A. Pagnoni, R. Pasunuru, P. Rodriguez, J. Nguyen, B. Muller, M. Li, C. Zhou, L. Yu, J. E. Weston, L. Zettlemoyer, et al. (2025) Byte latent transformer: patches scale better than tokens. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 9238–9258. Cited by: §5.3. B. Peng, C. Li, P. He, M. Galley, and J. Gao (2023) Instruction tuning with gpt-4. arXiv preprint arXiv:2304.03277. Cited by: 1st item. W. Qi, Y. Yan, Y. Gong, D. Liu, N. Duan, J. Chen, R. Zhang, and M. Zhou (2020) ProphetNet: predicting future n-gram for sequence-to-sequencepre-training. In Findings of the Association for Computational Linguistics: EMNLP 2020, p. 2401–2410. Cited by: §5.1. X. Qu, S. Wang, Z. Huang, K. Hua, F. Yin, R. Zhu, J. Zhou, Q. Min, Z. Wang, Y. Li, et al. (2025) Dynamic large concept models: latent reasoning in an adaptive semantic space. arXiv preprint arXiv:2512.24617. Cited by: §5.3. M. Samragh, A. Kundu, D. Harrison, K. Nishu, D. Naik, M. Cho, and M. Farajtabar (2025) Your llm knows the future: uncovering its multi-token prediction potential. arXiv preprint arXiv:2507.11851. Cited by: §5.1. M. Stern, N. Shazeer, and J. Uszkoreit (2018) Blockwise parallel decoding for deep autoregressive models. Advances in Neural Information Processing Systems 31. Cited by: §5.2. M. Sun, Z. Liu, A. Bair, and J. Z. Kolter (2023) A simple and effective pruning approach for large language models. arXiv preprint arXiv:2306.11695. Cited by: §5.2. L. Xiaomi, B. Xia, B. Shen, D. Zhu, D. Zhang, G. Wang, H. Zhang, H. Liu, J. Xiao, J. Dong, et al. (2025) MiMo: unlocking the reasoning potential of language model–from pretraining to posttraining. arXiv preprint arXiv:2505.07608. Cited by: §5.1. S. Yang, S. Huang, X. Dai, and J. Chen (2025) Multi-candidate speculative decoding. In CCF International Conference on Natural Language Processing and Chinese Computing, p. 335–348. Cited by: §5.2. S. Yang, B. Wang, Y. Zhang, Y. Shen, and Y. Kim (2024) Parallelizing linear transformers with the delta rule over sequence length. Advances in neural information processing systems 37, p. 115491–115522. Cited by: §5.2. L. Zheng, L. Yin, Z. Xie, C. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, et al. (2024) Sglang: efficient execution of structured language model programs. Advances in neural information processing systems 37, p. 62557–62583. Cited by: §5.2. J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou (2023a) Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911. Cited by: 1st item. Y. Zhou, K. Lyu, A. S. Rawat, A. K. Menon, A. Rostamizadeh, S. Kumar, J. Kagy, and R. Agarwal (2023b) Distillspec: improving speculative decoding via knowledge distillation. arXiv preprint arXiv:2310.08461. Cited by: §5.2. Z. M. Zuhri, E. H. Fuadi, and A. F. Aji (2025) Predicting the order of upcoming tokens improves language modeling. arXiv preprint arXiv:2508.19228. Cited by: §5.1.