Paper deep dive
Personalized LLM Recommendation with Semantic IDs: Autoregressive Beam Search vs. Multi-Token Prediction
Sejoon Oh, Fernando Amat Gil, Evan Smothers, Dan Iter, Mark Thornburg, Dawit Mureja Argaw, Sourabh Medapati, Thea Wang, Ria Verma, Qiuling Xu, Harshit Syal, Ehsan Gholami, Moumita Bhattacharya, Ashish Rastogi
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large-scale personalization is shifting from embedding-table retrieval to LLM-based recommenders that generate the Semantic ID (SID) of the item to recommend, where each SID is a short sequence of discrete hierarchical content codes. A single instruction-tuned model can then unify retrieval, content understanding, and language, and recommend brand-new titles from their content alone using SID; then, how should the model generate the SID? Traditionally, models rely on a single language modeling head that generates tokens autoregressively, typically using constrained beam search. In contrast, an emerging body of work explores multi-token prediction (MTP), where dedicated heads predict all tokens in parallel, substantially reducing the decoding latency associated with beam search. We report an industry-scale, apples-to-apples comparison of these two generation paradigms: single-head autoregressive decoding vs. MTP. We introduce a new auxiliary ranking loss, defined over the joint SID score, to train the MTP objective. We evaluate the SID training objectives on both quality and training/inference cost, inside a multi-task LLM that jointly does personalized retrieval, content question-answering (QA), and instruction following over a large streaming-media catalog. Our experiments show single-head beam search consistently beats MTP on most product-critical metrics: ranking and content QA, and trains at roughly 2.6x lower dollar cost on Qwen3-8B scale; MTP wins only on raw inference throughput. Around this finding, we detail an efficient training and inference stack and report end-to-end gains over the previous non-SID baseline equipped with an auxiliary ranking head: +77% item cold-start ranking, +22% content-QA accuracy, +2.8x faster training, with language metric regression traced to SID vocabulary expansion. We address the language regression by reinforcement learning post training.
Tags
Links
- Source: https://arxiv.org/abs/2610.15864v1
- Canonical: https://arxiv.org/abs/2610.15864v1
Trouble viewing inline? Open PDF directly →
Full Text
43,782 characters extracted from source content.
Expand or collapse full text
Personalized, aligned, long-term memory for AI systems Personalized LLM Recommendation with Semantic IDs: Autoregressive Beam Search vs. Multi-Token Prediction at Industry Scale Sejoon Oh †thanks: Project lead / corresponding author: sejoono@netflix.com. Fernando Amat Gil Evan Smothers Dan Iter Mark Thornburg Affiliation: Dawit Mureja Argaw Sourabh Medapati Thea Wang Ria Verma Qiuling Xu Affiliation: Harshit Syal Ehsan Gholami Moumita Bhattacharya Ashish Rastogi Affiliation: Netflix Personalization Foundations Team Abstract Large-scale personalization is shifting from embedding-table retrieval to LLM-based recommenders that generate the Semantic ID (sid) of the item to recommend, where each sid is a short sequence of discrete hierarchical content codes. A single instruction-tuned model can then unify retrieval, content understanding, and language, and recommend brand-new titles from their content alone using sid; then, how should the model generate the sid? Traditionally, models rely on a single language modeling head that generates tokens autoregressively, typically using constrained beam search. In contrast, an emerging body of work explores multi-token prediction (MTP), where dedicated heads predict all tokens in parallel, substantially reducing the decoding latency associated with beam search. We report an industry-scale, apples-to-apples comparison of these two generation paradigms: single-head autoregressive decoding vs. MTP. We introduce a new auxiliary ranking loss, defined over the joint SID score, to train the MTP objective. We evaluate the sid training objectives on both quality and training/inference cost, inside a multi-task LLM that jointly does personalized retrieval, content question-answering (QA), and instruction following over a large streaming-media catalog. Our experiments show single-head beam search consistently beats MTP on most product-critical metrics: ranking and content QA, and trains at roughly 2.6x lower dollar cost on Qwen3-8B scale; MTP wins only on raw inference throughput. Around this finding, we detail an efficient training and inference stack and report end-to-end gains over the previous non-sid baseline equipped with an auxiliary ranking head: +77% item cold-start ranking, +22% content-QA accuracy, +2.8x faster training, with language metric regression traced to sid vocabulary expansion. We address the language regression by reinforcement learning post training. 1 Introduction Recent large-scale personalization systems increasingly leverage a large language model (LLM) to unify retrieval, ranking, content understanding, and natural-language understanding. Such an LLM ingests a member’s interaction history and context and must (i) retrieve relevant catalog entities, (i) understand title content well enough to answer natural-language questions about it (content QA), and (i) follow general instructions. We treat content understanding as a first-class training and evaluation objective alongside ranking, since it lets the model reason about why a title fits a member, ground recommendations in metadata (not just popularity), and generalize to titles with little or no engagement history. Modern systems refer to catalog entities not by arbitrary integer ids but by Semantic IDs (sids): short sequences of discrete codes produced by residual quantization (Lee et al., 2022; Tay et al., 2022; Rajput et al., 2023; Singh et al., 2024) of a content embedding, so that semantically similar titles share early-level codes. The model then generates the target’s sid on its own output head, which removes the catalog-sized classification head, unifies retrieval with the language and content-understanding tasks, and greatly improves cold-start because a new title inherits a meaningful (non-random) sid from its content embedding. This recipe now underlies several industrial systems (YouTube’s PLUM (He et al., 2025), Spotify’s GLIDE (D’Amico et al., 2026), Kuaishou’s OneRec (Deng et al., 2025), Meta’s generative recommenders (Zhai et al., 2024)), which report substantial online gains. This raises the question at the focus of this paper: how should the LLM generate sid? (a) Single-head autoregressive decoding emits the codes one at a time on the standard LM head with constrained beam search, which is also the classical recipe of DSI and TIGER (Tay et al., 2022; Rajput et al., 2023) and the fine-tuning stage of PLUM (He et al., 2025) and GLIDE (D’Amico et al., 2026). (b) Multi-token prediction (MTP) attaches L dedicated heads to a shared trunk and predicts all codes in parallel (Gloeckle et al., 2024). Leveraging this parallel decoding, MTP can generate sid faster than the beam search in inference time. Only a few studies such as RPG (Hou et al., 2025) and STAMP (Zhan et al., 2026) have applied MTP in generative recommendations. Those studies achieve higher ranking metrics or training efficiency by leveraging parallel sid prediction. However, these arguments are often made on small, retrieval-only academic benchmarks. Using an open-source model with an industry-scale streaming dataset as a controlled testbed, we reach the conclusion in the regime that matters for deployment: single-head constrained beam search beats MTP on most product-critical metrics (ranking and content QA), with a significantly lower training cost on Qwen3-8B scale. Meanwhile, MTP outperforms on inference throughput only. Our contributions are: • The industry-scale, apples-to-apples comparison of single-head autoregressive decoding vs. MTP as sid generation paradigms. A quality-and-cost comparison of training objectives, inside a unified multi-task LLM and large-scale streaming dataset, on ranking, content QA, language, and training/inference performance (Section 4.2). • An auxiliary ranking loss for MTP, a new objective that optimizes the item-level joint sid score so that MTP’s parallel heads produce calibrated, rankable item scores despite their conditional-independence assumption (Section 3.2.2). • An efficient training and inference stack: Muon (Jordan et al., 2024), on-the-fly packing with cross-rank batch mixing, Cut Cross-Entropy (Wijmans et al., 2024), and HSDP (Zhao et al., 2023) for training (>2×>2× faster at −73%-73\% cost); and vLLM-based constrained, catalog-aware beam-search decoding for inference (Kwon et al., 2023), whose trie- and catalog-level constraints guarantee every candidate names a real, request-eligible entity while prefix caching reuses the long member-history KV cache across beams (Sections 3.3–3.4). • A comprehensive evaluation and ablations covering new user and item recommendations, plus controlled ablations of LoRA (Hu et al., 2022) vs. full fine-tuning, Muon (Jordan et al., 2024) vs. AdamW (Loshchilov and Hutter, 2019) optimizers, and content QA tasks (Section 4). 2 Related Work Generative retrieval and semantic IDs. The Differentiable Search Index (Tay et al., 2022) first framed retrieval as sequence-to-sequence generation of document identifiers. TIGER (Rajput et al., 2023) introduced semantic IDs for recommendation, obtaining discrete codes from an RQ-VAE (Lee et al., 2022) over item content embeddings and autoregressively generating the next item’s id. (Singh et al., 2024) show semantic IDs generalize better than random atomic ids for ranking. At industrial scale, PLUM (He et al., 2025) adapts a pre-trained LLM to generative retrieval via continued pre-training on sid-tokenized behavior, GLIDE (D’Amico et al., 2026) frames podcast discovery as instruction-following over a sid-discretized catalog, and OneRec (Deng et al., 2025) unifies retrieval and ranking in one generative model. We adopt this representation but place the sid vocabulary on a general-purpose instruction-tuned LLM’s own head, so that retrieval, content QA, and language tasks share a single decoder, rather than training a bespoke retrieval-only transformer. Autoregressive beam search vs. parallel/MTP decoding. MTP was introduced as an auxiliary objective that predicts n future tokens with n independent heads on a shared trunk, improving sample efficiency and enabling self-speculative decoding (Gloeckle et al., 2024); it is now a component of frontier LLMs (DeepSeek-AI, 2024). In generative recommendation (GR), a recent wave applies this idea to sids to escape the autoregressive bottleneck. RPG (Hou et al., 2025) predicts long (up to 64-token) unordered sids in parallel with an MTP loss and graph-constrained decoding for higher efficiency and ranking quality; STAMP (Zhan et al., 2026) introduces an auxiliary multi-token prediction objective to densify supervision, improving representation learning while semantic pruning reduces training cost. (Wu et al., 2025) question constrained decoding’s generalization theoretically. We revisit these conclusions empirically and at industry scale (Section 4.2): within a unified multi-task LLM, and even after strengthening MTP with an auxiliary ranking loss, single-head beam search delivers better ranking and content-QA quality at lower training cost, while MTP’s advantage is limited to inference throughput and full-catalog ranking capability11 1 It is computationally infeasible for the beam search to generate probabilities across entire catalog.. Speculative and self-speculative decoding. A distinct line of work uses multi-token or draft heads purely as a lossless accelerator: Medusa (Cai et al., 2024) adds parallel decoding heads verified by tree attention, Speculative Streaming (Bhendawade et al., 2024) folds future n-gram drafting into the target model, and (Zagyva et al., 2025) combine Medusa with distillation for an industrial travel application. Crucially, these methods speed up inference while preserving the underlying autoregressive model’s output distribution, so they do not trade quality for speed; they are orthogonal to, and can be layered on top of, our single-head decoder rather than being an alternative to it. NEZHA (Wang et al., 2026) makes the distinction explicit: it accelerates sid beam search with an autoregressive self-drafting head plus a hash-set verifier, and reports that a plain MTP decoder is faster but markedly worse in quality because it cannot exploit the sequential dependency inherent in the sid. We instead compare single-head vs. MTP as generation paradigms (a quality-and-cost question), whereas this line targets quality-preserving acceleration (a speed question). Efficient LLM training and inference. Our stack combines existing components: Muon (Jordan et al., 2024), Cut Cross-Entropy (Wijmans et al., 2024), sequence packing (Krell et al., 2023), and HSDP (Zhao et al., 2023) for both single-head and MTP training, and vLLM’s PagedAttention and prefix caching (Kwon et al., 2023) for inference with the single-head and beam search. Our contribution here is not any single component but their integration and measured joint effect on a large-scale run, plus the trie- and catalog-constrained decoding layer we build on vLLM. 3 Method We first define how entities are represented and generated as Semantic IDs (Section 3.1), then the two training paradigms we compare: single-head and MTP, the latter equipped with our auxiliary ranking loss (Section 3.2). We also introduce their corresponding inference procedures (Section 3.3), and finally the efficient training/inference stack (Section 3.4). 3.1 Semantic-ID Generation Let ℰE be the catalog of entities. Each entity e∈ℰe is mapped to a Semantic ID SID(e)=(c1(e),c2(e),…,cL(e)),cℓ(e)∈1,…,Kℓ,SID(e)= (c_1(e),c_2(e),…,c_L(e) ), c_ (e)∈\1,…,K_ \, (1) a length-L sequence of discrete codes drawn from per-level codebooks of sizes K1,…,KLK_1,…,K_L, obtained by residual quantization (Lee et al., 2022) of a content embedding ϕ(e)∈ℝdφ(e) ^d. The quantizer is trained so that the first code captures the coarsest semantic partition and later codes refine it, so entities close in content space share early-level codes. The framework is agnostic to the sid shape: level sizes may be uniform (e.g., 1024×41024× 4, meaning Kℓ=1024K_ =1024 for all L=4L=4 levels) or pyramidal (e.g., 1024×512×256×1281024×512×256×128, spending more capacity on the coarse levels); our experiments use the uniform 1024×41024× 4 shape. The ∑ℓKℓ _ K_ sid tokens are appended to the LLM vocabulary V, giving +=∪V^+=V with ||=∑ℓ=1LKℓ|S|= _ =1^LK_ . While sid collisions rarely happen (e.g., ≤1%≤ 1\%), we resolve the collision by returning the first entity in the sid bucket. We place the sid vocabulary on the LM head and generate the target’s codes, rather than scoring a catalog-sized auxiliary head as the baseline did. This gives (i) a simpler architecture that has one model, one head, with no catalog-sized layer to provision or re-train as the catalog grows; (i) stronger cold-start, since a new title’s sid is computed from its content embedding and can be recommended before any member interacts with it, by sharing sid prefixes with established titles22 2 For “extreme” cold-start entities whose sids are never exposed during training, we fall back to retrieval-augmented prompting or lightweight daily fine-tuning., and (i) shared representations, since retrieval, content QA, and instruction following reuse the same head. 3.2 Training Every training example (e.g., retrieval, content QA, or instruction following) is a token sequence over +V^+: retrieval targets are written as the target’s sid tokens, content QA references titles by both text and sid, and instruction data is unchanged. We compare two ways of producing the sid codes; both share the same backbone, tokenizer, and data (Figure 1). Figure 1: Multi-token-prediction (MTP) training. While the single-head approach uses a LM head with a next-token loss (Eq. 2), MTP adds L parallel sid heads trained with per-code cross-entropy and our auxiliary ranking loss over the joint sid score (Eq. 5). 3.2.1 Single-Head (LM Head) Training The classical design uses the single next-token LM head to emit sid codes autoregressively, so training is one next-token problem whose per-example loss splits into text and sid positions, ℒSH=−∑i∈textlogpθ(yi∣y<i)−λ∑i∈sidlogpθ(yi∣y<i),L^SH=-\!\! _i _text\!\! p_θ(y_i y_<i)\;-\;λ\!\! _i _sid\!\! p_θ(y_i y_<i), (2) where textT_text and sidT_sid index text and sid target positions and λ≥1λ≥ 1 up-weights the sparser sid positions. Crucially, this autoregressive training enforces dependencies between the SID levels, where predicting the current code cℓc_ is conditioned on previous codes c<ℓc_< . During inference, beam search retains multiple early prefixes that later scores can rerank. 3.2.2 Multi-Token Prediction (MTP) Training Following (Gloeckle et al., 2024), MTP keeps the next-token LM head and adds L parallel heads g1,…,gLg_1,…,g_L on the shared last-hidden state hxh_x for query x, with head gℓg_ responsible for code cℓc_ . Because all codes are predicted in parallel from hxh_x, the standard MTP retrieval term is a sum of independent per-code cross-entropies, ℒcode=−∑ℓ=1Llogpgℓ(cℓ(e+)∣hx),L_code\;=\;- _ =1^L p_g_ \! (c_ (e^+) h_x ), (3) for the positive item e+e^+. This factorization is exactly the weakness of MTP for ranking: it optimizes each code’s marginal likelihood but never the joint item score, so an off-target item whose individual codes are each independently probable can outscore the true target. Auxiliary ranking loss. We address this with a new auxiliary objective that directly optimizes the joint, item-level score. Define the MTP score of a candidate e as the sum of its per-code log-probabilities, sMTP(e∣x)=∑ℓ=1Llogpgℓ(cℓ(e)∣hx),s_MTP(e x)\;=\; _ =1^L p_g_ \! (c_ (e) h_x ), (4) and train the heads so the positive item ranks above a set (x)N(x) of sampled in-batch/hard negatives from the eligible catalog, via a sampled-softmax 33 3 If the entire catalog is small (e.g., ≤1M≤ 1M), normal Softmax without any sampled negatives can be used here. ranking loss: ℒrank=−logexp(sMTP(e+∣x))∑e∈e+∪(x)exp(sMTP(e∣x)),ℒMTP=ℒnext+αℒcode+βℒrank,L_rank\;=\;- \! (s_MTP(e^+ x)) _e∈\e^+\ (x) \! (s_MTP(e x)), ^MTP=L_next+α\,L_code+β\,L_rank, (5) with importance hyperparameters α,βα,β, and ℒnextL_next the text next-token loss retained from the LM head. Unlike standard MTP (Gloeckle et al., 2024), which uses per-token cross-entropy only, and RPG (Hou et al., 2025), which pairs MTP with graph-constrained decoding at inference, ℒrankL_rank teaches the independent heads to produce jointly calibrated, rankable item scores at training time, which partially compensates for the missing autoregressive coupling and gives MTP its strongest form. We empirically confirm that having ℒrankL_rank improves ranking metrics, while increasing training cost. 3.3 Inference 3.3.1 Single-Head: Constrained, Catalog-Aware Beam Search A single-head retrieval call generates the target sid token-by-token. Let T be the sid trie of all prefixes realized by some entity, and x⊆ℰC_x the per-request eligible catalog for query x. At step ℓ with prefix c<ℓc_< , we restrict the next-token distribution to the admissible set (c<ℓ,x)=k:(c<ℓ,k)∈and∃e∈x,SID(e)≻(c<ℓ,k),A(c_< ,x)\;=\; \\,k\;:\;(c_< ,k) \ and\ ∃\,e _x,\ SID(e)\! \!(c_< ,k)\, \, (6) where ≻ denotes “has as a prefix”. Constrained decoding guarantees we never emit a sid that names no real entity; catalog-aware decoding guarantees every candidate is eligible for the request, sharpening local ranking metrics without post-hoc filtering. Beam search keeps the top-N partial hypotheses per level, scoring a complete sid by its length-normalized log-probability s(e∣x)=1L∑ℓ=1Llogpθ(cℓ(e)∣x,c<ℓ(e)),s(e x)\;=\; 1L _ =1^L p_θ\! (c_ (e) x,c_< (e) ), (7) and returns the top-K entities. We implement this on vLLM (Kwon et al., 2023) that reuses the long member-history prompt’s KV cache across beams and decoding steps, so wide beams are affordable, and the trie/catalog constraints are applied as a logit mask at each step. 3.3.2 MTP: Parallel Decoding MTP produces all L per-code distributions from hxh_x in a single forward pass and scores candidates by sMTPs_MTP (Eq. 4); the top-K eligible entities are selected by evaluating sMTPs_MTP over the trie-restricted candidate set, without sequential beam expansion. This removes the per-code forward passes of beam search, which is the source of MTP’s inference-throughput advantage. However, it also removes the autoregressive coupling crucial for accurate retrieval and ranking. Thus, we utilize the ranking loss ℒrankL_rank to recover the retrieval/ranking ability. MTP is particularly useful for scenarios requiring dense, full-catalog scores with non-zero probability mass on every title. 3.4 Efficient Training and Inference Stack The comparison above is enabled by a stack that makes large-scale fine-tuning and inference cheap. For training, we combine four standard components and report their integrated effect (Section 4.3): the Muon optimizer (Jordan et al., 2024), which orthogonalizes matrix gradient updates and stores only momentum, roughly halving optimizer-state memory and converging in fewer steps than our AdamW Loshchilov and Hutter (2019) baseline; on-the-fly sequence packing with block-diagonal masks (Krell et al., 2023) plus cross-rank batch mixing that assigns different tasks to different data-parallel ranks in one global batch, improving throughput and stability; Cut Cross-Entropy (Wijmans et al., 2024), a fused kernel that avoids materializing the large-vocabulary logits tensor; and HSDP (Zhao et al., 2023), which shards state within a node and replicates across nodes, confining the heaviest communication to high-bandwidth intra-node links (∼ 7% absolute MFU gain on the 8B backbone). For inference we serve on vLLM (Section 3.3.1). 4 Experiments 4.1 Experimental Setup Confidentiality and training data. We study the streaming-media recommendation domain. By industry-scale we mean an 8B-parameter production backbone fine-tuned over a proprietary catalog on ≥10M≥ 10M sampled examples per task drawn from production logs, evaluated under a production training- and inference-cost budget rather than on a small retrieval-only academic benchmark. To respect confidentiality we report relative rather than absolute figures for proprietary quantities (catalog size, corpus sizes, and dollar costs) and reported metrics (e.g., MRR, IFEval, MMLU). A retrieval task uses member interaction history data (≥10M≥ 10M) snapshotted at a fixed date-hour; a content QA task uses an internal content QA corpus (≥10M≥ 10M) derived from our rich content metadata; an instruction-following task uses a mix of open-source datasets(Taori et al., 2023; Lambert et al., 2025) (≥10M≥ 10M). sids come from the latest quantizer based on RQ-VAE Rajput et al. (2023) snapshot. Models and baseline. All the experiments are based on open-weight Qwen3-8B backbone (Yang and Team, 2025) with an optional 0.6B variant. We fine-tune such pre-trained models by supervised fine-tuning (SFT) plus LoRA (Hu et al., 2022) on our training data, and we conduct additional reinforcement learning (RL) after the SFT for language metric enhancement. We use up to 256 NVIDIA H200 and A100-80GB GPUs for our model training and inference. The baseline model in this paper refers to the previous non-sid model: a LoRA-only recipe with a catalog-sized auxiliary head for ranking. The additional head was necessary for the baseline to generate logits across our entire catalog. However, this ranking head can degrade training efficiency and increase maximum memory usage. Content QA corpus. Content QA answers natural-language questions about a title (e.g., “What is Movie A about?”). We internally developed a large-scale Content QA corpus in which each title is referenced by both its text and its sid, so the entity representation matches what the model sees during retrieval. The corpus is generated synthetically by prompting a frontier teacher model (e.g., GPT 5.2) with internal title++sid context and spans four complementary question types: title-grounded descriptions, sid-centric semantics, member-like query grounding, and two-title comparisons. To reduce sensitivity to prompt style, we sample system prompts at training time, including a 20% probability of an empty system message. Evaluation. Ranking is measured with normalized Mean Reciprocal Rank (nMRR). For queries Q, MRR=1||∑x∈1rankx,nMRR=MRRMRRideal,MRR= 1|Q| _x 1rank_x, = MRRMRR_ideal, (8) where rankxrank_x is the position of the first relevant entity and the normalizer accounts for the number of eligible relevant items per query. We report the average ranking performance using an internal test set across 7 different offsets (i.e., predict +1−+7+1-+7 days future interactions). We also offer detailed metrics computed only on discoveries (i.e., watching new content) and new users (e.g., users with ≤K≤ K engagement) and items (e.g., titles launched after training cutoff), along with overall ranking. Content QA is scored by an LLM judge (Qwen2.5-32B-Instruct) (Zheng et al., 2023) on a stratified test set. Among numerous language benchmarks, we report IFEval (Zhou et al., 2023) and MMLU (Hendrycks et al., 2021) in this paper. Table 1: Single-head beam search vs. multi-token prediction. Relative quality change of the single-head over the MTP; positive values favor the single-head. Metrics are grouped by task family. Task family Metric Qwen3-8B Qwen3-0.6B Ranking Overall ranking (nMRR) ↑ +2.0% ↑ +3.5% Discovery (nMRR) ↑ +7.3% ↑ +12.6% New user (nMRR) ↑ +2.1% ↑ +4.2% New item (nMRR) ↓ -0.8% ↑ +6.8% Content Understanding QA Accuracy ↑ +2.0% ↑ +12.7% Language IFEval (accuracy) ↑ +3.3% ↑ +2.3% MMLU (accuracy) ↑ +5.7% ↑ +4.8% Table 2: Single-head vs. MTP: training and inference system metrics. Relative quality change of the single-head over the MTP. Single-head trains cheaper; MTP only wins inference throughput. Backbone Training Runtime ↓ MFU ↑ Tokens/sec ↑ Dollar Cost ↓ Inference/sec ↑ Qwen3-8B 0.37×0.37× 1.15×1.15× 1.18×1.18× 0.38×0.38× 0.57×0.57× Qwen3-0.6B 0.50×0.50× 1.21×1.21× 1.20×1.20× 0.97×0.97× 0.18×0.18× 4.2 Core Study: Single-Head Beam Search vs. Multi-Token Prediction This is our central result. We take the same backbone, sid tokenizer, data mixture, and training budget per token and train the model in two different ways: (1) single-head autoregressive training (Eq. 2) and constrained beam search inference (Eq. 7) and (2) multi-token prediction with our auxiliary ranking loss (Eq. 5), i.e., MTP in its strongest form. This isolates the generation paradigm as the only variable, making the comparison apples-to-apples at production scale. Quality: single-head wins where it counts. Single-head beam search beats MTP on all ranking and content-QA metrics (Table 1). On the Qwen 3-8B model, it improves discovery (↑ 7.3%), new user recommendation (↑ 2.1%), overall retrieval (↑ 2.0%), content understanding (↑ 2.0%), and general-language benchmarks (IFEval ↑ 3.3%, MMLU ↑ 5.7%). On the Qwen3-0.6B model, the single-head advantage is uniform and larger including gains on discovery (↑ 12.6%) and new item recommendations (↑ 6.8%). Across both scales, we attribute the ranking/QA boost of the single-head to autoregressive coupling: letting code cℓc_ condition on c<ℓc_< lets the model correct early mistakes, whereas MTP’s parallel heads must commit to all codes from a single hidden state under a conditional-independence assumption. Cost: single-head is far cheaper to train, but MTP is much faster at inference. The quality advantage does not come with a training-cost premium; in fact, the reverse (Table 2). For the 8B model, single-head trains 2.7×2.7× faster than MTP with 2.6×2.6× lower dollar cost, and higher model FLOPs utilization (MFU) & token per second. The only win for MTP is raw inference throughput (↑ 75%), consistent with the efficiency motivation of the MTP literature (Gloeckle et al., 2024; Hou et al., 2025). In our deployment, this throughput edge does not offset the quality and training-cost gaps, so we adopt single-head beam search as the default and keep MTP only for applications that need dense full-catalog scores. Reconciling with prior work. We confirm MTP’s inference advantage but overturn the quality claim that parallel prediction matches autoregressive generation. Two features of our regime explain the gap: a moderate sid length (L=4L=4), where autoregressive decoding is cheap and its coupling most valuable (vs. the long L≥64L≥64 sids that make beam search prohibitive in RPG); and multi-task training, which lets the single head share representations across retrieval, content QA, and language. Table 3: Overall metric comparison (Qwen3-8B) of the single-head with sid vs. the non-sid baseline with an auxiliary ranking head. Relative quality change of the sid model over the baseline; positive values favor the sid model. Metrics are grouped by task family. We observe enhanced ranking and content understanding, while language regression due to sid vocab extension. We confirm the language regression can be recovered by RL post training (e.g., IFEval ↑ +12.8%). Task family Metric Single-head with sid vs. Non-sid baseline Ranking Overall ranking (nMRR) ↑ +4.7% Discovery (nMRR) ↑ +5.3% New user (nMRR) ↑ +4.7% New item (nMRR) ↑ +77.3% Content Understanding QA Accuracy ↑ +21.7% Language IFEval (accuracy) ↓ -7.3% MMLU (accuracy) ↓ -10.0% Table 4: Training system metrics between the single-head with sid vs. the non-sid baseline, where both models are based on Qwen3-8B. Multipliers are relative to the non-sid baseline. Model Runtime ↓ MFU ↑ Tokens/sec ↑ Dollar Cost ↓ Single-head with sid 0.36× 4.75× 4.0× 0.27× 4.3 Single-head Beam Search with sid vs. the non-sid Baseline Having established the generation paradigm, we report the end-to-end gains of the full recipe (single-head ++ native sid ++ new training/inference stack) over the previous non-sid baseline. Overall comparison. The single-head model with sid outperforms the non-sid baseline across ranking and content QA (Table 3). The headline gains are ↑ 77.3% on new item nMRR, ↑ 21.7% on content-QA accuracy, and ↑ ∼ 5% on overall ranking & discovery. The significant new item nMRR boost is direct evidence that sid prefix sharing lets the model recommend titles it never saw engagement for. We observe regression on language metriccs such as IFEval (↓ 7.3%) and MMLU (↓ 10.0%). The key reason is that expanding the vocabulary with LKLK sid tokens (assuming L-level codebook with K tokens per level) and co-training on them perturbs the instruction-following distribution. To address this regression, we perform reinforcement learning (RL) with instruction-following datasets after the SFT stage, which yields ↑ +12.8% improvement of the IFEval metric. System efficiency. The training optimizations we introduced transform training economics (Table 4). Compared to the non-sid baseline without any training optimizations (Section 3.4), single-head model trains 2.8×2.8× faster, raises MFU ∼4.75× 4.75× and tokens/sec ∼4.0× 4.0×, and cuts training cost by 73%. These savings let us launch more model and ablation runs per unit budget. 4.4 Ablation Studies Table 5: LoRA vs. full fine-tuning on the single-head setup. Relative quality change of LoRA over full fine-tuning; positive values favor LoRA. Metrics are grouped by task family. We choose LoRA by default because of better ranking metrics, especially the cold-start item recommendation. Task family Metric Qwen3-8B Qwen3-0.6B Ranking Overall ranking (nMRR) ↑ +0.3% ↑ +0.1% Discovery (nMRR) ↑ +1.9% 0.0%0.0\% New user (nMRR) ↑ +0.9% 0.0%0.0\% New item (nMRR) ↑ +6.2% ↑ +4.5% Content Understanding QA Accuracy 0.0%0.0\% 0.0%0.0\% Language IFEval (accuracy) ↓ -5.2% ↑ +3.3% MMLU (accuracy) ↓ -6.1% ↑ +3.1% Table 6: System metrics for the LoRA FT vs. full FT ablation. Multipliers are relative to LoRA of the same backbone. Full FT is on-par-or-better than LoRA across all system metrics and model sizes, as it avoids the runtime memory overhead and kernel launch latencies introduced by LoRA. Backbone Variant Runtime ↓ MFU ↑ Tokens/sec ↑ Dollar Cost ↓ Qwen3-8B Full FT 1.0×1.0× 1.13×1.13× 1.02×1.02× 1.0×1.0× Qwen3-0.6B Full FT 0.87×0.87× 1.12×1.12× 1.17×1.17× 0.9×0.9× LoRA vs. full fine-tuning (FT). We choose LoRA by default compared to full FT, as LoRA ranking metrics are on-par-or-better than full FT (Table 5), especially the cold-start item recommendation metric. According to Table 6, full FT is on-par-or-better than LoRA because its standard dense matrix operations avoid the extra kernel launch overheads and lower arithmetic intensity introduced by LoRA. On the 8B model, LoRA shows a language-metric drop relative to full FT because we apply LoRA to the output layer, whereas on the 0.6B model (where LoRA is not applied to the output layer) the effect reverses. We plan to revisit full FT given its simplicity and system performance. Table 7: Optimizer and content-QA-data ablations against single-head LoRA training. The columns report Muon relative to AdamW and all tasks relative to all tasks without content QA, respectively; positive values favor the first variant. Metrics are grouped by task family. Task family Metric Muon vs. AdamW All tasks vs. no Content QA Ranking Overall ranking (nMRR) ↓ -0.1% ↓ -0.2% Discovery (nMRR) ↓ -0.9% 0.0%0.0\% New user (nMRR) ↑ +0.1% ↓ -0.1% New item (nMRR) ↑ +5.2% ↑ +4.0% Content QA Accuracy ↑ +0.1% ↑ +211.4% Language IFEval (accuracy) ↓ -1.9% ↓ -3.2% MMLU (accuracy) ↓ -0.6% ↓ -1.8% Muon vs AdamW optimizer. Holding everything else fixed, we compare two popular optimizers: Muon Jordan et al. (2024) and AdamW Loshchilov and Hutter (2019). As shown in Table 7, they reach similar quality, while Muon adds a small item cold-start gain, but Muon’s system performance is clearly superior; e.g., ∼20% 20\% higher MFU and throughput, lower memory consumption, and larger training batch size. Thus, we set Muon as the default for both single-head and MTP training. Contribution of content-QA data. Table 7 tests whether content understanding helps ranking. Removing all content-QA tasks leaves ranking nearly unchanged while collapsing content QA accuracy notably. We do not observe the synergy between ranking and content QA tasks in the current SFT-only recipe. This motivates moving content QA into a CPT stage followed by ranking-focused SFT, which is a direction for future iterations, and one consistent with the CPT+SFT of PLUM (He et al., 2025). Other ablations. We also found that introducing a separate weight for sid tokens in (2) and having the auxiliary ranking loss for MTP (5) significantly improve ranking metrics. Note that we leave other non-critical ablations such as various sid codebook shape (e.g., fan-in or fan-out), sid level (e.g., large 2-level or small 8-level), or different task mixture ratios as future work. 5 Conclusion Inside a unified multi-task LLM at industry scale, single-head autoregressive sid generation with constrained beam search beats multi-token prediction on most product-critical metrics (ranking and content QA) and trains at less than half the cost on Qwen3-8B scale, while MTP leads only on inference throughput. Even after we strengthen MTP with a new auxiliary ranking loss, the single-head wins. Furthermore, the end-to-end recipe, from efficient training optimizations with sid to vLLM-served constrained catalog-aware beam search, more than halves training time and cost and lifts cold-start (↑ +77%) and content-QA (↑ +22%) over the non-sid baseline. Our findings have some limitations: (1) most experiments are based on offline training and test data, and online A/B test results might exhibit different trends, (2) the LLM could be overfit to our tasks and input format, and the output quality could be sub-optimal for diverse input format and tasks, and (3) missing continual pre-training (CPT) or mid-training stage that can warm-up the sid token embeddings and reduce the training data volume for the SFT stage. We will probe the crossover in sid shape and length, catalog size, and task mixture at which it tips. We will also extend both single-head and MTP frameworks to multi-stage training such as CPT+SFT and CPT+SFT+RL. Finally, we will diversify our input prompt style and format and utilize a larger number of tasks during the SFT, to make our LLM robust. References [1] N. Bhendawade, I. Belousova, Q. Fu, H. Mason, M. Rastegari, and M. Najibi (2024) Speculative streaming: fast LLM inference without auxiliary models. In NeurIPS Efficient Natural Language and Speech Processing (ENLSP) Workshop, Cited by: §2. [2] T. Cai, Y. Li, Z. Geng, H. Peng, J. D. Lee, D. Chen, and T. Dao (2024) Medusa: simple LLM inference acceleration framework with multiple decoding heads. In International Conference on Machine Learning (ICML), Cited by: §2. [3] DeepSeek-AI (2024) DeepSeek-v3 technical report. arXiv preprint arXiv:2412.19437. Cited by: §2. [4] J. Deng, S. Wang, K. Cai, L. Ren, Q. Hu, W. Ding, Q. Luo, and G. Zhou (2025) OneRec: unifying retrieve and rank with generative recommender and iterative preference alignment. arXiv preprint arXiv:2502.18965. Cited by: §1, §2. [5] E. D’Amico, M. De Nadai, P. Chandar, G. Penha, G. Fazelnia, B. Huber, M. Lalmas, P. N. Bennett, et al. (2026) Deploying semantic id-based generative retrieval for large-scale podcast discovery at spotify. arXiv preprint arXiv:2603.17540. Cited by: §1, §1, §2. [6] F. Gloeckle, B. Youbi Idrissi, B. Rozière, D. Lopez-Paz, and G. Synnaeve (2024) Better & faster large language models via multi-token prediction. In International Conference on Machine Learning (ICML), Cited by: §1, §2, §3.2.2, §3.2.2, §4.2. [7] R. He, L. Heldt, L. Hong, et al. (2025) PLUM: adapting pre-trained language models for industrial-scale generative recommendations. arXiv preprint arXiv:2510.07784. Cited by: §1, §1, §2, §4.4. [8] D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt (2021) Measuring massive multitask language understanding. In International Conference on Learning Representations (ICLR), Cited by: §4.1. [9] Y. Hou, J. Li, A. Shin, J. Jeon, A. Santhanam, W. Shao, K. Hassani, N. Yao, and J. McAuley (2025) Generating long semantic ids in parallel for recommendation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD), Cited by: §1, §2, §3.2.2, §4.2. [10] E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen (2022) LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), Cited by: 4th item, §4.1. [11] K. Jordan, Y. Jin, V. Boza, J. You, F. Cesista, L. Newhouse, and J. Bernstein (2024) Muon: an optimizer for hidden layers in neural networks. Note: https://kellerjordan.github.io/posts/muon/ Cited by: 3rd item, 4th item, §2, §3.4, §4.4. [12] M. M. Krell, M. Kosec, S. P. Perez, and A. W. Fitzgibbon (2023) Efficient sequence packing without cross-contamination: accelerating large language models without impacting performance. In International Conference on Learning Representations, External Links: Link Cited by: §2, §3.4. [13] W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica (2023) Efficient memory management for large language model serving with pagedattention. In ACM Symposium on Operating Systems Principles (SOSP), Cited by: 3rd item, §2, §3.3.1. [14] N. Lambert, J. Morrison, V. Pyatkin, S. Huang, H. Ivison, F. Brahman, L. J. V. Miranda, A. Liu, N. Dziri, S. Lyu, Y. Gu, S. Malik, V. Graf, J. D. Hwang, J. Yang, R. L. Bras, O. Tafjord, C. Wilhelm, L. Soldaini, N. A. Smith, Y. Wang, P. Dasigi, and H. Hajishirzi (2025) Tulu 3: pushing frontiers in open language model post-training. External Links: 2411.15124, Link Cited by: §4.1. [15] D. Lee, C. Kim, S. Kim, M. Cho, and W. Han (2022) Autoregressive image generation using residual quantization. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), Cited by: §1, §2, §3.1. [16] I. Loshchilov and F. Hutter (2019) Decoupled weight decay regularization. In International Conference on Learning Representations, External Links: Link Cited by: 4th item, §3.4, §4.4. [17] S. Rajput, N. Mehta, A. Singh, R. H. Keshavan, T. Vu, L. Heldt, L. Hong, Y. Tay, V. Q. Tran, J. Samost, M. Kula, E. H. Chi, and M. Sathiamoorthy (2023) Recommender systems with generative retrieval. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §1, §2, §4.1. [18] A. Singh, T. Vu, N. Mehta, R. Keshavan, M. Sathiamoorthy, Y. Zheng, L. Hong, L. Heldt, L. Wei, D. Tandon, E. H. Chi, and X. Yi (2024) Better generalization with semantic ids: a case study in ranking for recommendations. arXiv preprint arXiv:2306.08121. Cited by: §1, §2. [19] R. Taori, I. Gulrajani, T. Zhang, Y. Dubois, X. Li, C. Guestrin, P. Liang, and T. B. Hashimoto (2023) Alpaca: a strong, replicable instruction-following model. Stanford Center for Research on Foundation Models. https://crfm. stanford. edu/2023/03/13/alpaca. html 3 (6), p. 7. Cited by: §4.1. [20] Y. Tay, V. Q. Tran, M. Dehghani, J. Ni, D. Bahri, H. Mehta, Z. Qin, K. Hui, Z. Zhao, J. Gupta, T. Garg, Y. Sung, M. Wang, and D. Metzler (2022) Transformer memory as a differentiable search index. In Advances in Neural Information Processing Systems (NeurIPS), Cited by: §1, §1, §2. [21] Y. Wang, S. Zhou, J. Lu, Z. Liu, L. Liu, M. Wang, W. Zhang, F. Li, W. Su, P. Wang, J. Xu, and X. Zhao (2026) NEZHA: a zero-sacrifice and hyperspeed decoding architecture for generative recommendations. In Proceedings of the ACM Web Conference (W), Cited by: §2. [22] E. Wijmans, B. Huval, A. Hertzberg, V. Koltun, and P. Krähenbühl (2024) Cut your losses in large-vocabulary language models. arXiv preprint arXiv:2411.09009. Cited by: 3rd item, §2, §3.4. [23] S. Wu, Z. Wei, Z. Chen, P. Ren, and M. de Rijke (2025) Constrained auto-regressive decoding constrains generative retrieval. arXiv preprint arXiv:2504.09935. Cited by: §2. [24] A. Yang and Q. Team (2025) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §4.1. [25] D. Zagyva, E. Stergiadis, L. van der Maas, A. Dokic, E. Fainman, I. Gusev, and M. Beladev (2025) Speed without sacrifice: fine-tuning language models with medusa and knowledge distillation in travel applications. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (ACL): Industry Track, Cited by: §2. [26] J. Zhai, L. Liao, X. Liu, Y. Wang, R. Li, X. Cao, L. Gao, Z. Gong, F. Gu, M. He, Y. Lu, and Y. Shi (2024) Actions speak louder than words: trillion-parameter sequential transducers for generative recommendations. In International Conference on Machine Learning (ICML), Cited by: §1. [27] T. Zhan, K. Fu, C. Lv, Z. Lv, and S. Zhang (2026) Semantic trimming and auxiliary multi-step prediction for generative recommendation. arXiv preprint arXiv:2604.05329. External Links: 2604.05329, Link Cited by: §1, §2. [28] Y. Zhao, A. Gu, R. Varma, L. Luo, C. Huang, M. Xu, L. Wright, H. Shojanazeri, M. Ott, S. Shleifer, A. Desmaison, C. Balioglu, P. Damania, B. Nguyen, G. Chauhan, Y. Hao, A. Mathews, and S. Li (2023) PyTorch fsdp: experiences on scaling fully sharded data parallel. Proceedings of the VLDB Endowment 16 (12). Cited by: 3rd item, §2, §3.4. [29] L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica (2023) Judging llm-as-a-judge with mt-bench and chatbot arena. Advances in Neural Information Processing Systems (NeurIPS). Cited by: §4.1. [30] J. Zhou, T. Lu, S. Mishra, S. Brahma, S. Basu, Y. Luan, D. Zhou, and L. Hou (2023) Instruction-following evaluation for large language models. arXiv preprint arXiv:2311.07911. Cited by: §4.1.