Paper deep dive
TSPORec: Token Selection via Preference Optimization for LLM-Based Sequential Recommendation
Wenqiao Zhu, Chao Xu, Haipang Wu, Ji Liu
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large Language Models (LLMs) have emerged as powerful tools for improving recommendation systems. The effectiveness of LLMs arises from their ability to harness rich textual information and their capacity to model heterogeneous user preferences based on users' interaction history. However, due to the large-scale and deep architectures, LLM-based sequential recommendation approaches generally incur high inference costs, resulting in a low return on investment. To mitigate this cost, many existing approaches resort to using only the first few tokens of item descriptions, which inadvertently discards valuable information contained in the full text, thereby leading to suboptimal recommendation performance. To address this limitation, we propose a novel Token Selection approach for Preference Optimization in LLM-based sequential Recommendation, i.e., TSPORec, which accurately pinpoints informative tokens throughout the entire textual content to improve recommendation performance. Specifically, we design a three-stage pipeline to select informative tokens and introduce a novel proxy reward to facilitate the implementation. TSPORec not only enhances recommendation performance but also improves computational efficiency. Extensive experiments across two models and datasets demonstrate the superb performance (up to 31.25%) and efficiency (up to 63.4%) of our approach compared with six baseline approaches. Code is available at this https URL.
Tags
Links
- Source: https://arxiv.org/abs/2608.09605v1
- Canonical: https://arxiv.org/abs/2608.09605v1
Trouble viewing inline? Open PDF directly →
Full Text
58,993 characters extracted from source content.
Expand or collapse full text
TSPORec: Token Selection via Preference Optimization for LLM-Based Sequential Recommendation Wenqiao Zhu * Chao XuHaipang WuJi Liu Abstract Large Language Models (LLMs) have emerged as powerful tools for improving recommenda- tion systems. The effectiveness of LLMs arises from their ability to harness rich textual infor- mation and their capacity to model heteroge- neous user preferences based on users’ interac- tion history. However, due to the large-scale and deep architectures, LLM-based sequential recommendation approaches generally incur high inference costs, resulting in a low return on investment. To mitigate this cost, many ex- isting approaches resort to using only the first few tokens of item descriptions, which inadver- tently discards valuable information contained in the full text, thereby leading to suboptimal recommendation performance. To address this limitation, we propose a novel Token Selec- tion approach for Preference Optimization in LLM-based sequential Recommendation, i.e., TSPORec, which accurately pinpoints informa- tive tokens throughout the entire textual con- tent to improve recommendation performance. Specifically, we design a three-stage pipeline to select informative tokens and introduce a novel proxy reward to facilitate the implemen- tation. TSPORec not only enhances recommen- dation performance but also improves computa- tional efficiency. Extensive experiments across two models and datasets demonstrate the su- perb performance (up to 31.25%) and efficiency (up to 63.4%) of our approach compared with six baseline approaches. Code is avaliable at https://github.com/WNQzhu/TSPORec.git. 1 Introduction Recommendation systems have become indispens- able for mitigating information overload across a wide range of commercial platforms (Zhai et al., 2024; Zhu et al., 2025a,c, 2022). A key factor in delivering personalized item recommendations lies in accurately modeling the heterogeneous prefer- * Correspondence: zhuwnq@outlook.com ences of users. Recently, Large Language Mod- els (LLMs) have demonstrated remarkable effec- tiveness in capturing such preferences (Kim et al., 2024; Liao et al., 2024; Zheng et al., 2024), owing to their vast parameter scales and strong contextual understanding. In particular, Sequential Recom- mendation (SR), which models diverse interests of users based on their recent interaction histories, has proven highly effective in enhancing recommen- dation performance when integrated with LLMs (Chen et al., 2024; Liu et al., 2024a; Chen et al., 2025). The goal of SR is to predict the next items a user is likely to prefer by modeling sequential patterns in their historical interactions. A key challenge in this field lies in simultaneously capturing item- level sequential dependencies and modeling the complex and heterogeneous preferences of users derived from these interaction sequences. Owing to the pivotal role in advancing recom- mendation systems, SR has attracted sustained at- tention from both academia and industry (Rendle et al., 2010; Cheng et al., 2013; Chen et al., 2024; Liu et al., 2024a). Existing approaches (Kang and McAuley, 2018; Li et al., 2020; Wu et al., 2020) predominantly leveraged item IDentification (ID) features, overlooking the abundant textual content typically present in real-world settings. Further- more, their reliance on shallow network architec- tures limit their capacity to model the complex and heterogeneous nature of user preferences. Recently, LLMs have emerged as effective so- lutions for overcoming the limitations of previous approaches, demonstrating strong performance in SR tasks (Chen et al., 2024; Liu et al., 2024a; Chen et al., 2025). The core idea behind these approaches is to leverage the rich textual features and power- ful representational capabilities of LLMs to extract robust representations of items and users. For ex- ample, HLLM (Chen et al., 2024) employs sep- arate item and user LLMs, which are based on arXiv:2608.09605v1 [cs.IR] 10 Aug 2026 LLMs such as TinyLlama (Zhang et al., 2024b) or Baichuan (Baichuan, 2023), to learn comprehen- sive item and user representations. However, existing LLM-based SR approaches are highly resource-intensive. For instance, given an interaction history of lengthSwith each item represented byntokens, a single inference step re- quires processing S× n tokens, which consists of both computation-intensive and memory-intensive operations. To mitigate this cost, existing ap- proaches (Chen et al., 2024; Liu et al., 2024a) typi- cally exploit only the first few tokens of each item, failing to fully exploit their rich textual features. This limitation raises an important research ques- tion: Can we identify and select the most infor- mative tokens from textual features to maximize information utilization? Addressing this question could benefit SR systems in two key ways. First, it enables effective usage of critical information within the text. Second, it reduces computational overhead through compressing the overall token sequence length. In this paper, we propose a novel token selec- tion approach for preference optimization in LLM- based Sequential Recommendation, i.e., TSPORec. TSPORec is composed of three stages to select the most informative tokens from textual features. First, we pre-train an SR model based on an LLM. Next, we freeze the LLM backbone and attach a policy head, which is then trained using a newly designed proxy reward function. Using the trained policy, we extract informative tokens from item texts and subsequently retrain the LLM on the re- fined input sequences. We conduct extensive ex- periments across six recommendation approaches and two publicly available benchmark datasets to demonstrate the effectiveness and generality of our approach. The key contributions of this paper are summarized as follows: •We propose a new token selection pipeline that automatically identifies informative to- kens from textual features. This pipeline shows strong generalization across LLMs and datasets. •We design a novel proxy reward method to guide policy learning in an effective and tar- geted manner. This method can identify mean- ingful token importance, leading to an effec- tive policy. •Extensive experimental results demonstrate the superb performance (up to 29.29% in terms of recall and 31.25% in terms of NDCG) and efficiency (up to 63.4% inference over- head reduction) of TSPORec compared with six State-of-the-Art baseline approaches. 2 Preliminaries Modern recommendation systems typically repre- sent users and items as learned embedding vec- tors. Lete u ande i denote the embeddings of a user and an item, respectively. In SR, the user em- bedding is derived from their interaction history: e u = g(e i ),whereg(·)is commonly a shal- low neural network or a LLM ande i represents a positive item. A positive iteme i for Useruis an item that Useruhas interacted with. In this work, we denote the LLM exploited to aggregate item embeddings into a user representation as the user LLM, and the LLM employed to generate item embeddings via token selection as the item LLM. A widely adopted training objective in recom- mendation systems is the InfoNCE loss (van den Oord et al., 2018), as defined in Formula (1): L InfoNCE (e i ) =− log exp(⟨e u , e i ⟩) exp(⟨e u , e i ⟩) + P N j=1 exp(⟨e u , e j i ⟩) , (1) where⟨·,·⟩denotes a similarity measure, i.e., dot product, ande 1 i , . . . , e N i are embeddings ofNneg- ative items. Negative items are typically the items that the user has not interacted with. We denote E(e i ) = ⟨e u , e i ⟩, ⟨e u , e 1 i ⟩, . . . , ⟨e u , e N i ⟩ the vector of similarity scores between the user embedding and the positive item embedding along withNnega- tive items. We takelabel = [1, 0,..., 0]as the corresponding one-hot label vector. The InfoNCE loss can then be equivalently expressed as a cross- entropy objective: L InfoNCE (e i ) =L CE softmax(E(e i )), label , (2) whereL CE denotes the Cross-Entropy loss. 3 Method In this section, we first formulate the problem to address in this paper. Then, we detail our approach, i.e., TSPORec, including the new token selection pipeline and the novel proxy reward method. 3.1 Problem Formulation We study the problem of token selection in LLM- based SR. Given an item represented by a token Policy Head (θ) Large Language Model t 1 t 2 t 3 t 4 ... t m−2 t m−1 t m [ITEM] Stage 2: Learning a Policy to Select Tokens. t i t i+1 t i+2 t d ... t w−2 t w−1 t w t j t j+1 t j+2 t d ... t v−2 t v−1 t v Prediction Head Large Language Model t 1 t 2 t 3 t 4 ... t m−2 t m−1 t m [ITEM] Stage 1: Pretrain LLM Using Item Tokens. Input Sampling Cross Entropy i Cross Entropy j Reward = ( 1 If Cross Entropy i ≤ Cross Entropy j , −1 Else. Reward Updated Frozen Prediction Head Large Language Model t a t b t c t d ... t k−2 t k−1 t k [ITEM] Select Stage 3: Retrain LLM Using Selected Item Tokens. Figure 1: Overview of TSPORec’s Three-Stage Training Pipeline: (1) LLM Pretraining, (2) Token Selection Policy Learning, (3) Token Selection and Retraining. sequenceT =t 1 ,t 2 ,...,t n , conventional LLM- based approaches typically derive an item embed- ding using only the first k tokens: e i = LLM(t 1 ,t 2 ,...,t k ), wherek ≤ n. Instead, we aim to identify an infor- mative subset ofktokens, denoted asT k ⊆ T, and feed them to the LLM to compute a refined item embedding:e ′ i = LLM(T k ).Our goal is to en- hance downstream recommendation performance by leveraginge ′ i , achieving superior results com- pared to those obtained using the original prefix- based embeddinge i . The selection ofT k is guided by a learnable policy, which prioritizes tokens that contribute most to user preference modeling. The problem addressed in this paper can be formulated as maximizing the expected reward: max T k R(θ).(3) We defer the definition of the rewardR(θ)to the next section, where it is given in Formula (16). 3.2 Informative Tokens Selection In this section, we detail our approach, i.e., TSPORec. To maximize flexibility, we perform informative token identification at the chunk level. A chunk is a subset of tokens consisting of several consecutive tokens in a token sequence. Specifi- cally, we select⌊k/c⌋chunks from the token se- quenceT, where each chunk consists ofcconsecu- tive tokens: h (t 1 1 ,t 2 1 ,...,t c 1 ),..., (t 1 ⌊k/c⌋ ,t 2 ⌊k/c⌋ ,...,t c ⌊k/c⌋ ) i , wherecdenotes the predefined chunk size, and⌊·⌋ is the floor function, which returns the greatest inte- ger less than or equal to its argument. Token-level selection is a special case of this formulation, corre- sponding toc = 1. This chunk-wise strategy allows for controllable granularity in capturing local se- mantic structures for downstream recommendation tasks. As shown in Figure 1, the overall pipeline of TSPORec consists of three stages: Pretraining, To- ken Selection Policy Learning, and Token Selection and Retraining. Pretraining In this stage, we pretrain both the user LLM (denoted asg) and the item LLM (de- noted asf) employing the InfoNCE loss defined in Eq. (1) to obtain a foundation model that can be leveraged for reward computation. For item LLM pretraining, we append a special [ITEM] token to the end of the textual token se- quence of each item. The item embedding is then extracted as the final-layer hidden state correspond- ing to this [ITEM] token. Specifically, given an input sequenceT = t 1 ,t 2 ,...,t k , [ITEM], the item LLM produces hidden states: [h 1 , h 2 ,..., h k+1 ].(4) We utilizeh k+1 , i.e., the representation at the [ITEM] position, as the item embedding. Token Selection Policy Learning In this stage, we introduce our novel token selection policy. Given the full set of textual features for all items, exhaustively evaluating every possible token sub- set is computationally infeasible due to its expo- nential time complexity. To mitigate this burden, we propose a differentiable proxy reward that ap- proximates the utility of a token subset, enabling efficient optimization of the selection policy. We begin by quantifying the informativeness of each tokent i through its relevance to the item embedding. Specifically, we compute a scalar im- portance score, referred to as theinfo(·), using a query-key attention mechanism: info(t i ) = ⟨W Q h i ,W K h k+1 ⟩ √ d ,(5) whereθ = W Q ,W K denotes a set of learnable parameter matrices inR d×d , anddis the dimension- ality of the hidden embeddings. The resulting score reflects the alignment between the token represen- tationh i and the item-level representationh k+1 . We then define the probability to select Tokent i as p i = exp (info(t i )) P k j=1 exp (info(t j )) .(6) Given a sequence ofMitems that a user has in- teracted with, denoted asI M =T 1 ,T 2 ,...,T M , whereT j represents the token sequence of thej-th item. We model the joint probability of the interac- tion sequence as P(I M ) = M Y j=1 k Y ℓ=1 p jℓ ,(7) wherep jℓ is the probability of theℓ-th token inT j . Based on these definitions, we outline the proce- dure for training the policy parametersθ. For each token sequenceT j in the interaction historyI M of a user, we perform two independent sampling passes to extract distinct subsets of token chunks, yielding two derived sequences denotedT ′ j andT ′ j . This process results in two new token sequence sets: I ′ M =T ′ 1 ,T ′ 2 ,...,T ′ M (8) I ′ M =T ′ 1 ,T ′ 2 ,...,T ′ M .(9) Using these sampled sequences, we construct two corresponding user embeddings via the pretrained, frozen LLM: e ′ u = g frozen f frozen (I ′ M ) (10) e ′ u = g frozen f frozen (I ′ M ) .(11) We construct the full item embedding based on the full token sequence as e i = f frozen (I M ).(12) We compute two cross-entropy values by contrast- ing the user embeddingse ′ u ande ′ u with the full item embeddinge i , exploiting a set ofNnegative item embeddingse j i N j=1 . This yields two cross- entropy values,L 1 ce andL 2 ce : L 1 ce =− log exp(⟨e ′ u , e i ⟩) exp(⟨e ′ u , e i ⟩) + P N j=1 exp(⟨e ′ u , e j i ⟩) (13) L 2 ce =− log exp(⟨e ′ u , e i ⟩) exp(⟨e ′ u , e i ⟩) + P N j=1 exp(⟨e ′ u , e j i ⟩) . (14) The reward signal r is then defined as: r = ( 1ifL 1 ce ≤L 2 ce , −1 otherwise. (15) Our objective is to optimize the policy parame- ters θ by maximizing the expected reward: R(θ) = E h r· logP(I ′ M | θ)− logP(I ′ M | θ) i (16) Given the above description of TSPORec, we establish the following theorem: Theorem 1. Under the framework of TSPORec, the following properties hold: •Smaller values of the cross-entropy losses L 1 ce andL 2 ce in Eq.(13)and Eq.(14)yield a tighter approximation to the ground-truth preference distribution in terms of KL diver- gence. •Token chunks shared between the two sampled sequencesI ′ M andI ′ M do not contribute to the gradient updates of the policy parameters θ. •The objective in Eq.(16)increases the like- lihood of selecting informative token chunks while suppressing less informative ones. Proof. We defer the detailed proof to Appendix F. DatasetMethodR@5R@10R@50N@5N@10N@50Impr. (avg) Amazon Books SASRec3.385.0911.592.242.794.20+0.0 % HSTU2.884.5111.061.892.413.83-11.46% LLMinit3.295.0511.742.182.744.19-1.14% HLLM(random)3.295.1712.962.172.774.45+2.14 % HLLM(topk logits)3.395.3413.352.242.864.59+5.36% HLLM(first-k)4.16 6.2914.492.803.485.25+24.40% TSPORec (Ours)4.376.5514.982.943.645.47+29.86% Best Impr.29.29%28.68%29.25%31.25%30.47%30.24%N/A Pixel SASRec2.413.779.631.592.033.29+0.0% HSTU2.143.418.851.401.812.97-10.22% LLMinit2.654.1010.531.732.193.58+8.92% HLLM(random)2.82 4.3911.071.842.343.78+15.71% HLLM(topk logits)2.794.3611.021.822.333.76+14.89% HLLM(first-k)2.804.4111.151.832.353.80+15.88% TSPORec (Ours)2.884.5111.201.882.403.85+18.15% Best Impr. 19.50%19.63%16.30%18.24%18.23%17.02%N/A Table 1: Performance comparison of various methods on the Amazon Books and Pixel datasets, using Qwen3- Embedding-0.6B as the backbone model, with text sequences truncated to 64 tokens. Token Selection and Retraining After optimiz- ing the policy parameters, we proceed to select informative token chunks as follows. We define the probability of a chunkcasP(c) = Q |c| i=1 p i , wherep i denotes the generation probability of the i-th token in the chunk, and|c|is the number of tokens inc. Chunks with higher probability scores according toP(c)are selected. Using this set of high-probability chunks, we construct a refined dataset and retrain the model on this updated rep- resentation to improve both its performance and efficiency. 4 Experiments In this section, we first describe the experimental setup, followed by a series of comprehensive ex- periments designed to evaluate the effectiveness and the efficiency of TSPORec. Finally, we present a case study that illustrates the specific tokens se- lected by TSPORec, providing insights into its op- erational mechanism. 4.1 Experimental Setup We conduct a comprehensive evaluation of TSPORec against four state-of-the-art baselines: (1) traditional ID-based sequential recommenda- tion models, such as SASRec (Kang and McAuley, 2018) and HSTU (Zhai et al., 2024); (2) se- mantic initialization methods that leverage large language model (LLM)-derived embeddings to initialize item ID representations, exemplified by LLMinit (Harte et al., 2023); and (3) hi- erarchical LLM-based frameworks, specifically HLLM (Chen et al., 2024). In addition, to study the impact of token selection, we adapt these baselines with different token selection policies, resulting in six baseline variants for compari- son.We evaluate our approach on two pub- licly available datasets: the Amazon Book Re- views dataset (McAuley et al., 2015) and the Pixel dataset (Cheng et al., 2023). For backbone LLMs, we employ Qwen3-Embedding-0.6B (Zhang et al., 2025b) and TinyLlama-1.1B (Zhang et al., 2024b). Performance is evaluated using Recall@K (R@K) and NDCG@K (N@K). Additional details on the experimental setup are given in Appendix A. 4.2 Experimental Results Overall Performance As shown in Table 1, TSPORec significantly outperforms all baseline approaches by a substantial margin. In this setting, input sequences are truncated to 64 tokens, and the Qwen3-Embedding-0.6B model serves as the frozen LLM backbone. For TSPORec, the chunk size is set to 8. On the Amazon Books dataset, TSPORec improves upon SASRec by up to 29.29% in Recall@K and 31.25% in NDCG@K, yielding an average gain of 29.43% across metrics. On the Pixel dataset, it achieves improvements of up to 19.63% in Recall@K and 18.24% in NDCG@K, with an average increase of 16.79%. The Importance of Tokens in LLM-based Rec- ommendation Given the significant advances achieved by LLM-based sequential recommenda- tion methods, we investigate the impact of different types of input tokens. Specifically, we compare random tokens, top-klogit tokens, the firstkto- kens, and TSPORec. For random tokens, we ran- domly selectktokens from the item text. For top-k logit tokens, we select thektokens with the high- est logits. The firstktokens is the default strategy employed by many existing methods. In contrast, TSPORec selects k tokens using a learned policy. As shown in Table 1, TSPORec achieves the most significant performance gain by leveraging the learned token selection policy, substantially outperforming existing token selection strategies. On the Amazon Books dataset, randomly selecting tokens reduces the performance improvement of HLLM from 24.40% to just 2.14%, highlighting the critical role of input token selection in LLM- based recommendation. These results demonstrate that the choice of in- put tokens has a profound impact on model ef- fectiveness—a factor largely overlooked by prior approaches. Surprisingly, the top-klogit tokens do not outperform the simple first-ktokens baseline, suggesting that more principled token selection mechanisms are needed. This limitation may stem from the unidirectional nature of standard LLM attention, which prevents hidden states from effec- tively capturing meaningful collaborative signals across the sequence. In contrast, as illustrated in Eq.(5), TSPORec leverages a bidirectional scoring mechanism. The importance score of each token is learned from both (i) the standard attention mecha- nism, which encodes semantic meaning, and (i) the pretrained item embeddingh k+1 , which provides a collaborative filtering signal. This dual-source design enables TSPORec to jointly capture rich semantic information and sequential collaborative patterns, thereby facilitating the identification of the most informative tokens and ultimately improv- ing model performance. The Impact of Token Sequence Length in LLM- based Recommendation In this section, we fur- ther investigate how the length of input tokens im- pacts recommendation performance. To this end, we conduct extensive experiments with varying in- put token lengths, as shown in Figure 2. We denote each configuration as Method−k, wherekrepre- sents the number of input tokens used. For instance, HLLM-128indicates the use of the first 128 tokens from the input sequence, while TSPORec-64de- notes the selection of 64 tokens according to the learned token selection policy. As illustrated in Figure 2, on the Amazon Books dataset, increasing the token length leads to performance improvements for both HLLM and TSPORec, suggesting that longer input sequences may contain additional informative tokens that ben- Token S I t (ms)U t (ms)Total (ms) 6429927326 12850927536 25681726843 Table 2: Inference time cost during evaluation.Token S denotes the length of the input token sequence,I t the inference time of the item LLM, andU t the inference time of the user LLM, using Qwen3-Embedding-0.6B. efit recommendation quality. In contrast, on the Pixel dataset, while moderate increases in token length initially enhance HLLM’s performance, fur- ther lengthening the input sequence eventually de- grades performance. This indicates that not all to- kens contribute positively to recommendations, and including irrelevant or noisy tokens can be detri- mental. This phenomenon further underscores the necessity of effective token selection to improve recommendation performance. Moreover, on both the Amazon Books and Pixel datasets, TSPORec with only 64 selected tokens achieves performance on par with or superior to the corresponding HLLM variants using 256 tokens. These results demon- strate that TSPORec not only enhances recommen- dation accuracy but also improves computational efficiency by enabling effective performance with significantly fewer tokens. The Efficiency of TSPORecTo evaluate the im- pact on inference efficiency of TSPORec, we an- alyze the computational cost across varying input sequence lengths. By selecting informative tokens, TSPORec effectively reduces the required input length without sacrificing performance, thereby lowering overall computational overhead. We measure the inference time of LLM on an NVIDIA H100 GPU using the Amazon Books dataset, with a batch size of 32 and an item se- quence length of 10. The results are summarized in Table 2. We observe that the item LLM dom- inates the inference time, and its computational share grows significantly as the input token se- quence length increases. Since TSPORec matches the performance of HLLM using only 64 tokens with HLLM using 256 tokens, it reduces inference cost by 63.4% in scenarios where item embeddings are precomputed offline, and by 61.3% when both the item and user LLMs are served online. The Generalization capability of TSPORecTo evaluate the generalization ability of TSPORec, we conduct experiments using a different LLM backbone—TinyLlama-1.1B, as well as a differ- ent downstream SR model, LLMinit (Harte et al., HLLM-64 HLLM-128HLLM-256 TSPORec-64 TSPORec-128 4.0 4.1 4.2 4.3 4.4 4.5 4.16 4.23 4.374.37 4.38 Recall@5 (a) Amazon Book, Recall@5 HLLM-64 HLLM-128HLLM-256 TSPORec-64 TSPORec-128 2.7 2.8 2.9 3.0 2.8 2.84 2.93 2.942.94 NDCG@5 (b) Amazon Book, NDCG@5 HLLM-64 HLLM-128HLLM-256 TSPORec-64 2.6 2.7 2.8 2.9 2.8 2.87 2.86 2.88 Recall@5 (c) Pixel, Recall@5 HLLM-64 HLLM-128HLLM-256 TSPORec-64 1.7 1.8 1.9 1.83 1.861.86 1.88 NDCG@5 (d) Pixel, NDCG@5 HLLM-64 HLLM-128HLLM-256 TSPORec-64 TSPORec-128 6.1 6.2 6.3 6.4 6.5 6.6 6.29 6.4 6.57 6.55 6.6 Recall@10 (e) Amazon Book, Recall@10 HLLM-64 HLLM-128HLLM-256 TSPORec-64 TSPORec-128 3.4 3.5 3.6 3.7 3.48 3.55 3.643.64 3.65 NDCG@10 (f) Amazon Book, NDCG@10 HLLM-64 HLLM-128HLLM-256 TSPORec-64 4.4 4.5 4.6 4.41 4.47 4.39 4.51 Recall@10 (g) Pixel, Recall@10 HLLM-64 HLLM-128HLLM-256 TSPORec-64 2.3 2.4 2.35 2.37 2.35 2.4 NDCG@10 (h) Pixel, NDCG@10 Figure 2: Performance of HLLM and TSPORec on Amazon Books and Pixel with different token sequence lengths, using Qwen3-Embedding-0.6B as the backbone model. MethodR@5R@10N@5N@10Impr. HLLM3.996.072.663.33+0.0% TSPORec4.166.302.783.46+4.12% Table 3: Performance comparison of HLLM and TSPORec on the Amazon Books dataset, using TinyLlama-1.1B as the backbone model, with text se- quences truncated to 64 tokens. 2023). We compare TSPORec against the HLLM variant with the same LLM backbone, and the re- sults are presented in Table 3. TSPORec consis- tently outperforms HLLM across all Recall and NDCG metrics, achieving an average improve- ment of 4.12%. Furthermore, as shown in Table 4, when TSPORec-selected tokens are used to initial- ize LLMinit, the model performance improves by 3.76%, demonstrating that the selected tokens carry informative signals for recommendation. These results confirm that TSPORec not only enhances downstream performance but also exhibits strong generalization across different LLM architectures and recommendation models. 4.3 Case Study In this section, we conduct a case study to examine the underlying mechanisms of token selection. An illustrative example is shown in Figure 3, with ad- ditional cases provided in Appendix B for further analysis. We compare the tokens selected by the first-kstrategy and TSPORec on the Pixel dataset. As depicted in Figure 3, notable differences exist between the two strategies, highlighted in green (tokens selected by TSPORec but not by first-k) and blue (tokens selected by first-kbut not by TSPORec). We observe that TSPORec removes the text seg- MethodR@5R@10Impr. LLMinit(first-k)3.295.05+ 0.0% LLMinit(TSPORec)3.425.23+3.76% Table 4: Performance comparison of LLMinit on the Amazon Books dataset, with different tokens for LLM embedding generation. ment “released in the summer of In accordance” and instead selects new content such as “cucum- ber, endless mode, Penny”. The newly selected tokens are predominantly content words—nouns, adjectives, or named entities—that convey specific and semantically rich information about the item. In contrast, the removed tokens consist largely of function words (e.g., prepositions, pronouns, and articles) that carry limited discriminative power. By prioritizing content words, TSPORec enriches item representations with more informative fea- tures, thereby enhancing downstream recommen- dation performance. Figure 4 further illustrates that TSPORec tends to filter out frequent (“hot”) tokens. This behavior is meaningful: hot tokens are shared across many items and thus exhibit low semantic specificity, making it difficult for the user LLM to distinguish between items and learn effective representations. 5 Related Work LLM-based SR LLMs possess extensive world knowledge and strong reasoning capabilities, offer- ing two principal advantages for sequential recom- mendation: rich semantic understanding and pow- erful modeling capacity derived from their high- dimensional parameter space. These characteristics enable LLMs to effectively capture complex user behavior patterns and thereby enhance recommen- dation performance (Sun et al., 2024; Zhang et al., Title: [Official Setting Book] The Story Behind Zombies (I) Tag: Mobile Games Description: Plants vs. Zombies is the sequel to Plants vs. Zombies, released in the summer of In accor- dance with the classic plant defense zombie gameplay based on the addition of energy beans, tactical cu- cumbers, endless mode, Penny chase and other elements and gameplay, the player will be able to experience more than a dozen play a variety of space and time maps the game at the same time integrated with real- time strategy, tower defense battles and card collection and other elements, the player to control the plants to fend off zombie attacks, to protect the space and time! Compress the following sentence into embedding: Title: [Official Setting Book] The Story Behind Zom- bies (I) Tag: Mobile Games Description: Plants vs. Zombies is the sequel to Plants vs. Zombies, released in the summer of In ac- cordance with the classic plant defense zombie game- play based on the addition of energy beans, tactical Compress the following sentence into embedding: Title: [Official Setting Book] The Story Behind Zom- bies (I) Tag: Mobile Games Description: Plants vs. Zombies is the sequel to Plants vs. Zombies with the classic plant defense zombie gameplay based on the addition of energy beans, tactical cucumbers, endless mode, Penny t o k e n s 6 4 f i r s t T S P O R e c s e l e c t e d 6 4 t o k e n Figure 3: Case study on token selection for item “i9019” in the Pixel dataset, with 64 tokens selected by each method. 0K 100K 200K 300K Ġthe Ġof Ġin ĠIn Ġsummer um Ġmode Ġendless Ġaccordance bers ĠPenny released Ġcuc Tokens Count Figure 4: Comparison of token distributions selected by first-k and TSPORec ( ̇ G is a leading space token). 2024a, 2025a; Qiao et al., 2024; Liu et al., 2024b; Xu et al., 2024). In the context of SR, LLM-based approaches can be broadly categorized into two groups. The first category leverages LLMs as semantic initializers to enrich ID-based item embeddings (Harte et al., 2023; Liu et al., 2024a; Hu et al., 2025). A key challenge in these methods arises from the dimensional mismatch between the high- dimensional LLM embeddings and the typically lower-dimensional ID embeddings used in rec- ommendation systems. To address this, dimen- sionality reduction techniques are commonly em- ployed—such as Principal Component Analysis in LLMEmb (Liu et al., 2024a) and Singular Value Decomposition in AlphaFuse (Hu et al., 2025). The second category adopts LLMs in an end- to-end manner, where the model directly utilizes the LLM and tokenized item sequences to predict the next item in the interaction sequence of users. HLLM (Chen et al., 2024) demonstrates signifi- cant performance improvements over traditional methods. This end-to-end integration enables mod- eling of both semantic content and collaborative signals from user interactions, thereby advancing the recommendation performance. However, due to computational constraints, only the first portion of the textual description of each item is used. This truncation limits the model’s ability to capture the full richness of the seman- tic information present in longer descriptions, po- tentially leading to suboptimal performance. In contrast, our method selects the most informative tokens from item descriptions. 6 Conclusion In this paper, we propose a novel Token Selection approach tailored for LLM-driven Sequential Rec- ommendation systems, i.e., TSPORec. We develop a three-stage workflow. First, we pre-train the foun- dational LLM-based sequential recommendation model to establish a robust baseline. Second, we train a dedicated policy to identify informative to- kens; additionally, a proxy reward function is de- signed to facilitate chunk-oriented token selection, addressing the challenges of granularity in token identification. Finally, leveraging the identified to- kens, we retrain the pre-trained baseline model to optimize its recommendation performance. Com- prehensive experimental results demonstrate the effectiveness (up to 31.25%) and efficiency (up to 63.4%) of the proposed method. Limitations The proposed TSPORec employs a three-stage pipeline to select the most informative tokens from the item’s textual features. While TSPORec sig- nificantly improves both recommendation perfor- mance and inference efficiency compared to stan- dard training methods, it incurs additional training time due to Stages 2 and 3. However, this overhead is practically acceptable, as models are typically trained once and deployed over an extended pe- riod—rendering inference efficiency a more critical concern than the one-time training cost. References Baichuan. 2023. Baichuan 2: Open large-scale lan- guage models. arXiv preprint arXiv: 2309.10305. Junyi Chen, Lu Chi, Bingyue Peng, and Zehuan Yuan. 2024. Hllm: Enhancing sequential recommendations via hierarchical large language models for item and user modeling. arXiv preprint arXiv:2409.12740. Junyi Chen, Lu Chi, Siliang Xu, Shiwei Ran, Bingyue Peng, and Zehuan Yuan. 2025. Hllm-creator: Hier- archical llm-based personalized creative generation. arXiv preprint arXiv:2508.18118. Chen Cheng, Haiqin Yang, Michael R. Lyu, and Irwin King. 2013. Where you like to go next: Successive point-of-interest recommendation. In IJCAI, pages 2605–2611. Yu Cheng, Yunzhu Pan, Jiaqi Zhang, Yongxin Ni, Aixin Sun, and Fajie Yuan. 2023. An image dataset for benchmarking recommender systems with raw pixels. arXiv preprint arXiv:2309.06789. Mingkai Deng, Jianyu Wang, Cheng-Ping Hsieh, Yihan Wang, Han Guo, Tianmin Shu, Meng Song, Eric P Xing, and Zhiting Hu. 2022. Rlprompt: Optimizing discrete text prompts with reinforcement learning. arXiv preprint arXiv:2205.12548. Kawin Ethayarajh, Winnie Xu, Niklas Muennighoff, Dan Jurafsky, and Douwe Kiela. 2024. Kto: Model alignment as prospect theoretic optimization. arXiv preprint arXiv:2402.01306. Mohammad Gheshlaghi Azar, Zhaohan Daniel Guo, Bi- lal Piot, Remi Munos, Mark Rowland, Michal Valko, and Daniele Calandriello. 2024. A general theoret- ical paradigm to understand learning from human preferences. In AISTATS. Chi Han, Qifan Wang, Hao Peng, Wenhan Xiong, Yu Chen, Heng Ji, and Sinong Wang. 2024. Lm- infinite: Zero-shot extreme length generalization for large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 3991–4008. Jesse Harte, Wouter Zorgdrager, Panos Louridas, Aster- ios Katsifodimos, D. Jannach, and Marios Fragkoulis. 2023. Leveraging large language models for sequen- tial recommendation. Proceedings of the 17th ACM Conference on Recommender Systems. Balazs Hidasi, Alexandros Karatzoglou, Linas Bal- trunas, and Domonkos Tikk. 2015. Session-based recommendations with recurrent neural networks. arXiv preprint arXiv:1511.06939. Jiwoo Hong, Noah Lee, and James Thorne. 2024. ORPO: Monolithic preference optimization without reference model. In EMNLP. Guoqing Hu, An Zhang, Shuo Liu, Zhibo Cai, Xun Yang, and Xiang Wang. 2025. Alphafuse: Learn id embeddings for sequential recommendation in null space of language embeddings. arXiv preprint arXiv:2504.19218. Wang-Cheng Kang and Julian J. McAuley. 2018. Self- attentive sequential recommendation.In ICDM, pages 197–206. Sein Kim,Hongseok Kang,Seungyoon Choi, Donghyun Kim, Minchul Yang, and Chanyoung Park. 2024. Large language models meet collaborative filtering: An efficient all-round llm-based recom- mender system. In Proceedings of the 30th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, KDD ’24, page 1395–1406, New York, NY, USA. Association for Computing Machinery. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. arXiv preprint arXiv:2104.08691. Jiacheng Li, Yujie Wang, and Julian McAuley. 2020. Time interval aware self-attention for sequential rec- ommendation. In WSDM, pages 322–330. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. arXiv preprint arXiv:2101.00190. Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. 2024. Llara: Large language-recommendation assis- tant. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR ’24, page 1785–1795, New York, NY, USA. Association for Computing Machinery. Qidong Liu, Xian Wu, Wanyu Wang, Yejing Wang, Yuanshao Zhu, Xiangyu Zhao, Feng Tian, and Yefeng Zheng. 2024a. Large language model empowered embedding generator for sequential recommendation. arXiv preprint arXiv:2409.19925. Qidong Liu, Xian Wu, Yejing Wang, Zijian Zhang, Feng Tian, Yefeng Zheng, and Xiangyu Zhao. 2024b. Llm-esr: Large language models enhancement for long-tailed sequential recommendation. Advances in Neural Information Processing Systems, 37:26701– 26727. Zhiwei Liu, Yongjun Chen, Jia Li, Philip S. Yu, Ju- lian McAuley, and Caiming Xiong. 2021. Con- trastive self-supervised sequential recommenda- tion with robust augmentation.arXiv preprint arXiv:2108.06479. Junru Lu, Jiazheng Li, Siyu An, Meng Zhao, Yulan He, Di Yin, and Xing Sun. 2024. Eliminating bi- ased length reliance of direct preference optimiza- tion via down-sampled kl divergence. arXiv preprint arXiv:2406.10957. Julian McAuley, Christopher Targett, Qinfeng Shi, and Anton van den Hengel. 2015. Image-based recom- mendations on styles and substitutes. In Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval, page 43–52. Yu Meng, Mengzhou Xia, and Danqi Chen. 2024. Simpo: Simple preference optimization with a reference-free reward. In NeurIPS. Shutong Qiao, Chen Gao, Junhao Wen, Wei Zhou, Qun Luo, Peixuan Chen, and Yong Li. 2024. Llm4sbr: A lightweight and effective framework for integrating large language models in session-based recommen- dation. arXiv preprint arXiv:2402.13840. Rafael Rafailov, Archit Sharma, Eric Mitchell, Christo- pher D Manning, Stefano Ermon, and Chelsea Finn. 2023. Direct preference optimization: Your language model is secretly a reward model. In NeurIPS. Steffen Rendle, Christoph Freudenthaler, and Lars Schmidt-Thieme. 2010. Factorizing personalized markov chains for next-basket recommendation. In W, pages 811–820. Fei Sun, Jun Liu, Jian Wu, Changhua Pei, Xiao Lin, Wenwu Ou, and Peng Jiang. 2019. Bert4rec: Se- quential recommendation with bidirectional encoder representations from transformer. In Proceedings of the 28th ACM International Conference on Informa- tion and Knowledge Management, page 1441–1450. Zhongxiang Sun, Zihua Si, Xiaoxue Zang, Kai Zheng, Yang Song, Xiao Zhang, and Jun Xu. 2024. Large language models enhanced collaborative filtering. In Proceedings of the 33rd ACM International Confer- ence on Information and Knowledge Management, pages 2178–2188. Yong Kiam Tan, Xinxing Xu, and Yong Liu. 2016. Im- proved recurrent neural networks for session-based recommendations. In Proceedings of the 1st Work- shop on Deep Learning for Recommender Systems, pages 17–22. Jiaxi Tang and Ke Wang. 2018. Personalized top-n se- quential recommendation via convolutional sequence embedding. In Proceedings of the Eleventh ACM International Conference on Web Search and Data Mining, page 565–573. Aaron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. Representation learning with contrastive predictive coding. arXiv preprint arXiv:1807.03748. Liwei Wu, Shuqing Li, Cho-Jui Hsieh, and James Sharp- nack. 2020. Sse-pt: Sequential recommendation via personalized transformer. In RecSys, pages 328–337. Xu Xie, Fei Sun, Zhaoyang Liu, Shiwen Wu, Jinyang Gao, Bolin Ding, and Bin Cui. 2021. Contrastive learning for sequential recommendation.arXiv preprint arXiv:2010.14395. Wujiang Xu, Qitian Wu, Zujie Liang, Jiaojiao Han, Xuy- ing Ning, Yunxiao Shi, Wenfang Lin, and Yongfeng Zhang. 2024. Slmrec: Distilling large language mod- els into small for sequential recommendation. arXiv preprint arXiv:2405.17890. Zhengyi Yang, Jiancan Wu, Zhicai Wang, Xiang Wang, Yancheng Yuan, and Xiangnan He. 2023.Gen- erate what you prefer: Reshaping sequential rec- ommendation via guided diffusion. arXiv preprint arXiv:2310.20453. Jiaqi Zhai, Lucy Liao, Xing Liu, Yueming Wang, Rui Li, Xuan Cao, Leon Gao, Zhaojie Gong, Fangda Gu, Jiayuan He, Yinghai Lu, and Yu Shi. 2024. Actions speak louder than words: trillion-parameter sequen- tial transducers for generative recommendations. In Proceedings of the 41st International Conference on Machine Learning. Chao Zhang, Shiwei Wu, Haoxin Zhang, Tong Xu, Yan Gao, Yao Hu, and Enhong Chen. 2024a. Notellm: A retrievable large language model for note recommen- dation. In Companion Proceedings of the ACM Web Conference 2024, pages 170–179. Chao Zhang, Haoxin Zhang, Shiwei Wu, Di Wu, Tong Xu, Xiangyu Zhao, Yan Gao, Yao Hu, and Enhong Chen. 2025a. Notellm-2: Multimodal large represen- tation models for recommendation. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pages 2815–2826. Peiyuan Zhang, Guangtao Zeng, Tianduo Wang, and Wei Lu. 2024b. Tinyllama: An open-source small language model. arXiv preprint arXiv: 2401.02385. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025b. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Chenzhuo Zhao, Ziqian Liu, Xingda Wang, Junting Lu, and Chaoyi Ruan. 2025. Pmpo: Probabilistic metric prompt optimization for small and large language models. arXiv preprint arXiv:2505.16307. Zhi Zheng, Wen Shuo Chao, Zhaopeng Qiu, Heng- shu Zhu, and Hui Xiong. 2024. Harnessing large language models for text-rich sequential recommen- dation. In W 2024 - Proceedings of the ACM Web Conference, W 2024 - Proceedings of the ACM Web Conference, pages 3207–3216. Association for Computing Machinery, Inc. Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019. Deep interest evolution network for click-through rate prediction. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence and Thirty- First Innovative Applications of Artificial Intelligence Conference and Ninth AAAI Symposium on Educa- tional Advances in Artificial Intelligence. Jie Zhu, Zhifang Fan, Xiaoxie Zhu, Yuchen Jiang, Hangyu Wang, Xintian Han, Haoran Ding, Xinmin Wang, Wenlin Zhao, Zhen Gong, Huizhi Yang, Zheng Chai, Zhe Chen, Yuchao Zheng, Qiwei Chen, Feng Zhang, Xun Zhou, Peng Xu, Xiao Yang, and 2 others. 2025a. Rankmixer: Scaling up ranking models in industrial recommenders. In Proceedings of the 34th ACM International Conference on Information and Knowledge Management, CIKM ’25, New York, NY, USA. Association for Computing Machinery. Wenqiao Zhu, Ji Liu, Lulu Wang, Jun Wu, and Yulun Zhang. 2025b. Sgdpo: Self-guided direct preference optimization for language model alignment. Wenqiao Zhu, Lulu Wang, and Jun Wu. 2025c. Address- ing cold-start problem in click-through rate predic- tion via supervised diffusion modeling. Wenqiao Zhu, Yesheng Xu, Xin Huang, Qiyang Min, and Xun Zhou. 2022. Spherical graph embedding for item retrieval in recommendation system. In Proceed- ings of the 31st ACM International Conference on Information and Knowledge Management, CIKM22, page 4752–4756. A More Experimental Setup Dataset#Users#Items#Interactions Amazon Books694,898686,62410,053,086 Pixel200,00096,2823,965,656 Table 5: Statistics of the Pixel and Amazon Books Datasets. In this section, we detail the experimental setup used to evaluate our approach. We conduct experiments on two publicly available datasets: the Amazon Book Reviews dataset (McAuley et al., 2015) and the Pixel dataset (Cheng et al., 2023). We present the statis- tics of the datasets used in Table 5. The Amazon Books dataset contains 694,898 users and 686,624 items, while the Pixel dataset contains 200,000 users and 96,282 items. Both datasets exhibit high sparsity, with user-item interaction densities of 2.1× 10 −5 and2.1× 10 −4 , respectively. Items with missing attributes are filtered out. We adopt a leave-one-out evaluation protocol: the most recent interaction is used for testing, the second-to-last for validation, and all earlier interactions for training. Performance is evaluated using Recall@K (R@K) and NDCG@K (N@K). Following the setup of HLLM (Chen et al., 2024), we set the learning rate to1 × 10 −4 for all baseline models. HLLM is trained for 5 epochs, while other models are trained for up to 200 epochs with early stopping to prevent overfitting. We fix the number of negative samples at 128 and set the maximum sequence length to 10. In the policy learning phase, the model is trained for 5 epochs, with a learning rate 1× 10 −3 . To investigate the impact of textual input length, we truncate item descriptions to 64, 128, and 256 to- kens, respectively. For the Amazon Books dataset, we use the title and description fields. For the Pixel dataset, we utilize the title, tag, and description text features. Since starting tokens occupy a distinct feature space (Han et al., 2024), TSPORec retains the first prefix size to 16 tokens from all sampled sequences. We train all models on a GPU cluster equipped with 8 H100 and 8 A800 GPUs. Each experiment is conducted three times, and we report the average performance to ensure statistical reliability. B Case Study We conduct an additional case study to further ex- amine the token selection behavior of the compet- ing methods. The results for item “i113948” are il- lustrated in Figure 5. Consistent with earlier obser- vations, significant differences are evident between the two strategies: tokens selected exclusively by TSPORec are highlighted in green, while those se- lected only by the first-kstrategy are marked in blue, allowing for a fine-grained comparison of their selection patterns. In this instance, TSPORec removes tag-related segments such as “tag: Cover Song” and “Fill in the lyrics cover original song: model original song”, and instead selects the segment “Ronghao Lyrics: Lok Yang, Cold Ashes Singing: Cold test service to play Penglai play”. This selection is semantically meaningful: for a music item, meta- data about the lyricist and composer provides more concrete, fine-grained, and discriminative signals than generic categorical tags. By prioritizing such specific metadata, TSPORec enriches the item rep- resentation, thereby improving downstream recom- mendation performance. As shown in Figure 6, consistent with the previ- ous case study, TSPORec tends to filter out high- frequency tokens. This behavior encourages the item LLM to learn more discriminative embed- dings, as frequent tokens often carry less item- specific semantic information. Please note that, whereas Figure 4 shows the token distribution across the full set of text features, Figure 6 depicts the distribution of token counts in the selection re- sults. The results reveal that—under both counting metrics—the proposed method consistently filters out frequent (“hot”) tokens. C More Experimental Results The Impact of Training and Selection Chunk SizesSince the training and selection chunk sizes may differ, we conduct an ablation study to inves- tigate the impact of training and selection chunk sizes on recommendation performance. we denote each configuration as TSPORec-Tk 1 Sk 2 , where the model is trained with chunk sizek 1 and performs token selection during inference with chunk size k 2 . We evaluate TSPORec on the Amazon Books dataset using an input sequence length of 128 to- kens. The results, shown in Figure 7, demonstrate that: (1) across all configurations, the proposed TSPORec consistently outperforms the HLLM Title: [Cold Embers] Sword three to Penglai friendly land blessing of the new sect on the line Tag: Cover Song Description: Fill in the lyrics cover original song: model original song: Li Ronghao Lyrics: Zhou Yaohui Composer: Li Ronghao Lyrics: Lok Yang, Cold Ashes Singing: Cold Ashes Late: Cold Ashes POV: Lok Yang Inspiration: Geng Zhi Men to the former in the test service to play Penglai play my blackout, special issue this song, congratulations on the new sect Penglai is about to go on line oh yes, or to say a little bit, I’m not Cold Ashes, Cold Ashes in the b station’s id is called the mountains are warm and the wind is light I’m not cold ashes I’m not cold ashes I’m not cold ashes I’m not cold ashes I’m not cold ashes he opened the live broadcast I’m not opening live broadcast I’m going to go to his live room occasionally and his play the game chirp chirp chirp chirp chirp chirp chirp chirp chirp! Compress the following sentence into embedding: Title: [Cold Embers] Sword three to Penglai friendly land blessing of the new sect on the line Tag: Cover Song Description: Fill in the lyrics cover original song: model original song: Li Ronghao Lyrics: Zhou Yaohui Composer: Li Compress the following sentence into embedding: Title: [Cold Embers] Sword three to Penglai friendly land blessing of the new sect on the lin : Zhou Yaohui Composer: Li Ronghao Lyrics: Lok Yang, Cold Ashes Singing: Cold test service to play Penglai play t o k e n s 6 4 f i r s t T S P O R e c s e l e c t e d 6 4 t o k e n Figure 5: Case study on token selection for item “i113948” in the Pixel dataset, with 64 tokens selected by each method. baseline by up to 3.54%, indicating robustness to varying parameter settings. (2) On average, TSPORec-T16S1, TSPORec- T16S8, TSPORec-T16S16, and TSPORec-T16S32 outperform HLLM by 3.54%, 2.15%, 2.54%, and 0.53%, respectively. A general downward trend in performance is observed as the selection chunk size increases, indicating that smaller chunks yield more effective token selection. We further validate this trend using top-klogit-based selection, with results presented in Table 7. The consistent perfor- mance degradation with larger chunk sizes across both methods suggests a shared preference for finer- grained selection units. This observation further implies that the limited effectiveness of top-klogit selection may stem from its unidirectional nature, rather than from disruptions to semantic sentence structure. Training TimeSince TSPORec employs a three- stage training pipeline, we report the training time for each stage in Table 6. The model is trained on a GPU cluster equipped with 8 NVIDIA H100 GPUs. Due to the additional stages, TSPORec requires approximately 21.5 hours more than a single-stage baseline. This overhead is acceptable in practice, as the full training procedure is performed only once and the resulting model is deployed for an extended period. D More Related Work D.1 Sequential Recommendation (SR) Sequential recommendation (SR) captures the di- verse preferences and evolving interests of users by modeling their recent interaction sequences (Hi- dasi et al., 2015; Tan et al., 2016; Zhou et al., 2019; Kang and McAuley, 2018; Li et al., 2020; Wu et al., 2020; Zhai et al., 2024; Harte et al., 2023; Chen et al., 2024). Traditional SR Traditional sequential recom- mendation methods primarily focus on enhanc- ing neural architectures to more accurately model users’ diverse interests. Representative examples include Caser (Tang and Wang, 2018), SASRec (Kang and McAuley, 2018), BERT4Rec (Sun et al., 2019), CLS2Rec (Xie et al., 2021), and CoSeRec (Liu et al., 2021). While effective, these approaches often struggle to scale to large parameter counts. To address this limitation, generative recommen- dation methods such as DreamRec (Yang et al., 2023) and HSTU (Zhai et al., 2024) have been proposed, which are inherently scalable to larger models. However, these generative approaches typ- ically fail to leverage the rich semantic knowledge pre-encoded in pre-trained LLMs, thereby limiting their potential to further improve recommendation performance. In contrast, our proposed TSPORec effectively harnesses the full semantic knowledge embedded in pre-trained LLMs, enabling more ac- curate sequential recommendations. Ġthe Ġto Ġin Ġsong ing ĠSong ĠCover Ġoriginal ĠR Ġplay ĠLi ai ĠP Ġcover ĠYang e R es Ġlyrics Ġtest Ġservice ĠSing Ġmodel ĠCold ĠAsh ĠFill ĠLok Description engl Tag Ċ 0K 50K 100K 150K Tokens Count Figure 6: Comparison of token distributions selected by first-k and TSPORec. HLLM-128 TSPORec-T16S1TSPORec-T16S8 TSPORec-T16S16TSPORec-T16S32 4.2 4.3 4.4 4.5 4.23 4.39 4.32 4.264.26 Recall@5 (a) Recall@5 HLLM-128 TSPORec-T16S1TSPORec-T16S8 TSPORec-T16S16TSPORec-T16S32 6.3 6.4 6.5 6.6 6.7 6.4 6.6 6.53 6.6 6.45 Recall@10 (b) Recall@10 HLLM-128 TSPORec-T16S1TSPORec-T16S8 TSPORec-T16S16TSPORec-T16S32 2.8 2.9 3.0 2.84 2.95 2.91 2.94 2.85 NDCG@5 (c) NDCG@5 HLLM-128 TSPORec-T16S1TSPORec-T16S8 TSPORec-T16S16TSPORec-T16S32 3.5 3.6 3.7 3.55 3.67 3.62 3.65 3.56 NDCG@10 (d) NDCG@10 Figure 7: Performance of HLLM and TSPORec on Amazon Books with different training and selection chunk sizes, using Qwen3-Embedding-0.6B as the backbone model. D.2 Prompt Engineering and Preference Optimization Prompt Engineering A substantial body of re- search shows that fine-grained control over individ- ual tokens can unlock more intelligent, efficient, and trustworthy LLM applications, even under con- strained token budgets (Lester et al., 2021; Li and Liang, 2021; Zhao et al., 2025; Deng et al., 2022). Prompt Tuning (Lester et al., 2021) and Prefix- Tuning (Li and Liang, 2021) operate in the model’s continuous embedding space, using gradient-based optimization to learn effective prompt represen- tations. RL-Prompt (Deng et al., 2022) reformu- lates discrete prompt optimization as a reinforce- ment learning problem, where token sequences are selected to maximize task-specific rewards. The method achieves strong performance across a range of NLP tasks and intriguingly reveals that optimal prompts often deviate from standard grammatical conventions to enhance effectiveness. In contrast to these methods—which optimize prompts to steer model outputs—our work focuses on identifying informative tokens directly from the input. Preference Optimization Preference optimiza- tion aims to align large language models (LLMs) with human preferences and values. Direct Pref- erence Optimization (DPO) (Rafailov et al., 2023) is a representative method that increases the like- lihood of outputs preferred by humans while de- creasing the likelihood of disfavored ones. DPO has inspired numerous follow-up studies (Meng et al., 2024; Gheshlaghi Azar et al., 2024; Lu et al., 2024; Ethayarajh et al., 2024; Hong et al., 2024; Zhu et al., 2025b). Unlike these approaches, our method does not modify the output probability dis- tribution of LLMs. Instead, we adjust token-level Stage 1: PretrainingStage 2: Token Selection Policy LearningStage 3: Token Selection and Retraining 13h 49m 1s8h 56m 24s12h 40m 52s Table 6: Training time of TSPORec for each stage. MethodR@5R@10Impr. (avg) HLLM-64-S13.395.34+0.0% HLLM-64-S83.165.03-6.29% Table 7: Performance comparison of HLLM with top-k token selection under different selection chunk sizes on the Amazon Books dataset, using Qwen3-Embedding- 0.6B as the backbone and truncating text sequences to 64 tokens. probabilities in the input space to identify which input segments, when emphasized, can enhance downstream recommendation performance. E Algorithm Framework We present the pseudocode of TSPORec in Algo- rithm 1. TSPORec consists of three main stages. In Stage 1, an LLM-based recommendation model (e.g., HLLM) is pretrained on user-item interaction data using textual features. In Stage 2, the LLM backbone is frozen, and a policy head is introduced; TSPORec then learns a token selection policy using the proposed proxy reward. In the final stage, the learned policy is applied to select the most infor- mative tokens from the dataset, and the base model is retrained on the refined item representations. F Proof Theorem 1. Under the framework of TSPORec, the following properties hold: •Smaller values of the cross-entropy lossesce 1 andce 2 in Eq.(13)and Eq.(14)yield a tighter approximation to the ground-truth preference distribution in terms of KL divergence. •Token chunks shared between the two sampled sequencesI ′ M andI ′ M do not contribute to the gradient updates of the policy parameters θ. •The objective in Eq.(16)increases the like- lihood of selecting informative token chunks while suppressing less informative ones. Proof. LetL =−R(θ). Algorithm 1: The TSPORec pipeline Input :Training Instances:D 0 , Number of Epochs: U , Initial Model: π 0 . Output : Final Model: π U , Updated Training Instances: D U // Pretrain LLM-based recommendation model π 1 1 π 1 ← π 0 2 for i← 1 to U do 3x∼D 0 // Sample training data from D 0 4 π 1 ← UPDATE(π 1 , x) 5 end // Freeze the LLM backbone and initialize the policy head. 6 π θ ← FREEZE-AND-INIT-POLICY(π 1 ,θ) 7 for i← 1 to U do 8x∼D 0 // Sample training data from D 0 9s 0 ∼ π θ (x) // Sampling Item Tokens. 10s 1 ∼ π θ (x) 11 c 0 ← CROSS-ENTROPY(s 0 ,π 1 ) 12 c 1 ← CROSS-ENTROPY(s 1 ,π 1 ) 13 r = COMPUTE-REWARD(c 0 ,c 1 ) 14 π θ ← UPDATE(π θ ,r) 15 end 16 D U ← SELECT(π θ ,D 0 ) 17 π U ← π 0 18 for i← 1 to U do 19x∼D U // Sample training data from D U 20 π U ← UPDATE(π U , x) 21 end 22 return π U ,D U (1) We denote the preference distribution asso- ciated withe ′ u asp ′ , that associated withe ′ u as p ′ , and the ground-truth preference distribution as q. By definition, the Kullback–Leibler (KL) diver- gences are given by KL(q∥p ′ ) = Z q(x) log q(x) p ′ (x) dx(17) = ce 1 − H(q),(18) and KL(q∥p ′ ) = Z q(x) log q(x) p ′ (x) dx(19) = ce 2 − H(q),(20) whereH(q) = − R q(x) logq(x) dxdenotes the entropy of q. SinceH(q)is constant with respect to the model parameters, the comparison betweence 1 andce 2 is equivalent to the comparison betweenKL(q∥p ′ ) andKL(q∥p ′ ). In other words, a smaller cross- entropy value corresponds to a tighter approxima- tion of the ground-truth preference distribution in terms of KL divergence. (2) Assume there exists a common chunkC shared betweenP 1 = P(I ′ M | θ)andP 2 = P(I ′ M | θ). The gradient of the loss with respect to C is given by: ∂L ∂C =−r ∂(logP 1 − logP 2 ) ∂C =−r ∂(logP 1 (C)− logP 2 (C)) ∂C + 0 = 0(21) Hence, the gradient ∂L ∂C vanishes. (3) Ifce 1 < ce 2 , indicating that the sequence associated withP 1 = P(I ′ M | θ)yields lower cross-entropy thanP 2 = P(I ′ M | θ)and is thus more informative, we set the rewardr = 1. In this case, the optimization objective is: L = logP 2 − logP 1 (22) Minimizing Eq. (22) increases the likelihood ofP 1 while decreasing that of P 2 . Conversely, ifce 1 > ce 2 , implying thatP 2 cor- responds to the more informative sequence, we set r =−1. The resulting loss becomes: L = logP 1 − logP 2 (23) Minimizing Eq. (23) promotesP 2 and suppresses P 1 . In both scenarios, the training objective effec- tively increases the probability of the more informa- tive sequence, thereby guiding the policy network to select token chunks that preserve semantically salient information.