Paper deep dive
AgenticRec: End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender Agents
Tianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, Hui Li
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/26/2026, 2:31:36 AM
Summary
AgenticRec is a ranking-oriented recommender agent framework that optimizes the entire decision-making trajectory—including reasoning, tool invocation, and ranking—using a two-stage reinforcement learning approach: List-Wise Group Relative Policy Optimization (GRPO) for end-to-end alignment and Progressive Preference Refinement (PPR) for sharpening fine-grained preference boundaries.
Entities (5)
Relation Signals (3)
AgenticRec → implements → Progressive Preference Refinement
confidence 95% · we introduce Progressive Preference Refinement (PPR) to resolve fine-grained preference ambiguities
AgenticRec → integrates → ReAct
confidence 95% · design a suite of recommendation-specific tools integrated into a ReAct loop
AgenticRec → utilizes → List-Wise GRPO
confidence 95% · we propose theoretically unbiased List-Wise Group Relative Policy Optimization (list-wise GRPO) to maximize ranking utility
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Recommender agents built on Large Language Models offer a promising paradigm for recommendation. However, existing recommender agents typically suffer from a disconnect between intermediate reasoning and final ranking feedback, and are unable to capture fine-grained preferences. To address this, we present AgenticRec, a ranking-oriented agentic recommendation framework that optimizes the entire decision-making trajectory (including intermediate reasoning, tool invocation, and final ranking list generation) under sparse implicit feedback. Our approach makes three key contributions. First, we design a suite of recommendation-specific tools integrated into a ReAct loop to support evidence-grounded reasoning. Second, we propose theoretically unbiased List-Wise Group Relative Policy Optimization (list-wise GRPO) to maximize ranking utility, ensuring accurate credit assignment for complex tool-use trajectories. Third, we introduce Progressive Preference Refinement (PPR) to resolve fine-grained preference ambiguities. By mining hard negatives from ranking violations and applying bidirectional preference alignment, PPR minimizes the convex upper bound of pairwise ranking errors. Experiments on benchmarks confirm that AgenticRec significantly outperforms baselines, validating the necessity of unifying reasoning, tool use, and ranking optimization.
Tags
Links
- Source: https://arxiv.org/abs/2603.21613v1
- Canonical: https://arxiv.org/abs/2603.21613v1
Trouble viewing inline? Open PDF directly →
Full Text
73,163 characters extracted from source content.
Expand or collapse full text
AgenticRec: End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender Agents Tianyi Li ∗ Xiamen University Xiamen, China litianyi@stu.xmu.edu.cn Zixuan Wang ∗ Xiamen University Xiamen, China williamzixuan@stu.xmu.edu.cn Guidong Lei Xiamen University Xiamen, China leiguidong@stu.xmu.edu.cn Xiaodong Li Xiamen University Xiamen, China xdli@xmu.edu.cn Hui Li † Xiamen University Xiamen, China hui@xmu.edu.cn Abstract Recommender agents built on Large Language Models offer a promising paradigm for recommendation. However, existing rec- ommender agents typically suffer from a disconnect between inter- mediate reasoning and final ranking feedback, and are unable to capture fine-grained preferences. To address this, we present Agen- ticRec, a ranking-oriented agentic recommendation framework that optimizes the entire decision-making trajectory (including in- termediate reasoning, tool invocation, and final ranking list genera- tion) under sparse implicit feedback. Our approach makes three key contributions. First, we design a suite of recommendation-specific tools integrated into a ReAct loop to support evidence-grounded reasoning. Second, we propose theoretically unbiased List-Wise Group Relative Policy Optimization (list-wise GRPO) to maximize ranking utility, ensuring accurate credit assignment for complex tool-use trajectories. Third, we introduce Progressive Preference Refinement (PPR) to resolve fine-grained preference ambiguities. By mining hard negatives from ranking violations and applying bidirectional preference alignment, PPR minimizes the convex up- per bound of pairwise ranking errors. Experiments on benchmarks confirm that AgenticRec significantly outperforms baselines, vali- dating the necessity of unifying reasoning, tool use, and ranking optimization. ACM Reference Format: Tianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, and Hui Li. 2026. AgenticRec: End-to-End Tool-Integrated Policy Optimization for Ranking- Oriented Recommender Agents. In . ACM, New York, NY, USA, 14 pages. https://doi.org/10.1145/n.n ∗ Equal contribution. † Corresponding author. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. Conference’17, Washington, DC, USA © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-x-x-x/Y/M https://doi.org/10.1145/n.n AgenticRec Training-free Agent ... Tool Call Recommendations Think / Observation No learning from RS feedback User Request + Context Interaction Log ... End-to-End Training Feedback static / heuristic policy learned policy User Request + Context Figure 1: Compared to existing training-free recommender agents like RecMind [30] and InteRecAgent [16] (left), Agen- ticRec (right) learns outcome-driven tool use and evidence- grounded ranking policies from interaction logs under implicit-feedback list-wise rewards. 1 Introduction Recommender Systems (RS) are indispensable for mitigating infor- mation overload in the digital era [33,41]. Recently, Large Lan- guage Models (LLMs) have significantly affected recommenda- tion [10,11,13] due to their strong natural language understanding and reasoning capabilities [6, 23, 24], enabling semantic matching between complex user intents and diverse item descriptions. How- ever, recommendation is ultimately a ranking problem grounded in large-scale implicit feedback and collaborative signals, which cannot be directly captured by language priors alone [1, 18]. Early attempts adapt LLMs to recommendation via fine-tuning, typically casting them as next-token prediction, such as text-to-text generation [4,11] or implicit-feedback supervision [1,18]. Although effective, these methods remain largely one-shot predictors that output an item or a ranked list from the given context. In practice, real-world systems often involve incomplete and heterogeneous in- formation: ambiguous user intent [9], missing item information [41], and collaborative signals not fully accessible from text alone [18]. To offer reliable recommendation, the system needs to perform multi- step reasoning [8,30] and actively acquire additional evidence and integrate such evidence into the final decision [16, 43]. Therefore, recent work has explored recommender agents [25, 42] that equip LLMs with structured decision loops (e.g., ReAct [35]) arXiv:2603.21613v1 [cs.IR] 23 Mar 2026 Conference’17, July 2017, Washington, DC, USATianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, and Hui Li and tools to perform “Think before Recommendation” for evidence- aware recommendation [16,30,43]. However, despite their promis- ing progress, existing recommender agents still face two challenges: •Disconnection between Reasoning and Ranking Feedback: First, their reasoning and acting behaviors are not optimized end-to-end under recommendation feedback: intermediate rea- soning steps and tool invocations are often driven by generic language priors or handcrafted prompt heuristics [16,30] (the left part of Fig. 1). Since tool use is intertwined with reasoning, not only when/what to invoke but also how the retrieved evi- dence is integrated into the final decision must be determined via reasoning, posing a challenge to the design. •Inability to Resolve Fine-Grained Preferences: Second, they typically lack a mechanism to progressively sharpen fine-grained preference boundaries on hard, highly confusable candidates. In practice, implicit feedback provides sparse supervision, and inaccurate recommendations may arise from subtle preference differences among top-ranked candidates, which are difficult to correct using coarse list-level signals alone. As a result, agents may learn globally reasonable behaviors, yet remain brittle in challenging cases. In this paper, we study recommender agents in a ranking setting, where the agent outputs a top-퐾ordered list from a candidate set under implicit feedback supervision. The candidate set is ready be- forehand (e.g., from a retrieval stage [14]). We propose AgenticRec, a ranking-oriented agentic recommendation training framework that optimizes the entire agent’s decision trajectory, including in- termediate reasoning, tool invocations, and generating the final list of top-퐾recommendation. To enable tool-integrated, evidence- grounded ranking decisions, we design a suite of recommendation- oriented tools (including accessing user profiles, item information, behavioral statistics, and collaborative information) tightly inte- grated into the agent’s reasoning chain. We adopt a two-stage reinforcement learning training paradigm. In the first stage, we introduce List-Wise Policy Optimization via Group Relative Policy Optimization [5] (i.e., list-wise GRPO) to align multi-step trajec- tories, including both reasoning and tool invocation, with final ranking list generation. In the second stage, we propose Progres- sive Preference Refinement (PPR) to further sharpen fine-grained preference boundaries among highly confusable candidates via turning the agent’s own challenging mistakes into supervision. Our main contributions are summarized as follows: •Tool-Integrated Ranking Reasoning: We propose Agenti- cRec, a ranking-oriented recommender agent framework that integrates a suite of designed recommendation-oriented tools into a ReAct-style reasoning chain to support evidence-grounded recommendation decision making. • List-Wise Policy Optimization: We introduce list-wise GRPO to optimize AgenticRec based on implicit feedback in an end- to-end manner, jointly training multi-step reasoning, tool-use decision-making, and the final ranked list generation under list- wise ranking rewards. •Progressive Preference Refinement: We design a self- bootstrapped refinement strategy that mines hard competitors from the agent’s ranking violations and performs bidirectional preference reasoning, progressively strengthening fine-grained preference discrimination among highly confusable candidates. •Theoretical Analysis: We show that list-wise GRPO gradient estimator is unbiased: it guarantees that the ranking quality is back-propagated to guide reasoning and tool-invocation steps, and it provides stability in learning from sparse feedback. We also prove that bidirectional preference reasoning tightens the error bound more effectively than positive-only supervision alone. Extensive experiments on benchmarks confirm that Agenti- cRec significantly outperforms baselines, validating the necessity of unifying reasoning, tool use, and ranking optimization. 2 Related Work LLMs for Recommendation. Early works utilized LLMs as general-purpose recommenders via prompting [10,13]. To address objective mismatches between LLMs and the recommendation task, subsequent research focused on fine-tuning, either by unifying tasks under a text-to-text paradigm [11] or aligning LLMs with implicit feedback [1] and collaborative signals [18]. Recently, inspired by test-time scaling [40], studies have explored reasoning-augmented recommendation to optimize multi-step reasoning [8,19,21]. De- spite these advances, current methods primarily improve one-shot predictions or representation quality, while lacking mechanisms to progressively refine decision-making based on list-wise supervision during inference, a gap AgenticRec addresses. Recommender Agents. The success of LLM-powered agents has sparked increasing interest in using recommender agents to better understand user-item interactions [25,42]. Existing ap- proaches can be categorized into three types [25]: simulation- oriented agents [29,36,38,39] that emulate user behaviors for analysis and data generation, interaction-oriented agents [7,22,44] that conduct human-like conversations to provide explanation, and recommender-oriented agents [2,15,16,30,32] that improve rec- ommendation via enhanced planning, reasoning, memory, and tool usage. AgenticRec also falls into recommender-oriented agents. While promising, these recommender agents suffer from two limi- tations: (1) a reliance on LLM priors and prompt-driven inference that often neglect recommender-specific signals (e.g., implicit feed- back, collaborative relations, and ranking constraints), and (2) static, prompt-driven tool usage that is not explicitly optimized for rank- ing. In contrast, AgenticRec learns when/what to call tools jointly with list-wise ranking optimization, enabling outcome-driven tool use aligned with real ranking feedback. 3 Problem Definition We study how to train a recommender agent on implicit feedback (e.g., purchases or clicks). For each recommendation request, the input consists of a user interaction history푥 푢 =[푣 1 ,푣 2 , . . .,푣 푡 ]and a candidate item setC=푐 1 ,푐 2 , . . .,푐 푛 . The task is ranking-oriented andCis ready beforehand (e.g., from a retrieval stage [14]). The implicit feedback log provides the next interacted item푦 ∈ Cas supervision. Given(푥 푢 ,C), the agent outputs a ranked top-퐾list 푟 퐾 =[푐 휎(1) ,푐 휎(2) , . . .,푐 휎(퐾) ], where휎denotes a permutation over candidates and퐾 ≤ 푛. Our goal is to learn an agent policy휋 휃 that AgenticRec : End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender AgentsConference’17, July 2017, Washington, DC, USA Stage 1: Tool-Integrated Policy Optimization Next Interacted Item Candidate Set 퐀 User History 퐀 퐀 Stage 2: Progressive Preference Refinement Ranking list 퐀 퐀 1 3 5 7 9 2 4 6 8 10 c10 c8 c15 c6 c11 c1 c3 c4 c17 c9 Ranking Violation Which is more preferred? Which is less preferred? ... Recommendation Tool Suite User Profile Tool Collaborative Tool Item Info Tool Sample Trajectories Hard Negative Mining Bidirectional Preference Reasoning c + c - Ranking Reward (NDCG@K) 퐀 rollouts per request Format Constraint Policy Optimization via List-wise GRPO Tool Reward 1 3 2 4 c8 c15 c3 c4 c + : c10 items ranked above c + c - : c3 Policy Optimization via Pair-wise GRPO Binary Reward ( 0 / 1 ) Format Constraint Random Sample Reasoning Trajectory Reasoning Trajectory <think> To solve this problem ... We now retrieve the user's profile to understand their preferences in more detail. </think> <tool_call> "name": "get_user_profile","arguments": </tool_call> <observation> The user strongly prefers ... (stop gradient) <think> ... </think> ... Now, we can build a list of 10 items: <answer> [8, 3, 15, 4, 10, 1, 6, 17, 11, 9] </answer> Think Act Obs Behaviour Tool = Target item 퐀/퐀 + = Hard negative 퐀 − = Stop-gradient observation c + c - Figure 2: Overview of AgenticRec. maximizes the expected list-wise utility under real-world feedback: max 휃 E (푥 푢 ,C,푦)∼D 푅(푟 퐾 ,푦) , where푅(·)measures the ranking quality induced by implicit feed- back (e.g., NDCG@K). In addition, the agent is allowed to query external tools for auxiliary evidence during decision-making. 4 Our Proposed AgenticRec 4.1 Overview AgenticRec unifies interactive reasoning with outcome-driven learning. Fig. 2 provides an overview of AgenticRec. Given a user history푥 푢 and a candidate setC, the agent follows a ReAct- style loop to optionally query tools before generating a valid top-퐾 ranked list푟 퐾 ⊆ C. AgenticRec applies a two-stage training: first aligning the agent’s tool-integrated decision trajectory with list- wise ranking rewards, and subsequently sharpening fine-grained preferences via self-bootstrapped hard negative mining. Overall, AgenticRec injects collaborative filtering and preference signals directly into the agent’s reasoning process while simultaneously learning outcome-driven tool invocation policies. Appendix A pro- vides the prompts used in AgenticRec. 4.2 Tool-Integrated Ranking Reasoning In the recommendation scenario, high-quality ranking often hinges on recommender-specific signals and external information that cannot be reliably recovered from language priors. We therefore equip the agent with tool-integrated ranking reasoning to retrieve such evidence on demand and ground its list-wise ranking decisions on the retrieved observations, rather than heuristically inferring them from text alone. Specifically, tool invocation is treated as part of the agent’s de- cision trajectory, so reinforcement learning can directly optimize when to call tools and which tools to call according to the rewards of the downstream task. As a result, tool usage becomes outcome- driven rather than static, improving alignment with practical rec- ommendation constraints and supervision from implicit feedback. Tool Interaction Budget. Tool invocation introduces non- negligible latency and may cause the agent to over-rely on external evidence. We therefore impose a strict decision budget on tool us- age: the maximum number of tool interactions is fixed to푇 max =10. If the trajectory exceeds this budget, we mark the request as invalid and assign the penalty as invalid-format outputs. This constraint encourages the agent to query tools only when necessary and to make efficient, evidence-grounded ranking decisions under realistic cost constraints. ReAct-based Tool Use. We follow the ReAct paradigm [35] to interleave reasoning and tool usage within a single inference loop. Starting from(푥 푢 ,C), the agent iteratively generates Think tokens and optionally executes an Act by selecting a tool, receiving its Observation (Obs), and finally outputs a ranked list: Think 1 → Act 1 → Obs 1 →·→ Rank. Unlike the standard ReAct, we replace the final answer generation with list-wise ranking overC. We treat tool observations as environment-provided context and mask them out from training: observation tokens are fed to the agent but excluded from the policy likelihood, i.e., gradients are applied only to agent-generated Think/Act/Rank tokens. Conference’17, July 2017, Washington, DC, USATianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, and Hui Li Tool Suite for Recommendation. We equip the agent with a set of typical recommendation tools that cover complementary evi- dence sources commonly used in practical systems, including user preference signals, item-side attributes, behavioral statistics, and collaborative information. Notably, our framework is tool-agnostic: additional tools can be seamlessly integrated as long as they ex- pose a query-response interface, and their utility can be learned end-to-end under the same list-wise objective. Details of tools and their invocation examples are provided in Appendix B. (1) User Profile Tool. It retrieves a user profile that summarizes user information and persistent preferences, such as favored cate- gories and interests inferred from historical interactions. We use LLM to pre-generate the profile text for each request. Beyond re- trieval, this tool can be upgraded to profile update after each recom- mendation request is completed: based on the observed feedback and the current profile consistency, the agent decides whether to refine the long-term profile to better reflect newly revealed prefer- ences. (2) Item Information Tool. It includes two tools empowering the agent to autonomously query detailed information about specific items, such as descriptions, prices, etc. This capability enables the agent to access precise item attributes, ensuring a comprehensive understanding of individual items during the reasoning process. (3) Behavioral Statistics Tool. It includes two tools extracting fea- tures helpful for recommendation based on user behavior data. This design turns manual feature engineering into an agentic process, maximizing the utility of vast behavioral data. (4) Collaborative Information Tool. It surfaces collaborative fil- tering signals via a lightweight sequential recommender (e.g., SAS- Rec [17]). It maintains user and item embeddings and supports fast retrieval in two directions: • User-to-user (U2U): Identify users with similar historical behavior. •Item-to-item (I2I): Discover similar items and recurring co- interaction patterns. Such collaborative evidence is essential for resolving preference ambiguity and capturing relational patterns of the form “users like you are interested in ...”, which are difficult to reliably infer from LLM priors alone. 4.3 List-Wise Group Relative Policy Optimization While tool-integrated ranking reasoning enables the agent to ac- tively acquire recommender-specific evidence, a key challenge re- mains: how to optimize these multi-step behaviors using sparse implicit feedback. To address this problem, we propose to optimize the agent with List-Wise Group Relative Policy Optimization (list- wise GRPO), where the reward is computed at the ranking list level and tool usage is optimized jointly as part of the agent trajectory. List-Wise Ranking Policy. Following the ReAct-based tool use described above, the agent produces the ranked list as the final action of a trajectory. AgenticRec generates the ranking as structured text tokens (e.g., an ordered list of candidate IDs), which enables direct policy optimization on the ranked list. Importantly, the policy likelihood is computed over the entire generation trace, so tool decisions are optimized implicitly through the same list-wise reward. Reward Design. Reward signals directly guide the agent to produce executable rankings that align with implicit feedback. We apply a hard format gate to ensure validity, and optimize ranking quality only when the output is well-formed. Specifically, if the ranked list is invalid, we ignore the ranking reward and assign a fixed negative reward. In addition, when the agent achieves Hit@1 and has invoked at least one tool, we provide a small bonus to encourage outcome-driven tool use. Formally, the overall reward is defined as: 푅(푟 퐾 ,푦)= 푅 rank (푟 퐾 ,푦)if valid(푟 퐾 ) and푦 ∈ 푟 퐾 , −0.5if valid(푟 퐾 ) and푦∉ 푟 퐾 , −1otherwise, (1) wherevalid(푟 퐾 )indicates that푟 퐾 is a well-formed top-퐾ranked list overC. (1) Ranking Reward. We adoptNDCG@퐾as the list-wise ranking reward: 푅 rank (푟 퐾 ,푦)= NDCG@퐾(푟 퐾 ;푦),(2) computed from implicit feedback in the current context. Specifically, we treat the logged next interacted item푦as the only positive target, and all other candidates as negatives. Under this single-positive setting,NDCG@퐾(푟 퐾 ;푦)=0 if푦∉ 푟 퐾 . In this case, the overall reward is set to−0.5 as defined in Eq. 1, which encourages the agent to rank the relevant item within the top positions. (2) Format Constraint. Invalid outputs (e.g., too many tool invo- cations, invalid items, or items outsideC) lead to execution failures in downstream pipelines. We therefore assign a fixed reward of−1 when the format is incorrect, regardless of ranking quality, enforc- ing strict validity of generated rankings. (3) Tool-use Reward. To encourage outcome-driven tool use, we add a small bonus when the agent achieves Hit@1 and has invoked at least one tool in the trajectory. Concretely, after computing 푅(푟 퐾 ,푦)in Eq. 1, we add+0.1 ifrank 푟 퐾 (푦)= 1 and푁 tool (휏)>0, and add 0 otherwise, where 푁 tool (휏) counts tool invocations in 휏 . GRPO for Trajectory-Level Optimization. Optimizing list-wise ranking policies via reinforcement learning is challenging due to sparse rewards and the enormous combinatorial action space of permutations, especially when the agent performs multi-step tool- use reasoning before outputting the final ranking. We employ GRPO to stabilize training and reduce variance. For each training instance(푥 푢 ,C,푦) ∼ D, we sample퐺inde- pendent trajectories휏 (푔) 퐺 푔=1 from the current policy, each ending with a ranked list푟 (푔) 퐾 parsed from휏 (푔) . We compute the list-wise rewards푅 (푔) 퐺 푔=1 as: 푅 (푔) = 푅 푟 (푔) 퐾 ,푦 . We then define the group-relative advantage as: ˆ 퐴 (푔) = 푅 푟 (푔) 퐾 ,푦 − 1 퐺 퐺 ∑︁ 푗=1 푅 푟 (푗) 퐾 ,푦 . AgenticRec : End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender AgentsConference’17, July 2017, Washington, DC, USA The GRPO objective is: L GRPO (휃)=−E (푥 푢 ,C,푦)∼D " 1 퐺 퐺 ∑︁ 푔=1 ˆ 퐴 (푔) · log푝 휃 (휏 (푔) | 푥 푢 ,C) # . where the expectation is taken over sampled trajectories. By using relative comparisons within each group of rollouts, GRPO signifi- cantly lowers variance compared with standard REINFORCE [31] while directly optimizing list-wise ranking quality. Importantly, the log-probability is computed over the entire generation trace, so tool invocation decisions are optimized implicitly through the same list-wise reward, enabling stable credit assignment for tool-use tra- jectories. Since early training often yields uniformly low rewards and thus noisy updates, inspired by the dynamic sampling strat- egy in DAPO [37], we drop training instances whose rollout group contains no informative trajectories, i.e., all sampled trajectories receive negative rewards. 4.4 Progressive Preference Refinement Although list-wise GRPO directly optimizes the overall ranking quality, list-level rewards can still be insufficient for learning fine- grained preference boundaries among highly similar candidates, due to coarse supervision and ambiguous credit assignment within the ranked list. To further strengthen the agent’s discriminative ability, we in- troduce an additional Progressive Preference Refinement (PPR) stage after list-wise GRPO training. We first train the agent to obtain a globally reasonable ranking policy, and then perform a self-bootstrapped preference refinement procedure by mining hard negatives from its predicted rankings and applying bidirec- tional preference reasoning alignment on the resulting hard pairs to sharpen fine-grained preference boundaries. This refinement stage requires no extra annotations beyond implicit feedback, yet provides targeted supervision that sharpens preference boundaries on challenging candidate sets. Hard Negative Mining from Ranking Violations. We exploit the agent’s own ranking outputs to automatically mine hard nega- tives. For each input(푥 푢 ,C), we identify the ground-truth positive item푐 + = 푦 ∈ Cbased on implicit feedback. After the agent gen- erates a ranked list푟 퐾 , we check whether푐 + is placed at the top position. If푐 + is not ranked first, we treat this case as a ranking violation, since at least one competing item is ranked above the positive, revealing an informative preference boundary. Concretely, letrank 푟 퐾 (푐)denote the position of item푐in the ranked list푟 퐾 (top-퐾). We construct a hard-negative candidate set by collecting items within the top-퐾that are ranked above the positive; if the positive item is not in푟 퐾 , we use the entire top-퐾 list as the hard-negative pool: 퐻(푥 푢 ,C)= ( 푐 − | 푐 − ∈ 푟 퐾 , rank 푟 퐾 (푐 − )< rank 푟 퐾 (푐 + ),if 푐 + ∈ 푟 퐾 , 푟 퐾 ,if 푐 + ∉ 푟 퐾 . Instead of deterministically picking the top-ranked item, we ran- domly sample one negative item from퐻to form a hard preference pair(푐 + ,푐 − ). This stochastic mining increases negative diversity and avoids overly-adversarial pairs that can yield ambiguous super- vision, providing a more stable refinement signal while remaining focused on competitive negatives. Importantly, the proposed min- ing procedure is purely interaction-driven and requires no external labeling beyond the implicit feedback used to identify 푐 + . Bidirectional Preference Reasoning. Existing recommendation methods often treat predicting the next interacted item as the sole positive target, while leaving the complementary negative prefer- ence signal under-exploited. However, ranking errors in practice often stem from subtle confusions among top candidates, where the agent not only needs to identify what the user is likely to click, but also to explicitly recognize what the user is less likely to choose under the same context. Training only on the positive direction provides one-sided supervision and may yield weak preference boundaries when hard competitors are highly similar. To overcome the limitations of one-sided supervision, we pro- pose bidirectional preference reasoning, a novel training mechanism designed to fully exploit the preference signals latent in hard nega- tive pairs. Given a mined hard preference pair(푐 + ,푐 − )under the same user context푥 푢 , we refine the agent via bidirectional prefer- ence reasoning from two symmetric perspectives: • Positive Direction: We first task the agent with the standard objective: identifying the item that the user is more likely to interact with next. This reinforces the agent’s ability to recognize matching attributes between the user context and the ground- truth item 푐 + . •Negative Direction: We introduce a counter-perspective task where the agent must explicitly identify the item the user is less likely to choose. This forces the agent to actively deliberate on the specific shortcomings or mismatching features of the hard negative item푐 − relative to the user’s history, rather than simply treating it as a background non-positive sample. Both directions share the same structured output format but differ in the queried perspective, thereby providing complementary learning signals for sharpening fine-grained preference boundaries. We optimize the two directions using a GRPO-style objective (pair-wise GRPO) analogous to the list-wise policy optimization stage. For each direction, we sample퐺rollouts, assign a binary reward푅 ∈ 0,1indicating whether the prediction matches the target, and perform GRPO updates with group-normalized advan- tages. This refinement stage reuses interaction-derived hard pairs to strengthen preference reasoning from two angles, thereby sharpen- ing the agent’s fine-grained preference boundaries and improving its robustness on challenging candidate sets. 4.5 Theoretical Analysis The list-wise GRPO gradient estimator theoretically remains unbi- ased despite the subtraction of the group-average baseline: Proposition 4.1 (Unbiasedness of List-wise GRPO Gradient). The list-wise GRPO gradient estimator, which utilizes the list-wise ranking metric푅(푟 퐾 ,푦)(i.e., NDCG@K) as the reward and the group average ranking score as the baseline, provides an unbiased estimate of the gradient for the expected list-wise utility objective퐽(휃)= E 휏∼휋 휃 [푅(푟 퐾 ,푦)]. Conference’17, July 2017, Washington, DC, USATianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, and Hui Li Table 1: Performance comparison among baselines and AgenticRec. The best performance score is denoted in bold. Model CDsInstrumentsOfficeGames H@1H@5N@5H@10 N@10H@1H@5N@5H@10 N@10H@1H@5N@5H@10 N@10H@1H@5N@5H@10 N@10 Conventional Sequential Recommenders Caser0.0488 0.2526 0.1487 0.5089 0.2307 0.1825 0.4644 0.3275 0.6384 0.3828 0.1771 0.4374 0.3139 0.5541 0.3511 0.1535 0.3953 0.2813 0.5220 0.3217 GRU4Rec0.0512 0.2348 0.1403 0.4851 0.2202 0.1960 0.4534 0.3291 0.6446 0.3907 0.1642 0.4324 0.3054 0.5705 0.3490 0.1708 0.3781 0.2817 0.5239 0.3288 SASRec0.0536 0.2348 0.1413 0.4660 0.2145 0.1813 0.4546 0.3238 0.6323 0.3807 0.1662 0.4270 0.3018 0.5863 0.3525 0.1535 0.3915 0.2762 0.5374 0.3234 ReaRec0.0357 0.1871 0.1082 0.4576 0.1942 0.1507 0.3786 0.2633 0.6188 0.3400 0.1716 0.4265 0.3049 0.5551 0.3457 0.1459 0.3723 0.2298 0.5873 0.2987 Training-free LLM-based Recommenders LLMRank0.0501 0.2265 0.1373 0.5995 0.2569 0.0417 0.2745 0.1533 0.6017 0.2584 0.0713 0.3102 0.1872 0.5576 0.2665 0.0403 0.2745 0.1519 0.6660 0.2767 InteRecAgent 0.0782 0.2547 0.1660 0.4857 0.2392 0.2063 0.4735 0.3433 0.6515 0.4004 0.1911 0.4467 0.3211 0.6056 0.3717 0.1782 0.4107 0.3009 0.5599 0.3477 Trainable LLM-based Recommenders TALLRec0.1144 0.3539 0.2313 0.6007 0.3102 0.0649 0.2683 0.1618 0.5147 0.2406 0.1539 0.3592 0.2543 0.5522 0.3318 0.1362 0.3819 0.2608 0.6468 0.3474 LLaRA0.2324 0.5172 0.3736 0.7234 0.4394 0.2512 0.5404 0.4043 0.6833 0.4510 0.2416 0.5007 0.3928 0.6496 0.4398 0.2763 0.6199 0.4588 0.7332 0.4952 S-DPO0.1764 0.4183 0.2988 0.6293 0.3669 0.0723 0.2671 0.1657 0.5220 0.2475 0.1573 0.3528 0.2513 0.5421 0.3522 0.2006 0.4529 0.3292 0.6967 0.4079 ReRe0.2073 0.4648 0.3371 0.6901 0.4080 0.2584 0.5049 0.3900 0.6765 0.4447 0.2601 0.5052 0.3970 0.6962 0.4579 0.2921 0.5528 0.4473 0.7102 0.4976 AgenticRec 0.2992 0.6472 0.4795 0.8093 0.5324 0.2586 0.6115 0.4393 0.8052 0.5021 0.2494 0.5755 0.4123 0.7773 0.4775 0.3282 0.6468 0.4887 0.8157 0.5445 Table 2: Ablation study of different variants of AgenticRec. R: pure reasoning, TIRR: tool-integrated ranking reasoning, PPR: progressive preference refinement. Dataset Setting VariantH@1 H@5 N@5 CDs Frozen R0.19070.46960.3303 TIRR0.23310.54760.3955 Agentic R0.23240.53990.3865 TIRR0.28370.61620.4606 TIRR + PPR 0.2992 0.6472 0.4795 Instr. Frozen R0.19730.56130.3818 TIRR0.19480.52770.3673 Agentic R0.23280.57970.4141 TIRR0.24630.60910.4321 TIRR + PPR 0.2586 0.6115 0.4393 Office Frozen R0.21470.50170.3640 TIRR0.19630.49130.3471 Agentic R0.22760.53740.4004 TIRR0.23260.55720.4040 TIRR + PPR 0.2494 0.5755 0.4123 Proposition 4.1 confirms that the computed gradient direction aligns mathematically with maximizing the expected list-wise util- ity (i.e., NDCG@K). This unbiasedness is critical for AgenticRec for two reasons: (1) First, it ensures accurate credit assignment for tool usage, guaranteeing that the “credit” from a high-quality ranking is correctly back-propagated to the specific intermediate reasoning and tool-invocation steps, thereby driving the agent to learn outcome-driven behaviors. (2) Second, it provides stability in sparse feedback environments; by anchoring the gradient update to a group baseline, the optimization focuses on relative list improve- ment rather than unstable absolute scores, which is essential for convergence when learning from noisy implicit ranking signals. Additionally, the bidirectional preference reasoning in PPR stage can theoretically sharpen the fine-grained preference boundaries among highly confusable candidates: Proposition 4.2 (Error Bound Minimization via Bidirec- tional Preference Reasoning). Optimizing the bidirectional pref- erence reasoning objective on mined hard negative pairs(푐 + ,푐 − ) minimizes the upper bound of the pairwise ranking error probability 푃 푟푎푛푘 푟 퐾 (푐 − )< 푟푎푛푘 푟 퐾 (푐 + ) . Proposition 4.2 shows that, by explicitly training on ranking violations (where initiallyΔ푠= 푠(푐 + )− 푠(푐 − )<0) and applying the bidirectional update, the agent maximizes the score margin Δ푠. The “Negative Direction” task provides an additional gradient flow explicitly penalizing the salient features of푐 − that caused the violation, thereby tightening the error bound more effectively than positive-only supervision alone. We provide the proofs of Proposition 4.1 and Proposition 4.2 in Appendix C and Appendix D, respectively. 5 Experiments 5.1 Experimental Setup 5.1.1 Datasets. We use four subsets of the Amazon 2023 bench- mark [20], including CDs, Instruments, Office, and Games. Details of the datasets are provided in Appendix E. 5.1.2Task Setting. We focus on the ranking task in recommenda- tion. Following prior work [3,18], for each evaluation instance, we construct a candidate set of 20 items, consisting of one ground-truth next item (positive) and 19 negative items randomly sampled from the global item pool. The sampled negatives do not overlap with the user’s historical interactions. We randomly shuffle the candidate items in each instance before feeding them to the model, so that the input order does not provide any positional bias. Given the user interaction sequence, the model is required to output a top-10 ranked list from the candidate set, and recommendation quality is evaluated based on the ranked results. 5.1.3Baselines. We use three groups of baselines: AgenticRec : End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender AgentsConference’17, July 2017, Washington, DC, USA Sequential recommendation methods include classic baselines Caser [28], GRU4Rec [12], and SASRec [17], which model user pref- erences from interaction sequences using convolutional, recurrent, and self-attention architectures, respectively. We further consider ReaRec [27], an inference-enhanced recommender that introduces additional reasoning steps at test time. Training-free LLM-based methods mainly rely on prompting or in-context reasoning without parameter updates. In particular, LLMRank [13] directly performs candidate ranking by prompting an off-the-shelf LLM with user history and item descriptions, serving as a strong zero-shot ranking baseline. InteRecAgent [16] represents autonomous recommender agents that leverage an LLM to interact with external tools (i.e., SASRec) for recommendation decision making, while remaining training-free for the LLM backbone. Trainable LLM-based methods adapt LLMs to recommendation via fine-tuning or reinforcement learning. TALLRec [1] improves recommendation by tuning LLMs with recommendation-oriented supervision under limited data. LLaRA [18] further integrates tex- tual and ID-style representations to align LLMs with recommen- dation signals. S-DPO [3] introduces a multi-negative, softmax- style DPO objective to better leverage implicit preference data be- yond positive-only supervision. ReRe [26] applies constrained beam search and preference rewards to improve ranking in generative recommendation. 5.1.4Evaluation Metrics. In our experiments, we adopt two standard top-K ranking metrics for evaluation: NDCG@K (N@K) and Hit Ratio@K (H@K), with퐾 ∈ 1, 5, 10, computed over the candidate sets. 5.1.5Implementation Details. For training-free LLM-based baselines, we use GPT-4 as the LLM for all experiments. For train- able LLM-based baselines, to ensure a fair comparison, we adopt Qwen3-4B-Instruct [34] as the backbone across all methods, includ- ing AgenticRec, with a learning rate of 1×10 −5 and a batch size of 128, while keeping other hyperparameters and configurations as their default settings. For AgenticRec, the training follows a two- stage paradigm with 3 and 1 epochs, respectively, and we set the group size to 8, the batch size to 64, and the learning rate to 1×10 −6 . Due to page limitations, we defer other details of implementations to Appendix F. 5.2 Overall Performance Tab. 1 reports the overall results, and we can observe that: •For conventional sequential recommenders, we observe stable but relatively limited performance, reflecting that purely sequence- modeling approaches are effective at capturing short-term dy- namics but struggle to leverage rich evidence at decision time. •For training-free LLM-based recommenders, LLMRank performs poorly despite using a strong LLM backbone, mainly because it relies on a one-shot prediction paradigm without reasoning and explicit optimization toward ranking feedback. InteRecAgent improves over direct prompting by incorporating tool use, yet its overall performance is constrained by the quality of the attached conventional sequential recommender (SASRec) that provides collaborative signals. 0200400600 Training Steps 3 4 5 6 7 8 9 10 Average Tool Call Count Average Tool Call Count Tool Call Rate (in H@10) 0.70 0.75 0.80 0.85 0.90 0.95 1.00 Tool Call Rate (in H@10) (a) Tool use 0200400600 Training Steps 0.600 0.625 0.650 0.675 0.700 0.725 0.750 0.775 0.800 H@10 (b) Recommendation quality Figure 3: Training statistics on the Office dataset: (a) tool usage statistics, where the blue line denotes the average num- ber of tool calls per trajectory and the orange line indicates the percentage of positively rewarded trajectories that have tool invocation; (b) recommendation performance measured by H@10 during training. 248 Group Size 0.54 0.56 0.58 0.60 0.62 H@5 Games H@5 N@5 248 Group Size 0.57 0.58 0.59 0.60 0.61 0.62 H@5 CDs H@5 N@5 0.40 0.42 0.44 0.46 0.48 N@5 0.42 0.43 0.44 0.45 0.46 N@5 Figure 4: Effect of group size in list-wise GRPO •Trainable LLM-based recommenders significantly outperform the above two groups, showing the importance of adapting LLMs to implicit feedback and ranking supervision. However, they still lack the tool-integrated reasoning capability required by LLMs, which leads to suboptimal results compared to AgenticRec. • We can see that AgenticRec consistently achieves the best per- formance across all datasets and evaluation metrics. Agenti- cRec optimizes the entire decision trajectory end-to-end under list-wise rewards, so that when/what to query and how to use retrieved evidence are both outcome-driven. Moreover, the pro- gressive refinement stage further strengthens discrimination among hard candidates. 5.3 Ablation Study We conduct an ablation study to examine the effects of reasoning (R), tool-integrated ranking reasoning (TIRR), and progressive pref- erence refinement (PPR) on AgenticRec. Tab. 2 presents the results, and we can see that: •Under the frozen (training-free) setting, TIRR does not consis- tently outperform pure reasoning (R). In some cases (e.g., Instru- ments and Office), naive tool integration even leads to perfor- mance degradation, indicating that tool usage driven solely by language priors is insufficient and may introduce noise. •When trained end-to-end under list-wise recommendation feed- back (agentic setting), TIRR consistently improves over R across all datasets, demonstrating that outcome-driven optimization is critical for learning effective tool-integrated ranking decisions. For example, on CDs, TIRR improves H@1 from 0.2324 to 0.2837, showing substantial gains at the top rank. Conference’17, July 2017, Washington, DC, USATianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, and Hui Li GamesInstruments 0.15 0.20 0.25 0.30 H@1 GamesInstruments 0.40 0.45 0.50 0.55 0.60 0.65 0.70 H@5 GamesInstruments 0.60 0.65 0.70 0.75 0.80 0.85 H@10 GamesInstruments 0.30 0.35 0.40 0.45 0.50 N@5 GamesInstruments 0.35 0.40 0.45 0.50 0.55 N@10 Qwen3-1.7BQwen3-4BQwen3-8B Figure 5: Performance of AgenticRec with varying backbone sizes. Candidate Products History Purchase LLM Reasoning Training-free Agent AgenticRec Training Torecommendthebestitem,Ineedtomatchthe user'shistorywithcandidates. Step1AnalyzeHistory:UserboughtaGameCube MemoryCardandController...So,onlyitemswith "Gamecube","Wii","GBA",or"controller"arerelevant. Step2FilterCandidates:Item19(PokémonLegends: Arceus)isaSwitchgame.TheuserusesGameCube. Notrelevant...Item14(LinkCable)connectsGBAto GameCube.Highlyrelevant... Theuserisclearlyinterestedinlegacyhardware maintenance...Ishouldexcludemodernconsole games. FinalAnswer:Item14(GBALinkCable),... TargetItem19NotinTop10 First,Ineedtoanalyzetheuserprofile&behavior... <tool>get_user_profile</tool> <tool>get_rating_behavior</tool> <obs>UserisaenthusiastoflegacyNintendo systems(GameCube/Wii)...</obs> Ineedcollaborativefilteringsignal... <tool>i2i("GameCubeCard")</tool> <obs>UserswhoboughtGameCubeitemsoften buySwitchgames... </obs> FinalRanking: -Mostlikely:ItemsdirectlytiedtoGameCube/Wii hardware(14,16)andaccessories(8,10). -Secondary:Itemswithcollaborativesignals(the toolsuggestsalinktoSwitchgames). -Exclude:3DS,PS4,Xbox,PCitems—notin user'shistoryorinterest. FinalAnswer:Item16(Screwdriver),... TargetItem19ranked10 th First,Ineedtoanalyzetheuserprofile... <tool>get_user_profile</tool> <obs>UserisadedicatedenthusiastoflegacyNintendosystems (GameCube/Wii)...</obs> Ineedcollaborativefilteringsignal... <tool>i2i("GameCubeCard")</tool> <obs>UserswhoboughtGameCubeoftenbuySwitchgames... </obs> Overall Rating: 4.8 / 5.0 ★ Popularity: 24,098 views Thecollaborativefilteringshowa cross-platforminterestinNintendo‘s ecosystem...Therefore,Ishouldsearch forNintendo-relatedcandidates. <tool>search(PokémonLegends)</tool>... <obs>Found1itemnamedPokémonLegends:Arceus...</obs> FinalRanking: - Mostlikely:Item19 (Pokémon),high-popularity,high-ratedSwitch gamewhichislikelytobeofinterestgiventhecollaborativefiltering. -Secondary:GameCubeandWiiitems. -Lessrelevant:PlayStationandXboxitems. FinalAnswer:Item19(PokémonLegends:Arceus),... TargetItem19ranked1 st Ground Truth: Item19[PokémonLegends:Arceus-NintendoSwitch] Question: Givenauser'sinteractionhistoryandalistof20 candidateitems,youmustselecttheTOP10items mostlikelytobeboughtandrankthemfrommost likelytoleastlikely. N@10 = 0.2891 N@10 = 1.0 N@10 = 0.0 Figure 6: An example showing the effectiveness of AgenticRec. Textured highlights denote critical reasoning steps that most significantly influence the final ranking and illustrate the differences between methods. •Finally, incorporating PPR further boosts performance on all datasets, especially on H@1 and H@5. This suggests that refining fine-grained preference boundaries among confusable candidates effectively complements list-wise supervision. Overall, these results confirm that both tool-integrated rank- ing reasoning and progressive preference refinement are essential components of AgenticRec. 5.4 Further Analysis 5.4.1 Analysis on Tool Invocation. Fig. 3(a) shows the statistics of tool invocation during the first stage training (list-wise GRPO) on Office. The tool invocation rate among positively rewarded trajectories (orange line) increases rapidly in early training and re- mains consistently high thereafter, indicating that the policy quickly learns to leverage tools for effective decision-making. Meanwhile, the average number of tool calls (blue line) gradually rises and then stabilizes, suggesting that the model develops a relatively stable tool use strategy rather than indiscriminately increasing tool calls. Fig. 3(b) shows the change of H@10 on training trajectories during the first stage training, reflecting steady improvements in the learned policy. The synchronized progression between stabilized tool use and increasing H@10 suggests that list-wise GRPO brings more effective tool planning, gradually leading to better recommendation. 5.4.2 Analysis of Group Size. As illustrated in Fig. 4, increasing the group size in list-wise GRPO generally leads to more stable and improved recommendation performance. As the group size increases, the model benefits from richer intra-group comparisons, enabling more effective list-wise credit assignment and more accu- rate preference modeling. However, the performance gains tend to become marginal when the group size grows further, suggesting a saturation effect where additional samples provide diminishing optimization benefits. Overall, a moderate group size strikes a good balance between optimization stability and training efficiency. 5.4.3 Scaling Ability. To examine the scalability of AgenticRec with respect to model capacity, we conduct experiments using Qwen3 models across varying parameter sizes (1.7B, 4B, and 8B). While our primary implementation utilizes Qwen3-4B-Instruct, we adopt the Qwen3 series for this analysis to investigate scaling behavior, simultaneously verifying the method’s generalization on foundation models. As shown in Fig. 5, performance improves consistently as model size increases across datasets and evaluation metrics. Larger models achieve stronger results in both hit-based and ranking-based mea- sures, indicating that AgenticRec can effectively utilize increased model capacity to enhance list-wise reasoning and preference mod- eling. The stable upward trend across datasets further demonstrates that AgenticRec shows robust optimization behavior and scales reliably with backbone size. 5.4.4 Case Study. To intuitively demonstrate the effectiveness of AgenticRec, we present a sample from the Games dataset in Fig. 6. AgenticRec : End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender AgentsConference’17, July 2017, Washington, DC, USA The user history consists of legacy Nintendo purchases (Game- Cube accessories), while the actual next item purchased is Pokémon Legends: Arceus, a game for the Nintendo Switch. The vanilla rea- soning LLM exhibits an over-reliance on literal string matching (e.g., strictly filtering for keywords like GameCube or Wii), failing to mine user latent interests or grasp the intrinsic semantics of items. By invoking tools, the training-free agent utilizes collaborative signals to effectively improve ranking quality; however, without tool-integrated training and preference refinement, it fails to fur- ther explore nuanced differences among items and remains trapped by literal semantics in the final decision, resulting in suboptimal performance. In contrast, AgenticRec leverages collaborative signals to as- tutely infer the user’s latent interest in the Nintendo ecosystem. Building on this insight, it proactively invokes tools to acquire de- tailed specifications of promising candidates, enabling fine-grained differentiation for ranking. Ultimately, it flexibly synthesizes multi- source information to deliver precise recommendations. 6 Conclusion In this paper, we proposed AgenticRec, a ranking-oriented agentic recommendation framework. Through list-wise GRPO and PPR, AgenticRec enables ranking-driven tool integrated reasoning and sharpens fine-grained preference discrimination without additional supervision. Extensive experiments on benchmarks confirm that AgenticRec significantly outperforms baselines, validating the necessity of unifying reasoning, tool use, and ranking optimization. Overall, this work demonstrates that recommender agents can move beyond static, one-shot prediction toward decision-making entities that actively reason, acquire evidence, and optimize outcomes under recommendation feedback. In the future, we plan to explore the memory mechanism [25] and enable AgenticRec to handle long user history interaction sequences. References [1] Keqin Bao, Jizhi Zhang, Yang Zhang, Wenjie Wang, Fuli Feng, and Xiangnan He. 2023. TALLRec: An Effective and Efficient Tuning Framework to Align Large Language Model with Recommendation. In RecSys. 1007–1014. [2]Shihao Cai, Chongming Gao, Haoyan Liu, Wentao Shi, Jianshan Sun, Ruim- ing Tang, and Fuli Feng. 2025.MGFRec: Towards Reinforced Reasoning Recommendation with Multiple Groundings and Feedback. arXiv Preprint https://arxiv.org/abs/2510.22888 (2025). [3]Yuxin Chen, Junfei Tan, An Zhang, Zhengyi Yang, Leheng Sheng, Enzhi Zhang, Xiang Wang, and Tat-Seng Chua. 2024. On Softmax Direct Preference Optimiza- tion for Recommendation. In NeurIPS. [4]Zeyu Cui, Jianxin Ma, Chang Zhou, Jingren Zhou, and Hongxia Yang. 2022. M6- Rec: Generative Pretrained Language Models are Open-Ended Recommender Systems. arXiv Preprint (2022). https://arxiv.org/abs/2205.08084 [5]DeepSeek-AI. 2024. DeepSeek-V3 Technical Report. arXiv Preprint (2024). https: //arxiv.org/abs/2412.19437 [6]DeepSeek-AI. 2025. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning. arXiv Preprint (2025). https://arxiv.org/abs/2501.12948 [7]Jiabao Fang, Shen Gao, Pengjie Ren, Xiuying Chen, Suzan Verberne, and Zhaochun Ren. 2024. A Multi-Agent Conversational Recommender System. arXiv Preprint (2024). https://arxiv.org/abs/2402.01135 [8] Yi Fang, Wenjie Wang, Yang Zhang, Fengbin Zhu, Qifan Wang, Fuli Feng, and Xiangnan He. 2025. Reason4Rec: Large Language Models for Recommendation with Deliberative User Preference Alignment. arXiv Preprint (2025). https: //arxiv.org/abs/2502.02061 [9]Chongming Gao, Wenqiang Lei, Xiangnan He, Maarten de Rijke, and Tat-Seng Chua. 2021. Advances and Challenges in Conversational Recommender Systems: A Survey. arXiv Preprint (2021). https://arxiv.org/abs/2101.09459 [10]Yunfan Gao, Tao Sheng, Youlin Xiang, Yun Xiong, Haofen Wang, and Jiawei Zhang. 2023. Chat-REC: Towards Interactive and Explainable LLMs-Augmented Recommender System. arXiv Preprint (2023). https://arxiv.org/abs/2303.14524 [11]Shijie Geng, Shuchang Liu, Zuohui Fu, Yingqiang Ge, and Yongfeng Zhang. 2022. Recommendation as Language Processing (RLP): A Unified Pretrain, Personalized Prompt & Predict Paradigm (P5). In RecSys. 299–315. [12]Balázs Hidasi, Alexandros Karatzoglou, Linas Baltrunas, and Domonkos Tikk. 2016. Session-based Recommendations with Recurrent Neural Networks. In ICLR. [13]Yupeng Hou, Junjie Zhang, Zihan Lin, Hongyu Lu, Ruobing Xie, Julian J. McAuley, and Wayne Xin Zhao. 2024. Large Language Models are Zero-Shot Rankers for Recommender Systems. In ECIR, Vol. 14609. 364–381. [14]Junjie Huang, Jizheng Chen, Jianghao Lin, Jiarui Qin, Ziming Feng, Weinan Zhang, and Yong Yu. 2025. A Comprehensive Survey on Retrieval Methods in Recommender Systems. ACM Trans. Inf. Syst. 44, 28 (2025), 28:1–28:43. [15]Jiani Huang, Xingchen Zou, Lianghao Xia, and Qing Li. 2025. MR.Rec: Synergizing Memory and Reasoning for Personalized Recommendation Assistant with LLMs. arXiv Preprint https://arxiv.org/abs/2510.14629 (2025). [16]Xu Huang, Jianxun Lian, Yuxuan Lei, Jing Yao, Defu Lian, and Xing Xie. 2025. Recommender AI Agent: Integrating Large Language Models for Interactive Recommendations. TOIS 43, 4 (2025), 96:1–96:33. [17]Wang-Cheng Kang and Julian J. McAuley. 2018. Self-Attentive Sequential Rec- ommendation. In ICDM. 197–206. [18] Jiayi Liao, Sihang Li, Zhengyi Yang, Jiancan Wu, Yancheng Yuan, Xiang Wang, and Xiangnan He. 2024. LLaRA: Large Language-Recommendation Assistant. In SIGIR. 1785–1795. [19]Jiacheng Lin, Tian Wang, and Kun Qian. 2025. Rec-R1: Bridging Generative Large Language Models and User-Centric Recommendation Systems via Reinforcement Learning. TMLR 2025 (2025). [20] Qidong Liu, Xiangyu Zhao, Yejing Wang, Zijian Zhang, Howard Zhong, Chong Chen, Xiang Li, Wei Huang, and Feng Tian. 2025. Bridge the Domains: Large Language Models Enhanced Cross-domain Sequential Recommendation. In SIGIR. 1582–1592. [21] Zhanyu Liu, Shiyao Wang, Xingmei Wang, Rongzhou Zhang, Jiaxin Deng, Honghui Bao, Jinghao Zhang, Wuchao Li, Pengfei Zheng, Xiangyu Wu, Yifei Hu, Qigen Hu, Xinchen Luo, Lejian Ren, Zixing Zhang, Qianqian Wang, Kuo Cai, Yun- fan Wu, Hongtao Cheng, Zexuan Cheng, Lu Ren, Huanjie Wang, Yi Su, Ruiming Tang, Kun Gai, and Guorui Zhou. 2025. OneRec-Think: In-Text Reasoning for Gen- erative Recommendation. arXiv Preprint (2025). https://arxiv.org/abs/2510.11639 [22]Guangtao Nie, Rong Zhi, Xiaofan Yan, Yufan Du, Xiangyang Zhang, Jianwei Chen, Mi Zhou, Hongshen Chen, Tianhao Li, Ziguang Cheng, Sulong Xu, and Jinghe Hu. 2024. A Hybrid Multi-Agent Conversational Recommender System with LLM and Search Engine in E-commerce. In RecSys. 745–747. [23]OpenAI. 2023. GPT-4 Technical Report. arXiv Preprint (2023). https://arxiv.org/ abs/2303.08774 [24] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. 2022. Train- ing language models to follow instructions with human feedback. In NeurIPS. [25] Qiyao Peng, Hongtao Liu, Hua Huang, Qing Yang, and Minglai Shao. 2025. A Survey on LLM-powered Agents for Recommender Systems. In EMNLP. [26] Junfei Tan, Yuxin Chen, An Zhang, Junguang Jiang, Bin Liu, Ziru Xu, Zhu Han, Jian Xu, Bo Zheng, and Xiang Wang. 2025. Reinforced Preference Optimization for Recommendation. arXiv Preprint https://arxiv.org/abs/2510.12211 (2025). [27] Jiakai Tang, Sunhao Dai, Teng Shi, Jun Xu, Xu Chen, Wen Chen, Wu Jian, and Yuning Jiang. 2025. Think Before Recommend: Unleashing the Latent Reasoning Power for Sequential Recommendation. arXiv Preprint (2025). https://arxiv.org/ abs/2503.22675 [28]Jiaxi Tang and Ke Wang. 2018. Personalized Top-N Sequential Recommendation via Convolutional Sequence Embedding. In WSDM. 565–573. [29]Lei Wang, Jingsen Zhang, Hao Yang, Zhiyuan Chen, Jiakai Tang, Zeyu Zhang, Xu Chen, Yankai Lin, Hao Sun, Ruihua Song, Xin Zhao, Jun Xu, Zhicheng Dou, Jun Wang, and Ji-Rong Wen. 2025. User Behavior Simulation with Large Language Model-based Agents. TOIS 43, 2 (2025), 55:1–55:37. [30]Yancheng Wang, Ziyan Jiang, Zheng Chen, Fan Yang, Yingxue Zhou, Eunah Cho, Xing Fan, Yanbin Lu, Xiaojiang Huang, and Yingzhen Yang. 2024. RecMind: Large Language Model Powered Agent For Recommendation. In NAACL. 4351–4364. [31]Ronald J. Williams. 1992. Simple Statistical Gradient-Following Algorithms for Connectionist Reinforcement Learning. Mach. Learn. 8 (1992), 229–256. [32] Chenghao Wu, Ruiyang Ren, Junjie Zhang, Ruirui Wang, Zhongrui Ma, Qi Ye, and Wayne Xin Zhao. 2025. STARec: An Efficient Agent Framework for Recommender Systems via Autonomous Deliberate Reasoning. In CIKM. 3355–3365. [33]Likang Wu, Zhi Zheng, Zhaopeng Qiu, Hao Wang, Hongchao Gu, Tingjia Shen, Chuan Qin, Chen Zhu, Hengshu Zhu, Qi Liu, Hui Xiong, and Enhong Chen. 2024. A survey on large language models for recommendation. World Wide Web (W) 27, 5 (2024), 60. [34]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, Chujie Zheng, Dayiheng Liu, Fan Zhou, Fei Huang, Feng Hu, Hao Ge, Haoran Wei, Huan Lin, Jialong Conference’17, July 2017, Washington, DC, USATianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, and Hui Li Tang, Jian Yang, Jianhong Tu, Jianwei Zhang, Jian Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keqin Bao, Kexin Yang, Le Yu, Lianghao Deng, Mei Li, Mingfeng Xue, Mingze Li, Pei Zhang, Peng Wang, Qin Zhu, Rui Men, Ruize Gao, Shixuan Liu, Shuang Luo, Tianhao Li, Tianyi Tang, Wenbiao Yin, Xingzhang Ren, Xinyu Wang, Xinyu Zhang, Xuancheng Ren, Yang Fan, Yang Su, Yichang Zhang, Yinger Zhang, Yu Wan, Yuqiong Liu, Zekun Wang, Zeyu Cui, Zhenru Zhang, Zhipeng Zhou, and Zihan Qiu. 2025. Qwen3 Technical Report. arXiv Preprint (2025). https://arxiv.org/abs/2505.09388 [35] Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. 2023. ReAct: Synergizing Reasoning and Acting in Language Models. In ICLR. [36]Se-eun Yoon, Zhankui He, Jessica Maria Echterhoff, and Julian J. McAuley. 2024. Evaluating Large Language Models as Generative User Simulators for Conversa- tional Recommendation. In NAACL. 1490–1504. [37]Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Tiantian Fan, Gaohong Liu, Lingjun Liu, Xin Liu, Haibin Lin, Zhiqi Lin, Bole Ma, Guangming Sheng, Yuxuan Tong, Chi Zhang, Mofan Zhang, Wang Zhang, Hang Zhu, Jinhua Zhu, Jiaze Chen, Jiangjie Chen, Chengyi Wang, Hongli Yu, Weinan Dai, Yuxuan Song, Xiangpeng Wei, Hao Zhou, Jingjing Liu, Wei-Ying Ma, Ya-Qin Zhang, Lin Yan, Mu Qiao, Yonghui Wu, and Mingxuan Wang. 2025. DAPO: An Open-Source LLM Reinforcement Learning System at Scale. In NeurIPS. [38] An Zhang, Yuxin Chen, Leheng Sheng, Xiang Wang, and Tat-Seng Chua. 2024. On Generative Agents in Recommendation. In SIGIR. 1807–1817. [39]Junjie Zhang, Yupeng Hou, Ruobing Xie, Wenqi Sun, Julian J. McAuley, Wayne Xin Zhao, Leyu Lin, and Ji-Rong Wen. 2024. AgentCF: Collaborative Learning with Autonomous Language Agents for Recommender Systems. In W. 3679–3689. [40]Qiyuan Zhang, Fuyuan Lyu, Zexu Sun, Lei Wang, Weixu Zhang, Zhihan Guo, Yufei Wang, Irwin King, Xue Liu, and Chen Ma. 2025. What, How, Where, and How Well? A Survey on Test-Time Scaling in Large Language Models. arXiv Preprint (2025). https://arxiv.org/abs/2503.24235 [41]Shuai Zhang, Lina Yao, Aixin Sun, and Yi Tay. 2019. Deep Learning Based Recommender System: A Survey and New Perspectives. ACM Comput. Surv. 52, 1 (2019), 5:1–5:38. [42]Yu Zhang, Shutong Qiao, Jiaqi Zhang, Tzu-Heng Lin, Chen Gao, and Yong Li. 2025. A Survey of Large Language Model Empowered Agents for Recommendation and Search: Towards Next-Generation Information Retrieval. arXiv Preprint (2025). https://arxiv.org/abs/2503.05659 [43]Yuyue Zhao, Jiancan Wu, Xiang Wang, Wei Tang, Dingxian Wang, and Maarten de Rijke. 2024. Let Me Do It For You: Towards LLM Empowered Recommendation via Tool Learning. In SIGIR. 1796–1806. [44] Yaochen Zhu, Harald Steck, Dawen Liang, Yinhan He, Nathan Kallus, and Jundong Li. 2025. Llm-based conversational recommendation agents with collaborative verbalized experience. EMNLP (2025), 2207–2220. A Prompts Used in AgenticRec Following are the prompts used in AgenticRec: System Prompt for AgenticRec You are an helpful recommendation system assistant equipped with external tools. You can optionally call the tools to assist with the task. Tools You may call one or more functions to assist with the user query. You are provided with function signatures within <tools></tools> XML tags: <tools> tools_schema </tools> For each function call, return a json object with function name and arguments within <tool_call></tool_call> XML tags: <tool_call> "name": <function-name>, "arguments": <args-json-object> </tool_call> Prompt for Recommendation Solve the following problem step by step. You now have the ability to selectively invoke tools to gather information to assist your reasoning. The last part of your response should be in the following format: <answer> ’The final answer goes here.’ </answer> User question: Given a user’s interaction history and a list of 20 candidate items, you must select the TOP 10 items most likely to be bought and rank them from most likely to least likely. Answer Format: 1. Output ONLY the [...]result with no additional commentary in your final answer. - Example: [3, 17, 8, 1, 12, 5, 19, 2, 14, 6] 2. Your final output MUST include exactly 10 item indices (no more, no less). 3. Each index must appear exactly once (no duplicates). 4. All indices must be valid (between 1 and 20). Here is the history of items the user has bought: history_items_str Here is the list of candidate items: candidate_items_str Remember to place the final answer in the last part using the format: <answer> ’The final answer goes here.’ </answer> AgenticRec : End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender AgentsConference’17, July 2017, Washington, DC, USA Prompt for Progressive Preference Refinement Solve the following problem step by step. You need to selectively invoke tools to gather information to assist your reasoning. The last part of your response should be in the following format: <answer> ’The final answer goes here.’ </answer> User question: Given a user’s interaction history and two candidate items, you must determine which item the user is [MORE / LESS] LIKELY to buy next. Answer Format: 1. Output ONLY the Aor Bresult in your final answer. 2. Your answer must be exactly one of: A or B. Here is the history of items the user has bought: history_items_str Here are the two options: Item A: item_a_info Item B: item_b_info Remember to place the final answer in the last part using the format: <answer> ’The final answer goes here.’ </answer> B Tool Invocation Example The following are examples of tool invocation in AgenticRec: •User Profile Tool: Fig. 7 illustrates the user profile tool. We utilize the Qwen3-4B-Instruct (consistent with our backbone) to pre-generate user profiles. The input consists of the user’s interaction history prior to the timestamp of each sample in our processed dataset. User Profile Tool ▶ Tool Call <tool_call> " name" : " get_user_profile" , " arguments" : </tool_call> ◀ Tool Output The user is a dedicated enthusiast of Nintendo legacy systems, particularly favoring Switch controllers and related accessories. They consistently prefer high-functionality products with fea- tures like vibration, motion control, and macro capabilities. Figure 7: An example of user profile tool invocation. •Item Information Tool:AsshowninFig.8, item_info_searchtoolretrievesdetailedinformation about a specific item based on the dataset’s item metadata. Additionally, we construct a candidate analysis function to support another toolcandidates_analyzethat aggregates candidate items by category or price, providing a structured summary that enables the agent to quickly grasp the composi- tion of the candidate list. This dual-granularity design facilitates high-quality list-wise ranking by combining specific item details with a candidate overview. Item Information Tool ▶ Tool Call <tool_call> " name" : " item_info_search" , " arguments" : " item_name" : " Plants vs. Zombies Garden Warfare 2 - Xbox One" </tool_call> ◀ Tool Output Found 1 item named "Plants vs. Zombies Garden Warfare 2 - Xbox One" (Categories: Xbox One, Games / Store: Electronic Arts / Price: $16.99) Overall Rating: 4.6 / 5.0 Popularity: 1625 reviews ▶ Tool Call <tool_call> " name" : " candidates_analyze" , " arguments" : </tool_call> ◀ Tool Output Candidate Analysis by Category: - Legacy Systems > Nintendo Systems > Game Boy Systems > Game Boy: – Item 13: [EverDrive GBA Mini] - Legacy Systems > Nintendo Systems > Nintendo DS > Games: – Item 4: [Call of Duty: Black Ops] – Item 17: [Cake Mania NDS] - Legacy Systems > PlayStation Systems > PlayStation 2 > Ac- cessories: – Item 6: [PS2 Lava Glow 2.4 GHZ RF Wirel...] – Item 12: [Wireless 2.4G Controller Gamep...] - Nintendo Switch > Games: ... Figure 8: An example of item information tool invocation. Conference’17, July 2017, Washington, DC, USATianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, and Hui Li •Behavioral Statistics Tool: Fig. 9 illustrates the return in- formation of this tool. Upon invocation, it computes statis- tics on relevant metadata fields to generate statistical sum- maries of user recent behavior. Specifically, for Session Behavior (get_session_behavior), the user’s dynamic short-term inter- est patterns are extracted using item metadata from recent ses- sions; meanwhile, for Rating Behavior (get_rating_behavior), items rated by the user are categorized into high, neutral, and low groups to serve as explicit preference signals. Behavioral Statistics Tool ▶ Tool Call <tool_call> " name" : " get_session_behavior" , " arguments" : </tool_call> ◀ Tool Output Below are the extracted interests for the user’s most recent 2 sessions, listed chronologically from oldest to newest: 3.3h ago, the user was interested in Games, Legacy Systems. 1.8h ago, the user was interested in PlayStation 4, Games. This information reflects the user’s historical interests, interest evolution, or potential current focus. ▶ Tool Call <tool_call> " name" : " get_rating_behavior" , " arguments" : </tool_call> ◀ Tool Output User’s historical rating records are as follows: Five-star items (=5): PowerA MOGA XP5-X Plus Bluetooth Controller for Mo- bile & Cloud Gaming on Android/PC... Neutral items (>=3): sunwaytek Syozen Z2 Wireless Mechanical Controller for Nintendo Switch... Wireless Switch Pro Controller with 9 Color Adjustable LED/Unique Crack/Turbo/Vibration/Motion Control/Wake- Up... Dishonored: The Death of the Outsider... Low-rated items (<3): PowerA Enhanced - Seafoam Fade, Gamepad, Wired Video Game Controller, Gaming Controller, Xbox Series X|S, Xbox One (Only at Amazon)... Figure 9: An example of behavioral statistics tool invocation. •Collaborative Information Tool: Fig. 10 illustrates the in- formation returned by the tool. We employ SASRec[17] on the training set to model collaborative information and construct the embedding space, which is utilized for collaborative retrieval. Collaborative Information Tool ▶ Tool Call <tool_call> " name" : " get_similar_items" , " arguments" : " item_title" : " Tiger Woods PGA Tour 2005" </tool_call> ◀ Tool Output Based on collaborative filtering, users who enjoy the following items are more likely to purchase [Tiger Woods PGA Tour 2005]: [1] HORI Nintendo Switch Mario Kart 8 Deluxe Wheel (Cate- gories: Nintendo Switch, Accessories, Controllers) [2] GIRIAITUS Wii Motion Plus Adapter (Categories: Legacy Systems, Nintendo Systems, Wii) [3] Resident Evil 5: Gold Edition... ▶ Tool Call <tool_call> " name" : " get_similar_users" , " arguments" : </tool_call> ◀ Tool Output The following users have similar preferences to the current user. Their recent purchases include: User [1]: [Madden NFL 10 - Playstation 3], [Dirt - Xbox 360], [NBA Hangtime - PlayStation]... Figure 10: An example of collaborative information tool in- vocation. C Proof of Proposition 4.1 In AgenticRec, the optimization goal is to maximize the expected quality of the generated top-퐾list푟 퐾 given user context푥 푢 . The trajectory휏encompasses the entire reasoning chain, including tool invocations (Act) and observations (Obs), culminating in the final ranking action 푟 퐾 . Thus, the objective is: ∇퐽(휃)= E 휏∼휋 휃 [푅 푟푎푛푘 (푟 퐾 ,푦)·∇ log휋 휃 (휏)](3) where푅 푟푎푛푘 (푟 퐾 ,푦)is the list-wise metric (i.e., NDCG@K) derived from implicit feedback푦 (Eq. 2). To address the high variance inherent in list-wise ranking re- wards, we employ the GRPO estimator using a group of퐺sampled AgenticRec : End-to-End Tool-Integrated Policy Optimization for Ranking-Oriented Recommender AgentsConference’17, July 2017, Washington, DC, USA trajectories휏 (푔) 퐺 푔=1 . The estimated gradient is: ∇ ˆ 퐽 퐺푅푃푂 (휃)= 1 퐺 퐺 ∑︁ 푔=1 (푅(푟 (푔) 퐾 ,푦)−푏)∇ log휋 휃 (휏 (푔) )(4) where the baseline푏= 1 퐺 Í 퐺 푗=1 푅(푟 (푗) 퐾 ,푦) is the average list-wise utility of the sampled group. Proof.To proveE[∇ ˆ 퐽 퐺푅푃푂 ]=∇퐽, we examine the expectation of the baseline term. Since samples are i.i.d., the expected gradient contribution of the baseline is: E " 1 퐺 퐺 ∑︁ 푔=1 푏·∇ log휋 휃 (휏 (푔) ) # = E[푏]· E " 퐺 ∑︁ 푔=1 ∇ log휋 휃 (휏 (푔) ) # . Using the likelihood ratio identity, we have: E 휏 [∇ log휋 휃 (휏)]= ∫ 휋 휃 (휏)·∇ log휋 휃 (휏)푑휏= ∫ 휋 휃 (휏)· ∇휋 휃 (휏) 휋 휃 (휏) 푑휏 = ∫ ∇휋 휃 (휏)푑휏=∇ ∫ 휋 휃 (휏)푑휏 =∇(1)= 0, (5) where ∫ 휋 휃 (휏)푑휏= 1 since the integral of any probability distribu- tion over its entire domain must sum to 1. From Eq. 5, we can see that the baseline term vanishes in ex- pectation. Thus, the gradient direction remains aligned with the true objective of maximizing NDCG@K, while the group-relative subtraction(푅 (푔) −푏)reduces variance by focusing on the relative ranking quality within the group. □ D Proof of Proposition 4.2 Let푠(푐|푥 푢 )be the agent’s scoring potential for an item푐given context푥 푢 and we use푠(푐)for short if there is no ambiguity. A ranking violation occurs if푠(푐 − )> 푠(푐 + )for a hard negative푐 − . We define the score difference as: Δ푠= 푠(푐 + )−푠(푐 − ). A ranking error occurs whenΔ푠< 0. Proof.The fundamental goal of ranking optimization is to min- imize the misordering of pairs. The discrete pairwise ranking error is given by the indicator function: L 0−1 (Δ푠)= I(Δ푠< 0)= ( 1 ifΔ푠< 0 0 ifΔ푠 ≥ 0 Directly optimizingL 0−1 is intractable due to its discontinuity and zero gradients almost everywhere, which prevents effective gradient-based updates via the agent’s policy. For a pair(푐 + ,푐 − ), our bidirectional preference reasoning creates two complementary tasks: •Positive Direction: Maximize likelihood of choosing푐 + . Let this probability be: 푃 푝표푠 = 푒 푠(푐 + ) 푒 푠(푐 + ) +푒 푠(푐 − ) •Negative Direction: Maximize likelihood of rejecting푐 − (iden- tifying it as “less likely”). This effectively maximizes: 푃 푛푒푔 = 푒 −푠(푐 − ) 푒 −푠(푐 + ) +푒 −푠(푐 − ) = 푒 푠(푐 + ) 푒 푠(푐 + ) +푒 푠(푐 − ) Optimizing these tasks via GRPO is equivalent to minimizing the negative log-likelihood (cross-entropy) of the preference prob- ability푃(푐 + ≻ 푐 − ). For the pair(푐 + ,푐 − ), the combined lossL 퐵푖 is proportional to: L 퐵푖 ≈− log(푃 푝표푠 )− log(푃 푛푒푔 )=−2 log 1 1+푒 −(푠(푐 + )−푠(푐 − )) = 2 log(1+푒 −Δ푠 ). The above formulation represents the logistic loss. We observe that the derived logistic loss functionL(Δ푠)= 2log(1+푒 −Δ푠 )serves as a convex upper bound to the 0-1 ranking error lossL 0−1 : •Upper Bound: For anyΔ푠, 2log(1+푒 −Δ푠 ) ≥ log( 2)· I(Δ푠<0). Specifically, whenΔ푠<0 (an error), the logistic loss grows linearly with the magnitude of the violation (−Δ푠), imposing a heavy penalty. WhenΔ푠>0 (correct ranking), the loss ap- proaches zero smoothly. •Convexity: Unlike the step functionI(·), the logistic lossL(Δ푠) is smooth and convex. This ensures that the optimization land- scape is well-behaved, providing non-vanishing gradients even when the ranking is currently incorrect (i.e., whenΔ푠 ≪ 0). By mining hard negatives whereΔ푠<0 and applying the bidi- rectional update, AgenticRec effectively minimizes 2log(1+푒 −Δ푠 ). Since this function is a tight convex upper bound to the indicator error function, minimizing the bidirectional loss theoretically guar- antees the minimization of the empirical pairwise ranking error rate, thereby sharpening the fine-grained preference boundaries among highly confusable candidates. □ E Details of Data We conduct experiments on four subsets of the Amazon 2023 bench- mark 1 [20], including CDs and Vinyl (CDs), Musical Instruments (Instruments), Office Products (Office), and Video Games (Games). To keep the training time tractable, we restrict the interaction data to the period from October 2022 to October 2023. For sequence construction, we follow Liao et al. [18]to cap the maximum length of user interaction history to 10. Table 3: Statistics of the preprocessed datasets. Dataset#Users #Items #Inters. Sparsity CDs5,4378,78513,82699.71% Instruments7,5935,27915,74699.61% Office27,13011,51147,33399.85% Games6,2513,00311,45799.39% We adopt a common chronological split with a ratio of 8:1:1 for training/validation/testing by timestamp, where interactions 1 https://cseweb.ucsd.edu/~jmcauley/datasets.html#amazon_reviews Conference’17, July 2017, Washington, DC, USATianyi Li, Zixuan Wang, Guidong Lei, Xiaodong Li, and Hui Li are sorted by time and split accordingly. The statistics of these processed datasets are summarized in Tab. 3. F Details of Implementations For sequential recommendation baselines, we use a batch size of 1,024 and optimize all models with binary cross-entropy loss and a learning rate of 0.001. For ReaRec, the number of reasoning steps during training is set to 3. For training-free LLM-based baselines, we use GPT-4 as the LLM for all experiments. For trainable LLM- based baselines, to ensure a fair comparison, we adopt Qwen3-4B- Instruct [34] as the backbone across all methods, including Agen- ticRec, with a learning rate of 1×10 −5 and a batch size of 128, while keeping other hyperparameters and configurations as their default settings. Since TALLRec [1] was originally designed for point-wise ranking, we adapt it to our setting by reformulating the task as selecting the next item from a given candidate set. Similarly, ReRe [26], which was originally proposed for sequential recom- mendation, is adapted to our setting by constraining the model to recommend from a given candidate set. For AgenticRec, the train- ing follows a two-stage paradigm with 3 and 1 epochs, respectively, and we set the group size to 8, the maximum generation length to 6,144, the batch size to 64, and the learning rate to 1× 10 −6 . For baselines that explicitly output item relevance scores, we obtain the ranked list by scoring all candidate items and sorting them in descending order. For TALLRec, LLaRA, S-DPO and ReRe we rank the candidate items according to their joint generation probability. All experiments are conducted on 4 NVIDIA A800 GPUs (80GB memory each).