Paper deep dive
WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search
Yingtao Ren, Aolin Li, Cheng Cheng, Huiyun Hu, Huasheng Liang, Baoxing Xie, Jun Yuan, Haijun Wu, Chenxing Wang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Short-video search requires compact query rewriters that infer personalized intent from brief, ambiguous queries under strict latency constraints. The challenge is to decide when personalization is warranted and generate a reformulation that preserves the user's immediate information need. A rewriter optimized for fluency or engagement may still introduce unsupported details and redirect the search toward a different intent. Existing methods often underutilize users' actual reformulations and behavioral traces, relying instead on synthetic reasoning and indirect behavioral rewards that may produce more engaging candidates but cause intent drift. To build an effective rewriter that remains faithful to user intent, we propose WeFair, Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in WeChat short-video search. WeFair assigns real user behavioral signals to a progression from demonstration to preference to verification. We first mine actual human reformulations from search sessions, then apply multidimensional LLM screening and contextual verification to filter unsupported transitions without extensive manual annotation. During training, we begin with a supervised curriculum that progressively builds rewriting capabilities from large-scale human personalized query demonstrations, and then use direct preference optimization to align with users' subsequent consumption. In the final on-policy optimization stage, we construct a strict verification reward based on actual human reformulations to further boost performance with group relative policy optimization. Further analysis shows that auxiliary rewards can activate more rollout groups in GRPO but reduce intent fidelity. Online A/B testing in WeChat short-video search shows a 1.576% relative increase in effective clicked-query rate (VV>10s) and a 1.731% relative reduction in query reformulation rate.
Tags
Links
- Source: https://arxiv.org/abs/2610.16927v1
- Canonical: https://arxiv.org/abs/2610.16927v1
Trouble viewing inline? Open PDF directly →
Full Text
33,446 characters extracted from source content.
Expand or collapse full text
WeFair: Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in Short-Video Search Conference: ; ; Yingtao Ren email: yingtaoren@tencent.com Note: These authors contributed equally to this work. Affiliation: Weixin Group, Tencent, China , Aolin Li email: churenli@tencent.com Affiliation: Weixin Group, Tencent, China , Cheng Cheng email: trcheng@tencent.com Affiliation: Weixin Group, Tencent, China , Huiyun Hu email: huiyunhu@tencent.com Affiliation: Weixin Group, Tencent, China , Huasheng Liang email: watsonliang@tencent.com Affiliation: Weixin Group, Tencent, China , Baoxing Xie email: baoxingxie@tencent.com Affiliation: Weixin Group, Tencent, China , Jun Yuan email: junjyuan@tencent.com Affiliation: Weixin Group, Tencent, China , Haijun Wu email: ethanhjwu@tencent.com Affiliation: Weixin Group, Tencent, China and Chenxing Wang email: magnuswang@tencent.com Note: Corresponding author. Affiliation: Weixin Group, Tencent, China © none Abstract. Short-video search requires compact query rewriters that infer personalized intent from brief, ambiguous queries under strict latency constraints. The challenge is to decide when personalization is warranted and generate a reformulation that preserves the user’s immediate information need. A rewriter optimized for fluency or engagement may still introduce unsupported details and redirect the search toward a different intent. Existing methods often underutilize users’ actual reformulations and behavioral traces, relying instead on synthetic reasoning and indirect behavioral rewards that may produce more engaging candidates but cause intent drift. To build an effective rewriter that remains faithful to user intent, we propose WeFair, Fidelity-Aware Alignment from Human Feedback for Personalized Query Rewriting in WeChat short-video search. WeFair assigns real user behavioral signals to a progression from demonstration to preference to verification. We first mine actual human reformulations from search sessions, then apply multidimensional LLM screening and contextual verification to filter unsupported transitions without extensive manual annotation. During training, we begin with a supervised curriculum that progressively builds rewriting capabilities from large-scale human personalized query demonstrations, and then use direct preference optimization to align with users’ subsequent consumption preferences. In the final on-policy optimization stage, we construct a strict verification reward based on actual human reformulations to further boost performance with group relative policy optimization. Further analysis shows that auxiliary rewards can activate more rollout groups in on-policy optimization but reduce intent fidelity. Online A/B testing in WeChat short-video search shows a 1.576% relative increase in effective clicked-query rate (V>10>10s) and a 1.731% relative reduction in query reformulation rate. Keywords: Personalized query rewriting, short-video search 1. Introduction Figure 1. Overview of intent drift in WeChat short video search. The original query “crepe cake” is redirected toward recipes instead of the intended eating videos. WeFair uses human feedback through demonstration, preference, and verification to mitigate intent drift. Personalized query rewriting is essential when a brief search query does not fully express the user’s information need. In short-video search, the intended meaning may depend on recent searches, viewed content, or other context available when the query is issued. Rewriting can expose this information to retrieval by producing a more informative expression, but it must also satisfy strict latency constraints (Gong et al., 2026; Chu et al., 2026). Therefore, the practical objective is training a compact rewriter model that can resolve ambiguity without replacing the user’s current intent and needs. Furthermore, the rewriter model also needs to accurately decide when to rewrite or preserve the original query. Although general-purpose large language models (LLMs) exhibit strong reasoning capabilities, they fail to accurately understand diverse user intents and require several seconds per inference, which is unacceptable for real-time search. To address these limitations, some approaches use synthetic Chain-of-Thought traces to distill reasoning capabilities from large teacher LLMs into a smaller rewrite model, and then use large-scale poster consumption feedback to align them with platform-specific user preferences. Figure 2. Overview of WeFair: (a) Search sessions yield screened reformulation and preservation targets. (b) Curriculum SFT builds rewriting capabilities, DPO learns preferences from reformulations and consumption feedback, and GRPO applies strict target verification to constrain intent drift. (c) The compact policy directly rewrites or preserves the query using query-time context, without explicit chain-of-thought generation. However, consumption-oriented alignment does not necessarily preserve the user’s immediate information need. When insufficiently constrained by direct evidence of user intent, a rewriter may favor queries that retrieve engaging content while redirecting the original search, resulting in intent drift. To better capture personalized intent, recent methods incorporate actual user reformulations, which provide direct evidence of how users clarify their information needs. PicQue uses historical reformulations as context for generating training examples that subsequently undergo manual verification, while WeWrite directly uses mined reformulations for supervised fine-tuning and reward construction in GRPO. Despite this progress, how to exploit the distinct roles of real user signals across training stages remains underexplored. While these advances demonstrate the value of real user signals, reformulations and subsequent consumption feedback differ in how directly they reflect personalized intent, and their fine-grained use across training stages remains underexplored. To bridge the gap, we propose WeFair, a fidelity-aware alignment framework for training compact personalized query rewriters in WeChat short-video search. Its central idea is to match supervision fidelity to the requirements of each training stage, improving how human feedback guides a small model toward faithful and efficient rewriting. As illustrated in Figure 1, this progression can enable a compact model to acquire personalized query rewriting capabilities without intent drift. WeFair combines automated data construction with three complementary training stages. We mine actual reformulations from search sessions and apply LLM screening and contextual verification to retain demonstrations grounded in user intent. Training then proceeds through three stages, each with a distinct purpose. First, a supervised fine-tuning (SFT) curriculum progresses from session-to-query prediction to context-conditioned reasoning and finally direct query rewriting, building intent understanding before consolidating rewrite-or-preserve behavior. The broad coverage of human demonstrations prepares the compact policy model to generate useful candidates for later optimization. Second, direct preference optimization (DPO) learns from candidate pairs distinguished by subsequent consumption outcomes, increasing the probability of rewrites associated with better user experiences. This stage uses consumption feedback as comparative evidence to refine candidate preferences. Finally, group relative policy optimization (GRPO) applies a strict verification reward based on exact match with actual human reformulations or preservation. This final stage anchors policy updates to observed human expressions without adding graded similarity, retrieval, or other judge rewards. At inference time, the rewrite model directly outputs a short rewrite or a preservation decision, avoiding explicit CoT generation. Figure 2 shows the overview of WeFair. Our further analysis suggests that the three training stages differ in their tolerance for supervision with varying intent fidelity. SFT can tolerate lower-fidelity supervision to acquire broad rewriting capabilities, while DPO can incorporate preferences inferred from subsequent consumption to refine candidate selection. In contrast, final GRPO optimization requires strict fidelity to the intended target: auxiliary rewards can activate more rollout groups while introducing intent drift. Experiments demonstrate WeFair’s effectiveness in personalized rewriting. Overall, our main contributions can be summarized as follows: (1) We examine the role of supervision fidelity in personalized query rewriting, showing how consumption-based optimization can induce intent drift. (2) We propose WeFair, a fidelity-aware alignment framework that matches real user feedback to different training stages, combining broad capability acquisition, preference learning, and strict verification to mitigate intent drift. (3) We evaluate WeFair through online A/B testing to assess rewriting effectiveness. 2. Related Work 2.1. Personalized Query Rewriting LLM-based query rewriting improves retrieval by enriching query representations and adapting them to search domains. Query2doc generates pseudo-documents, while CardRewriter incorporates platform knowledge through knowledge cards (Wang et al., 2023; Gong et al., 2026). Domain adaptation further aligns query expressions with retrieval requirements. BEQUE combines supervised rewriting with retrieval feedback(Peng et al., 2024), and negation-aware rewriting preserves explicit constraints (Guo et al., 2025b). MAAQR and QP-OneModel extend this direction through adaptive refinement and unified query understanding (Zheng et al., 2025; Huang et al., 2026). Beyond representation enrichment, IterQR and Think-Then-Rewrite incorporate reasoning into retrieval-oriented training(Chen et al., 2025; Li et al., 2026a), while MiniELM combines lightweight distillation with simulated feedback (Nguyen et al., 2025). For personalized search, these capabilities additionally require evidence of what the current user intends to find, which utilizes user context and observed behavior to resolve ambiguity and determine when rewrite is essential. PicQue combines hybrid reasoning supervision with GRPO to address over-rewriting without online CoT generation (Chu et al., 2026), while Think-to-Personalize uses purchase history to support personalized retrieval (Jiang et al., 2026). WeWrite mines actual user reformulations and trains with SFT and GRPO using lexical similarity and conditional query-volume rewards (Cheng et al., 2026). Together, these studies establish complementary roles for contextual evidence and users’ expressed reformulations in understanding personalized intent. Behavioral feedback also raises a distinction between search utility and fidelity to an issued query. OneBar emphasizes content-grounded recommendations, while short-video and conversational query suggestion address consumption utility and intent coverage (Tang et al., 2026; Tian et al., 2026; Liu et al., 2026). DVCQR aligns separate rewrites with sparse and dense retrieval (Li et al., 2026b). These objectives can improve discovery, but their gains alone do not establish intent preservation. 2.2. LLM Post-Training and Feedback Alignment Post-training methods provide different mechanisms for learning from demonstrations, preferences, and rewards. SFT learns demonstrated responses (Ouyang et al., 2022), whereas DPO optimizes chosen–rejected preferences without separately training a reward model (Rafailov et al., 2023). PPO uses clipped policy updates (Schulman et al., 2017), and GRPO estimates advantages within sampled groups without a learned value network (Guo et al., 2025a). Beyond algorithm selection, recent work examines which supervision is useful at different learning stages. Stage-specific data selection distinguishes capability acquisition from reinforcement (He et al., 2026). SmartSearch uses a process-reward-guided curriculum (Wen et al., 2026a). Hindsight self-distillation further derives corrections and preferences from follow-up user interactions (Kleine Buening et al., 2026). This work motivates matching supervision to both the policy’s capabilities and the evidence available from users. In search applications, this allocation also depends on what feedback measures. AWHCP combines SFT, DPO, and GRPO with quality, format, and recall rewards (Ou et al., 2026), while CoRe uses deployed relevance feedback (Wen et al., 2026b). Such signals align generation with search outcomes, but provide different evidence about intended reformulations. Comparisons under compute constraints further show that on-policy rewriting need not outperform prompted alternatives (Xu et al., 2026), motivating examination of the optimization signals alongside the algorithms. Reward-based optimization consequently requires considering the reliability of candidate-level feedback. TTRL learns from majority-voted pseudo-labels and shows that reward accuracy can remain high despite inaccurate labels: an incorrect label can still assign correct negative rewards to other incorrect answers (Zuo et al., 2025). This separates source-label accuracy from the reliability of induced rewards. 3. Methodology 3.1. Task Formulation and Framework Overview Task Formulation. In short-video search system, given a query q and user context c available at query time, a policy πθ(y∣x) _θ(y x), where x=(c,q)x=(c,q), generates a reformulation q q or a preservation marker ⊥ . The reformulation serves as an additional retrieval query, and the preservation marker ensures that only the original query is retained. We define intent drift as changing or omitting information required by the user, or introducing personalized constraints unsupported by x. The objective is to generate context-supported reformulations while preserving the original query. Framework Overview WeFair organizes training according to intent fidelity: how reliably supervision reflects the user’s immediate information need. Human reformulations provide direct but incomplete evidence of intent, whereas subsequent consumption offers indirect preference signals. Accordingly, curriculum SFT builds rewriting capabilities from broad demonstrations, DPO learns relative preferences from consumption-grounded pairs, and GRPO anchors final refinement through strict agreement with screened reformulation and preservation targets. 3.2. Human Feedback Construction WeFair utilize context-to-query records =(c,q)C=\(c,q)\ and user reformulations for demonstrations, preference learning, and verification. We first extract context-to-query records by pairing user-entered queries with preceding context and requiring at least one retrieved video to be watched for more than ten seconds. For Reformulation candidates ℛR, we link eligible adjacent queries (q,q′)(q,q ) within the same session to the context preceding q. Both sources exclude queries issued through system suggestions. Session adjacency alone does not establish intent continuity. Therefore, we remove tagged suggestions and lookup-table fragment replacements from c to obtain screening context csc_s, then strip selected headers and history-entry prefixes to derive rule evidence e. These views serve screening and leave the policy input unchanged. Algorithm 1 summarizes the overall construction process. Algorithm 1 Human feedback construction 1: Candidates ℛR, natural pool N, parameters Θ , exclusions X 2: Reformulation pool P, preservation pool ℬB 3: ←∅P← ; ℋ←∅H← 4: for each valid (q,q′,c)∈ℛ(q,q ,c) do 5: (cs,e)←PrepareEvidence(c)(c_s,e)← PrepareEvidence(c) 6: if Rules 1–4 reject under Θ then continue 7: d←ScreenCandidate(q,q′,cs)d← ScreenCandidate(q,q ,c_s) 8: if d=0d=0 then 9: ℋ←ℋ∪(q,q′,c)H ∪\(q,q ,c)\ 10: else if d=1d=1 then 11: t←PreservationGate(q,q′,cs)t← PreservationGate(q,q ,c_s) 12: if t=0t=0 then ←∪((c,q),q′)P ∪\((c,q),q )\ 13: end if 14: end for 15: ℬ←SelectHard(ℋ)∪SelectNatural()B← SelectHard(H)∪ SelectNatural(N) 16: return CleanAndExcludeConflicts(,ℬ,) CleanAndExcludeConflicts(P,B,X) Hard-rule screening. After getting the initial candidates, we then conduct a rule-based screening. Let S(t)S(t) denote a string’s character set, a=s(q)a=s(q), b=s(q′)b=s(q ), and U=S(b)∖S(a)U=S(b) S(a), where s removes whitespace, non-word characters, and underscores. Reject a candidate if any condition holds: (1) No character gain: U=∅U= . (2) Function-word-only gain: the added characters belong to a fixed Chinese function-word lexicon and introduce a function word. (3) Insufficient overlap: J(q,q′)=|S(q)∩S(q′)|/|S(q)∪S(q′)|≤0.06J(q,q )=|S(q)∩ S(q )|/|S(q)∪ S(q )|≤ 0.06, with the threshold calibrated on a small annotated set. (4) Unsupported change: no contextual match passes the new trigram, bigram, single-character, or common-prefix fallback conditions. The contextual alternatives use occurrence thresholds in e and are evaluated in that order. Only candidates passing all vetoes reach LLM screening. These lexical tests provide coarse evidence, which requires subsequent semantic validation. Accordingly, only candidates that pass all these hard-rule vetoes proceed to LLM screening, where such semantic validation is carried out. LLM screening. This step operationalizes the semantic validation by checking three conditions: intent continuity I, information gain G, and contextual bridging B. These conditions are evaluated by Gemini 3.5 Flash: KLLM(q,q′,cs)=I(q,q′)∧G(q,q′)∧B(q,q′,cs).K_LLM(q,q ,c_s)=I(q,q ) G(q,q ) B(q,q ,c_s). Continuity preserves the subject, task, answer type, and essential constraints, and gain requires useful retrieval information or supported entity recovery. Bridging rejects unsupported evidence (Level 1) and isolated signals such as target replay or broad interests (Level 2). Level 3 requires at least two independent, consistent signals supporting a stable attribute or nearly unique entity recovery; Level 4 requires direct evidence linking the original subject or task to the added detail. Stable disambiguation admits Level 3; relational requests generally require Level 4. The model returns acceptance d=1d=1 or rejection d=0d=0 with a brief reason. Malformed judgments are discarded. Approximately one third of valid decisions accept the candidate. Lexical retention and transfer verification. For LLM-accepted candidates, PreservationGate retains pairs where the preprocessed original query is a substring, ordered subsequence, or character multiset contained in the target. These lexical shortcuts preserve characters, not necessarily intent. Other pairs reuse an existing semantic-transfer judgment for the same record or receive a new one; only a valid non-transfer label (t=0t=0) passes. Approximately seven tenths of LLM-accepted candidates enter P, retaining the observed target y∗=q′y^*=q . Preservation labels. For preserved query candidates, natural examples come from an existing non-reformulation pool N, and the hard examples are selected from valid LLM rejections ℋH. Both receive ((c,q),⊥)((c,q), ) in ℬB. Rejected candidates and screening explanations are excluded from policy input. These labels prescribe conservative preservation without establishing that every possible rewrite is invalid. Rule failures, malformed judgments, and semantic-transfer rejections are not automatically converted into preservation examples. Finally, all candidates undergo validity checks, evaluation exclusion, and label-conflict removal before sample records are assigned to mutually exclusive training stages. This partitioning is at the record level and does not imply request-level or user-level separation. 3.3. Fidelity-Aware Progressive Alignment The training of WeFair follows demonstration, preference, and verification, with each stage initializing the next. Demonstration: curriculum SFT. The curriculum consists of three phases. Phase 1 trains the model to predict the query q from preceding context c. Phase 2 trains it to learn context-conditioned rationales that explain either a reformulation or a preservation decision. Phase 3 trains it to predict the final target directly from x=(c,q)x=(c,q). Context-to-query records provide broad behavioral coverage, while the smaller adaptation subset teaches evidence-supported rewriting and the serving format. The rationales are auxiliary explanations generated by Gemini-3.1-Pro-Preview, not observed human reasoning. For phase k∈1,2,3k∈\1,2,3\ and input–target sequence pair (u,v)(u,v), the objective function is: (1) ℒSFT(k)=−(u,v)∼S(k)∑t=1|v|logπθ(vt∣u,v<t).L_SFT^(k)=-E_(u,v) _S^(k) _t=1^|v| _θ(v_t u,v_<t). Each phase continues from the preceding checkpoint; after the final direct-prediction phase, we obtain πSFT _SFT. Preference: DPO with zero-consumption negatives. For each preference input, the policy generates 64 candidates (x)Y(x), and we construct up to two DPO pairs from the same input. Let ri(x,y)r_i(x,y) be the rank of the i-th subsequent relevant consumed video in candidate y’s retrieval results. Using up to three such videos as user preference, a chosen candidate y+y^+ must have at least one relevant consumed video appear in its top-20 retrieval results. A reject candidate y−y^- satisfies no relevant consumed video in its top-100 retrieval results. The first pair uses the human reformulation q′q as the positive, yielding (x,q′,y1−)(x,q ,y^-_1). The second pair uses a consumption-supported chosen candidate, yielding (x,ycons+,y2−)(x,y^+_cons,y^-_2). If only one valid negative candidate is available, we set y1−=y2−y^-_1=y^-_2 and share it across both pairs. For preserved samples, we simply use the preservation marker ⊥ and any generated candidate y−y^-, yielding (x,⊥,y−)(x, ,y^-). DPO increases the relative likelihood of human reformulations or consumption-supported candidates over observed zero-consumption alternatives. Consumption thus guides comparative preference learning with relatively high fidelity. Because engagement can differ from the immediate information need, final alignment returns to explicit reformulation and preservation markers. Verification: human-grounded GRPO. Verification is the final mechanism for constraining optimization to direct target evidence with GRPO. Starting from the DPO policy, we sample 32 responses for each (x,y∗)∈V(x,y^*) _V. After deterministic parsing, a valid response y receives (2) R(y,y∗)=wr,y=y∗,y∗≠⊥,wp,y=y∗=⊥,0,otherwise,R(y,y^*)= casesw_r,&y=y^*, y^*≠ ,\\ w_p,&y=y^*= ,\\ 0,&otherwise, cases where wr,wp>0w_r,w_p>0 weight exact matching with the retained human query q′q and preservation, and any other responses receive zero reward, ensuring the highest fidelity signal supervision. Our further analysis shows that the high-fidelity reward function outperforms other designs with graded similarity, retrieval, or learned-judge rewards. This conservative final stage anchors updates to human reformulations and preservation labels, mitigating consumption-driven intent drift. 4. Experiment 4.1. Online Evaluation We evaluate WeFair through online A/B testing in WeChat short-video search. Relative to the control, the deployed model improves the effective clicked-query rate (V>10>10s) by 1.576% and reduces query reformulation rate (QRR) by 1.731%. Here, V>10>10s measures the fraction of exposed queries with at least one video consumption exceeding ten seconds. These results indicate improved effective consumption and fewer subsequent query revisions. 5. Conclusion We presented WeFair, a fidelity-aware alignment framework for compact personalized query rewriting in short-video search. WeFair addresses intent drift by assigning human feedback to complementary training roles: curriculum SFT builds rewriting capabilities, DPO learns relative preferences, and GRPO anchors final refinement to strict agreement with human reformulations and preservation targets. The resulting policy directly rewrites or preserves the query without generating reasoning traces. Online A/B testing shows a 1.576% relative increase in effective clicked-query rate and a 1.731% relative reduction in query reformulation rate, supporting the framework’s practical value for personalized search. References Chen et al. (2025) S. Chen, X. Jia, Y. Zhang, S. Zhang, X. Li, and W. Lin IterQR: An Iterative Framework for LLM-based Query Rewrite in e-Commercial Search System. arXiv preprint arXiv:2504.05309. External Links: 2504.05309, Link Cited by: §2.1. Cheng et al. (2026) C. Cheng, C. Wang, A. Li, H. Wu, H. Hu, J. Wang, and D. Liao When & How to Write for Personalized Demand-aware Query Rewriting in Video Search. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 4528–4533. External Links: Document, Link Cited by: §2.1. Chu et al. (2026) X. Chu, A. Li, J. Zhang, W. Li, Z. Tan, D. Yin, S. Wang, and D. Shi Accurate and Efficient Personalized Query Rewriting in Baidu Search. In Proceedings of the ACM Web Conference 2026, p. 7902–7913. External Links: Document, Link Cited by: §1, §2.1. Gong et al. (2026) P. Gong, F. Zhu, Y. Yin, C. Dai, C. Zhang, K. Zheng, W. Bao, J. Mao, and Y. Zhang CardRewriter: Leveraging Knowledge Cards for Long-Tail Query Rewriting on Short-Video Platforms. In Proceedings of the ACM Web Conference 2026, p. 8029–8040. External Links: Document, Link Cited by: §1, §2.1. Guo et al. (2025a) D. Guo, D. Yang, H. Zhang, J. Song, P. Wang, Q. Zhu, R. Xu, R. Zhang, S. Ma, X. Bi, et al. DeepSeek-r1 incentivizes reasoning in llms through reinforcement learning. Nature 645 (8081), p. 633–638. External Links: Link Cited by: §2.2. Guo et al. (2025b) M. Guo, M. Al-Darabsah, C. H. Teo, J. May, T. Agarwal, and R. Bhagat Learning to Rewrite Negation Queries in Product Search. In Proceedings of the 31st International Conference on Computational Linguistics: Industry Track, Abu Dhabi, UAE, p. 575–582. External Links: Link Cited by: §2.1. He et al. (2026) C. He, R. Zhang, Z. Wang, and X. Li Learning What to Learn: Stage-Specific Data Sets for SFT-then-RL in Small Language Model Reasoning. arXiv preprint arXiv:2606.04466. External Links: 2606.04466, Link Cited by: §2.2. Huang et al. (2026) J. Huang, X. Huang, F. Zhao, Y. Liu, H. Zhang, F. Shi, C. Li, Z. Sun, Y. Wu, Y. Hu, Y. Bai, and S. Cao QP-OneModel: A Unified Generative LLM for Multi-Task Query Understanding in Xiaohongshu Search. arXiv preprint arXiv:2602.09901. External Links: 2602.09901, Link Cited by: §2.1. Jiang et al. (2026) A. Jiang, G. Zhang, J. Song, K. Qi, D. Chen, W. Lin, and D. Lian Think-to-Personalize: Unifying Reasoning and Retrieval for User-Centric Personalized Dense Retrieval. arXiv preprint arXiv:2608.18855. Note: Accepted at CIKM 2026 External Links: 2608.18855, Link Cited by: §2.1. Kleine Buening et al. (2026) T. Kleine Buening, J. Hübotter, B. Pásztor, I. Shenfeld, G. Ramponi, and A. Krause Aligning Language Models from User Interactions. In Conference on Language Modeling, External Links: Link Cited by: §2.2. Li et al. (2026a) A. Li, Y. Shi, Y. Si, Y. Wu, M. Cai, X. Tan, Y. Wang, C. Sun, X. Liu, and K. Kuang Think Then Rewrite: Reasoning Enhanced Query Rewriting for Domain Specific Retrieval. Proceedings of the AAAI Conference on Artificial Intelligence 40 (18), p. 15045–15053. External Links: Document, Link Cited by: §2.1. Li et al. (2026b) C. Li, X. Tu, and Z. Wang DVCQR: Dual-View Conversational Query Rewriting with Stage-wise Reinforcement Learning. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 22993–23014. External Links: Document, Link Cited by: §2.1. Liu et al. (2026) X. Liu, L. Ma, J. Qiao, M. Zhou, L. Li, X. Bian, H. Chen, X. Jiang, and G. Jiang Generative Query Suggestion via Intent Coverage and Query-Level Credit Assignment. arXiv preprint arXiv:2609.19209. External Links: 2609.19209, Link Cited by: §2.1. Nguyen et al. (2025) D. A. Nguyen, R. K. Mohan, S. Yang, P. S. Akash, and K. C. Chang MiniELM: A Lightweight and Adaptive Query Rewriting Framework for E-Commerce Search Optimization. In Findings of the Association for Computational Linguistics: ACL 2025, p. 6952–6964. External Links: Document, Link Cited by: §2.1. Ou et al. (2026) R. Ou, K. Wang, J. Shao, T. Zhang, and C. Huo Aligning Query Rewriting with Human Cognition and Preference in E-Commerce Search. In Proceedings of the ACM Web Conference 2026, p. 7933–7942. External Links: Document, Link Cited by: §2.2. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. F. Christiano, J. Leike, and R. Lowe Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35, p. 27730–27744. External Links: Document, Link Cited by: §2.2. Peng et al. (2024) W. Peng, G. Li, Y. Jiang, Z. Wang, D. Ou, X. Zeng, D. Xu, T. Xu, and E. Chen Large Language Model based Long-tail Query Rewriting in Taobao Search. In Companion Proceedings of the ACM Web Conference 2024, p. 20–28. External Links: Document, Link Cited by: §2.1. Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn Direct Preference Optimization: Your Language Model is Secretly a Reward Model. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36, p. 53728–53741. External Links: Document, Link Cited by: §2.2. Schulman et al. (2017) J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov Proximal Policy Optimization Algorithms. arXiv preprint arXiv:1707.06347. External Links: 1707.06347, Link Cited by: §2.2. Tang et al. (2026) Y. Tang, Y. Yang, B. Chen, Y. Ma, Z. Liang, C. Lei, W. Ou, and J. Liu OneBar: An End-to-End Content-Grounded Generative Query Recommendation Framework for E-Commerce Video Feeds. arXiv preprint arXiv:2606.15330. External Links: 2606.15330, Link Cited by: §2.1. Tian et al. (2026) M. Tian, Xuye, L. Meng, L. Chen, Z. Qin, and Y. Wang From Short Video to Clickable Search: RLVR-Enabled Listwise Query Suggestion with Retrieval-Augmented Context. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 6: Industry Track), p. 552–562. External Links: Document, Link Cited by: §2.1. Wang et al. (2023) L. Wang, N. Yang, and F. Wei Query2doc: Query Expansion with Large Language Models. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 9414–9423. External Links: Document, Link Cited by: §2.1. Wen et al. (2026a) T. Wen, G. Dong, and Z. Dou SmartSearch: Process Reward-Guided Query Refinement for Search Agents. In Proceedings of the 49th International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 1971–1982. External Links: Document, Link Cited by: §2.2. Wen et al. (2026b) Y. Wen, R. Yang, X. Chang, H. Sun, G. Tang, C. Liu, J. Chen, Z. Ma, L. Qiu, X. Fan, C. Yu, Q. Zhou, Y. Chen, and Z. Wang CoRe: A Continuously Reward-Finetuned LLM Query Rewriter for Multi-Stage Context-Aware Relevance in Web-Scale Video Search. arXiv preprint arXiv:2606.14127. External Links: 2606.14127, Link Cited by: §2.2. Xu et al. (2026) Z. Xu, S. Zhuang, X. Ma, B. Chen, Y. Tian, F. Mo, T. Li, J. Cao, and V. Srikumar Rethinking On-policy Optimization for Query Augmentation. Transactions on Machine Learning Research. External Links: ISSN 2835-8856, Link Cited by: §2.2. Zheng et al. (2025) Q. Zheng, M. Zhong, S. Gong, H. Jiang, K. Wu, H. Liu, J. Xu, and L. Mo MAAQR: An LLM-based Multi-Agent Framework for Adaptive Query Rewriting in Alipay Search. In Proceedings of the 48th International ACM SIGIR Conference on Research and Development in Information Retrieval, p. 4289–4293. External Links: Document, Link Cited by: §2.1. Zuo et al. (2025) Y. Zuo, K. Zhang, L. Sheng, S. Qu, G. Cui, X. Zhu, H. Li, Y. Zhang, X. Long, E. Hua, B. Qi, Y. Sun, Z. Ma, L. Yuan, N. Ding, and B. Zhou TTRL: Test-Time Reinforcement Learning. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38, p. 131459–131483. External Links: Document, Link Cited by: §2.2.