Paper deep dive

SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations

Yu Wang, Yuchen Li, Rui Kong, Xinran Chen, Jiamin Chen, Hengyi Cai, Shuaiqiang Wang, Jiashu Zhao, Yulun Zhang, Zhonghao Lyu, Haoyi Xiong, Linghe Kong, Jimmy Xiangji Huang, Dawei Yin

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 94

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 9/11/2026, 5:15:30 AM

Summary

The paper introduces SWRouter, a framework for multi-turn Large Language Model (LLM) routing that addresses challenges in context segmentation and evaluation. SWRouter uses a similarity-based mechanism to construct prompts from dialogue history, reducing information loss and confusion, and employs a dual-metric evaluation framework to decouple construction accuracy from router performance. Experiments show SWRouter outperforms baselines and individual LLMs.

Entities (12)

SWRouter · method · 100%Information Confusion · challenge · 95%Information Loss · challenge · 95%Attribution Bias · challenge · 90%Claude · llm · 90%Conv-ID Context · baseline · 90%DeepSeek · llm · 90%Gemini · llm · 90%GPT · llm · 90%Llama · llm · 90%C2MAB-V · baseline · 85%RouterDC · baseline · 85%

Relation Signals (9)

SWRouter → outperforms → Conv-ID Context

confidence 95% · SWRouter consistently surpasses strong baselines, achieving ... an additional 8.22% gain over the Conv-ID Context baseline.

SWRouter → outperforms → best individual large language model

confidence 95% · achieving a 16.26% improvement in evaluation accuracy over the best individual large language model

SWRouter → uses → similarity-based context segmentation

confidence 95% · SWRouter combines a similarity-based context segmentation mechanism for prompt construction

SWRouter → uses → dual-metric evaluation framework

confidence 95% · SWRouter combines ... a dual-metric evaluation framework that decouples construction accuracy from router performance.

Claude → hasstrength → Code Generation

confidence 90% · Claude is particularly effective in code generation and understanding

GPT → hasstrength → Text Generation

confidence 90% · OpenAI’s GPT series is known for strong text generation ability

DeepSeek → hasstrength → Mathematical reasoning

confidence 90% · DeepSeek demonstrates strength in mathematical reasoning

C2MAB-V → istypeof → single-turn router

confidence 90% · methods that decide between small and large models, such as C2MAB-V

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models exhibit complementary strengths, motivating routing methods that dispatch each query to the most suitable model. Although existing routers are effective in single-turn settings, they do not directly transfer to multi-turn dialogue, where routing performance critically depends on how historical context is segmented, retained, and incorporated into the current prompt. This introduces two fundamental challenges: preventing information loss and information confusion during context construction, and evaluating routing quality without conflating model selection with prompt construction quality. In this paper, we propose SWRouter, a Similarity-Contractive Window Router for multi-turn large language model routing. SWRouter combines a similarity-based context segmentation mechanism for prompt construction with a dual-metric evaluation framework that decouples construction accuracy from router performance. Experiments on multi-turn dialogue benchmarks demonstrate that SWRouter consistently surpasses strong baselines, achieving a 16.26% improvement in evaluation accuracy over the best individual large language model and an additional 8.22% gain over the Conv-ID Context baseline. Our results highlight that multi-turn large language model routing requires a joint design of context construction and evaluation, rather than a direct extension of single-turn routing methods.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

91,184 characters extracted from source content.

Expand or collapse full text

SWRouter: Similarity-Contractive Window Routing for Multi-Turn Large Language Model Conversations CCS: Information systems Retrieval models and rankingCCS: Computing methodologies Natural language generationCCS: Computing methodologies Machine learning approaches Yu Wang email: wangyv123@sjtu.edu.cn Affiliation: Department of Big Data Management and Application, Shanghai Jiao Tong University, Shanghai, China , Yuchen Li email: yuchenli1230@gmail.com Affiliation: School of Computer Science, Shanghai Jiao Tong University, Shanghai, China Affiliation: Baidu Inc., Beijing, China , Rui Kong email: monster119120@gmail.com Affiliation: Baidu Inc., Beijing, China , Xinran Chen email: fantastique0910@gmail.com Affiliation: Baidu Inc., Beijing, China , Jiamin Chen email: jmchen26-c@my.cityu.edu.hk Affiliation: Baidu Inc., Beijing, China , Hengyi Cai email: hengyi1995@gmail.com Affiliation: Baidu Inc., Beijing, China , Shuaiqiang Wang email: shqiang.wang@gmail.com Affiliation: Baidu Inc., Beijing, China , Jiashu Zhao email: jzhao@wlu.ca Affiliation: Wilfrid Laurier University, Waterloo, Canada , Yulun Zhang email: yulun100@gmail.com Affiliation: School of Computer Science, Shanghai Jiao Tong University, Shanghai, China , Zhonghao Lyu email: lzhon@kth.se Affiliation: Department of Computer Science, The Hang Seng University of Hong Kong, Hong Kong SAR, China , Haoyi Xiong email: haoyi.xiong.fr@ieee.org Affiliation: Baidu Inc., Beijing, China , Linghe Kong email: linghe.kong@sjtu.edu.cn Affiliation: School of Computer Science, Shanghai Jiao Tong University, Shanghai, China , Jimmy Xiangji Huang email: jhuang@yorku.ca Affiliation: York University, Toronto, Canada and Dawei Yin email: yindawei@acm.org Affiliation: Baidu Inc., Beijing, China © , 2025 Abstract. Large language models exhibit complementary strengths, motivating routing methods that dispatch each query to the most suitable model. Although existing routers are effective in single-turn settings, they do not directly transfer to multi-turn dialogue, where routing performance critically depends on how historical context is segmented, retained, and incorporated into the current prompt. This introduces two fundamental challenges: preventing information loss and information confusion during context construction, and evaluating routing quality without conflating model selection with prompt construction quality. In this paper, we propose SWRouter, a Similarity-Contractive Window Router for multi-turn large language model routing. SWRouter combines a similarity-based context segmentation mechanism for prompt construction with a dual-metric evaluation framework that decouples construction accuracy from router performance. Experiments on multi-turn dialogue benchmarks demonstrate that SWRouter consistently surpasses strong baselines, achieving a 16.26% improvement in evaluation accuracy over the best individual large language model and an additional 8.22% gain over the Conv-ID Context baseline. Our results highlight that multi-turn large language model routing requires a joint design of context construction and evaluation, rather than a direct extension of single-turn routing methods. Keywords: Large Language Models, Multi-turn Dialogue, LLM Routing, Context Segmentation, Evaluation Framework, Model Selection 1. Introduction Large Language Models (LLMs) have become core infrastructure for a wide range of intelligent applications, including conversational assistants, code generation, content creation, and increasingly, AI search and agentic question answering (Li et al., 2025; Chen et al., 2025; Li et al., 2026a). Their rapid adoption has also stimulated growing efforts to improve the efficiency of long-context processing and inference (Li et al., 2026b; Li et al., 2026c). Meanwhile, many organizations have developed their own LLM families, such as OpenAI’s ChatGPT (OpenAI, 2022), DeepSeek’s DeepSeek (DeepSeek-AI, 2024), Meta’s LLaMA (Touvron et al., 2023a), Google’s Gemini (Gemini Team, 2023), and Anthropic’s Claude (Anthropic, 2024). However, no single model consistently performs best across all tasks and applications. For example, OpenAI’s GPT series is known for strong text generation ability (OpenAI, 2022), DeepSeek demonstrates strength in mathematical reasoning (Shao et al., 2024), and Claude is particularly effective in code generation and understanding (Anthropic, 2024). Such heterogeneity has motivated a growing line of work on LLM routing, which dynamically dispatches each query to the most suitable model rather than relying on a fixed model for all inputs. Existing routing methods have achieved promising results in single-turn settings, where the routing decision is made from a single prompt in isolation. Representative examples include routers that select among models of similar scale, such as RouterDC (Chen et al., 2024), as well as methods that decide between small and large models, such as C2MAB-V (Dai et al., 2024). However, directly applying these routing strategies to multi-turn dialogue is fundamentally difficult, because model selection in conversational systems depends not only on the current user query, but also on how the dialogue history is segmented, retained, and incorporated into the prompt. Figure 1. Illustration of the two challenges in multi-turn large language model routing. (a) The context-construction stage receives a dialogue history and the current user turn, then decides which historical turns should be retained before routing. When this step drops necessary history, the routed model receives an under-specified prompt and suffers from information loss; when it carries obsolete turns across a topic shift, the prompt contains misleading context and causes information confusion. (b) The evaluation stage observes only the final routed response score, but that score is jointly determined by prompt construction and model selection. A high-quality window can raise the absolute scores of all candidate LLMs, whereas a poor window can depress them even if the router still chooses the relatively best model, producing attribution bias unless construction quality and router performance are measured separately. A two-part schematic showing context segmentation failures and attribution bias in multi-turn LLM routing evaluation. In multi-turn dialogue, naive use of historical context introduces two fundamental challenges. • First, there is a lack of context segmentation mechanism. When historical information is not properly segmented and integrated, the router may fail to distinguish relevant from irrelevant past context. This leads to two concrete failure modes: information loss, where critical historical information required for the current response is omitted, and information confusion, where outdated context is mistakenly carried into a new topic. As illustrated in Figure 1, missing prior context can cause failure on rule-dependent queries, while incorrect reuse of obsolete context can degrade performance after topic shifts. • Second, there is an attribution bias in multi-turn router evaluation. In multi-turn settings, final response quality is jointly determined by two factors: the quality of context construction and the selected model’s performance under the constructed prompt. A poorly constructed prompt may substantially degrade the responses of all candidate models, regardless of which model is selected. Conversely, even with well-constructed context, poor model selection can still lead to unsatisfactory responses. As illustrated in Figure 1, a well-combined prompt and a poorly combined prompt may lead to dramatically different response quality across all candidate models (e.g., 0.9/0.7/0.3 vs. 0.09/0.07/0.03). However, the observed inference accuracy alone does not indicate whether such performance variation comes from the context construction stage or from errors in the trained router. This conflates construction quality with routing quality, making faithful evaluation of multi-turn routing systems difficult. To address these issues, we propose SWRouter, a multi-turn LLM routing framework that jointly addresses context construction, router training, and evaluation. SWRouter consists of three tightly coupled components: (1) a similarity-based context segmentation mechanism that selectively incorporates semantically relevant historical turns when constructing the prompt, thereby reducing both information loss and information confusion; (2) a pluggable router backbone trained on the constructed prompts with contrastive objectives to select the most suitable model; and (3) a decoupled evaluation framework that separates construction accuracy from router performance, enabling faithful assessment of each component independently. We evaluate SWRouter on multi-turn dialogue benchmarks and compare it against representative routing baselines and strong individual LLMs. Empirically, SWRouter consistently matches or surpasses the Conv-ID Context baseline, improves average accuracy by 16.26% over the best individual LLM, and further achieves an 8.22% gain over the Conv-ID Context baseline on multi-turn dialogue benchmarks. Our contributions are summarized as follows: • We identify that the lack of context segmentation in multi-turn dialogue routing leads to two concrete failure modes—information loss and information confusion. To address this issue, we propose a similarity-based prompt construction mechanism that integrates historical and current information. • We reveal a classification bias in existing router evaluation protocols and introduce a decoupled evaluation framework with three complementary metrics: evaluation accuracy, construction accuracy, and router performance. • We construct a strong Conv-ID Context baseline and conduct extensive in-distribution and out-of-distribution evaluations. SWRouter achieves 16.26% and 8.22% gains over the best single LLM and Conv-ID Context on multi-turn benchmarks, and outperforms Conv-ID Context by 2.68% on OOD tasks. Decoupled metric analysis further confirms that the gains stem from improved construction accuracy (w¯ w), while router performance (Prouter>1P_router>1) validates effective model selection across all settings. 2. Background and Motivation 2.1. Problem Formulation In conventional single-turn LLM routing, the routing decision is made from the current prompt alone: given a user query, the router selects the candidate model that is expected to produce the best response. In multi-turn dialogue, however, this problem becomes fundamentally more challenging. The router must not only determine which model to select, but also decide what historical context should be preserved and how that context should be incorporated into the current prompt. Formally, let ℳ=Mtt=1TM=\M_t\_t=1^T denote a pool of T candidate LLMs, and let train=(xi,yi)i=1nD_train=\(x_i,y_i)\_i=1^n denote the training set, where xix_i is the current user input and yiy_i is the corresponding target answer. In a single-turn setting, routing is performed directly on xix_i. In a multi-turn setting, by contrast, the current turn should be interpreted together with relevant dialogue history. Therefore, the effective routing input is no longer the raw query xix_i, but a context-enhanced query cic_i, which is constructed by combining the current turn with selected historical information. More specifically, for each current query xix_i, we construct a combined query cic_i by integrating it with semantically relevant historical turns. This yields a transformed training set train=(ci,yi)i=1n,C_train=\(c_i,y_i)\_i=1^n, where cic_i captures the multi-turn context on which routing should be based. The router then produces a probability distribution over the candidate model pool ℳM and selects the model that is most suitable for answering under the constructed prompt. Frequently used symbols are summarized in Table 1. Table 1. Symbols and Definitions. Symbol Definition ℳ=Mtt=1TM=\M_t\_t=1^T Pool of T candidate LLMs available for routing. xix_i Current user query or turn before context construction. yiy_i Target answer or supervision associated with xix_i. HiH_i Historical window retained for the i-th query after similarity-based segmentation. cic_i Context-enhanced query obtained by merging xix_i with the selected historical window. trainD_train Original training set before window-based prompt construction. trainC_train Transformed training set composed of context-enhanced queries. E⁡(⋅,w)E(·;w) Encoder that maps a query or constructed prompt into a dense representation. eie_i Embedding of xix_i produced by E⁡(⋅,w)E(·;w). sim⁡(⋅,⋅)sim(·,·) Cosine similarity used for window partitioning and model matching. τ Similarity threshold controlling whether adjacent turns are merged or split. ktk_t Learnable embedding representing candidate model MtM_t. R⁡(ci,θ)R(c_i;θ) Router distribution over candidate LLMs for constructed prompt cic_i. Ii+I_i^+, Ii−I_i^- Positive and negative candidate model sets used by the sample-LLM contrastive loss. Lsample-LLML_sample-LLM Contrastive loss that separates top-performing and bottom-performing LLMs for each query. Lsample-sampleL_sample-sample Contrastive loss that aligns semantically related constructed prompts. wi,tw_i,t True score of model MtM_t on constructed prompt cic_i, normalized to [0,1][0,1]. si(t)s_i^(t) Binary indicator showing whether the router selects model MtM_t for cic_i. si¯ si Evaluation accuracy, i.e., the average true score of routed responses. w¯ w Construction accuracy, i.e., the average true score over all candidate models and prompts. ProuterP_router Router performance, defined as si¯/w¯ si/ w. This formulation highlights a key distinction between single-turn and multi-turn routing: in multi-turn dialogue, routing quality depends not only on model selection, but also on prompt construction. If relevant history is omitted, the routed model may fail because of information loss; if irrelevant or outdated history is retained, the routed model may instead suffer from information confusion. As illustrated in Figure 1, these two failure modes arise before the routing decision itself and directly affect the quality of downstream responses. 2.2. Single-turn LLM Routing and Its Limitation Among existing single-turn routing methods, RouterDC (Chen et al., 2024) is a representative approach. It learns a router that maps an input query to a probability distribution over candidate LLMs, and optimizes the routing process with dual contrastive objectives. This design is effective in single-turn scenarios because the router only needs to reason over one prompt in isolation (Chen et al., 2024). However, this assumption does not hold in multi-turn dialogue. A single-turn router such as RouterDC (Chen et al., 2024) takes the current prompt as the routing unit, while ignoring the fact that the prompt itself may be under-specified or even misleading if dialogue history is not properly segmented and incorporated. As a consequence, even when the router selects the relatively best model for the observed input, the end-to-end system can still fail because the routing input is poorly constructed. This limitation is particularly important in our setting for two reasons. First, multi-turn dialogue introduces a context selection problem: the system must decide which historical turns are relevant to the current request before routing can be performed reliably. Second, conventional router evaluation is largely based on relative ranking signals, which are suitable for stable optimization but insufficient for measuring end-to-end usefulness when prompt construction quality varies. As discussed in Section 4.4, two prompts can induce the same relative ranking over candidate models while yielding drastically different true scores. Therefore, directly extending a single-turn router to multi-turn dialogue is insufficient. These observations motivate a routing framework that jointly addresses context construction and routing evaluation. In the next section, we introduce SWRouter , which augments router learning with similarity-based window partitioning for prompt construction and a decoupled evaluation framework for faithful assessment in multi-turn settings. 3. Preliminaries and Problem Setup This section formalizes the multi-turn routing setting studied in this paper. The goal is to make explicit the objects that are optimized by the router and the objects that are produced by the context-construction module. Unless otherwise specified, the notation follows Table 1. 3.1. Multi-turn Dialogue Instances We consider a collection of multi-turn dialogues. For a routing instance indexed by i, let i=(ui,1,ai,1),(ui,2,ai,2),…,(ui,mi−1,ai,mi−1),ui,miZ_i=\(u_i,1,a_i,1),(u_i,2,a_i,2),…,(u_i,m_i-1,a_i,m_i-1),u_i,m_i\ denote the dialogue observed before producing the next assistant response. Here, ui,ju_i,j is the j-th user turn, ai,ja_i,j is the corresponding assistant turn, and ui,miu_i,m_i is the current user request. For consistency with the rest of the paper, we write xi=ui,mix_i=u_i,m_i for the current query. The raw dialogue history before xix_i is denoted by ℋiraw=(ui,1,ai,1),…,(ui,mi−1,ai,mi−1).H^raw_i=\(u_i,1,a_i,1),…,(u_i,m_i-1,a_i,m_i-1)\. The router does not directly operate on ℋirawH^raw_i. Instead, a context-construction function g⁡(⋅)g(·) first selects and organizes the relevant historical information, producing a retained semantic window HiH_i and a context-enhanced prompt (1) ci=g⁡(ℋiraw,xi).c_i=g(H^raw_i,x_i). In this paper, g⁡(⋅)g(·) is instantiated by similarity-based window construction, but the problem setup allows other context selectors such as full-history concatenation, current-turn-only prompting, or retrieval-based context selection. This distinction is important because xix_i alone may be under-specified. For example, the current request may contain pronouns, omitted constraints, or references to entities introduced in previous turns. At the same time, the entire raw history may contain outdated goals, corrected assumptions, or off-topic turns. Therefore, a valid multi-turn routing instance is not merely a user query; it is the pair consisting of the current request and the constructed prompt that determines what the candidate LLMs actually observe. 3.2. Candidate Models and Router Objective Let ℳ=Mtt=1TM=\M_t\_t=1^T be a fixed pool of T candidate LLMs. Given a context-enhanced prompt cic_i, a router with parameters θ outputs a probability distribution R⁡(ci,θ)=[R1(ci,θ),R2(ci,θ),…,RT(ci,θ)],R(c_i;θ)= [R_1(c_i;θ),R_2(c_i;θ),…,R_T(c_i;θ) ], where Rt(ci,θ)R_t(c_i;θ) is the probability of selecting candidate model MtM_t. At inference time, the selected model index is (2) t^i=arg⁡maxt∈1,…,TRt(ci,θ), t_i= _t∈\1,…,T\R_t(c_i;θ), and the final response is generated by Mt^i(ci)M_ t_i(c_i). The ideal routing decision depends on the response quality that each candidate model would obtain under the same constructed prompt. Let wi,t∈[0,1]w_i,t∈[0,1] denote the normalized quality score of model MtM_t on prompt cic_i. The oracle candidate for this prompt is (3) ti⋆=arg⁡maxt∈1,…,T⁡wi,t.t_i = _t∈\1,…,T\w_i,t. The routing objective is therefore to learn a model selector that approaches the oracle candidate while using only the constructed prompt at inference time. Equivalently, the end-to-end objective can be written as (4) maxθ,g⁡1n∑i=1nwi,t^i,t^i=arg⁡maxtRt(g⁡(ℋiraw,xi),θ). _θ,g\; 1n _i=1^nw_i, t_i, t_i= _tR_t(g(H^raw_i,x_i);θ). This expression makes explicit that multi-turn routing has two coupled components: the context-construction function g⁡(⋅)g(·) and the router R⁡(⋅,θ)R(·;θ). If g⁡(⋅)g(·) removes necessary history, even a strong router may select a model using an incomplete prompt. If g⁡(⋅)g(·) introduces irrelevant history, the candidate models may all produce lower-quality responses, making the routing decision less meaningful. 3.3. Training Supervision and Evaluation Scores During training and evaluation, each constructed prompt cic_i is paired with responses from all candidate models. A judge model then assigns an integer score to each response, which is normalized to obtain wi,tw_i,t. These scores serve two roles. First, they provide relative supervision for router learning by identifying strong and weak candidate models for each prompt. For example, the top-scoring models form the positive set Ii+I_i^+, while the bottom-scoring models form the negative set Ii−I_i^-. Second, they provide absolute response-quality measurements for decoupled evaluation. The distinction between relative and absolute signals is central to our setting. Relative supervision is useful for training because it tells the router which candidate models should be preferred for a particular prompt. However, relative ranking alone cannot determine whether the constructed prompt itself is good. If all candidate models receive low absolute scores because the prompt omits necessary history, the best candidate model may still be easy to identify, but the final user-facing answer remains poor. For this reason, Section 4.4 separately measures evaluation accuracy, construction accuracy, and router performance. 3.4. Scope We focus on routing among a fixed pool of already available LLMs. The router does not modify the candidate models, and the context-construction module operates before generation. This scope matches practical deployments in which different models have complementary strengths and costs, but invoking all models for every user request is undesirable. The problem studied here is therefore not how to train a stronger LLM, but how to construct an informative multi-turn prompt and select an appropriate model for that prompt. 4. Methodology 4.1. Overview SWRouter is designed to address the two central challenges of multi-turn LLM routing introduced in Section 1: (1) how to construct an effective routing prompt from dialogue history, and (2) how to evaluate routing quality without conflating it with prompt construction quality. To this end, SWRouter consists of three stages: (1) similarity-based window partitioning to construct combined prompts from multi-turn dialogue history (Section 4.2); (2) model routing based on the constructed prompts, optimized with contrastive training objectives that rely on relative, ranking-based scores (Section 4.3); and (3) decoupled evaluation that distinguishes construction accuracy from router performance using absolute true scores after prompt construction (Section 4.4). In the first stage, SWRouter organizes multi-turn dialogue history into semantically coherent windows. Instead of directly concatenating the entire dialogue history, it measures the semantic similarity between adjacent user turns and dynamically decides whether the current turn should be merged into the existing window or begin a new one. This design allows SWRouter to preserve relevant historical information while reducing the risk of introducing irrelevant or outdated context. In the second stage, the constructed context-enhanced prompt is fed into a pluggable router backbone that selects the most suitable model from the candidate LLM pool. Concretely, the router backbone encodes the combined prompt into a dense representation and computes its similarity with a set of learnable model embeddings, yielding a probability distribution over candidate LLMs. The router backbone is optimized with contrastive objectives that encourage it to distinguish strong candidate models from weak ones under multi-turn settings. In the third stage, SWRouter adopts a decoupled evaluation framework tailored to multi-turn routing. Since existing routing evaluations are typically based on relative ranking signals, they are suitable for optimization but insufficient for measuring end-to-end usefulness when prompt construction quality varies. Therefore, SWRouter separately reports construction accuracy, which reflects the average quality of all candidate models under the constructed prompt, and router performance, which measures how much the router improves over random selection. Overall, these three stages form a unified framework for multi-turn LLM routing. The similarity-based window partitioning stage determines what historical context should be preserved, the routing stage determines which model should be selected under the constructed prompt, and the decoupled evaluation stage determines how the resulting system should be assessed. Pipeline diagram of SWRouter showing similarity-based window partitioning, router training and inference, and decoupled evaluation. Figure 2. Overview of SWRouter . Given a multi-turn dialogue, SWRouter first encodes adjacent user turns and computes their semantic similarity to decide whether the current turn should extend the existing window or start a new one. The retained window is then merged into a context-enhanced prompt and passed to a pluggable router, which represents the prompt with an encoder, compares it with learnable candidate-model embeddings, and selects the LLM with the highest routing score. During training, the router is optimized with sample-LLM and sample-sample contrastive objectives so that high-quality models and semantically related prompts are pulled closer in representation space. During evaluation, the generated responses are scored by judge models and decomposed into construction accuracy, evaluation accuracy, and router performance, allowing prompt construction quality and model-selection ability to be analyzed separately.Pipeline diagram of SWRouter showing similarity-based window partitioning, router training and inference, and decoupled evaluation. 4.2. Similarity-based Window Partitioning A core challenge in multi-turn LLM routing is that the current user query should not be interpreted in isolation. Instead, the router should determine which historical turns remain semantically relevant to the current request and should therefore be retained in the routing prompt. To address this issue, SWRouter employs a similarity-based window partitioning algorithm that organizes dialogue history into semantically coherent windows. This formulation follows the broader view of sequential structure analysis and segmentation in formal-language and string-processing algorithms (Aho and Ullman, 1972; Gusfield, 1997; Chandra et al., 1981), while using neural semantic similarity rather than symbolic parsing rules. Let E⁡(x,w)E(x;w) denote an encoder that maps an input utterance x into an embedding in ℝpR^p. Given the current user query xix_i, we first compute its embedding ei=E⁡(xi,w).e_i=E(x_i;w). We then measure the similarity between the current query and the previous query: (5) similarity=sim⁡(ei,ei−1),similarity=sim(e_i,e_i-1), where sim⁡(⋅,⋅)sim(·,·) denotes cosine similarity, eie_i is the embedding of the current query xix_i, and ei−1e_i-1 is the embedding of the previous query xi−1x_i-1. Based on this similarity score, SWRouter updates the historical window HiH_i as follows: (6) Hi=xi,if sim(ei,ei−1)<τ,Hi−1∪xi,if sim(ei,ei−1)≥τ,H_i= cases\x_i\,&if sim(e_i,e_i-1)<τ,\\ H_i-1∪\x_i\,&if sim(e_i,e_i-1)≥τ, cases where τ is a similarity threshold. Intuitively, if the current turn is insufficiently similar to the previous one, SWRouter starts a new window; otherwise, it appends the current turn to the existing window. In this way, the dialogue history is partitioned into locally coherent semantic segments rather than being naively concatenated into a single undifferentiated context. After the historical window is determined, SWRouter constructs a context-enhanced prompt cic_i for routing by combining the current query with the selected dialogue history in HiH_i. This design enables the router to make decisions based on relevant multi-turn context, rather than relying solely on the current turn. Compared with naive history concatenation, the similarity-based windowing mechanism offers two advantages. First, it preserves relevant context needed for rule-dependent or context-dependent requests, thereby reducing information loss. Second, it avoids carrying obsolete or off-topic history into the new prompt, thereby reducing information confusion. These two effects correspond exactly to the two failure modes illustrated in Figure 1. The similarity threshold τ controls the granularity of context segmentation. A smaller τ tends to merge more turns into the same window, which may preserve more context but also increases the risk of introducing irrelevant information. A larger τ produces more aggressive segmentation, which can better isolate topic shifts but may also discard useful dependencies. Therefore, τ governs the trade-off between context retention and context purity, and its impact will be analyzed in Section 6.5. Algorithm 1 summarizes the similarity-window construction procedure. The procedure is applied independently to each dialogue. It processes user turns in chronological order, compares each user turn with the immediately preceding user turn, and updates the active semantic window according to the threshold τ. Assistant turns that lie inside the retained window are preserved when constructing the final prompt, so that the generated prompt keeps both the user’s constraints and the assistant-side facts that may be needed for resolving follow-up references. Algorithm 1 Similarity-based window construction for multi-turn routing. The algorithm scans a dialogue from left to right, starts a new semantic window when adjacent user turns are weakly related, and otherwise extends the current window. For each current request, it outputs a context-enhanced prompt that contains only the active semantic window. Input: Dialogue i=(ui,1,ai,1),…,(ui,mi−1,ai,mi−1),ui,miZ_i=\(u_i,1,a_i,1),…,(u_i,m_i-1,a_i,m_i-1),u_i,m_i\; encoder E⁡(⋅,w)E(·;w); threshold τ Output: Context-enhanced prompt cic_i and retained window HiH_i for the current request xi=ui,mix_i=u_i,m_i 1 H←ui,1,ai,1H←\u_i,1,a_i,1\; 2 eprev←E⁡(ui,1,w)e_prev← E(u_i,1;w); 3 for j=2j=2 to mim_i do 4 ej←E⁡(ui,j,w)e_j← E(u_i,j;w); 5 ρj←sim⁡(ej,eprev) _j (e_j,e_prev); 6 if ρj<τ _j<τ then 7 H←ui,jH←\u_i,j\ ; // topic shift; reset the semantic window 8 else 9 H←H∪ui,jH← H∪\u_i,j\ ; // same local context; extend the window 10 if j<mij<m_i and ui,j∈Hu_i,j∈ H then 11 H←H∪ai,jH← H∪\a_i,j\ ; // retain assistant-side context inside the window 12 eprev←eje_prev← e_j; 13 Hi←H_i← H; 14 ci←FormatPrompt⁡(Hi,xi)c_i (H_i,x_i); 15 return ci,Hic_i,H_i; 4.3. Routing and Training For a query xix_i, after being transformed into a combined query cic_i using the similarity-based window partitioning algorithm, SWRouter generates a selection probability distribution over T candidate LLMs: (7) R⁡(ci,θ)=softmax⁡([sim⁡(E⁡(ci,w),k1)sim⁡(E⁡(ci,w),k2)sim⁡(E⁡(ci,w),kT)]),R(c_i;θ)=softmax ( bmatrixsim(E(c_i;w),k_1)\\ sim(E(c_i;w),k_2)\\ \\ sim(E(c_i;w),k_T) bmatrix ), where θ≡w,k1,k2,…,kTθ≡\w,k_1,k_2,…,k_T\ denotes the parameters of SWRouter , and sim⁡(⋅,⋅)sim(·,·) denotes cosine similarity. A straightforward way to train the router is to align its output distribution with a score distribution: (8) min⁡∑(ci,yi)∈trainθ⁡KL⁡(R⁡(ci,θ),softmax⁡[si(1),…,si(T)]), _θ _(c_i,y_i) _trainKL (R(c_i;θ),\;softmax[s_i^(1),…,s_i^(T)] ), where KL⁡(⋅,⋅)KL(·,·) denotes the Kullback–Leibler divergence (Kullback and Leibler, 1951). This KL objective has recently been used in LLM routing (Lu et al., 2024). However, we argue that it is not an ideal proxy for router training in our setting, because the goal of routing is to assign queries to top-performing LLMs rather than to fit the full score distribution, especially for bottom-performing models. Following the contrastive learning spirit of RouterDC (Chen et al., 2024), SWRouter adopts contrastive training objectives over the context-enhanced query. 4.3.1. Sample-LLM Contrastive Loss For each combined query cic_i, SWRouter constructs positive and negative LLM index sets Ii+I_i^+ and Ii−I_i^-. The sample-LLM contrastive loss is defined as (9) Lsample-LLM(ci,yi,θ)=−log⁡Pi+Pi++Pi−,L_sample-LLM(c_i,y_i;θ)=- P_i^+P_i^++P_i^-, where (10) Pi+=∑t∈Ii+esim⁡(E⁡(ci,w),kt),P_i^+= _t∈ I_i^+e^sim(E(c_i;w),k_t), is the sum of the exponentiated similarities between the query embedding E⁡(ci,w)E(c_i;w) and the top-K+K_+ model embeddings, and (11) Pi−=∑t∈Ii−esim⁡(E⁡(ci,w),kt),P_i^-= _t∈ I_i^-e^sim(E(c_i;w),k_t), is the corresponding sum over the bottom-K−K_- model embeddings. Here, Ii+I_i^+ and Ii−I_i^- denote the index sets of the top-K+K_+ and bottom-K−K_- LLMs, respectively. 4.3.2. Sample-Sample Contrastive Loss In addition to contrasting candidate models for each query, SWRouter leverages unsupervised clustering to group semantically related queries and constructs a sample-sample contrastive loss: This representation-learning design is related to scalable log-linear optimization and predictive structure learning across related tasks (Andrew and Gao, 2007; Ando and Zhang, 2005). (12) Lsample-sample(ci,θ)=−log⁡Qi+Qi++Qi−,L_sample-sample(c_i;θ)=- Q_i^+Q_i^++Q_i^-, where (13) Qi+=esim⁡(E⁡(ci,w),E⁡(ci+,w))Q_i^+=e^sim(E(c_i;w),E(c_i^+;w)) represents the similarity between the embedding of query cic_i and that of a randomly chosen in-group query ci+c_i^+, and (14) Qi−=∑ci−∈Xi−esim⁡(E⁡(ci,w),E⁡(ci−,w))Q_i^-= _c_i^-∈ X_i^-e^sim(E(c_i;w),E(c_i^-;w)) denotes the summed similarities between cic_i and a set of out-group queries Xi−X_i^-. The overall training objective combines the above two contrastive losses: (15) min⁡∑θ⁡(αLsample-LLM+βLsample-sample), _θΣ (α L_sample-LLM+β L_sample-sample ), where α and β are hyperparameters that balance the two loss terms. Algorithm 2 summarizes the training and inference pipeline of SWRouter . The algorithm uses Algorithm 1 as a preprocessing step to transform raw multi-turn dialogues into context-enhanced prompts. Training then relies on model-level and sample-level contrastive supervision, while inference only requires a single router forward pass before calling the selected candidate LLM. Algorithm 2 Router training and inference for SWRouter . The procedure first converts raw dialogues into context-enhanced prompts using Algorithm 1, then builds contrastive supervision from candidate-model scores, optimizes the router with sample-LLM and sample-sample losses, and finally selects one candidate LLM for each test prompt. Input: Training dialogues trainD_train, test dialogues testD_test, LLM pool ℳ=Mtt=1TM=\M_t\_t=1^T, encoder E⁡(⋅,w)E(·;w), model embeddings ktt=1T\k_t\_t=1^T, threshold τ, hyperparameters K+K_+, K−K_-, N, λ, b, η Output: Trained router R⁡(⋅,θ)R(·;θ) and routed responses on the test set 1 Data preprocessing: 2 for i∈trainZ_i _train do 3 (ci,Hi)←WindowConstruct⁡(i,E,τ)(c_i,H_i) (Z_i,E,τ) using Algorithm 1; 4 for t=1t=1 to T do 5 Generate response ri,t←Mt(ci)r_i,t← M_t(c_i); 6 Score ri,tr_i,t with the judge model to obtain wi,tw_i,t; 7 train←(ci,wi,tt=1T)C_train←\(c_i,\w_i,t\_t=1^T)\; 8 Training: 9 Cluster queries cii=1n\c_i\_i=1^n into N groups 10 repeat 11 Sample mini-batch B from trainC_train 12 for (ci,yi)∈B(c_i,y_i)∈ B do 13 Construct Ii+I_i^+ and Ii−I_i^- using wi,tw_i,t 14 Compute Lsample-LLM(ci,yi,θ)L_sample-LLM(c_i,y_i;θ) 15 Sample in-group query ci+c_i^+ and out-group queries ci−\c_i^-\ 16 Compute Lsample-sample(ci,θ)L_sample-sample(c_i;θ) 17 L⁡(B,θ)←∑(ci,yi)∈B[Lsample-LLM+λLsample-sample]L(B;θ)← _(c_i,y_i)∈ B [L_sample-LLM+λ L_sample-sample ] 18 θ←θ−η∇θL(B,θ)θ←θ-η _θL(B;θ) 19 until converged; 20 Inference: 21 for i∈testZ_i _test do 22 (ci,Hi)←WindowConstruct⁡(i,E,τ)(c_i,H_i) (Z_i,E,τ); 23 t^i←arg⁡maxtRt(ci,θ) t_i← _tR_t(c_i;θ); 24 y^i←Mt^i(ci) y_i← M_ t_i(c_i); 25 return R⁡(⋅,θ)R(·;θ) and y^i\ y_i\; 4.4. Decoupled Evaluation Metrics For each combined query cic_i and each candidate model ℳtM_t (t=1,…,Tt=1,…,T), we obtain a true score 0≤wi,t≤10≤ w_i,t≤ 1 from judge models, which reflects the absolute response quality of ℳtM_t on cic_i. Evaluation Accuracy (absolute quality). Given the router’s selection indicator si(t)∈0,1s_i^(t)∈\0,1\, where si(t)=1s_i^(t)=1 if the router selects ℳtM_t for cic_i and 00 otherwise, the score of the selected model is (16) si=∑t=1Twi,t⋅si(t).s_i= _t=1^Tw_i,t· s_i^(t). We define evaluation accuracy as the average true score achieved by the routing system: (17) si¯=1n∑i=1nsi, si= 1n _i=1^ns_i, which reflects the absolute usefulness of the routed responses under the constructed prompts. Construction Accuracy (prompt quality). To characterize the overall quality of the constructed prompt across all candidate models, we define the average true score (18) w¯=1nT∑i=1n∑t=1Twi,t, w= 1nT _i=1^n _t=1^Tw_i,t, which reflects the average capability of the candidate model pool under a given prompt construction. A higher w¯ w indicates that the constructed prompt enables better responses across all candidate models. Router Performance (relative improvement). Based on si¯ si and w¯ w, we define router performance as (19) Prouter=si¯w¯.P_router= si w. Here, w¯ w reflects the average capability of the candidate model pool under a given prompt construction, while ProuterP_router measures how much the router improves over this average by selecting stronger models. A value greater than 1 indicates that the router outperforms random model selection. Diagram comparing construction accuracy, evaluation accuracy, and router performance for multi-turn routing. Figure 3. Illustration of the three complementary metrics used for decoupled evaluation. For each constructed prompt, all candidate LLMs are first evaluated with absolute true scores. Construction accuracy w¯ w averages these scores across the whole candidate pool and therefore measures whether the prompt construction stage preserves useful context for models in general. Evaluation accuracy si¯ si then records the true score of the single model selected by the router, representing the end-to-end quality observed by users. Router performance Prouter=si¯/w¯P_router= si/ w normalizes the selected-model score by the candidate average, showing whether the router improves over random model selection under the same constructed prompts. Diagram comparing construction accuracy, evaluation accuracy, and router performance for multi-turn routing. Discussion: three complementary metrics. For evaluation in multi-turn dialogue, we adopt three complementary metrics to characterize the system: evaluation accuracy (si¯ si), construction accuracy (w¯ w), and router performance (ProuterP_router). Figure 3 illustrates why these metrics are complementary. Under good prompt construction, the candidate models obtain scores of 0.9,0.7,0.3\0.9,0.7,0.3\, yielding an average construction accuracy of w¯=0.633 w=0.633. If the router selects the best model, then si¯=0.9 si=0.9 and Prouter=0.9/0.633=1.42P_router=0.9/0.633=1.42, indicating both strong construction quality and effective routing. In contrast, under poor prompt construction, information loss uniformly degrades the candidate scores to 0.09,0.07,0.03\0.09,0.07,0.03\, reducing construction accuracy to w¯=0.0633 w=0.0633. Even if the router still selects the best model, the final evaluation accuracy drops to si¯=0.09 si=0.09, while the router performance remains unchanged: Prouter=0.09/0.0633=1.42P_router=0.09/0.0633=1.42. This example shows that high router performance does not necessarily imply high end-to-end response quality. Instead, si¯ si reflects the final usefulness of the routed response, w¯ w measures the intrinsic quality of prompt construction, and ProuterP_router quantifies the router’s ability to select better-than-average candidates. Together, these metrics enable a decoupled analysis of construction quality and routing effectiveness. 5. Implementation We implement SWRouter following the standard router-training and evaluation pipeline for multi-turn LLM routing. Our implementation consists of candidate model construction, multi-turn dataset preprocessing, similarity-window segmentation, response scoring, and router optimization. Candidate LLMs. We evaluate SWRouter on seven open-source LLMs from HuggingFace: Mistral-7B (Jiang et al., 2023a), MetaMath-Mistral-7B (Yu et al., 2024), zephyr-7b-beta (Tunstall et al., 2023), Chinese-Mistral-7B (Zhou and Bai, 2024), dolphin-2.6-mistral-7b (Cognitive Computations, 2024a), Llama-3-8B (Llama Team, 2024), and dolphin-2.9-llama3-8b (Cognitive Computations, 2024b). The first five models are Mistral-based, while the last two are Llama-3-based. These models cover general-purpose, instruction-tuned, domain-tuned, and language-adapted variants, enabling us to evaluate whether SWRouter can select among LLMs with heterogeneous capabilities. The Llama-family candidates build on the broader open-foundation-model line represented by LLaMA and LLaMA 2 (Touvron et al., 2023a; Touvron et al., 2023b), and zephyr-7b-beta follows preference-optimization techniques such as direct preference optimization (Rafailov et al., 2023). Datasets. We conduct experiments on two multi-turn dialogue datasets, MTBench (Bai et al., 2024) and ShareGPT (Schmid, 2023). MTBench contains multi-turn conversations across diverse task categories, while ShareGPT consists of real user-AI interactions. We preprocess ShareGPT by removing noisy samples and filtering out non-multi-turn dialogues. For both datasets, we randomly split the data into 70% for training and 30% for testing. All training samples are combined as trainC_train for router training. Baselines. We compare SWRouter with three groups of baselines. First, Single Model directly uses one candidate LLM to answer each query without routing. This group includes all seven candidate LLMs listed above. Second, Conv-ID Context preserves the original dialogue segmentation according to conversation IDs and performs routing with the full original multi-turn context. This baseline serves as an ID-based static context reference. Third, ZOOTER (Lu et al., 2024) is used as a strong single-turn router backbone. To evaluate the generality of the proposed similarity-window mechanism, we combine ZOOTER with the same similarity-window segmentation and compare it with SWRouter . Evaluation protocol. We use the Language Model Evaluation Harness (Gao et al., 2023) for model evaluation, following common benchmark practice from MMLU-style multitask evaluation (Hendrycks et al., 2021). For open-ended generation, we generate M=10M=10 candidate responses using stochastic beam search, where beam expansion is sampled with temperature 0.2. The generated responses are then scored by judge models, and the router is evaluated using testing weighted accuracy. Similarity-window configurations. We use mDeBERTaV3-base (He et al., 2021) as the encoder E⁡(x,w)E(x;w), which contains approximately 86M backbone parameters. This choice follows the use of efficient neural encoders for structured text analysis (Rasooli and Tetreault, 2015). The similarity threshold is set to τ=0.91τ=0.91 for constructing similarity windows. The LLM embedding dimension is 768, and the number of clusters is set to N=5N=5. Router training configurations. The router is trained for 1,000 steps using AdamW (Loshchilov and Hutter, 2019). The learning rate is 5×10−55× 10^-5, the weight decay is 0.01, and the batch size is 16. All experiments are conducted on NVIDIA A100 80GB GPUs. 6. Evaluation We evaluate SWRouter to answer the following questions: (1) Can SWRouter improve routing performance on multi-turn dialogue tasks? (2) How well does SWRouter generalize to out-of-distribution (OOD) scenarios? (3) How much does each component contribute to the effectiveness of SWRouter ? (4) How sensitive is SWRouter to key hyperparameters? (5) What is the computational overhead introduced by SWRouter ? 6.1. Evaluation Setup 6.1.1. Prompts Used for Evaluation To evaluate the quality of routed responses in multi-turn dialogue, we employ a judge model with the following evaluation prompt: The prompt is written as a structured evaluation rubric following standard academic reporting conventions (American Psychological Association, 1983). You are given a multi-turn conversation, including the previous dialogue history, the current user request, and an assistant response generated by a candidate LLM. Please evaluate the quality of the assistant response with a single integer score from 1 to 10. Judge the response semantically rather than by exact wording: paraphrases, different ordering, or different surface expressions should be treated as acceptable when they preserve the same meaning and satisfy the user’s request. Do not reward verbosity by itself; longer answers should receive higher scores only when the additional content improves correctness, usefulness, or coverage. Use the following criteria when assigning the score. Intent satisfaction evaluates whether the response identifies the user’s actual goal in the current turn and provides a useful answer or action for that goal. Dialogue context usage evaluates whether the response correctly incorporates relevant prior turns, carries over established constraints, resolves references such as “it” or “that” appropriately, and avoids being distracted by obsolete or unrelated context. Instruction following evaluates whether the response obeys explicit requirements about format, language, style, scope, tools, refusal behavior, or output length. Factuality and reasoning evaluates whether the response is factually correct, logically consistent, and supported by the information available in the conversation or by reliable reasoning. Completeness evaluates whether all important parts of the request are addressed, including edge cases, caveats, or requested explanations. Clarity and usefulness evaluates whether the response is easy to understand, well organized, appropriately concise, and practically helpful to the user. Apply the following scoring guideline. A score of 10 should be reserved for a response that fully satisfies the user’s intent, uses all necessary dialogue context correctly, follows every instruction, is factually sound, complete, and clear. A score of 9 indicates an excellent response with only negligible wording or presentation issues. Scores of 7–8 indicate a mostly correct and useful response that satisfies the main request but has minor omissions, mild ambiguity, limited explanation, or only partial use of relevant context. Scores of 5–6 indicate a partially useful response that addresses the general topic but misses important constraints, overlooks meaningful dialogue history, contains noticeable but non-fatal factual or reasoning errors, or leaves substantial parts of the task incomplete. Scores of 3–4 indicate a weak response that is only loosely related to the request, substantially incomplete, poorly grounded in the dialogue, or affected by major factual, reasoning, or instruction-following errors. Scores of 1–2 indicate a response that is irrelevant, contradicts the conversation, refuses without justification, is unsafe when safety is required, fails to answer the user, or is dominated by hallucinated content. When several issues are present, choose the score that best reflects the overall usefulness of the response to the user. Penalize strongly for errors that would cause the user to take a wrong action, violate an explicit constraint, or misunderstand the task outcome. A response with a major factual error should not receive a score above 6, even if it is fluent. A response that ignores the current user request or relies on the wrong dialogue context should not receive a score above 4. A response that is correct but unnecessarily verbose, poorly organized, or missing minor details may still receive a high score if the user’s intent is satisfied. Return only the final score. The judge model returns an integer rating from 1 to 10, which we normalize to the range [0,1][0,1] as the true score wi,tw_i,t for evaluation metrics computation. 6.2. Overall Performance Ranked horizontal bar chart showing average evaluation accuracy for all single LLMs and routing methods, with markers for ShareGPT and MTBench scores. Figure 4. Overall evaluation accuracy (%) on ShareGPT and MTBench. Horizontal bars rank all candidate single LLMs and routing methods by average accuracy, while markers show the corresponding ShareGPT and MTBench scores. The dashed line marks the strongest single LLM, making the routing gain of SWRouter visually explicit.Ranked horizontal bar chart showing average evaluation accuracy for all single LLMs and routing methods, with markers for ShareGPT and MTBench scores. As shown in Figure 4, SWRouter achieves the best overall performance across the two multi-turn dialogue datasets. Compared with the strongest single LLM, dolphin-2.9-llama3-8b, SWRouter improves the average evaluation accuracy from 24.63% to 40.89%, corresponding to an absolute gain of 16.26%. The improvement is consistent across both datasets, with gains of 14.29% on ShareGPT and 18.22% on MTBench. Compared with Conv-ID Context, which preserves the original dialogue segmentation by conversation IDs, SWRouter further improves the average evaluation accuracy by 8.22%. This result suggests that exact ID-based partitioning is not necessarily the optimal context construction strategy for multi-turn routing. Instead, dynamically constructing semantic windows provides more useful routing inputs. Moreover, ZOOTER equipped with the similarity-window mechanism achieves 39.21% average evaluation accuracy, substantially outperforming Conv-ID Context. This indicates that the proposed similarity-window design is general and can improve different routing backbones by constructing better multi-turn contexts. 6.3. OOD Generalization Dumbbell chart comparing SWRouter and Conv-ID Context on PreAlgebra, MBPP, C-EVAL, and their average, with gain annotations. Figure 5. Evaluation accuracy (%) of SWRouter and Conv-ID Context on out-of-distribution datasets. PreAlgebra, MBPP, and C-EVAL cover mathematical reasoning, code generation, and Chinese knowledge-intensive evaluation, respectively. The connected markers compare SWRouter with Conv-ID Context on each task, and the right-side annotations report the absolute gain or loss. This visualization emphasizes that SWRouter improves the average OOD result despite not winning on every individual task.Dumbbell chart comparing SWRouter and Conv-ID Context on PreAlgebra, MBPP, C-EVAL, and their average, with gain annotations. We further evaluate the generalization ability of SWRouter under out-of-distribution scenarios, including PreAlgebra, MBPP, and C-EVAL. These datasets cover mathematical reasoning, code generation, and Chinese knowledge-intensive evaluation, respectively, and are related to established mathematical and Chinese multitask evaluation settings (Cobbe et al., 2021; Li et al., 2024a). As shown in Figure 5, SWRouter achieves the best average OOD performance, reaching 43.99% and outperforming Conv-ID Context by 2.68%. In particular, SWRouter improves Conv-ID Context by 10.10% on PreAlgebra and 4.28% on MBPP. Although Conv-ID Context performs better on C-EVAL, SWRouter exhibits stronger average robustness across heterogeneous OOD tasks. This result indicates that SWRouter does not simply fit ID-based routing labels. Instead, the learned selection strategy captures transferable routing patterns that generalize beyond the original dialogue distribution. 6.4. Performance Breakdown Bar chart showing accuracy drops from full SWRouter after removing similarity-window construction, sample-LLM loss, or sample-sample loss. Figure 6. Ablation study results measured by evaluation accuracy (%) of SWRouter . Each bar reports the accuracy drop relative to the full SWRouter model after removing one component. The full model obtains 27.98% on ShareGPT and 53.79% on MTBench. This drop-oriented view highlights that similarity-window construction is the dominant contributor, especially on MTBench, while both contrastive losses also provide consistent gains.Bar chart showing accuracy drops from full SWRouter after removing similarity-window construction, sample-LLM loss, or sample-sample loss. We conduct ablation studies to quantify the contribution of each component in SWRouter . As shown in Figure 6, removing any component consistently degrades performance, showing that all components are necessary for effective routing. Similarity window. Removing the similarity window causes the largest performance drop. Evaluation Accuracy decreases by 11.33% on ShareGPT and 46.57% on MTBench. This confirms that adaptive context construction is the most important component of SWRouter . Sample-LLM loss. Removing Lsample-LLML_sample-LLM reduces performance to 25.70% on ShareGPT and 38.17% on MTBench. This indicates that modeling the relationship between dialogue samples and candidate LLMs is critical for accurate routing. Sample-sample loss. Removing Lsample-sampleL_sample-sample also significantly hurts performance, especially on MTBench. This shows that sample-level representation alignment helps construct a more structured routing space. Actual score. We further observe that proxy routing scores can substantially overestimate the final end-to-end performance if prompt construction is ignored. This highlights the necessity of evaluating actual generated responses rather than relying only on intermediate routing objectives. 6.5. Sensitivity Analysis (a) Router performance ProuterP_router. (b) Average value of si¯ si. Figure 7. Effect of the similarity threshold τ on router performance ProuterP_router and the average value of si¯ si. The left subfigure shows how strongly the router improves over the average candidate-model score under each threshold, while the right subfigure reports the resulting end-to-end routed response quality. Together, the curves show that threshold selection affects both model-selection effectiveness and absolute response quality, with the best operating region concentrated around τ=0.91τ=0.91.Two line charts showing how the similarity threshold affects router performance and average evaluation accuracy. (a) MT-Bench. (b) ShareGPT. Figure 8. Effect of the similarity threshold τ on the average value of w¯ w for MT-Bench and ShareGPT. Since w¯ w averages the true scores of all candidate LLMs under the constructed prompts, these curves isolate the prompt-construction quality from the router’s model-selection behavior. The comparison shows how stricter or looser window partitioning changes the amount of useful context retained for the candidate model pool.Two line charts showing how the similarity threshold affects construction accuracy on MT-Bench and ShareGPT. We analyze the sensitivity of SWRouter to the similarity threshold τ. Effect of Similarity Threshold τ.We conduct a sensitivity study on the similarity threshold τ with respect to three key performance metrics: the average value of si¯ si, the average value of w¯ w, and router performance ProuterP_router. Average Value of si¯ si (si¯=1n∑i=1nsi si= 1n _i=1^ns_i). Figure 7(b) shows the effect of τ on si¯ si. The similarity-window construction achieves its best performance at τ=0.91τ=0.91, reaching 40.89% accuracy and outperforming the Conv-ID Context baseline of 32.67%. When τ moves away from this value, the performance decreases noticeably. For example, the accuracy drops to 25.6% at τ=0.85τ=0.85, 22.2% at τ=0.90τ=0.90, 20.2% at τ=0.92τ=0.92, and 18.3% at τ=0.95τ=0.95. These results indicate that si¯ si is highly sensitive to the similarity threshold, and that τ=0.91τ=0.91 provides the most effective similarity window. Average Value of w¯ w. Figure 8 presents the average value of w¯ w under different similarity thresholds τ on MT-Bench and ShareGPT. The results show that w¯ w is strongly affected by the similarity threshold. On MT-Bench, w¯ w increases from 9.60% at τ=0.85τ=0.85 to 12.10% at τ=0.87τ=0.87, and further reaches the maximum value of 23.70% at τ=0.91τ=0.91. However, when the threshold is increased to τ=0.95τ=0.95, w¯ w drops sharply to 13.70%, suggesting that an overly strict similarity constraint can substantially reduce the retained high-quality samples. A similar trend can also be observed on ShareGPT. The average value of w¯ w is 6.00% at τ=0.85τ=0.85, increases significantly to 14.90% at τ=0.87τ=0.87, and remains at a comparable level of 14.70% when τ=0.91τ=0.91. It then decreases to 12.80% at τ=0.95τ=0.95. These results suggest that ShareGPT is slightly more stable than MT-Bench around the optimal region, but its performance is still affected by threshold selection. Overall, the high-quality region is mainly concentrated around τ=0.91τ=0.91. MT-Bench achieves its best result at τ=0.91τ=0.91, while ShareGPT maintains competitive performance in the range of τ∈[0.87,0.91]τ∈[0.87,0.91]. Considering both datasets, τ=0.91τ=0.91 provides a stable trade-off between sample quality and coverage, and we therefore adopt it as the default threshold. Router Performance (ProuterP_router). Figure 7(a) illustrates the effect of the similarity threshold τ on router performance ProuterP_router. Overall, ProuterP_router is influenced by the choice of τ, although its variation is less pronounced than that of w¯ w. Specifically, ProuterP_router is 15.4% at τ=0.87τ=0.87, increases to its peak of 21.5% at τ=0.91τ=0.91, and then decreases to 14.7% at τ=0.92τ=0.92 and 12.1% at τ=0.95τ=0.95. These results indicate that router performance still depends on the similarity threshold, but the overall fluctuation is relatively moderate. Compared with w¯ w, ProuterP_router is less sensitive to changes in τ. Nevertheless, the best router performance is still achieved at τ=0.91τ=0.91, further supporting our choice of τ=0.91τ=0.91 as the default threshold. 6.6. Robustness Analysis 6.6.1. Extension: Top-K Retrieval for Long-Range Dependency Recovery As an extension to the core similarity-based windowing approach, we investigate whether augmenting it with a lightweight Top-K retrieval mechanism can further recover semantically related queries that are separated by unrelated turns. We simulate a scenario where a user issues a query a, then several unrelated queries, and finally a related query b. For each b, similarity-based Top-K retrieval is performed over the dialogue history to test whether a can be recalled. Table 2. Recall performance of similarity-based retrieval under different levels of semantic interference. Inserted queries denote unrelated turns placed between two semantically related user requests; Top-K recall measures whether the retrieval module can recover the earlier related turn from the dialogue history. Inserted Queries Top-1 Recall Top-3 Recall Top-5 Recall 1 79.43 100.00 100.00 3 65.16 91.31 100.00 5 59.84 77.84 94.15 We also test a more challenging case where 20 unrelated queries are inserted between a and b. The results are shown below. Table 3. Retrieval robustness under the high-difficulty setting with 20 distractor turns. This setting stresses long-range dependency recovery by separating two related requests with many unrelated turns, and the reported recall indicates whether Top-K retrieval can still recover the relevant earlier query. Inserted Queries Top-K Recall (%) Difficulty 20 5 66.13 Hard As shown in Tables 2 and 3, similarity-based retrieval substantially improves robustness under semantic drift. Top-3 retrieval provides an effective trade-off between recall and efficiency. Even under high-difficulty conditions with 20 distractors, Top-5 recall remains above 66%, indicating strong resilience. These results demonstrate that augmenting local similarity-based segmentation with retrieval mechanisms can further enhance context reconstruction in realistic multi-turn dialogue scenarios. 6.6.2. Semantic Coherence in Long Dialogues To further analyze the potential semantic drift as the dialogue length increases, we conducted a similarity analysis over multi-turn dialogues on the ShareGPT and MT-Bench datasets. Specifically, we computed the cosine similarity between the first and last user turns within each dialogue window under different minimum round sizes (20, 25, and 30). Table 4. Cosine similarity between the first and last user turns across dialogues of varying lengths. For each dataset, we group dialogues by minimum round size and report the proportion of long-range turn pairs above or below the default similarity threshold τ=0.91τ=0.91, together with the maximum, minimum, and average similarity. The table characterizes how quickly semantic drift appears as dialogue length increases. Dataset Min Group Size Total Pairs ≥ 0.91 (%) <<0.91 (%) Max Min Avg ShareGPT 20 19 73.68 26.32 0.9922 0.6774 0.9283 25 13 76.92 23.08 0.9922 0.6774 0.9211 30 7 71.43 28.57 0.9874 0.8035 0.9247 MT-Bench 20 7 57.14 42.86 0.9620 0.7961 0.8989 25 6 50.00 50.00 0.9620 0.7961 0.8928 30 4 25.00 75.00 0.9426 0.7961 0.8603 As shown in Table 4, in ShareGPT, over 70% of long-range pairs maintain a cosine similarity above 0.91, and the average similarity remains above 0.92, suggesting strong semantic consistency even across extended dialogues. In MT-Bench, the average similarity is slightly lower but still indicates limited semantic drift. Extremely low similarity cases (<<0.80) are rare, implying that our similarity-based segmentation remains semantically coherent over long dialogue windows. 6.7. Detailed Overhead Analysis We conduct a comprehensive analysis of the computational overhead introduced by SWRouter from three aspects: inference latency, training cost, and token generation cost. 6.7.1. Latency Analysis Table 5. Time consumption of SWRouter during inference. The table decomposes latency into similarity calculation for window construction, router selection, and downstream LLM inference, showing that the additional cost introduced by similarity-window routing is small relative to candidate-model generation. Time Similarity Calculation Router Selection LLM Inference SWRouter 50.6s 16min32s >>6h As shown in Table 5, the similarity calculation in SWRouter accounts for less than 1% (approximately 2.2‰) of the total inference time, indicating high computational efficiency. The dominant cost comes from LLM inference, which is consistent with all routing-based systems. 6.7.2. Training Cost Analysis The training process of SWRouter consists of four steps: (1) Dataset window partitioning: We use a lightweight encoder (microsoft/mdeberta-v3-base) with approximately 86M backbone parameters, which is much smaller than the 7B/8B candidate LLMs, making this step’s cost negligible. (2) Candidate response generation: We use the seven small models mentioned in the paper (Mistral-7B, MetaMath-Mistral-7B, zephyr-7b-beta, Chinese-Mistral-7B, dolphin-2.6-mistral-7b, Meta-Llama-3-8B, dolphin-2.9-llama3-8b) to generate candidate answers for each query. (3) Judge-model scoring: We use GPT for scoring. Since the output is just a single numerical score, this step does not significantly increase memory usage. (4) Router training: The similarity-window encoder introduces limited overhead: microsoft/mdeberta-v3-base has approximately 86M backbone parameters, which is much smaller than the 7B/8B candidate LLMs. Moreover, the encoder is used only for lightweight similarity computation, while the dominant cost remains candidate LLM inference. Router training is performed only once before deployment, making it a one-time overhead. 6.7.3. Token Generation Cost To evaluate the efficiency of model selection, we compare the token-level costs between SWRouter and the Conv-ID Context method. Due to the unavailability of precise pricing information for small-sized models such as 7B and 8B, we adopt token count as a unified proxy for cost estimation. Table 6. Overall evaluation accuracy and token cost comparison. Evaluation accuracy measures response quality after routing, while token cost counts the generated tokens used by each routing strategy as a model-agnostic proxy for serving cost. The comparison highlights the effectiveness–cost tradeoff between Conv-ID Context, SimWindow + ZOOTER, and SWRouter . Router Evaluation Acc. ↑ Token Cost ↓ Conv-ID Context 32.67 2834 SimWindow + ZOOTER 39.20 3405 SWRouter 40.89 3477 As shown in Table 6, SWRouter incurs approximately 1.23×1.23× the token cost of Conv-ID Context while achieving an 8.22%8.22\% absolute improvement in evaluation accuracy. Compared to ZOOTER, SWRouter requires only 1.02×1.02× the token cost, yet delivers a further 1.69%1.69\% accuracy gain. This result suggests that SWRouter achieves a reasonable trade-off between performance and efficiency, despite lacking access to ground-truth optimal selections. 6.7.4. Additional Analysis We further analyze the experimental results from the following four aspects: (1) Prompt Construction Impact. The similar performance patterns across models at different similarity thresholds suggest that prompt construction, rather than model architecture, is the primary performance driver. τ=0.91τ=0.91 performs best among tested thresholds; performance is relatively strong in the neighborhood around 0.91. (2) Router-Score Correlation. Analysis reveals a weak negative correlation (Pearson R = -0.2301) between router performance and true scores, indicating that higher true scores do not necessarily translate to better routing performance. This underscores the importance of decoupled evaluation metrics. (3) Prompt Construction Dominance. τ=0.91τ=0.91 performs best among tested thresholds; performance is relatively strong in the neighborhood around 0.91, emphasizing that prompt construction quality outweighs router performance in multi-turn dialogue systems. (4) Exact Partitioning is Not Necessarily Optimal. This can be compared from three aspects: (i) On Evaluation Accuracy (si¯ si), when τ=0.91τ=0.91 with 40.89%, it is 8.22% higher than Conv-ID Context (32.67%), proving that the router model trained with exact partitioning does not perform as well as SWRouter . (i) On Construction Accuracy (w¯ w), when the similarity threshold τ is 0.90, the average score of 7 individual models significantly outperforms the prompt constructed by Conv-ID Context, indicating that in multi-turn dialogue scenarios, strictly accurate context partitioning cannot be considered the optimal context segmentation method. (i) On Router Performance (ProuterP_router), the average performance for both τ=0.85τ=0.85 and τ=0.91τ=0.91 surpass Conv-ID Context, meaning that the prompt constructed by Conv-ID Context also fails to train the optimal router performance. Overall, SWRouter introduces limited additional computational overhead while substantially improving routing performance, demonstrating a favorable effectiveness–efficiency tradeoff for multi-turn LLM routing. 7. Related Work Large language models have become increasingly heterogeneous in capability, cost, latency, and domain specialization. This heterogeneity has motivated systems that use multiple LLMs rather than committing to a single model for all inputs. Existing work can be broadly grouped into LLM ensembling and cascading, single-turn LLM routing, and context-aware routing for multi-turn dialogue. 7.1. LLM Ensembling and Cascading LLM ensembling aims to improve response quality by consulting multiple models or multiple generations before producing the final answer. A simple and widely used strategy is voting or self-consistency, where multiple reasoning paths or model outputs are aggregated to reduce variance and improve reliability (Li et al., 2024b; Wang et al., 2023). More structured ensemble methods go beyond voting: LLM-Blender (Jiang et al., 2023b) first ranks candidate outputs with PairRanker and then synthesizes a final response with GenFuser, showing that complementary model outputs can be combined into a stronger answer. Another line of work focuses on cascades, where models are queried sequentially according to estimated difficulty, confidence, or cost. FrugalGPT (Chen et al., 2023) studies cost-effective model selection through cascaded calls, while language-model cascades further explore uncertainty-aware and token-level routing policies (Gupta et al., 2024; Yue et al., 2024). Online cascade learning extends this idea to streaming or adaptive inference settings (Nie et al., 2024). These methods reduce cost compared with invoking all available models, but they may still require multiple model calls for a single user request. In contrast, SWRouter follows the routing paradigm: it aims to choose one suitable candidate LLM after constructing the multi-turn prompt, thereby avoiding repeated generation from many models at inference time. 7.2. Single-turn LLM Routing LLM routing selects the most suitable model for a query without necessarily calling all candidate LLMs. Early routing studies often formulate the problem as correctness prediction or reward prediction for each candidate model. Shnitzer et al. (Shnitzer et al., 2023) construct benchmark datasets for LLM routing and train model-specific binary classifiers to estimate whether each model will answer correctly. ZOOTER (Lu et al., 2024) aligns a router with reward-model supervision and shows that learned routers can outperform static model selection. Cost-aware routing methods such as C2MAB-V (Dai et al., 2024) further incorporate online decision making and reward signals to balance model quality and inference cost. Recent work also studies representation-based routing. LoraRetriever (Zhao et al., 2024) routes inputs by predicting task identity and retrieving suitable LoRA modules, while Srivatsa et al. (Srivatsa et al., 2024) analyze classifier-based and clustering-based routing strategies. RouterDC (Chen et al., 2024) introduces a dual contrastive learning objective that models both query-model relations and query-query relations, making it a strong single-turn router backbone. These methods provide useful supervision and representation-learning tools for model selection, and SWRouter builds on this direction. However, they generally assume that each routing instance is already a complete prompt. This assumption is reasonable for single-turn benchmarks, but it becomes fragile in multi-turn dialogue because the routing input itself depends on how historical context is selected and assembled. 7.3. Context Construction for Multi-turn Dialogue Multi-turn dialogue introduces a context-construction problem before routing can even begin. A conversational request may depend on previous constraints, definitions, or preferences, but the full dialogue history may also contain obsolete or off-topic content. Directly concatenating all previous turns can therefore increase prompt length and introduce irrelevant information, whereas using only the current turn can omit essential context. Long-context benchmarks such as MT-Bench-101 (Bai et al., 2024) and real dialogue data such as ShareGPT (Schmid, 2023) highlight that multi-turn tasks often require tracking user intent across turns rather than treating each prompt independently. Several adjacent research areas offer partial solutions. Retrieval-based context construction can recover long-range dependencies by searching dialogue history for semantically related turns. Sequence segmentation and similarity-based partitioning, rooted in broader sequence analysis and string-processing ideas (Aho and Ullman, 1972; Gusfield, 1997; Chandra et al., 1981), provide a way to divide a conversation into locally coherent windows. Neural encoders such as DeBERTa (He et al., 2021) can then map turns into dense representations for semantic matching. Nevertheless, these techniques are usually studied as prompt-construction or retrieval components rather than as part of a routing evaluation framework. SWRouter integrates similarity-based window construction with router learning so that the selected model is conditioned on a prompt that is explicitly designed for multi-turn relevance. 7.4. Evaluation of Routing Under Constructed Prompts Most routing evaluations focus on whether the router selects a high-scoring model for a given input. This is appropriate when the input prompt is fixed and complete, but it can be misleading when prompt construction varies. In multi-turn dialogue, a poorly constructed prompt can lower the absolute scores of all candidate LLMs even if their relative ranking remains unchanged. Conversely, a better prompt can improve all candidate responses while leaving router-selection difficulty similar. Standard aggregate metrics such as benchmark accuracy (Hendrycks et al., 2021; Li et al., 2024a; Cobbe et al., 2021) or reward-model score therefore conflate two factors: whether the constructed prompt preserves enough information, and whether the router chooses the best model under that prompt. The key difference between SWRouter and prior routing work is this explicit separation. Instead of evaluating only the selected model’s final score, SWRouter reports evaluation accuracy, construction accuracy, and router performance. This decoupled view makes it possible to determine whether improvements come from better prompt construction, better model selection, or both. As a result, SWRouter extends single-turn LLM routing to a setting where context segmentation is a first-class part of the routing problem rather than a preprocessing detail. 8. Conclusion This paper presents SWRouter , a similarity-contractive window router that addresses two fundamental challenges in multi-turn LLM routing: information loss and information confusion caused by improper context segmentation, and attribution bias in router evaluation that conflates prompt construction quality with routing effectiveness. To tackle these challenges, SWRouter introduces a similarity-based window partitioning mechanism that dynamically constructs semantically coherent prompts from dialogue history, and a decoupled evaluation framework comprising three complementary metrics: evaluation accuracy (si¯ si), construction accuracy (w¯ w), and router performance (ProuterP_router). Extensive experiments on ShareGPT and MTBench demonstrate that SWRouter achieves 16.26% and 8.22% gains over the best individual LLM and the Conv-ID Context baseline, respectively. On out-of-distribution tasks (PreAlgebra, MBPP, and C-EVAL), SWRouter further outperforms Conv-ID Context by 2.68% on average, demonstrating strong generalization. Decoupled metric analysis confirms that the gains are primarily driven by improved construction accuracy, while Prouter>1P_router>1 consistently validates effective model selection. These results highlight that multi-turn LLM routing requires a joint design of context construction and evaluation, rather than a direct extension of single-turn routing methods. Despite these advances, the current framework is evaluated on 7B/8B scale models; its behavior under larger or proprietary LLMs remains to be studied. Future work may extend the routing framework to heterogeneous model pools spanning different scales and explore its applicability to proprietary LLMs. References Aho and Ullman (1972) A. V. Aho and J. D. Ullman The theory of parsing, translation, and compiling. Vol. 1, Prentice-Hall, Englewood Cliffs, NJ. Cited by: §4.2, §7.3. American Psychological Association (1983) American Psychological Association Publications manual. American Psychological Association, Washington, DC. Cited by: §6.1.1. Ando and Zhang (2005) R. K. Ando and T. Zhang A framework for learning predictive structures from multiple tasks and unlabeled data. Journal of Machine Learning Research 6, p. 1817–1853. External Links: Link Cited by: §4.3.2. Andrew and Gao (2007) G. Andrew and J. Gao Scalable training of L1-regularized log-linear models. In Proceedings of the 24th International Conference on Machine Learning, Corvallis, OR, USA, p. 33–40. External Links: Document Cited by: §4.3.2. Anthropic (2024) Anthropic Claude. Note: https://docs.anthropic.com/en/docs/welcome Cited by: §1. Bai et al. (2024) G. Bai, J. Liu, X. Bu, Y. He, J. Liu, Z. Zhou, Z. Lin, W. Su, T. Ge, B. Zheng, and W. Ouyang MT-bench-101: A fine-grained benchmark for evaluating large language models in multi-turn dialogues. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Bangkok, Thailand, p. 7421–7454. External Links: Document, Link Cited by: §5, §7.3. Chandra et al. (1981) A. K. Chandra, D. Kozen, and L. J. Stockmeyer Alternation. Journal of the ACM 28 (1), p. 114–133. External Links: Document Cited by: §4.2, §7.3. Chen et al. (2023) L. Chen, M. Zaharia, and J. Zou FrugalGPT: how to use large language models while reducing cost and improving performance. CoRR abs/2305.05176. External Links: Link, Document, 2305.05176 Cited by: §7.1. Chen et al. (2024) S. Chen, W. Jiang, B. Lin, J. T. Kwok, and Y. Zhang RouterDC: query-based router by dual contrastive learning for assembling large language models. In Advances in Neural Information Processing Systems, Vol. 37, Vancouver, BC, Canada, p. 66305–66328. External Links: Link Cited by: §1, §2.2, §2.2, §4.3, §7.2. Chen et al. (2025) X. Chen, Y. Li, H. Cai, Z. Ma, X. Chen, H. Xiong, S. Wang, B. He, L. Sun, and D. Yin Multi-agent proactive information seeking with adaptive llm orchestration for non-factoid question answering. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, p. 4341–4352. Cited by: §1. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. CoRR abs/2110.14168. External Links: Link, 2110.14168 Cited by: §6.3, §7.4. Cognitive Computations (2024a) Cognitive Computations Cognitivecomputations/dolphin-2.6-mistral-7b. Note: https://huggingface.co/cognitivecomputations/dolphin-2.6-mistral-7b Cited by: §5. Cognitive Computations (2024b) Cognitive Computations Cognitivecomputations/dolphin-2.9-llama3-8b. Note: https://huggingface.co/cognitivecomputations/dolphin-2.9-llama3-8b Cited by: §5. Dai et al. (2024) X. Dai, J. Li, X. Liu, A. Yu, and J. C. S. Lui Cost-effective online multi-LLM selection with versatile reward models. CoRR abs/2405.16587. External Links: Link, Document, 2405.16587 Cited by: §1, §7.2. DeepSeek-AI (2024) DeepSeek-AI DeepSeek-V3 technical report. CoRR abs/2412.19437. External Links: Link, Document, 2412.19437 Cited by: §1. Gao et al. (2023) L. Gao, J. Tow, B. Abbasi, S. Biderman, S. Black, A. DiPofi, C. Foster, L. Golding, J. Hsu, A. L. Noac’h, H. Li, K. McDonell, N. Muennighoff, C. Ociepa, J. Phang, L. Reynolds, H. Schoelkopf, A. Skowron, L. Sutawika, E. Tang, A. Thite, B. Wang, K. Wang, and A. Zou A framework for few-shot language model evaluation. Note: https://zenodo.org/records/10256836 External Links: Document Cited by: §5. Gemini Team (2023) Gemini Team Gemini: A family of highly capable multimodal models. CoRR abs/2312.11805. External Links: Link, Document, 2312.11805 Cited by: §1. Gupta et al. (2024) N. Gupta, H. Narasimhan, W. Jitkrittum, A. S. Rawat, A. K. Menon, and S. Kumar Language model cascades: token-level uncertainty and beyond. CoRR abs/2404.10136. External Links: Link, Document, 2404.10136 Cited by: §7.1. Gusfield (1997) D. Gusfield Algorithms on strings, trees, and sequences: computer science and computational biology. Cambridge University Press, Cambridge, UK. External Links: Document Cited by: §4.2, §7.3. He et al. (2021) P. He, X. Liu, J. Gao, and W. Chen DeBERTa: decoding-enhanced BERT with disentangled attention. In The Ninth International Conference on Learning Representations, Virtual Event, Austria. External Links: Link Cited by: §5, §7.3. Hendrycks et al. (2021) D. Hendrycks, C. Burns, S. Basart, A. Zou, M. Mazeika, D. Song, and J. Steinhardt Measuring massive multitask language understanding. In The Ninth International Conference on Learning Representations, Virtual Event, Austria. External Links: Link Cited by: §5, §7.4. Jiang et al. (2023a) A. Q. Jiang, A. Sablayrolles, A. Mensch, C. Bamford, D. S. Chaplot, D. de Las Casas, F. Bressand, G. Lengyel, G. Lample, L. Saulnier, L. R. Lavaud, M. Lachaux, P. Stock, T. L. Scao, T. Lavril, T. Wang, T. Lacroix, and W. E. Sayed Mistral 7b. CoRR abs/2310.06825. External Links: Link, Document, 2310.06825 Cited by: §5. Jiang et al. (2023b) D. Jiang, X. Ren, and B. Y. Lin LLM-Blender: ensembling large language models with pairwise ranking and generative fusion. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics, Toronto, Canada, p. 14165–14178. External Links: Document, Link Cited by: §7.1. Kullback and Leibler (1951) S. Kullback and R. A. Leibler On information and sufficiency. The Annals of Mathematical Statistics 22 (1), p. 79–86. External Links: Document Cited by: §4.3. Li et al. (2024a) H. Li, Y. Zhang, F. Koto, Y. Yang, H. Zhao, Y. Gong, N. Duan, and T. Baldwin CMMLU: measuring massive multitask language understanding in chinese. In Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, p. 11260–11285. External Links: Document, Link Cited by: §6.3, §7.4. Li et al. (2024b) J. Li, Q. Zhang, Y. Yu, Q. Fu, and D. Ye More agents is all you need. CoRR abs/2402.05120. External Links: Link, Document, 2402.05120 Cited by: §7.1. Li et al. (2025) Y. Li, H. Cai, R. Kong, X. Chen, J. Chen, J. Yang, H. Zhang, J. Li, J. Wu, Y. Chen, et al. Towards ai search paradigm. arXiv preprint arXiv:2506.17188. Cited by: §1. Li et al. (2026a) Y. Li, J. Chen, X. Chen, Z. Li, H. Zhang, R. Kong, J. Li, X. Ma, H. Cai, L. Su, et al. Retain to refine: adaptive online question answering via query routing and long-short memory. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, p. 2312–2322. Cited by: §1. Li et al. (2026b) Y. Li, R. Kong, X. Chen, C. Zhang, J. Chen, C. Deng, X. Ma, H. Zhang, T. Peng, H. Cai, et al. Probe-and-fetch: dynamic kv cache pruning for accelerated long-context inference in web-scale ai search. In Proceedings of the ACM Web Conference 2026, p. 8127–8137. Cited by: §1. Li et al. (2026c) Y. Li, R. Kong, Z. Lyu, Q. Li, X. Chen, H. Cai, L. Yan, S. Wang, J. Zhao, G. Zhu, et al. Flexspec: frozen drafts meet evolving targets in edge-cloud collaborative llm speculative decoding. IEEE Transactions on Mobile Computing. Cited by: §1. Llama Team (2024) Llama Team The llama 3 herd of models. CoRR abs/2407.21783. External Links: Link, Document, 2407.21783 Cited by: §5. Loshchilov and Hutter (2019) I. Loshchilov and F. Hutter Decoupled weight decay regularization. In International Conference on Learning Representations, New Orleans, LA, USA. External Links: Link Cited by: §5. Lu et al. (2024) K. Lu, H. Yuan, R. Lin, J. Lin, Z. Yuan, C. Zhou, and J. Zhou Routing to the expert: efficient reward-guided ensemble of large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Mexico City, Mexico, p. 1964–1974. External Links: Document, Link Cited by: §4.3, §5, §7.2. Nie et al. (2024) L. Nie, Z. Ding, E. Hu, C. M. Jermaine, and S. Chaudhuri Online cascade learning for efficient inference over streams. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, Vienna, Austria, p. 38071–38090. External Links: Link Cited by: §7.1. OpenAI (2022) OpenAI ChatGPT: optimizing language models for dialogue. Note: https://openai.com/blog/chatgpt Cited by: §1. Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn Direct preference optimization: your language model is secretly a reward model. In Advances in Neural Information Processing Systems, Vol. 36, New Orleans, LA, USA, p. 53728–53741. External Links: Link Cited by: §5. Rasooli and Tetreault (2015) M. S. Rasooli and J. R. Tetreault Yara parser: A fast and accurate dependency parser. CoRR abs/1503.06733. External Links: Link, 1503.06733 Cited by: §5. Schmid (2023) P. Schmid ShareGPT 90k raw dataset. Note: https://huggingface.co/datasets/philschmid/sharegpt-raw/tree/main/sharegpt_90k_raw_dataset Cited by: §5, §7.3. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. CoRR abs/2402.03300. External Links: Link, Document, 2402.03300 Cited by: §1. Shnitzer et al. (2023) T. Shnitzer, A. Ou, M. Silva, K. Soule, Y. Sun, J. Solomon, N. Thompson, and M. Yurochkin Large language model routing with benchmark datasets. CoRR abs/2309.15789. External Links: Link, Document, 2309.15789 Cited by: §7.2. Srivatsa et al. (2024) K. A. Srivatsa, K. K. Maurya, and E. Kochmar Harnessing the power of multiple minds: lessons learned from LLM routing. CoRR abs/2405.00467. External Links: Link, Document, 2405.00467 Cited by: §7.2. Touvron et al. (2023a) H. Touvron, T. Lavril, G. Izacard, X. Martinet, M. Lachaux, T. Lacroix, B. Rozière, N. Goyal, E. Hambro, F. Azhar, A. Rodriguez, A. Joulin, E. Grave, and G. Lample LLaMA: open and efficient foundation language models. CoRR abs/2302.13971. External Links: Link, Document, 2302.13971 Cited by: §1, §5. Touvron et al. (2023b) H. Touvron, L. Martin, K. Stone, P. Albert, A. Almahairi, Y. Babaei, N. Bashlykov, S. Batra, P. Bhargava, S. Bhosale, D. Bikel, L. Blecher, C. Canton-Ferrer, M. Chen, G. Cucurull, D. Esiobu, J. Fernandes, J. Fu, W. Fu, B. Fuller, C. Gao, V. Goswami, N. Goyal, A. Hartshorn, S. Hosseini, R. Hou, H. Inan, M. Kardas, V. Kerkez, M. Khabsa, I. Kloumann, A. Korenev, P. S. Koura, M. Lachaux, T. Lavril, J. Lee, D. Liskovich, Y. Lu, Y. Mao, X. Martinet, T. Mihaylov, P. Mishra, I. Molybog, Y. Nie, A. Poulton, J. Reizenstein, R. Rungta, K. Saladi, A. Schelten, R. Silva, E. M. Smith, R. Subramanian, X. E. Tan, B. Tang, R. Taylor, A. Williams, J. X. Kuan, P. Xu, Z. Yan, I. Zarov, Y. Zhang, A. Fan, M. Kambadur, S. Narang, A. Rodriguez, R. Stojnic, S. Edunov, and T. Scialom Llama 2: open foundation and fine-tuned chat models. CoRR abs/2307.09288. External Links: Link, Document, 2307.09288 Cited by: §5. Tunstall et al. (2023) L. Tunstall, E. Beeching, N. Lambert, N. Rajani, K. Rasul, Y. Belkada, S. Huang, L. von Werra, C. Fourrier, N. Habib, N. Sarrazin, O. Sanseviero, A. M. Rush, and T. Wolf Zephyr: direct distillation of LM alignment. CoRR abs/2310.16944. External Links: Link, Document, 2310.16944 Cited by: §5. Wang et al. (2023) X. Wang, J. Wei, D. Schuurmans, Q. V. Le, E. H. Chi, S. Narang, A. Chowdhery, and D. Zhou Self-consistency improves chain of thought reasoning in language models. In The Eleventh International Conference on Learning Representations, Kigali, Rwanda. External Links: Link Cited by: §7.1. Yu et al. (2024) L. Yu, W. Jiang, H. Shi, J. Yu, Z. Liu, Y. Zhang, J. T. Kwok, Z. Li, A. Weller, and W. Liu MetaMath: bootstrap your own mathematical questions for large language models. In The Twelfth International Conference on Learning Representations, Vienna, Austria. External Links: Link Cited by: §5. Yue et al. (2024) M. Yue, J. Zhao, M. Zhang, L. Du, and Z. Yao Large language model cascades with mixture of thought representations for cost-efficient reasoning. In The Twelfth International Conference on Learning Representations, Vienna, Austria. External Links: Link Cited by: §7.1. Zhao et al. (2024) Z. Zhao, L. Gan, G. Wang, W. Zhou, H. Yang, K. Kuang, and F. Wu LoraRetriever: input-aware LoRA retrieval and composition for mixed tasks in the wild. In Findings of the Association for Computational Linguistics: ACL 2024, Bangkok, Thailand, p. 4447–4462. External Links: Document, Link Cited by: §7.2. Zhou and Bai (2024) C. Zhou and Y. Bai Chinese-Mistral: an efficient and effective chinese large language model. Note: https://github.com/THU-ESIS/Chinese-Mistral Cited by: §5.

Instant research discovery