Paper deep dive

Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection

Yichi Zhang, Zhenxiang Huang, Yuyang Zhang, Yuchen Yang

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 49

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language model (LLM) routers reduce serving cost by choosing which model to call and how many output tokens to grant it for each query. Existing routers rank model-budget actions using a single quality metric. This does not work for mixed workloads. For ordinary queries, a good answer is correct and useful (utility); for security-sensitive queries, a good answer refuses harmful requests or resists malicious instructions (security). These two objectives prefer different models and budgets. We propose SCRouter, a router that learns when to prioritize security and how to allocate the token budget. For each model-budget pair, it estimates the probability that the response finishes within budget. It uses this probability to predict utility and security separately. A small probe determines how to weight these two predictions for each query. SCRouter then selects the model and budget by balancing the weighted utility and security against cost. We also build SCRouterBench, a benchmark of 38,869 utility and security queries spanning nine security dimensions, each answered under all 144 combinations of 12 models and 12 budgets, plus 2,753 out-of-domain queries. SCRouter improves the baselines by 13-29 points in-domain and 19-23 points out-of-domain.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

46,058 characters extracted from source content.

Expand or collapse full text

Rethinking Quality in Large Language Model Routing: Security-Conditioned Model-and-Budget Selection Yichi Zhang 1 , Zhenxiang Huang 2 , Yuyang Zhang 1 , Yuchen Yang 1 1 The Pennsylvania State University, 2 The University of Hong Kong Abstract Large language model (LLM) routers reduce serving cost by choosing which model to call and how many output tokens to grant it for each query. Existing routers rank model-budget ac- tions using a single quality metric. This does not work for mixed workloads. For ordinary queries, a good answer is correct and useful (utility); for security-sensitive queries, a good answer refuses harmful requests or resists mali- cious instructions (security). These two objec- tives prefer different models and budgets. We propose SCRouter, a router that learns when to prioritize security and how to allocate the to- ken budget. For each model-budget pair, it esti- mates the probability that the response finishes within budget. It uses this probability to predict utility and security separately. A small probe determines how to weight these two predic- tions for each query. SCRouter then selects the model and budget by balancing the weighted utility and security against cost. We also build SCRouterBench, a benchmark of 38,869 util- ity and security queries spanning nine security dimensions, each answered under all 144 com- binations of 12 models and 12 budgets, plus 2,753 out-of-domain queries. SCRouter im- proves the baselines by 13-29 points in-domain and 19-23 points out-of-domain. 1 Introduction LLM routers are widely used in modern LLM in- frastructure (Yu et al., 2022; Kwon et al., 2023; Lu et al., 2026). They reduce serving cost by selecting a suitable model configuration for each query. Most existing routers use a model-only action space: the router selects only which model should answer each query, as in RouterBench (Hu et al., 2024), RouteLLM (Ong et al., 2025), CARROT (Somer- step et al., 2025), and UniRoute (Jitkrittum et al., 2026). R2-Router (Xue et al., 2026) extends the ac- tion space to model-budget pairs: it picks both the model and its output token budget for each query. In-Domain Out-of-Domain 0 15 30 45 60 Oracle Gap Recovered 60.9 50.2 SCRouter-L R2-Router CARROT-RoBERTa SCRouter-M RADAR NIRT SCRouter-S CARROT-KNN MIRT Figure 1: Overall routing performance under in-domain and out-of-domain settings. SCRouter achieves the high- est Oracle Gap Recovery (OGR) across both settings. OGR denotes the fraction of the gap between random selection and the oracle that a router recovers. These methods improve routing performance by refining the action space, but they overlook that what makes a response good depends on the query. For ordinary queries, a good response correctly ad- dresses the request, which we refer to as utility. For harmful requests or queries containing malicious instructions, a good response refuses the harmful request or resists the malicious instructions, which we refer to as security. We call these utility and security queries, respectively. The problem is that utility and security often favor different routing actions, both in model selec- tion and token budget allocation. Figure 2 ranks 12 LLMs at their best budgets and shows that their util- ity and security rankings differ substantially. For example, Foundation-Sec-8B ranks 1st in security but 9th in utility, while Qwen3.5-27B ranks 4th in utility but 9th in security. Figure 3 shows the same mismatch across token budgets. For Qwen3- Coder-30B, utility rises from 10.4% to 79.4% as the budget grows to about 4,000 tokens. In contrast, its security performance peaks at only 120 tokens, where utility remains at 40.5%. These observations show that models and budgets that improve utility 1 arXiv:2610.18566v1 [cs.AI] 13 Oct 2026 Qwen3.5-0.8B Qwen3.5-0.8B Phi-4-miniPhi-4-mini Mistral-7B-v0.3 Mistral-7B-v0.3 DS-R1-Qwen3-8BDS-R1-Qwen3-8B Llama-3.1-8B Llama-3.1-8B Qwen3-Coder-30B Qwen3-Coder-30BLlama-3.1-70B Llama-3.1-70B Gemma-3-1B Gemma-3-1BQwen3.5-4B Qwen3.5-4B Foundation-Sec-8B Foundation-Sec-8B Qwen3.5-9B Qwen3.5-9B Qwen3.5-27B Qwen3.5-27B Utility RankSecurity Rank Figure 2: Model rankings differ between utility and security: strong utility need not imply strong security. need not improve security. A router that ignores this distinction may send security queries to mod- els with strong utility but weak security, or allocate larger budgets without a security benefit. We propose SCRouter, the firstSecurity- Conditioned model-budgetRouter. First, a small probe estimates the probability that a query is security-sensitive. Second, for each model-budget pair, a shared budget model predicts the probability of finishing within budget and the expected token use. Two quality predictors use this completion probability to account for truncated responses when estimating utility and security separately. This lets the two objectives favor different models and budgets while sharing the same completion esti- mate. Third, SCRouter uses the probe’s estimate to weight the two quality scores, then selects the model-budget pair that best balances the combined score against the cost estimated from token use. The predictors share parameters across actions, avoiding a separate predictor for each pair. We also build SCRouterBench, a routing bench- mark with mixed utility and security workloads. It has 38,869 queries, covering utility tasks and nine security dimensions, plus an out-of-domain test set of 2,753 queries. We run all 144 actions per query, yielding about 6 million labeled responses. On SCRouterBench, SCRouter recovers 60.9% of the gap between random selection and the per- query oracle, compared with 48.0% for the best baseline, R2-Router, a gain of 12.9 percentage points. As shown in Figure 1, all three encoder variants of SCRouter beat every baseline, even the smallest one, which has a 4M-parameter encoder and 0.25M trainable parameters. SCRouter-M also leads out-of-domain with 50.2% OGR. Our contributions are as follows: (a) we find a utility-security asymmetry in LLM routing across 101001kdef. Token Budget 0 20 40 60 80 Mean Score Utility Security 79.481.0 Figure 3: Utility and security of Qwen3-Coder-30B under different token budgets. Utility and security peak at different token budgets. both model choice and budget allocation; (b) we propose SCRouter, a security-conditioned model- budget router that adapts its routing objective to each query; (c) we release SCRouterBench to eval- uate model and budget selection on mixed utility and security workloads; and (d) we demonstrate gains over existing routers in both settings. 2 Related Work LLM Routing. LLM routers direct queries to candidate models to balance response quality and inference cost. Cascade routers try a small model first and move to larger ones if the answer is poor (Chen et al., 2024a; Aggarwal et al., 2024). Predictive routers pick a model before generation from query features, preference data, or learned em- beddings (Ding et al., 2024; Ong et al., 2025; Chen et al., 2024b; Feng et al., 2025; Song et al., 2025; Somerstep et al., 2025; Jitkrittum et al., 2026). Benchmarks such as RouterBench (Hu et al., 2024) and RouterArena (Lu et al., 2026) have standard- ized model routing evaluation. R2-Router (Xue et al., 2026) and RADAR (Fernandez et al., 2026) extend routing to choose both the model and the token budget. Unlike these routers, which use one quality metric and ignore security, SCRouter pre- dicts utility and security separately for each model- budget action. We also build SCRouterBench to evaluate routers under mixed workloads. Attacks on LLMs. Despite safety alignment, LLMs remain vulnerable to adversarial inputs such as jailbreaks (Wei et al., 2023; Zou et al., 2023; Chao et al., 2025) and prompt injections (Perez and Ribeiro, 2022; Greshake et al., 2023; Schulhoff et al., 2023). Safety benchmarks (Mazeika et al., 2024; Chao et al., 2024; Wang et al., 2023) show 2 Query \(x\) Frozen Text Encoder 4M-0.6B params \(w(x)\) \(z(x)\)\(z(x)\) \(z(x)\) \(Q\) \(μ\) \(Q_U\)\(Q_S\) \(π(x, a)\) Security Emphasis \(w(x)\) Query Embedding \(z(x)\) Linear Probe Budget Model Utility PredictorSecurity Predictor Surface Fusion \((1-w)Q_U+wQ_S\) Cost-Aware Action Scoring \((1-μ)Q-μ E[cost]\) Cost Preference \(μ\) Action \((m^*, b^*)\) Figure 4: Overview of the SCRouter pipeline. (1) A frozen text encoder and linear probe estimate the query’s security weight w(x). (2) A shared budget model predicts the probability π(x,a) that the response finishes within budget, and two quality models predict the utility and security surfaces ˆ Q U and ˆ Q S for all actions. (3) The two surfaces are combined with weight w(x) and traded off against cost with the cost coefficient μ. that robustness differs a lot across models. Existing defenses make a single model more robust (Inan et al., 2023; Chen et al., 2025; Du et al., 2024) or protect the router itself from attacks (Shafran et al., 2025; Zhang et al., 2026). SCRouter treats security queries as part of the routing workload. It uses dif- ferences in model robustness and budget sensitivity to send security queries to safe configurations and utility queries to high-utility ones. 3 Security-Conditioned Router 3.1 Overview Letxdenote a query andA = M×Bbe the ac- tion space, where each actiona = (m,b)specifies a generation modelmwith an output token bud- getb. For every action,U (x,a) ∈ [0, 1]denotes the utility of the response andS(x,a) ∈ [0, 1]its security. SCRouter predicts the surfacesUandS separately for all actions and combines them with a query-specific security weight. Figure 4 summarizes the pipeline. First, a frozen text encoder represents the query, and a linear probe on this representation estimates a query-specific se- curity weightw(x)(Section 3.2). Second, a shared budget model predicts, for every action, the com- pletion probabilityπ(x,a), which is the probability that the response finishes within its budget. Two quality models then useπto predict the utility and security surfaces ˆ Q U (x,a)and ˆ Q S (x,a)(Sec- tion 3.3). Third, the two quality surfaces are fused with security weightw(x)and traded off against the expected cost using a cost coefficientμset by the operator (Section 3.4). 3.2 Query Conditioning 10 1 10 2 10 3 10 4 #Router Params (M) 0 20 40 60 Oracle Gap Recovered In-Domain 10M100M1B #Encoder Params 10 1 10 2 10 3 10 4 #Router Params (M) 0 20 40 60 Out-of-Domain SCRouter-L R2-Router CARROT-RoBERTa SCRouter-M RADAR NIRT SCRouter-S CARROT-KNN MIRT Figure 5: Efficiency versus performance across routers. SCRouter gets higher OGR with fewer parameters and lower TTFT. The x-axis denotes router size, and circle size denotes text encoder size. Query Representationz(x). A frozen encoder maps each queryxto a representationz(x), which is used by the security probe, budget model, and quality predictors. Security Weightw(x). A logistic-regression probe on the query representationz(x)gives the security weight directly: w(x) = σ v ⊤ z(x) + v 0 .(1) We train the probe on binary labels (0 for utility, 1 for security), sow(x)is the predicted probability that a query is a security query. Its regularization strength is selected by minimizing validation log- loss. Figure 6 shows that the probe separates utility and security queries in-domain and generalizes to unseen sources. Encoder Variants.Prior work (Xue et al., 2026) uses large encoders such as Qwen3-Embedding- 0.6B (Zhang et al., 2025).These encoders 3 −20−1001020 logit w(x) 0 0.05 0.10 Density 99.6%99.6% In-Domain −20−1001020 logit w(x) 74.2%86.7% Out-of-Domain UtilitySecurity Figure 6: Performance of the security probe. The probe separates utility and security queries well in-domain and generalizes to unseen sources. are already close in size to the smallest candi- date LLMs and add to the time to first token (TTFT). We use smaller encoders to keep rout- ing cheap. We design three variants: SCRouter- L (Qwen3-Embedding-0.6B, 1024-d), SCRouter- M (all-MiniLM-L6-v2, 384-d), and SCRouter- S (BERT-tiny, 128-d). SCRouter-S uses a 4M encoder and 0.25M trainable parameters. Fig- ure 5 compares efficiency and performance across routers; Section 5.2 reports the results. 3.3 Budget-Conditioned Quality Surfaces A response may score low for two reasons. It may finish but give a weak answer, or it may reach the token limit before it finishes. We call the second case budget-induced truncation. For each objective t∈U,S, we model the two cases separately and mix them by the completion probability π, ˆ Q t = πF t + (1− π)T t ,(2) whereF t andT t are the predicted qualities of com- pleted and truncated responses. A shared budget model givesπand the expected number of gener- ated tokens. Two quality models, one for utility and one for security, giveF t andT t . In SCRouterBench and many other benchmarks, completion rates rise almost monotonically withbfor all models. We therefore deriveπfrom the number of tokens each response needs. Shared Budget Model. LetB ∗ (x,m,b)be the smallest number of output tokens with which model m, instructed with budgetb, completes its answer tox. The response is truncated exactly when B ∗ > b. Because a model writes shorter answers when told a budget,B ∗ depends onb. The budget changes the answer itself, not only where it stops. 0 1 Qwen3.5-0.8B MAE 0.016 Gemma-3-1B MAE 0.037 Phi-4-mini MAE 0.022 0 1 P(finish) Qwen3.5-4B MAE 0.005 Mistral-7B-v0.3 MAE 0.014 DS-R1-Qwen3-8B MAE 0.009 0 1 Llama-3.1-8B MAE 0.019 Foundation-Sec-8B MAE 0.026 Qwen3.5-9B MAE 0.007 101001k 0 1 Qwen3.5-27B MAE 0.010 101001k Token Budget Qwen3-Coder-30B MAE 0.021 101001k Llama-3.1-70B MAE 0.033 Fitted π Observed finish rate Figure 7: Budget model calibration across models and budgets. Lines denote the fitted completion probabil- ityπfrom the shared budget model, while markers represent observed completion rates. The predictedπ increases with b even though we do not force it to. A toweru L (x), a two-layer MLP, maps the query to a low-dimensional demand vector, and logB ∗ ∼N λ, s 2 ,(3) λ =⟨u L (x),A L m ⟩ + c L m + g m,b ,(4) whereA L m andc L m describe how many tokens modelmneeds,g m,b how strongly the instruction bcompresses this need, ands = s m,b is a learned spread per action. The default budget carries no in- struction, is capped atb = 4,000tokens, and keeps its own column ofgands. Withζ = (logb−λ)/s, the completion probability is π = P (B ∗ ≤ b) = Φ(ζ).(5) A completed response stops afterB ∗ tokens and a truncated one stops atb, so the model generates min(B ∗ ,b) tokens, whose expectation is ˆ E[tok] = e λ+s 2 /2 Φ(ζ− s) + b (1− π).(6) This one distribution gives both the completion probability and the expected cost. Figure 7 shows that the predictedπfollows the observed comple- tion rates as the budget grows. Completion/Truncation Quality Models. Fol- lowing multidimensional item response theory, for each objectivet∈U,Sa toweru t (x)maps the query to a demand vector, and each model has an ability vector in the same space. With the logit φ t =⟨u t ,A t,m ⟩ + c t,m + β t,m,b +⟨u t ,C t,b ⟩, (7) 4 1614440078412961936 #Action Space 1M 10M 100M 1B Trainable Parameters +14 per action +0.30M per action SCRouter-L R2-Router 1.63M 1.66M 43.7M 700M Figure 8: Parameter scaling across action spaces. Train- able parameters on a log scale fornmodels andnbud- gets excluding the frozen encoder, with our12× 12 setting dashed. R2-Router grows quadratically, from 43.7M to 700M parameters, while SCRouter stays at about 1.6M because its towers are shared. the predicted qualities of completed and truncated responses are given by F t = σ(φ t ),(8) T t = σ φ ′ t + κ t,m (logb− ̄ λ) ,(9) whereφ ′ t has the form of Eq. 7 with its own pa- rametersA ′ t,m ,c ′ t,m ,ρ t,m,b , andC ′ t,b . HereA t,m is modelm’s ability for objectivet,β t,m,b andρ t,m,b capture how the budget instruction itself changes quality,⟨u t ,C t,b ⟩lets the budget sensitivity depend on the query, andκ t,m gives partial credit accord- ing to how close the budget came to being suffi- cient. The shift ̄ λisλfrom Eq. 4 with its gradi- ent stopped, so the quality loss does not change the budget model. Eq. 2 then mixesF t andT t by π. Each tower is a two-layer MLP of width 512 with a 32-dimensional output. Because the tow- ers are shared across actions and the model and budget parameters are lookup tables, one forward pass gives the full|M|×|B|surface for each ob- jective. SCRouter therefore needs no per-action predictor, and its size grows much more slowly than R2-Router’s, as shown in Figure 8. Training. We call one (query, action) result a cell, and train each part only on the cells that tell us something about it. A cell either givesB ∗ exactly or gives only a lower bound. Specifically, with the budget instruction fixed, a completed response of lengthℓgives an observedB ∗ = ℓ, whereas a trun- cated response gives only the lower boundB ∗ > b. This observation does not imply completion under a different budget instruction. We therefore fit the budget model on all training cells with a censored log-normal likelihood, plus a binary cross-entropy loss betweenπand whether the response actually completed, which we read from its stop reason.F t is fitted on the completed cells andT t on the trun- cated cells of objective-tqueries, and an end-to-end binary cross-entropy aligns ˆ Q t with the observed score on all cells of objective-tqueries. As a re- sult, the utility model sees only utility queries and the security model sees only security queries. No query needs both labels. All terms are summed with unit weights and optimized with AdamW with early stopping based on the validation loss. 3.4 Surface Fusion and Cost-Aware Decision We then combine the two surfaces with the query’s security weight. For every action a, ˆ Q(x,a) = (1− w) ˆ Q U + w ˆ Q S .(10) Settingw = 0gives utility-only routing andw = 1 gives security-only routing. Values in between mix the two without retraining. The cost of an action is estimated before genera- tion from the known input lengthn in , the expected number of generated tokens (Eq. 6), and the per- token prices p in m and p out m of model m: ˆ C(x,a) = n in p in m + ˆ E[tok]p out m C ref ,(11) whereC ref is the largest nominal cost over all train- ing queries and actions. This nominal cost takes the output length to be the token budget, or the model’s mean training output length for the default budget. Given the operator’s cost coefficientμ ∈ [0, 1], SCRouter selects a ∗ = arg max a∈A (1−μ) ˆ Q(x,a)−μ ˆ C(x,a). (12) All|M||B|actions are scored without invoking any candidate generation model, and onlya ∗ is executed. In short,w(x)decides what counts as a good answer, andμdecides how much quality to give up for lower cost. 4 SCRouterBench SCRouterBench evaluates routers on mixed util- ity and security queries under cost constraints. As summarized in Table 1, utility queries come from six datasets in SPROUT (Somerstep et al., 2025); security queries cover nine dimensions from Decod- ingTrust (Wang et al., 2023), SALAD-Bench (Li et al., 2024), and HackAPrompt (Schulhoff et al., 2023). See Appendix A for details. 5 In-domainOOD DimensionSource#Q Source#Q Utility † SPROUT19,882 MixEval998 ToxicityDecodingTrust2,000 ToxiGen200 StereotypeDecodingTrust1,994 CrowS-Pairs200 Adv. demo.DecodingTrust1,996 BadChain200 PrivacyDecodingTrust2,000 Synthetic200 EthicsDecodingTrust2,000 ETHICS200 FairnessDecodingTrust1,997 BBQ300 Harmful req. SALAD-Bench4,732 HarmBench188 Cyber riskSALAD-Bench768 HarmBench67 Jailbreak/PIHackAPrompt1,500 Tensor Trust200 Total38,8692,753 Table 1: Overview of SCRouterBench. In-domain queries are split 70/10/20 into train/val/test; out-of- domain queries are used for evaluation only. Every query is scored under all 144 actions. † Six datasets: MMLU-Pro, GPQA, MuSR, MATH, OpenHermes, RAGBench. PI: prompt injection. Action Grid and Setup.Each query is evaluated across all|A| = 144actions formed by 12 open- weight models from 0.8B to 70B parameters and 12 token budgetsB. The token budgets include 11 instructed budgets from 10 to 4,000 tokens plus a default budget with no budget instruction. Each response gets a score in[0, 1]for its own objective: exact match for closed-form utility questions, gpt- oss-120b as a judge for open-ended ones, and the original benchmark protocol for security queries. Missing final answers receive a score of zero. Data Splits and Label Verification.The bench- mark contains 38,869 queries split 70/10/20 into train, validation, and test sets. We stratify splits by dimension and group them by template to avoid leakage. For out-of-domain evaluation, 2,753 ad- ditional queries are gathered from nine unseen sources. To ensure label quality, we compare util- ity labels with gold answers (97.7% agreement) and security labels with Qwen3Guard (Cohen’s κ ≥ 0.76). We drop queries whose labels cannot be decided, so every query has all 144 action labels. Routing Headroom.Per-query routing has a lot of room to improve on this data. In every dimen- sion, the best fixed action is well below the per- query oracle, and the oracle’s model choices are spread out, with an entropy of 2.5-3.4 bits. This equals spreading the choices evenly over 6 to 11 of the 12 models. No single model is best for most queries, so the router must choose per query. 5 Experiments 5.1 Experimental Setup Router Baselines.We compare with seven base- lines, each re-run with its original encoder, ar- chitecture, training objective, and decision rule. R2-Router (Xue et al., 2026) trains one MLP per model-budget action on Qwen3-Embedding- 0.6B. RADAR (Fernandez et al., 2026) fits a two- parameter IRT model over all actions on Qwen3- Embedding-8B, and we report both its Chebyshev rule and a linear rule. MIRT and NIRT (Song et al., 2025) use BERT-base. CARROT (Somerstep et al., 2025) has KNN, RoBERTa, and Linear variants. R2-Router and RADAR choose among the same 144 actions as SCRouter. MIRT, NIRT, and the CARROT variants choose among the 12 models at the default budget, as in their original form. Cost.Every router makes its decision using a cost estimate available before generation. The estimate uses the input tokens and the router’s own predic- tion of output tokens, priced with a fixed per-model price table and normalized by the largest nominal training costC ref defined in Section 3.4. Evalua- tion uses the realized cost of the chosen response, which reflects its actual input and output tokens at the same per-model prices. Metric. Sweeping the cost coefficientμtraces the router’s performance-cost frontierR(c). We summarize it with Oracle Gap Recovery (OGR): the area between the router and a random baseline, divided by the area between the oracle and the random baseline, over log cost: OGR = 100· R R(c)− Rnd(c) d logc R O(c)− Rnd(c) d logc , (13) whereO(c)is the oracle frontier andRnd(c)is a label-free baseline that uniformly samples from fixed actions with mean cost at mostc. The integral runs from the cost of the cheapest fixed action to that of the highest-scoring one. If a router cannot reach a costc, we setR(c) = Rnd(c). By defini- tion,OGRis 0 for random selection and 100 for the oracle. We report OGR on all queries and on the two subsets, averaged over three seeds. 5.2 Main Results Table 2 reports OGR in-domain and out-of-domain, and Figure 9 shows the performance-cost curves. 6 10 −5 10 −4 Cost per Query (USD) 0 25 50 75 100 Oracle Gap Recovered Overall 10 −5 10 −4 Cost per Query (USD) 0 25 50 75 100 Utility 10 −5 Cost per Query (USD) 0 25 50 75 100 Security SCRouter-L CARROT-RoBERTa SCRouter-M CARROT-Linear SCRouter-S NIRT R2-Router MIRT RADARCARROT-KNN Figure 9: Performance-cost trade-offs on in-domain evaluation. Each curve shows performance against log cost as μ varies. Random selection and the oracle are the lower and upper bounds. SCRouter is on top at most costs, with the largest gap on security and at low cost, where model-only baselines have no available action. In-DomainOut-of-Domain RouterAll Util. Sec.All Util.Sec. SCRouter-L60.9 52.972.0 48.149.647.3 SCRouter-M60.552.970.950.2 50.7 49.8 SCRouter-S59.9 53.9 66.6 46.9 48.4 46.1 R2-Router48.0 49.4 39.1 15.6 32.50.3 RADAR (linear)39.2 44.9 27.4 27.7 38.3 19.9 RADAR (Chebyshev) 36.7 41.3 30.0 27.4 37.0 15.8 CARROT-KNN42.1 39.0 38.8 23.7 34.11.2 CARROT-RoBERTa35.9 33.2 32.1 14.7 31.8-5.8 CARROT-Linear37.7 34.8 30.4 14.8 27.1-9.1 NIRT32.6 32.8 17.19.6 26.9 -14.3 MIRT32.0 32.4 17.67.3 25.8 -15.4 Table 2: OGR on SCRouterBench, in-domain and out- of-domain, for all queries (All) and for the utility (Util.) and security (Sec.) subsets. SCRouter variants beat all baselines in both settings. In-Domain. SCRouter-L has the highest over- all OGR, 12.9 points above the best baseline, R2- Router, with the same encoder and action space. It is better on both objectives, with gains of 32.9 points on security and 3.5 points on utility. As shown in Figure 5, the gains hold with smaller en- coders. SCRouter-M, with a 23M MiniLM encoder, is still 12.5 points ahead. SCRouter-S, with a 4M encoder, is 11.9 points ahead and has the best utility OGR of 53.9. It is behind the two larger variants only on the security subset. Figure 9 shows where the gains come from. SCRouter scores higher than R2-Router at most costs, especially low ones. Model-only baselines such as MIRT, NIRT, and CARROT cannot go be- low the cost of the cheapest model at its default budget. RADAR uses a single ability scale, which ranks actions well for utility but poorly for secu- rity, where its OGR drops to 27.4. On utility, all curves meet at high cost, where a strong model with a large budget answers most queries well. The routers differ mainly when the cost is tight. On security, SCRouter leads even at the lowest costs. A safe answer to a harmful request is often a short refusal, and Figure 3 shows that security peaks at only 120 tokens. Model-only baselines cannot lower the budget to use this. Utility versus Security. Every baseline in Ta- ble 2 scores lower on security than on utility in- domain. RADAR (linear) loses 17.5 points, and NIRT and MIRT lose about 15. SCRouter shows the opposite pattern, with all three variants higher on security. Compared with R2-Router, SCRouter gains at most 4.5 points on utility but 27.5 to 32.9 points on security. Most of its overall gain thus comes from security queries. A larger action space alone does not explain this. CARROT-KNN chooses among only 12 models but reaches 38.8 on security, close to the 39.1 of R2-Router with all 144 actions. SCRouter uses the same 144 actions as R2-Router and reaches 72.0, so the gain comes from its quality metric. Out-of-Domain.All routers do worse on the nine unseen sources, most of all on security. On security, R2-Router drops to a near-random 0.3, CARROT- KNN gets 1.2, and the other CARROT and IRT routers fall below random. RADAR is the strongest baseline under domain shift, reaching an overall score of 27.7. Its two-parameter IRT model is sim- ple and likely has less room to overfit the train- ing sources. RADAR (linear) loses 11.5 points overall from in-domain to out-of-domain, while R2-Router, which trains one MLP per action, loses 7 ComponentIn-Domain Out-of-Domain SCRouter-L (full)60.948.1 w/o Query-specificw(x)54.0 (-6.9) 40.7 (-7.4) w/o Budget terms58.3 (-2.6) 43.0 (-5.1) w/o Success/failure split55.0 (-5.9) 42.4 (-5.7) w/o Separate quality models 49.9 (-11.0) 37.3 (-10.8) Table 3: Ablation results. We report the overall OGR after removing each component from the full model. 32.4. All SCRouter variants still beat it on every metric. SCRouter-M has the best overall OGR of 50.2. It is 22.5 points above RADAR overall, 12.4 points on utility, and 29.9 points on security. SCRouter still does worse out-of-domain than in-domain. SCRouter-L drops from 60.9 to 48.1 overall, mainly because its security OGR falls from 72.0 to 47.3. Figure 6 explains much of this drop. The probe classifies 99.6% of security queries cor- rectly in-domain but only 74.2% on unseen sources, compared with 86.7% for utility queries. About a quarter of unseen security queries thus get a low w(x)and are routed like utility queries. The probe still separates most queries, which helps SCRouter keep a large lead over the baselines. 5.3 Ablation Study To measure what each part adds, we remove one part at a time from SCRouter-L and test in-domain and out-of-domain. Table 3 shows the results. Query-Specific Weight. Replacingw(x)with a fixed weight lowers OGR by 6.9 points in- domain and 7.4 points out-of-domain, so a per- query weight matters. A fixed weight applies the same trade-off to every query. It either undervalues security on harmful requests or undervalues utility on ordinary ones. Budget Modeling. Removing budget terms, such as the per-model compression table, query- dependent budget sensitivity, partial credit for failed responses, and auxiliary losses, reduces OGR by 2.6 points in-domain and 5.1 points out-of- domain. Modeling the budget helps in both settings, especially out-of-domain. Success/Failure Split. Predicting quality sepa- rately for successful and failed responses helps in both settings. Omitting this split reduces OGR by 5.9 points in-domain and 5.7 points out-of-domain. Failed utility responses are rarely useful, but failed security responses are often still safe, so the two cases need separate predictions. 0 15 30 45 60 Oracle Gap Recovered 54.9 SCRouter-L R2-Router CARROT-RoBERTa SCRouter-M RADAR NIRT SCRouter-S CARROT-KNN MIRT Figure 10: Leave-one-source-out evaluation across 15 training sources. We report the average held-out OGR for each method. All SCRouter variants beat the base- lines on held-out sources. Separate Quality Models.Using separate utility and security models matters most. Merging them into one model gives the largest drops, with OGR falling by 11.0 points to 49.9 in-domain and by 10.8 points to 37.3 out-of-domain. A single model tends to fit the more common utility labels and does worse on security. 5.4 Robustness under Domain Shift To test robustness to domain shift, we hold out each of the 15 training sources in turn and report the results in Figure 10. SCRouter variants attain 50.5-54.9 average held-out OGR, whereas base- lines RADAR, R2-Router, and CARROT-KNN reach only 34.7, 22.0, and 9.5 respectively. This 16–20 point gain over the best baseline matches our out-of-domain results in Table 2. Smaller encoders transfer better. SCRouter-S has the best held-out OGR of 54.9, and SCRouter-M has the best out-of- domain OGR in Table 2. SCRouter-L is best only in-domain. A larger encoder may fit the training sources too closely. A small encoder is thus a good default when the test workload is unknown. 6 Conclusion We present SCRouter, a router that picks both the model and token budget. It predicts utility and se- curity separately, weights them per query with a se- curity probe, and uses a shared budget model to es- timate success probability and cost. On SCRouter- Bench, SCRouter recovers 60.9% of the oracle gap in-domain and beats all baselines out-of-domain. These results demonstrate that routing mixed work- loads requires a per-query quality metric, not just a larger action space. 8 Limitations Following prior routing benchmarks (Hu et al., 2024; Somerstep et al., 2025), our evaluation fo- cuses on single-turn text queries and a pool of 12 open-weight models. Multi-turn and multimodal workloads are left for future evaluation. Cost com- parisons use actual token consumption and a fixed per-model price table shared by all routers. This provides a consistent comparison, but does not ac- count for time-varying prices or load-dependent serving costs. Ethical Considerations SCRouterBench is built entirely from publicly available benchmarks, used under their original licenses, and no new human data was collected. Its security queries and the corresponding model re- sponses contain toxic, biased, and harmful content, including jailbreak and prompt-injection attacks, which we use solely to evaluate routing and release only for research purposes. The per-model security results could also reveal which configurations are easiest to attack. We think the benefit of security- aware routing outweighs this risk. SCRouter is a cost-aware routing layer, not a safety guarantee. We used AI assistants to check grammar. References Abdelrahman Abouelenin, Atabak Ashfaq, Adam Atkin- son, Hany Awadalla, Nguyen Bach, et al. 2025. Phi-4-Mini technical report: Compact yet powerful multimodal language models via mixture-of-LoRAs. arXiv preprint arXiv:2503.01743. Pranjal Aggarwal, Aman Madaan, Ankit Anand, Sriv- idya Pranavi Potharaju, Swaroop Mishra, Pei Zhou, Aditya Gupta, Dheeraj Rajagopal, Karthik Kappa- ganthu, Yiming Yang, Shyam Upadhyay, Manaal Faruqui, and Mausam. 2024. AutoMix: Automati- cally mixing language models. In Advances in Neu- ral Information Processing Systems. Patrick Chao, Edoardo Debenedetti, Alexander Robey, Maksym Andriushchenko, Francesco Croce, Vikash Sehwag, Edgar Dobriban, Nicolas Flammarion, George J. Pappas, Florian Tramèr, Hamed Hassani, and Eric Wong. 2024. JailbreakBench: An open ro- bustness benchmark for jailbreaking large language models. In Advances in Neural Information Process- ing Systems. Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2025. Jailbreaking black box large language mod- els in twenty queries. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). Lingjiao Chen, Matei Zaharia, and James Zou. 2024a. FrugalGPT: How to use large language models while reducing cost and improving performance. Transac- tions on Machine Learning Research. Shuhao Chen, Weisen Jiang, Baijiong Lin, James Kwok, and Yu Zhang. 2024b.RouterDC: Query-based router by dual contrastive learning for assembling large language models. In Advances in Neural Infor- mation Processing Systems. Sizhe Chen, Julien Piet, Chawin Sitawarin, and David Wagner. 2025. StruQ: Defending against prompt injection with structured queries. In 34th USENIX Security Symposium. Dujian Ding, Ankur Mallick, Chi Wang, Robert Sim, Subhabrata Mukherjee, Victor Rühle, Laks V. S. Lak- shmanan, and Ahmed Hassan Awadallah. 2024. Hy- brid LLM: Cost-efficient and quality-aware query routing. In The Twelfth International Conference on Learning Representations. Yanrui Du, Sendong Zhao, Danyang Zhao, Ming Ma, Yuhan Chen, Liangyu Huo, Qing Yang, Dongliang Xu, and Bing Qin. 2024. MoGU: A framework for enhancing safety of LLMs while preserving their usability. In Advances in Neural Information Pro- cessing Systems. Tao Feng, Yanzhen Shen, and Jiaxuan You. 2025. GraphRouter: A graph-based router for LLM selec- tions. In The Thirteenth International Conference on Learning Representations. Nigel Fernandez, Branislav Kveton, Ryan A. Rossi, Andrew S. Lan, and Zichao Wang. 2026. RADAR: Reasoning-ability and difficulty-aware routing for reasoning LLMs. In The Fourteenth International Conference on Learning Representations. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, et al. 2025. Gemma 3 techni- cal report. arXiv preprint arXiv:2503.19786. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, et al. 2024. The Llama 3 herd of models.arXiv preprint arXiv:2407.21783. Kai Greshake, Sahar Abdelnabi, Shailesh Mishra, Christoph Endres, Thorsten Holz, and Mario Fritz. 2023. Not what you’ve signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection. In Proceedings of the 16th ACM Workshop on Artificial Intelligence and Security. Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, et al. 2025. DeepSeek-R1 incentivizes reasoning in LLMs through reinforcement learning. Nature. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021. Aligning AI with shared human values. In In- ternational Conference on Learning Representations. 9 Qitian Jason Hu, Jacob Bieker, Xiuyu Li, Nan Jiang, Benjamin Keigwin, Gaurav Ranganath, Kurt Keutzer, and Shriyash Kaustubh Upadhyay. 2024. Router- bench: A benchmark for multi-LLM routing system. In Agentic Markets Workshop at ICML 2024. Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, and Madian Khabsa. 2023. Llama guard: LLM- based input-output safeguard for human-AI conver- sations. arXiv preprint arXiv:2312.06674. WittawatJitkrittum,HarikrishnaNarasimhan, Ankit Singh Rawat, Jeevesh Juneja, Congchao Wang, Zifeng Wang, Alec Go, Chen-Yu Lee, Pradeep Shenoy, Rina Panigrahy, Aditya Krishna Menon, and Sanjiv Kumar. 2026.Universal model routing for efficient LLM inference. In The Fourteenth International Conference on Learning Representations. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gon- zalez, Hao Zhang, and Ion Stoica. 2023. Efficient memory management for large language model serv- ing with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles. Lijun Li, Bowen Dong, Ruohui Wang, Xuhao Hu, Wang- meng Zuo, Dahua Lin, Yu Qiao, and Jing Shao. 2024. SALAD-bench: A hierarchical and comprehensive safety benchmark for large language models. In Find- ings of the Association for Computational Linguistics: ACL 2024. Yifan Lu, Rixin Liu, Jiayi Yuan, Xingqi Cui, Shenrun Zhang, Hongyi Liu, and Jiarong Xing. 2026. Router- arena: An open platform for comprehensive compari- son of LLM routers. In The Fourteenth International Conference on Learning Representations. Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, Elham Sakhaee, Nathaniel Li, Steven Basart, Bo Li, David Forsyth, and Dan Hendrycks. 2024. Harmbench: A standardized eval- uation framework for automated red teaming and ro- bust refusal. In Forty-first International Conference on Machine Learning. Isaac Ong, Amjad Almahairi, Vincent Wu, Wei-Lin Chi- ang, Tianhao Wu, Joseph E. Gonzalez, M Waleed Kadous, and Ion Stoica. 2025. RouteLLM: Learning to route LLMs from preference data. In The Thir- teenth International Conference on Learning Repre- sentations. Alicia Parrish, Angelica Chen, Nikita Nangia, Vishakh Padmakumar, Jason Phang, Jana Thompson, Phu Mon Htut, and Samuel R. Bowman. 2022. BBQ: A hand-built bias benchmark for question answering. In Findings of the Association for Computational Linguistics: ACL 2022. Fábio Perez and Ian Ribeiro. 2022. Ignore previous prompt: Attack techniques for language models. In NeurIPS ML Safety Workshop. Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis- François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Kost, Christopher Carnahan, and Jordan Boyd-Graber. 2023. Ignore this title and HackAPrompt: Exposing systemic vulnerabilities of LLMs through a global prompt hacking competition. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Avital Shafran, Roei Schuster, Tom Ristenpart, and Vi- taly Shmatikov. 2025. Rerouting LLM routers. In Second Conference on Language Modeling. Seamus Somerstep, Felipe Maia Polo, Allysson Flavio Melo de Oliveira, Prattyush Mangal, Mírian Silva, Onkar Bhardwaj, Mikhail Yurochkin, and Subha Maity. 2025. CARROT: A cost aware rate optimal router. In ICLR 2025 Workshop on Founda- tion Models in the Wild. Wei Song, Zhenya Huang, Cheng Cheng, Weibo Gao, Bihan Xu, GuanHao Zhao, Fei Wang, and Runze Wu. 2025. IRT-router: Effective and interpretable multi-LLM routing via item response theory. In Pro- ceedings of the 63rd Annual Meeting of the Associa- tion for Computational Linguistics (Volume 1: Long Papers). Sam Toyer, Olivia Watkins, Ethan Adrian Mendes, Justin Svegliato, Luke Bailey, Tiffany Wang, Isaac Ong, Karim Elmaaroufi, Pieter Abbeel, Trevor Dar- rell, Alan Ritter, and Stuart Russell. 2024. Tensor trust: Interpretable prompt injection attacks from an online game. In The Twelfth International Confer- ence on Learning Representations. Boxin Wang, Weixin Chen, Hengzhi Pei, Chulin Xie, Mintong Kang, Chenhui Zhang, Chejian Xu, Zidi Xiong, Ritik Dutta, Rylan Schaeffer, Sang Truong, Simran Arora, Mantas Mazeika, Dan Hendrycks, Zi- nan Lin, Yu Cheng, Sanmi Koyejo, Dawn Song, and Bo Li. 2023. DecodingTrust: A comprehensive as- sessment of trustworthiness in GPT models. In Ad- vances in Neural Information Processing Systems. Sajana Weerawardhena, Paul Kassianik, Blaine Nel- son, Baturay Saglam, Anu Vellore, Aman Priyan- shu, Supriti Vijay, Massimo Aufiero, Arthur Gold- blatt, Fraser Burch, Ed Li, Jianliang He, Dhruv Ke- dia, Kojin Oshiba, Zhouran Yang, Yaron Singer, and Amin Karbasi. 2025. Llama-3.1-FoundationAI- SecurityLLM-8B-Instruct technical report. arXiv preprint arXiv:2508.01059. Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How does LLM safety training fail? In Advances in Neural Information Processing Systems. Jiaqi Xue, Qian Lou, Jiarong Xing, and Heng Huang. 2026. R2-router: A new paradigm for LLM routing with reasoning. In Forty-third International Confer- ence on Machine Learning. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, et al. 2025. Qwen3 technical report. arXiv preprint arXiv:2505.09388. 10 Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, Soo- jeong Kim, and Byung-Gon Chun. 2022. Orca: A distributed serving system for Transformer-Based generative models. In 16th USENIX Symposium on Operating Systems Design and Implementation (OSDI 22). Wenhui Zhang, Huiyu Xu, Zhibo Wang, Zhichao Li, Zeqing He, Xuelin Wei, and Kui Ren. 2026. Rerouteguard: Understanding and mitigating ad- versarial risks for llm routing.arXiv preprint arXiv:2601.21380. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, Fei Huang, and Jingren Zhou. 2025. Qwen3 embedding: Advancing text embedding and reranking through foundation models. arXiv preprint arXiv:2506.05176. Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Univer- sal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043. A SCRouterBench Details Table 5 lists the sources, dimensions, scoring cri- teria, and split sizes of SCRouterBench, including the out-of-domain evaluation set. Table 4 lists the 12 models in the action space. 11 TierModelDeveloperParamsFocus Small Qwen3.5-0.8BQwen0.8BGeneral Gemma-3-1B-it (Gemma Team et al., 2025)Google1BGeneral Phi-4-mini-instruct (Abouelenin et al., 2025)Microsoft3.8BGeneral Qwen3.5-4BQwen4BGeneral Medium Mistral-7B-Instruct-v0.3Mistral AI7BGeneral DeepSeek-R1-0528-Qwen3-8B (Guo et al., 2025)DeepSeek8BReasoning Llama-3.1-8B-Instruct (Grattafiori et al., 2024)Meta8BGeneral Foundation-Sec-8B-Instruct (Weerawardhena et al., 2025)Cisco8BSecurity Qwen3.5-9BQwen9BGeneral Large Qwen3.5-27BQwen27BGeneral Qwen3-Coder-30B-A3B-Instruct (Yang et al., 2025)Qwen30B (3B active)Coding Llama-3.1-70B-Instruct (Grattafiori et al., 2024)Meta70BGeneral Table 4: Model pool of SCRouterBench: 12 open-weight LLMs from 0.8B to 70B parameters, covering general- purpose instruction following, reasoning, coding, and security specialization. Each model is paired with 12 token budgets, giving|A| = 144 actions. SourceDimensionOutcome (score 1 if . . . )TrainValTest Utility (SPROUT) MMLU-ProKnowledgeextracted final answer matches the reference3,734534 1,067 GPQAExpert QAextracted final answer matches the reference1682448 MuSRMulti-step reasoningextracted final answer matches the reference2343467 MATHMathematicsextracted final answer matches the reference3,069439877 OpenHermesInstruction followingindependent judge (gpt-oss-120b) score6,139876 1,753 RAGBenchRetrieval-grounded QAindependent judge (gpt-oss-120b) score57283164 Security DecodingTrustToxicity1−max Detoxify toxicity over the full response1,400200400 DecodingTrustStereotype biasdoes not agree with the stereotype1,395200399 DecodingTrustAdv. demonstrationdoes not output the attacker’s target label1,397200399 DecodingTrustPrivacydoes not leak the target personal information1,400200400 DecodingTrustMachine ethicscorrect wrong / not-wrong judgment1,400200400 DecodingTrustFairnesscorrect income prediction (Adult)1,397200400 SALAD-Bench Harmful requestsMD-Judge rates the response safe3,311474947 SALAD-Bench Cyber riskMD-Judge rates the response safe53777154 HackAPromptJailbreak / prompt injection does not output the attacker’s target string1,050150300 Total (38,869 queries)27,203 3,891 7,775 Out-of-domainOOD MixEvalUtility exact answer match (640 multiple-choice) or in- dependent judge (358 open) 998 ToxiGenToxicity1−max Detoxify toxicity over the full response200 CrowS-PairsStereotype biaschooses the less stereotypical option200 BadChainAdv. demonstration answer matches the StrategyQA reference de- spite backdoored demonstrations 200 SyntheticPrivacydoes not leak the planted secret200 ETHICSMachine ethicscorrect commonsense-morality label200 BBQFairnessanswers “unknown” in an ambiguous context300 HarmBenchHarmful requestsHarmBench classifier: attack unsuccessful188 HarmBenchCyber riskHarmBench classifier: attack unsuccessful67 Tensor TrustJailbreak / prompt injection no hijacking or access-code extraction (official rules) 200 Total2,753 Table 5: Composition of SCRouterBench. Every query is answered under 12 models×12 budgets and scored in [0, 1]for its own objective only (higher is better; for security, safer); a response without a visible final answer scores 0. Counts are queries with all 144 outcomes. Splits are grouped by prompt template and near-duplicate text and stratified by dimension. The out-of-domain sources, including ETHICS (Hendrycks et al., 2021), BBQ (Parrish et al., 2022), HarmBench (Mazeika et al., 2024), and Tensor Trust (Toyer et al., 2024), share no data with the training sources and are scored with their released criteria. 12

Instant research discovery