Paper deep dive

Who Belongs in the Eval Set? A Capability-Taxonomy-Driven Pipeline for Curating Regression Eval Sets in Agent-Extensibility Platforms

Tezan Sahu, Aritra Das, Pankaj Mittal, Sudipta Das

Year: 2026Venue: arXiv preprintArea: cs.LGType: PreprintEmbeddings: 28

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Platform teams hosting agent-extensibility surfaces face a regression-economics paradox: every onboarding customer ships an evaluation set tuned to their domain, but the platform's regression set must live under a hard query-count ceiling bounded by release cadence. To our knowledge, no published industrial pipeline addresses this platform-side curation problem: existing evaluation frameworks are customer-side, and benchmark-compression work treats benchmarks as fixed pools rather than streams of incoming sets. We describe a capability-taxonomy-driven curation pipeline applied to declarative agents with custom actions in Microsoft 365 Copilot. It takes an agent specification and a customer's eval set as input, projects each query into a platform-owned capability taxonomy, and outputs per-query decisions (admit, drop, swap, or human review), under the philosophy that a healthy regression set is the minimal set of queries capturing the maximal spread of capability signatures -- distinct combinations of capabilities a query exercises together. Three components instantiate this: a classifier producing per-(query, capability) verdicts via a hybrid of deterministic specification-based extraction and large-language-model (LLM) semantic inference; an Invocation Quality (IQ) rater scoring how thoroughly a query exercises each capability, so a new query sharing a signature with an existing entry can still be recognized as a better test and displace it; and a consolidator comparing incoming queries against the regression set on coverage and quality through a rule-based decision cascade, backed by a conservative curator that only suggests evictions. The mechanism is taxonomy-agnostic and applies to any regression eval-set curation problem with a typed capability taxonomy, including taxonomies that evolve in response to the very evidence the pipeline surfaces.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

27,168 characters extracted from source content.

Expand or collapse full text

Who Belongs in the Eval Set? A Capability-Taxonomy-Driven Pipeline for Curating Regression Eval Sets in Agent-Extensibility Platforms Tezan Sahu MicrosoftHyderabadIndia tezansahu@microsoft.com , Aritra Das Indian Institute of Technology RoorkeeRoorkeeIndia aritra˙d@cs.iitr.ac.in , Pankaj Mittal MicrosoftHyderabadIndia pankajmi@microsoft.com and Sudipta Das MicrosoftHyderabadIndia sudiptad@microsoft.com Abstract. Platform teams hosting agent-extensibility surfaces face a regression-economics paradox: every onboarding customer ships an evaluation set tuned to their domain, but the platform’s regression set must live under a hard query-count ceiling structurally bounded by release cadence. To the best of our knowledge, no published industrial pipeline addresses this platform-side curation problem — existing evaluation frameworks (helm, ; agentbench, ; hal, ; langsmith, ; braintrust, ; foundryevals, ) are customer-side, and benchmark-compression work (essencebench, ; metabench, ; tinybench, ) treats benchmarks as fixed pools rather than streams of incoming sets. We describe a capability-taxonomy-driven curation pipeline being applied to declarative agents with custom actions in Microsoft 365 Copilot (copilotextend, ). It takes an agent specification and a customer’s eval set as input, projects each query into a platform-owned capability taxonomy (∼ 30 capabilities), and outputs per-query decisions (admit, drop, swap, or human review) under the philosophy that a healthy regression set is the minimal collection of queries that captures the maximal spread of capability signatures — distinct combinations of capabilities a query exercises together. Three components instantiate this: a classifier producing per-(query, capability) verdicts via a hybrid of deterministic specification-based extraction and large-language-model (LLM) semantic inference (wang2023selfcons, ); an Invocation Quality (IQ) rater that scores how thoroughly a query exercises each capability, so a new query sharing a signature with an existing entry can still be recognized as a better test and displace it; and a consolidator that compares incoming queries against the regression set on coverage and quality through a rule-based decision cascade, backed by a conservative curator that only suggests evictions. The mechanism is taxonomy-agnostic and applies to any regression eval-set curation problem with a typed capability taxonomy — including ones whose taxonomies evolve in response to the very evidence the pipeline surfaces. Figure 1. Pipeline overview. The classifier emits per-tag verdicts and confidences for each (query, capability) pair. The Invocation Quality (IQ) rater turns these into a uniform per-capability score and derives R-relative facts (champions, new coverage, wins, similarity, novelty). The consolidator’s pure-rule Verdict Engine emits one decision per query; the Curator is the only writer to the regression set R. 1. Introduction Agent-extensibility platforms expose a menu of composable capabilities — authentication modes, tool-invocation protocols (REST/API-based plugins, Model Context Protocol (mcp, ), MCP Apps (mcpapps, )), orchestration patterns (single-turn, multi-turn, parallel calls, branching), and response-format primitives — that downstream third parties compose into agents. Every onboarding customer brings an evaluation set tuned to their domain. Why customer eval sets, not synthetic ones? A reasonable alternative is for the platform to synthesize its own eval set by enumerating capability combinations. Two reasons rule this out. Combinatorial blowup: with ∼ 30 typed capabilities the joint configuration space is intractable, and uniform sampling produces configurations no real customer would ever build. LLM non-determinism over realistic phrasings: per-capability behavior is verified by unit and functional tests we run separately, but end-to-end agent behavior — multiple capabilities firing through an LLM under a real user’s wording — is governed by how the LLM responds to actual customer phrasings, a distribution synthetic queries cannot reproduce. Customer eval sets carry both the combinations real agents ship and the phrasings the LLM has to handle. Two structural pressures then bound this setting: Hard eval-set ceiling. The regression set is judged by a large language model (LLM) on every candidate release build and human-triaged on every regression failure, gating flight reviews, A/B tests, and release certification. Every admission competes for a finite slot bounded by release cadence. Tenant-and-mock-setup labor. Regression evals must deterministically exercise the platform’s code paths, including those touching third-party services — hitting real 3P endpoints is unviable, since they are flaky, rate-limited, costly, and behaviorally non-stationary. Each admitted query therefore carries a provisioned tenant, synthetic connectors, and mock data fixtures engineered to deterministically reproduce the expected 3P responses. This per-query reviewer labor often dominates the release’s token-and-compute cost. The naive “admit all of your queries” inflates this engineering tail; the equally naive “manual review committee” freezes intuition and degrades with drift. Two questions must be answered per incoming query: (Q1) which capabilities does it exercise? (Q2) given the live regression set, does it deserve a slot? Existing literature offers neither answer in this form. Customer-side LLM and agent evaluation frameworks — HELM (helm, ), AgentBench (agentbench, ), HAL (hal, ), together with observability platforms like LangSmith (langsmith, ), Braintrust (braintrust, ), and Azure AI Foundry Evaluation (foundryevals, ) — certify a single agent owner against their own success criteria; the notion of a platform-owned regression set curated across many customers under one fixed budget is simply not their setting. Benchmark-compression methods (essencebench, ; metabench, ; tinybench, ) achieve up to 200×200× pool reductions while preserving ranking, but operate on closed, fixed pools rather than an open stream feeding a long-lived, edit-in-place set. Test-suite minimization in software engineering (chvatal1979, ; smithtsm, ; ltm, ) supplies a set-cover framing on the coverage axis (NP-complete; greedy yields an HnH_n-approximation), but classical formulations carry no per-capability quality dimension — the very dimension a same-signature replacement decision needs. To the best of our knowledge, no published industrial pipeline addresses this platform-side eval-set curation problem. What we describe is therefore not a one-shot compression of a fixed pool but the curation policy for a living regression set that ingests new customer eval sets continuously. We close that gap. The rest of the paper develops a three-component pipeline — a hybrid classifier, an Invocation Quality (IQ) rater, and a rule-based consolidator with a conservative curator — that jointly answers Q1 and Q2 over a platform-owned capability taxonomy, and we are applying it to declarative agents with custom actions in Microsoft 365 Copilot (copilotextend, ). 2. Methodology The three components are chained in sequence (Fig. 1) over a platform-owned capability taxonomy and a live regression set R. Each component below first defines its outputs and the reason it exists; the design invariants underpinning the whole are summarized in §2.5. 2.1. Capability Taxonomy and Signatures Let =c1,…,cmC=\c_1,…,c_m\ be the platform’s capability taxonomy — a typed, hierarchical classification (the deployed instance has m≈30m≈ 30 leaves). Mutually-exclusive groups carry exactly-one constraints. A slice (compact form, sibling lists wrap by line): Capability Taxonomy Request Auth No Auth, API Key, OAuth, SSO Attachments Image, File, Audio Execution Orchestration single/multi-turn, parallel, branching Tool Output Size Small, Normal, Large Response Format Plain, Structured, URLs, Tables For a query q, the classifier emits a value vq,c∈TRUE,FALSEv_q,c∈\TRUE,FALSE\ for every c∈c . We call sig(q)=c:vq,c=TRUEsig(q)=\c:v_q,c=TRUE\ the capability signature of q. The regression set R is the curated portfolio of queries; the pipeline’s job is to keep R small while ⋃q∈Rsig(q) _q∈ Rsig(q) stays wide. The taxonomy is itself versioned. Platforms add capabilities as features ship, and reviewer feedback may also retro-add dimensions: when many incoming queries land outside C (caught as low-confidence, off-distribution clusters at projection time), the pipeline surfaces these as taxonomy-extension candidates — either truly emergent platform usage, capabilities the original taxonomy missed, or new combinations worth naming. Reviewer-approved extensions bump the taxonomy version and re-classify affected queries. 2.2. Classifier For each (q,c)(q,c) the classifier emits a verdict vq,cv_q,c and a confidence κq,c∈[0,1] _q,c∈[0,1] via two non-overlapping paths. Specification-based extractor. For capabilities the agent specification declares (authentication mode, tool inventory, declared platform capabilities), the truth is a lookup over the spec. The extractor returns the declared value with κq,c=1 _q,c=1. LLM with N-sample self-consistency (wang2023selfcons, ). For capabilities that surface only in query intent (response-format expectations, branching logic, tonality), the classifier draws N independent LLM samples, each returning a boolean vote v~i v_i and an exercise-probability PiP_i. Then: vq,c=majority(v~1,…,v~N),κq,c=1N∑i=1NPi.v_q,c=majority( v_1,…, v_N), _q,c= 1N _i=1^NP_i. The two paths are kept non-overlapping — the specification-based path is authoritative where the spec speaks; inference never overrides ground truth. Mutex arbitration over exactly-one groups runs after the paths resolve. Evaluation. We evaluate the LLM path (the specification-based path is a deterministic lookup) by per-capability correlation between predicted verdicts and human-labeled ground truth rather than raw accuracy. The choice is forced by class imbalance: each query exercises only a handful of the ∼ 30 capabilities, so an always-FALSE baseline would score >>90% accuracy by construction — an uninformative number. Per-capability correlation against labels captures the agreement structure correctly under this imbalance and avoids being inflated by the dominant negative class. 2.3. Invocation Quality (IQ) Rater The rater answers what binary coverage cannot: when a new query shares sig(q)sig(q) with an existing r∈Rr∈ R, is q a better test of those capabilities than r? Overlap is common; a shallow tester should yield to a deeper one. The rater splits into two halves — an intrinsic adapter (a fact about q alone) and a relational contextualizer (a fact about q versus the live R). The split is load-bearing: only the intrinsic half can be persisted; the relational half must be recomputed every cycle, because R itself changes on every admission and eviction. Adapter (intrinsic, stateless). For every c with vq,c=TRUEv_q,c=TRUE, the adapter emits cs(q,c):=κq,c∈[0,1],cs(q,c):= _q,c∈[0,1], the answer to “how thoroughly does q exercise c?”. cscs is a stored fact about q alone — a property of the query, not of the regression set. Contextualizer (relational, live). A pure intrinsic score is not enough to decide what to do with q: the consolidator’s rules need to know whether q fills a coverage gap, decisively beats an incumbent, or is just a near-duplicate of something we already keep. None of those questions can be answered from cs(q,⋅)cs(q,·) in isolation — every one of them refers to what is currently in R. The contextualizer is the bridge: it materializes, for q against the live R, six relational facts that encode exactly these questions. Two calibration knobs enter here: τ is the competence bar (the minimum cscs for a capability to count as competently covered), and δ is the winning margin (by how much an incoming score must beat the incumbent champion to count as decisive). Both live in configuration (§2.5, I4). Because R changes every cycle, the facts below are derived fresh and never persisted (§2.5, I2): champion(c)=arg⁡maxr∈R⁡cs(r,c), (c)= _r∈ R\,cs(r,c), the incumbent best test of c; champ_cs(c)=maxr∈R⁡cs(r,c), \_cs(c)= _r∈ R\,cs(r,c), current bar to beat on c; capsτ(q)=c:cs(q,c)≥τ, _τ(q)=\c:cs(q,c)≥τ\, q’s competent coverage (effective, not nominal); newCaps(q)=capsτ(q)∖⋃r∈Rcapsτ(r), (q)=caps_τ(q) _r∈ Rcaps_τ(r), “q fills a gap” (no one in R covers these well); wins(q)=c:cs(q,c)>champ_cs(c)+δ, (q)=\c:cs(q,c)>champ\_cs(c)+δ\, “q beats the champion” (by margin δ); maxSim(q)=maxr∈R⁡sim(q,r), (q)= _r∈ R\,sim(q,r), clone-detection signal; novel(q)=[sig(q)∉sig(r):r∈R]. (q)=1\! [sig(q)∉\sig(r):r∈ R\ ]. unseen capability combination. These six facts are the only inputs the Verdict Engine consumes about R; the engine itself touches no cscs values directly. Evaluation. Absolute quality scores for individual queries are not available in the abstract; what is available, by design, is reviewer action on the verdicts the pipeline emits. We therefore evaluate the rater at the verdict level: every admission, replacement, and eviction-candidate surfaced by the consolidator passes through a reviewer in the normal release flow, and their accept/reject call is logged. Verdict-level agreement between the pipeline and the reviewer is the quality metric. This reuses the review signal already produced for production purposes, so quality measurement imposes no additional dedicated annotation cycles. 2.4. Consolidator Given cs(q,⋅)cs(q,·) and the R-relative facts, a pure rule-based Verdict Engine emits one of INCLUDE,EXCLUDE,REPLACE,KEEP-BOTH\INCLUDE,EXCLUDE,REPLACE,KEEP -BOTH\, each tagged direct or review. Four further calibration knobs enter at this layer: σ is the clone-similarity cutoff (text similarity above which an incoming q is treated as a near-duplicate of some r∈Rr∈ R); N is the redundancy floor (minimum number of competent coverers R must retain per capability before any one of them can be evicted); k is the concentration cap (limit on how many capabilities a single r may be the sole competent coverer of); csmincs_ is the honesty floor (minimum top cscs across q’s TRUE tags below which a direct-admission verdict is downgraded to review). With these in hand, the cascade fires five gates in fixed order — (1) clone gate maxSim(q)≥σmaxSim(q)≥σ; (2) merit (newCaps∪wins∪c:novel(q)≠∅newCaps ∪\c:novel(q)\≠ ); (3) new competent coverage (newCaps≠∅newCaps≠ ); (4) replacement eligibility (subset-domination, N, k); and (5) honesty downgrade (admitting on direct route with maxc⁡cs(q,c)<csmin _ccs(q,c)<cs_ forces review). Each gate contributes at most one reason code; the full pseudocode is in Appendix B. Curator. The only writer to R. It stages admissions per query, prunes per cycle, and never auto-deletes: redundant un-pinned coverers are flagged as suggest-only eviction items that a human confirms. The asymmetry between the pure Verdict Engine and the write-capable Curator is the core safety property (§2.5, I3). 2.5. Design Invariants The pipeline rests on five invariants. With the components in hand, each becomes a constraint on the math we have just defined. I1: Atomicity. A query is the smallest curatable unit. If it is the best test of any one capability (its cscs is the champion on that capability and clears τ), it is pinned and cannot be evicted; we tolerate extra redundancy elsewhere instead. I2: Stored facts, derived roles. cs(q,c)cs(q,c) is a stored fact about q alone. The R-relative roles — champion, redundant, pinned, covered — are re-derived live whenever R changes. The intrinsic-cscs / R-relative-role line is the rater/consolidator boundary. I3: Admission cheap, eviction slow. A REPLACE verdict never deletes; it stages a candidate. The Curator’s prune turns a beaten resident into a suggested eviction item that a human confirms. Eviction removes evidence the platform was relying on, and must remain auditable. I4: No hard thresholds in code. All six knobs introduced above (τ, δ, σ, N, k, csmincs_ ) live in configuration, never in code. Their current calibration and the open problem of moving from uniform defaults to per-capability values are discussed in §3. I5: Honesty as route-downgrade, not admission. A low-confidence tag cannot admit a query; if rules yield an admitting verdict whose value rests on a low-confidence tag, the route is forced to review with reason LOW_CONFIDENCE. An EXCLUDE is never converted into an admission. 3. Discussion We are applying this pipeline to the curation of regression eval sets for declarative agents with custom actions in Microsoft 365 Copilot. Two threads stand out. Generalizability, with an evolving taxonomy. The mechanism is taxonomy-agnostic: the classifier, IQ rater, and consolidator make no assumption about which capabilities the taxonomy contains, only that it is typed and supports exactly-one groups. The same pipeline applies to any regression-eval-set curation problem where (a) a platform team owns a capability taxonomy, (b) downstream contributors ship eval sets, and (c) the curated set is living and bounded by a fixed budget. Crucially, the taxonomy itself is not fixed: platforms add capabilities over time, and the pipeline closes a loop in which queries that project to low-confidence, off-distribution regions cluster into taxonomy-extension candidates that reviewers can promote into the next taxonomy version (whether they correspond to emergent platform behavior, missed combinations, or capabilities the original taxonomy did not anticipate). Web service APIs, data pipelines, and other multi-tenant platforms with evolving capability menus are natural next targets. Per-capability calibration is the central tuning problem. The six knobs τ, δ, σ, N, k, csmincs_ are absolute thresholds, not batch-relative percentiles, and a value sensible for one capability is not automatically sensible for another — the bar for “competent coverage” of authentication is structurally different from the bar for response-format handling. The principled target is therefore a per-capability schedule of these knobs, learned from validated reviewer overrides. Our current workaround is a uniform, conservative default set across all capabilities, derived from a small pilot of reviewer-annotated runs; reviewer overrides on borderline verdicts are logged but do not yet feed an automated calibration loop. Identifying which override signal counts as “validated” — separating principled adjustments from reviewer disagreement noise — is the next architectural decision. Pipeline accessibility via an MCP server. We expose the pipeline behind a Model Context Protocol server, with tools mapped to each pipeline step (classify, rate, consolidate) and to common step-ranges (classify-to-verdict; cscs-to-verdict, for re-running the consolidator over an updated R). This lets any MCP-aware coding agent or chat assistant invoke the pipeline programmatically: reviewers triage incoming eval sets from their existing chat surface, and partner platforms can embed the pipeline into their own onboarding flows without re-implementing it. Combined with the taxonomy-agnostic design above, this is what makes the system reusable rather than Microsoft-365-specific. 4. Conclusion & Future Work We presented a capability-taxonomy-driven pipeline for the platform-side curation of regression eval sets, an industrial problem we believe is unaddressed in the published literature. Three components — a hybrid classifier, an intrinsic-versus-relative IQ rater, and a rule-based consolidator with a conservative curator — jointly maintain a small regression set with maximal capability-signature coverage on top of a versioned, evolving taxonomy. Future work spans per-capability threshold calibration, recency-decayed champion selection so stale-but-strong incumbents yield to fresher tests, cost-weighted set-cover when capabilities differ materially in setup labor, and an end-to-end study of how the pipeline transfers to non-agent platforms. References (1) Microsoft. Microsoft 365 Copilot Custom Actions and Agent extensibility. https://learn.microsoft.com/en-us/microsoft-365-copilot/extensibility/, 2026. (2) Anthropic. Model Context Protocol Specification. https://modelcontextprotocol.io/, 2024. (3) D. Pratinidhi. MCP Apps in Microsoft 365 Copilot Chat. Microsoft 365 Developer Blog, March 2026. (4) P. Liang et al. Holistic Evaluation of Language Models (HELM). TMLR, 2023. (5) X. Liu et al. AgentBench: Evaluating LLMs as Agents. ICLR, 2024. (6) B. Kapoor et al. Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation. arXiv:2510.11977, 2025. (7) LangChain. LangSmith: LLM Application Observability & Evaluation. https://smith.langchain.com/, 2024–2026. (8) Braintrust. AI Evaluation & Observability Platform. https://w.braintrust.dev/, 2024–2026. (9) Microsoft. Azure AI Foundry Agentic Evaluation. https://learn.microsoft.com/en-us/azure/ai-foundry/concepts/evaluation-overview, 2026. (10) Anonymous Authors. Rethinking LLM Evaluation: Can We Evaluate LLMs with 200× Less Data? (EssenceBench). arXiv:2510.10457, 2025. (11) Anonymous Authors. MetaBench: Item-Response-Theory-Based Selection for Efficient LLM Evaluation. ICLR, 2025. (12) F. Maia Polo et al. TinyBenchmark: Evaluating LLMs with Fewer Examples. arXiv:2402.14992, 2024. (13) V. Chvátal. A Greedy Heuristic for the Set-Covering Problem. Mathematics of Operations Research, 4(3):233–235, 1979. (14) G. Rothermel and M. J. Harrold et al. A Survey of Test-Suite Minimization, Selection, and Prioritization. IEEE Transactions on Software Engineering, 38(5):1019–1043, 2012. (15) Anonymous Authors. LTM: Scalable Black-Box Similarity-Based Test Suite Minimization with Language Models. ASE, 2024. (16) X. Wang et al. Self-Consistency Improves Chain-of-Thought Reasoning in Language Models. ICLR, 2023. Appendix Appendix A Worked Example A customer onboards a project-tracker agent (OAuth-authenticated, four read-type tools, one write-type tool) and ships a 20-query eval set with their onboarding bundle. At the start of this cycle, the regression set R already holds ∼ 120 queries accumulated across prior onboardings of similar agents: several entries exercise OAuth-gated read flows, but no incumbent strongly tests structured-output formatting (the best Structured Output coverer in R currently sits at cs=0.30cs=0.30, below the competence bar τ=0.50τ=0.50). We walk one of the 20 incoming queries through the pipeline: “Pull recent issues from the tracker filtered by my team’s tag and summarise them as a bulleted list.” Tables 1 and 2 give the classifier output and the resulting R-relative facts for this query. Table 1. Classifier output (TRUE tags only). Capability Path κ OAuth spec-extractor 1.00 Has Read Tools spec-extractor 1.00 Single-turn LLM (N=5N=5) 0.72 Structured Output LLM (N=5N=5) 0.45 Table 2. R-relative facts for q (τ=0.50τ=0.50, σ=0.80σ=0.80, δ=0.05δ=0.05). The Structured Output champion in R has champ_cs=0.30<τchamp\_cs=0.30<τ — below the competence bar. Fact Value capsτ(q)caps_τ(q) OAuth, Has Read Tools, Single-turn newCaps(q)newCaps(q) Structured Output wins(q)wins(q) ∅ maxSim(q)maxSim(q) 0.42 novel(q)novel(q) 0 Verdict. Clone gate passes (0.42<0.800.42<0.80), merit holds (newCapsnewCaps non-empty), and new competent coverage is non-empty, so the engine emits INCLUDE-direct with reason NEW_COVERAGE; the honesty check (maxc⁡cs(q,c)=1.0≥csmin _ccs(q,c)=1.0 _ ) does not downgrade. Curator action. Stages q into R, and on the next per-cycle prune the previous shallow Structured Output coverer (cs=0.30cs=0.30) is dethroned and surfaced as a suggest-only eviction item that a human confirms before deletion. Appendix B Verdict Engine: Formal Rules Algorithm 1 Verdict Engine 1:function Decide(q, RFactsRFacts, cfgcfg) 2: if maxSim(q)≥σmaxSim(q)≥σ then return (Exclude,direct,Clone)( Exclude,direct, Clone) 3: end if 4: merit←(newCaps≠∅)∨(wins≠∅)∨novel(q)merit←(newCaps≠ ) (wins≠ ) (q) 5: if ¬merit \,merit then return (Exclude,direct,Redundant)( Exclude,direct, Redundant) 6: end if 7: if newCaps≠∅newCaps≠ then 8: ←(Include,direct,New_Coverage)verdict←( Include,direct, New\_Coverage) 9: else if wins≠∅wins≠ then 10: E←champion(c):c∈winsE←\champion(c):c \ ⊳ evict candidates 11: if ∃r∈E:caps(r)⊈caps(q)∃ r∈ E:caps(r) (q) then 12: return (Keep-Both,direct,Split_or_Floor)( Keep-Both,direct, Split\_or\_Floor) 13: else if ∃c:#coverersR′(c)<N∃ c:\#coverers_R (c)<N on R′=(R∖E)∪qR =(R E)∪\q\ then 14: return (Keep-Both,direct,Split_or_Floor)( Keep-Both,direct, Split\_or\_Floor) 15: else if sole(q)>ksole(q)>k then 16: return (Replace,review,Concentration)( Replace,review, Concentration) 17: else 18: ←(Replace,direct,Dominates)verdict←( Replace,direct, Dominates) 19: end if 20: else 21: ←(Include,review,Novel_Combo)verdict←( Include,review, Novel\_Combo) 22: end if 23: if admitting(verdict) ∧ route == direct ∧maxc⁡cs(q,c)<csmin _ccs(q,c)<cs_ then 24: verdict.route ← review; append Low_Confidence 25: end if 26: return verdict 27:end function

Instant research discovery