Paper deep dive
Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction
Xinyi Hong, Pinjun Dong, Xinyang Yu, Binyan Jiang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/3/2026, 10:49:47 AM
Summary
The paper introduces HYSET, a set-level tool retrieval framework for LLM agents that formulates tool selection as query-conditioned hyperedge prediction on a tool co-invocation hypergraph. Unlike existing methods that score tools individually or sequentially, HYSET evaluates the joint utility of a candidate tool set using cardinality-specific interaction matrices. Experiments on ToolBench demonstrate that HYSET outperforms state-of-the-art baselines in retrieval performance and end-to-end task success, while supporting zero-shot and few-shot transfer.
Entities (8)
Relation Signals (7)
HYSET → evaluatedon → ToolBench
confidence 95% · Experiments on ToolBench demonstrate that HYSET consistently outperforms state-of-the-art baselines
HYSET → uses → Tool Co-invocation Hypergraph
confidence 95% · we formulate tool retrieval as query-conditioned hyperedge prediction on a tool co-invocation hypergraph
HYSET → captures → cardinality-specific interactions
confidence 92% · we capture size-dependent tool compatibility through cardinality-specific interactions
HYSET → outperforms → ToolGen
confidence 90% · HYSET consistently outperforms state-of-the-art baselines
HYSET → outperforms → COLT
confidence 90% · HYSET consistently outperforms state-of-the-art baselines
HYSET → outperforms → BM25
confidence 90% · HYSET consistently outperforms state-of-the-art baselines
HYSET → outperforms → Contriever
confidence 90% · HYSET consistently outperforms state-of-the-art baselines
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language model (LLM) agents increasingly rely on invoking external tools to complete real-world tasks. Tool retrieval, which selects a small task-relevant subset from a library of thousands of tools before the agent acts, has therefore become a critical component of LLM agent pipelines. However, existing retrievers either score each tool in isolation or assemble the tool set sequentially, so the joint utility of a candidate set is never evaluated as a whole. In this paper, we propose HYSET, short for HYperedge-based SEt-level Tool retrieval. Our contributions are threefold: (i) we formulate tool retrieval as query-conditioned hyperedge prediction on a tool co-invocation hypergraph, under which the tool set itself becomes the unit of scoring and most existing retrieval paradigms reduce to restricted instances; (ii) we capture size-dependent tool compatibility through cardinality-specific interactions; and (iii) we design HYSET as a pre-selection module requiring no modification to the downstream agent. Experiments on ToolBench demonstrate that HYSET consistently outperforms state-of-the-art baselines in both tool retrieval performance and end-to-end task success. Beyond the in-domain setting, HYSET further supports zero-shot/few-shot transfer, generalizing to held-out tools/categories and unseen domains with minimal supervision.
Tags
Links
- Source: https://arxiv.org/abs/2607.25718v2
- Canonical: https://arxiv.org/abs/2607.25718v2
Trouble viewing inline? Open PDF directly →
Full Text
46,993 characters extracted from source content.
Expand or collapse full text
Tools Are Not Islands: Set-Level Tool Retrieval for LLM Agents via Query-Conditioned Hyperedge Prediction Xinyi Hong1, Pinjun Dong2, Xinyang Yu2, Binyan Jiang2 Abstract Large language model (LLM) agents increasingly rely on invoking external tools to complete real-world tasks. Tool retrieval, which selects a small task-relevant subset from a library of thousands of tools before the agent acts, has therefore become a critical component of LLM agent pipelines. However, existing retrievers either score each tool in isolation or assemble the tool set sequentially, so the joint utility of a candidate set is never evaluated as a whole. In this paper, we propose HYSET, short for HYperedge-based SEt-level Tool retrieval. Our contributions are threefold: (i) we formulate tool retrieval as query-conditioned hyperedge prediction on a tool co-invocation hypergraph, under which the tool set itself becomes the unit of scoring and most existing retrieval paradigms reduce to restricted instances; (i) we capture size-dependent tool compatibility through cardinality-specific interactions; and (i) we design HYSET as a pre-selection module requiring no modification to the downstream agent. Experiments on ToolBench demonstrate that HYSET consistently outperforms state-of-the-art baselines in both tool retrieval performance and end-to-end task success. Beyond the in-domain setting, HYSET further supports zero-shot/few-shot transfer, generalizing to held-out tools/categories and unseen domains with minimal supervision. 1 Introduction Large language models (LLMs) are increasingly deployed as autonomous agents capable of invoking external tools to complete real-world tasks (Schick et al. 2023; Shen et al. 2023; Patil et al. 2023; Yao et al. 2022). As agents grow more capable and their invocation demands expand, tool libraries have scaled from dozens of handcrafted functions to large-scale API ecosystems containing thousands of real-world endpoints. ToolBench (Qin et al. 2023), for instance, includes 16,464 API endpoints spanning 49 categories from the RapidAPI Hub, a scale that renders exhaustive in-context presentation of every tool description impractical. One might expect that increasingly long context windows would eventually eliminate the need for explicit tool selection. However, empirical evidence shows that the effective use of information degrades sharply when context exceeds tens of thousands of tokens (Liu et al. 2023), and injecting the full tool library into every prompt would incur prohibitive latency and cost at agent scale. Moreover, the difficulty is not one of scale alone. Real-world tasks rarely depend on a single API, and a query is typically resolved by several APIs invoked jointly (Qin et al. 2023; Qu et al. 2024), so what must be retrieved is not a ranked list of individually relevant tools but a jointly useful tool set. Tool retrieval, which selects such a small task-relevant subset from a large API library before the agent acts, therefore remains a persistent infrastructure bottleneck rather than a transitional one. Figure 1: Overview and motivation of HYSET. (a) Tool-retrieval pipeline; (b) limitations of existing retrievers; and (c) our set-level view as query-conditioned hyperedge prediction. Existing tool retrievers address this bottleneck through three representative paradigms. The most common paradigm, semantic-matching retrieval, scores each API independently against the query. Sparse methods such as BM25 (Robertson and Zaragoza 2009) rank APIs by lexical overlap, while dense bi-encoders such as Contriever (Izacard et al. 2021) and Sentence-BERT-style models (Reimers and Gurevych 2019; Qin et al. 2023) score APIs by cosine similarity in a shared embedding space. All such methods treat each tool as an independently scorable item, so the score of a candidate set is simply an aggregate of per-tool signals. This treatment breaks down whenever tool utility is a joint property of the set rather than a sum of individual values. A tool that ranks low in isolation may become indispensable alongside other selected APIs, and a group of individually high-scoring tools may collectively fail to cover the full scope of a task. This gap is evident in practice. On ToolBench, a fine-tuned Contriever retriever reported by Qu et al. (2024) attains a Recall@3 of 68.6%, yet its COMP@3, the fraction of queries whose complete required tool set is covered by the top-3 results, is only 39.7%. To move beyond such independent scoring, the other two paradigms have recently been explored. Graph-enhanced retrievers such as COLT (Qu et al. 2024) augment semantic matching with dual-view collaborative learning over a bipartite query-scene-tool graph, improving the recovery of the complete required tool set. Nevertheless, their collaborative signal is distilled into per-tool embeddings during training, and inference still reduces to independent top-k ranking, so set-level coherence is only approximated rather than scored explicitly. Generative retrievers such as ToolGen (Wang et al. 2024), building on generative retrieval (Tay et al. 2022), instead emit tool identifiers directly from the language model. However, the tool set is assembled step by step from local conditional probabilities under a sequence-likelihood objective, so complete candidate sets are never explicitly compared, and whether the assembled set is coherent and complete can only be judged after generation ends. Figure 1 contrasts these paradigms with our set-level view. Supplementary Material H gives extended related work. Across all three paradigms, tools are either scored independently or generated sequentially, and no candidate set is ever assessed as a whole. We therefore argue that tools are not islands, and tool retrieval is inherently a set-level problem. This position raises two questions. (Q1) Can tool sets be scored as a whole, and does this improve retrieval? (Q2) Do tool co-invocation patterns vary with set size, and does modeling these differences help? Both are grounded in a representative ToolBench example. For the travel-planning query “I am flying from Chicago to Tokyo for five days next month. Find round-trip flights, book a hotel near Shinjuku, check the weather for those dates, and convert my 2,000 USD budget into yen”, whose ground-truth tools are Flight,Hotel,Weather,Currency\Flight,Hotel,Weather,Currency\, flight terms dominate the wording and a fine-tuned dense retriever scores Flight, CheapFlight, FlightTracker, Hotel, Weather and Currency at 0.920.92, 0.890.89, 0.870.87, 0.800.80, 0.550.55 and 0.320.32. Top-4 retrieval therefore returns Flight,CheapFlight,FlightTracker,Hotel\Flight,CheapFlight,FlightTracker,Hotel\ and leaves two subtasks uncovered. The failure is structural rather than a matter of calibration. Once Flight is selected, a second flight API contributes almost nothing, yet each per-tool score is assigned in ignorance of what else has been selected, which is the concern of (Q1) and is visible only when tools are scored as a whole. However, scoring the set jointly is still not enough, because the interaction among tools must itself depend on the cardinality of the set. A currency converter and a weather lookup belong together in the four-tool request above, but in a two-tool task they are co-invoked only under a contrived query such as “convert my budget into yen and tell me whether it will rain in Tokyo”. On ToolBench the two co-occur in 0.4%0.4\% of two-tool sets but in 23%23\% of four-tool sets, a gap far wider than the mechanical increase in pair count with set size would explain. This underlies (Q2) and motivates cardinality-specific interaction modeling. We therefore propose HYSET, short for HYperedge-based SEt-level Tool retrieval. Our main contributions are summarized as follows: • Formulation. To the best of our knowledge, we are the first to recast tool retrieval for LLM agents as query-conditioned hyperedge prediction over a tool co-invocation hypergraph, making the tool set itself the unit of scoring. We further provide a unified view where existing tool-retrieval paradigms arise as restricted instances. • Method. We design HYSET as a pre-selection module requiring no modification to the downstream agent, which scores candidate tool sets via cardinality-specific interaction matrices so that tool compatibility can vary with set size. • Experiments. On ToolBench, HYSET consistently outperforms strong baselines from all three paradigms, with relative improvements of 11.6% in COMP@5 and up to 13.1% in end-to-end pass rate over the strongest baselines. It further transfers zero-shot to held-out tools and categories, and recovers 93.2% of fully supervised performance with only 5 labeled examples per category. 2 Methodology Figure 2: HYSET framework. HYSET consists of two components: query-set alignment, measuring relevance to the input query, and set-level scoring, treating each candidate set as a hyperedge and modeling cardinality-specific interactions among its tools. 2.1 Preliminaries Let =t1,…,tNT=\t_1,…,t_N_T\ be a tool library. We assume each query x in the natural-language query space X can be fulfilled by jointly invoking a subset E⊆E , and we assume access to a training set tr=(xi,Ei⋆)i=1ND_tr=\(x_i,E_i )\_i=1^N, where Ei⋆E_i is the annotated tool set for xix_i. Each Ei⋆E_i is the ground truth for xix_i but need not be the unique feasible set, and it is unordered because invocation order is decided by the downstream agent rather than by the retriever. These sets admit a hypergraph representation (Battiston et al. 2020) that we call the tool co-invocation hypergraph ℋ=(,ℰ)H=(V,E), whose node set =V=T collects all tools and whose hyperedges ℰE are the observed Ei⋆E_i , so the unit of supervision shifts from the relevance of an individual tool to the joint utility of an entire set. Writing M=maxi|Ei⋆|M= _i|E_i | for the largest observed tool-set size, the admissible candidate hyperedge space ℰME_M collects all E⊆E with 1≤|E|≤M1≤|E|≤ M, assuming that tool sets required at inference do not exceed M. 2.2 Problem Formulation Set-level tool retrieval seeks a mapping ℛ:→ℰMR:X _M returning the complete tool set for a query. Since =V=T, every candidate set E∈ℰME _M is a hyperedge over V, so the task is query-conditioned hyperedge prediction. We model it through a parametric scoring family Fθ:×ℰM→ℝF_θ:X×E_M , where Fθ(x,E)F_θ(x,E) measures the joint utility of selecting E for x, and estimate θ^=argminθ∈Θℒ(θ;tr) θ= _θ∈ L(θ;D_tr) with ℒL the empirical loss of Section 2.4. For a new query xnewx_new, the learned scoring function F^=Fθ F=F_ θ predicts E^(xnew)=argmaxE∈ℰMFθ^(xnew,E). E(x_new)= _E _MF_ θ(x_new,E). (1) The resulting set E^(xnew) E(x_new) is subsequently provided to the frozen downstream LLM agent to complete the user query. 2.3 A Unified View of Tool Retrieval Under the hypergraph formulation and scoring family introduced in Section 2.2, major existing tool-retrieval paradigms can be subsumed by a unified set-scoring framework. (i) Semantic-matching retrievers induce Fmatch:×ℰM→ℝF_match:X×E_M with Fmatch(x,E)=∑t∈EFmatch(x,t)=∑t∈Esmatch(x,t)F_match(x,E)= _t∈ EF_match(x,\t\)= _t∈ Es_match(x,t), where t∈ℰM\t\ _M is the singleton hyperedge associated with tool t, and smatch(x,t)s_match(x,t) is typically a lexical or dense query-tool matching score. (i) Graph-enhanced retrievers induce Fgraph:×ℰM→ℝF_graph:X×E_M with Fgraph(x,E)=∑t∈EFgraph(x,t)=∑t∈Es(x,t)F_graph(x,E)= _t∈ EF_graph(x,\t\)= _t∈ Es_G(x,t), where s(x,t)s_G(x,t) is typically the similarity between a query embedding and a graph-contextualized embedding of the individual tool t. (i) Generative retrievers induce Fgen:×ℰM→ℝF_gen:X×E_M with Fgen(x,E)=log∑π∈(E)pϕ(π,EOS∣x)F_gen(x,E)= _π∈ S(E)p_φ(π,EOS x), where (E) S(E) collects the |E|!|E|! orderings of E, and pϕp_φ is typically the autoregressive probability of a language model over tool-identifier tokens. Its factorization is given in Supplementary Material A.2. The formulations above provide a unified mathematical view of existing paradigms. Supplementary Material A.2 derives the three induced scores and A.3 shows that they form a strict hierarchy of interaction orders. Algorithm 1 Training and Inference of HYSET Training input: Tool node set V, training set tr=(xi,Ei⋆)i=1ND_tr=\(x_i,E_i )\_i=1^N, frozen query encoder (⋅)r(·), frozen agent A, candidate-pool size KnegK_neg, maximum cardinality M, execution limit R, and weights η,λη,λ Inference input: Query xnewx_new and shortlist sizes K1<KpoolK_1<K_pool Output: Learned scoring function F F and predicted tool set E^(xnew) E(x_new) 0: Training phase 1: repeat 2: Sample a minibatch index set ℬ⊆1,…,NB \1,…,N\ 3: for each i∈ℬi do 4: Construct i⊆ℰMC_i _M of size KnegK_neg using the negative-sampling procedure in Section 2.4 5: Compute E^i E_i from iC_i via Eqs. (2) and (7) 6: Run A on xix_i using DFSDT (Qin et al. 2023) with tool set E^i E_i for at most R steps 7: Obtain the task reward ρi _i from the execution result 8: end for 9: Update θ via Eq. (11) 10: until convergence 11: Obtain θ^←θ θ←θ and F^=Fθ F=F_ θ 11: Inference phase 12: Construct the KpoolK_pool-tool shortlist S using Eqs. (12) and (13) 13: Compute E^(xnew) E(x_new) via Eq. (14) 14: return F F and E^(xnew) E(x_new) 2.4 The Framework of HYSET Scoring Function Parameterization. We parameterize the scoring function Fθ:×ℰM→ℝF_θ:X×E_M introduced in Section 2.2 through the decomposition Fθ(xi,E)=Fset(E)+Falign(xi,E),F_θ(x_i,E)=F_set(E)+F_align(x_i,E), (2) where FsetF_set models the internal interaction among the tools in E independently of the query, and FalignF_align models the alignment between xix_i and E. First, motivated by latent-space models of hypergraph data (Turnbull et al. 2019; Wu et al. 2024; Hong et al. 2026), we parameterize the set-interaction term FsetF_set as Fset(E)=∑1≤a<b≤mja⊤mjb,F_set(E)= _1≤ a<b≤ mz_j_a M_mz_j_b, (3) where E=tj1,…,tjm∈ℰME=\t_j_1,…,t_j_m\ _M is a candidate hyperedge of cardinality m=|E|m=|E|, ja∈ℝdzz_j_a ^d_z is the learnable embedding of tool node tjat_j_a, ∈ℝ||×dzZ ^|V|× d_z collects all tool embeddings as rows, and m∈ℝdz×dzM_m ^d_z× d_z is a symmetric interaction matrix shared across candidate hyperedges of cardinality m. A larger Fset(E)F_set(E) indicates that the tools of E better conform to the learned patterns of co-invocation and functional complementarity. For m≥2m≥ 2, the pair contribution ja⊤mjbz_j_a M_mz_j_b is allowed to vary with the cardinality of the set containing it, which is precisely what mm=2M\M_m\_m=2^M is introduced to model. To formalize the distinction, we introduce the notion of a pairwise-decomposable set function based on the Möbius transform (Grabisch et al. 2000). Specifically, for a given function f:ℰM∪∅→ℝf:E_M∪\ \ , the Möbius transform and its inverse are defined as f~(E):=∑T⊆E(−1)|E|−|T|f(T),f(E)=∑T⊆Ef~(T). f(E):= _T E(-1)^|E|-|T|f(T), f(E)= _T E f(T). Intuitively, the Möbius coefficient f~(T) f(T), whose interaction order is |T||T|, measures the joint contribution attributable specifically to T after subtracting the contributions of its proper subsets. Hence, we call f pairwise-decomposable if f~(E)=0 f(E)=0 whenever 3≤|E|≤M3≤|E|≤ M, meaning that it contains no interactions above order two. For instance, for any E∈ℰME _M with |E|=3|E|=3, its third-order interaction is F~set(E)=∑ti,tj⊆Ei⊤(3−2)j. F_set(E)= _\t_i,t_j\ Ez_i (M_3-M_2)z_j. Thus, 3−2M_3-M_2 induces the third-order effect without an explicit three-way tensor. More generally, we have the following theorem: Theorem 1. Assume ||≥M+2|V|≥ M+2. The set function Fset:ℰM∪∅→ℝF_set:E_M∪\ \ is pairwise-decomposable if and only if i⊤2j=⋯=i⊤Mj,∀ti,tj∈,i≠j.z_i M_2z_j=·s=z_i M_Mz_j, ∀\,t_i,t_j ,\ i≠ j. We prove Theorem 1 in Supplementary Material A.4. The theorem identifies the exact role of the cardinality-specific matrices. FsetF_set reduces to a fixed pairwise model precisely when every pair receives the same score at all cardinalities from 22 to M. Otherwise, its cardinality dependence induces interactions above order two. Thus, although FsetF_set is parameterized as a sum over pairs, it can represent structured joint effects at orders up to M and thereby capture multi-tool compatibility that an ordinary graph with fixed pairwise edge weights cannot express. These structured effects are generated by O(Mdz2)O(Md_z^2) parameters, rather than the O(dzM)O(d_z^M) parameters required by an unrestricted explicit M-way tensor. Second, we parameterize the query-set alignment FalignF_align as Falign(xi,E)=(xi)⊤(xi,E),F_align(x_i,E)=r(x_i) s(x_i,E), (4) where :→ℝdrr:X ^d_r is a frozen pre-trained language model that maps query xix_i to its embedding (xi)r(x_i) (Reimers and Gurevych 2019), and :×ℰM→ℝdrs:X×E_M ^d_r produces a query-conditioned representation (xi,E)s(x_i,E) of candidate hyperedge E. Specifically, we compute (xi,E)s(x_i,E) using cross-attention pooling with (xi)r(x_i) as the query and jkk=1m\Pz_j_k\_k=1^m as the keys and values (Vaswani et al. 2017), (xi,E)=∑k=1mαk(xi,E)jk,s(x_i,E)= _k=1^m _k(x_i,E)Pz_j_k, (5) where αk(xi,E)=exp(ℓ(xi,tjk))∑q=1mexp(ℓ(xi,tjq)) _k(x_i,E)= \! ( (x_i,t_j_k) ) _q=1^m \! ( (x_i,t_j_q) ) is the normalized attention weight assigned to tjkt_j_k within E, and ℓ(xi,tjk)=(xi)⊤jk (x_i,t_j_k)=r(x_i) Pz_j_k is the matching score between query xix_i and tool tjkt_j_k. The matrix ∈ℝdr×dzP ^d_r× d_z maps each tool embedding into the query space. Substituting Eq. (5) into Eq. (4) gives Falign(xi,E)=∑k=1mαk(xi,E)ℓ(xi,tjk).F_align(x_i,E)= _k=1^m _k(x_i,E) (x_i,t_j_k). (6) For m≥2m≥ 2, each attention weight depends on all tools in E, so the alignment term is itself a query-conditioned set-level score (Supplementary Material A.5). The trainable parameters are θ=(,mm=2M,)θ=(Z,\M_m\_m=2^M,P), with Z shared by the two terms. Training Objective. In practice, we construct the training objective of HYSET from negative-sampled set supervision and execution feedback, as depicted in Figure 2 and summarized in Algorithm 1. Since the admissible space of Section 2.1 contains |ℰM|=∑m=1M(||m)|E_M|= _m=1^M |V|m candidate hyperedges, exhaustive training is infeasible, and we use negative sampling both to build tractable candidate pools and to supply contrasting tool sets for the retrieval loss. For each query xix_i with ground-truth hyperedge Ei⋆E_i , we form the pool i=Ei⋆∪iC_i=\E_i \ _i with i⊆ℰM∖Ei⋆N_i _M \E_i \ and |i|=Kneg−1|N_i|=K_neg-1 distinct negatives, drawn from three sources in the fixed proportion 50%/30%/20%50\%/30\%/20\%: size-matched sets sampled uniformly from the |Ei⋆||E_i |-tool subsets of V, in-batch negatives taken from the ground-truth sets of the other queries of the minibatch (Oord et al. 2018), and hard negatives obtained by replacing one or two tools of Ei⋆E_i with their nearest neighbors under Z (Karpukhin et al. 2020; Robinson et al. 2020). Supplementary Material A.8 states the sampling procedure in full, gives an adaptive mixture recovering this one as the constant special case, and measures the resulting false-negative rate. Supplementary Material C.8 sweeps the proportion. Given iC_i, we score each candidate using Eq. (2) and select E^i=argmaxE∈iFθ(xi,E), E_i= _E _iF_θ(x_i,E), (7) The frozen LLM agent A then processes xix_i under the DFSDT search procedure (Qin et al. 2023) with its tool set restricted to E^i E_i for at most R steps, and its answer y^i y_i receives the execution reward ρi=ExecScore(xi,y^i)∈[0,1] _i=ExecScore(x_i, y_i)∈[0,1], whose definition, configuration and refresh schedule are given in Supplementary Material A.9. The candidate pool induces the retrieval loss ℒret(θ)=−∑i∈ℬlogexpFθ(xi,Ei⋆)∑E∈iexpFθ(xi,E),L_ret(θ)=- _i F_θ(x_i,E_i ) _E _i F_θ(x_i,E), (8) which raises Fθ(xi,Ei⋆)F_θ(x_i,E_i ) relative to Fθ(xi,E)F_θ(x_i,E) for every E∈iE _i. We further use execution feedback to reinforce successful model-selected sets through the reward-weighted self-training loss ℒself(θ)=−∑i∈ℬρilogexpFθ(xi,E^i)∑E∈iexpFθ(xi,E),L_self(θ)=- _i _i F_θ(x_i, E_i) _E _i F_θ(x_i,E), (9) where E^i E_i and ρi _i are treated as constants within each parameter update. Combining Eqs. (8) and (9), the full objective is ℒ(θ)=ℒret(θ)+ηℒself(θ)+λ∑m=2M‖m‖F2,L(θ)=L_ret(θ)+ _self(θ)+λ _m=2^M\|M_m\|_F^2, (10) where η>0η>0 controls the execution-feedback term and λ>0λ>0 controls the regularization of the cardinality-specific interaction matrices. We estimate θ^=argminθ∈Θℒ(θ)s.t.∥j∥2=1∀tj∈, θ= _θ∈ L(θ) .t. \|z_j\|_2=1 ∀\,t_j , (11) where ‖j‖2=1\|z_j\|_2=1 fixes the scale of each tool embedding. Method Sup. Recall@5 ↑ NDCG@5 ↑ COMP@5 ↑ Pass Rate ↑ GPT-4 Human BM25 A 41.0241.02 39.6839.68 22.3822.38 19.21±0.4419.21± 0.44 15.37±1.4715.37± 1.47 Contriever A 41.0841.08 38.3038.30 22.0422.04 18.80±0.4118.80± 0.41 14.17±1.4214.17± 1.42 ToolLLaMA-Ret A 68.8868.88 70.1270.12 61.1161.11 64.05±0.6864.05± 0.68 61.83±1.9861.83± 1.98 ToolRerank A 80.71±0.1780.71± 0.17 81.63±0.0881.63± 0.08 69.93±0.0769.93± 0.07 63.39±0.7263.39± 0.72 58.30±2.0158.30± 2.01 COLT A 77.07±0.1077.07± 0.10 83.84±0.0683.84± 0.06 68.98±0.1968.98± 0.19 63.54±0.6663.54± 0.66 59.36±2.0159.36± 2.01 ToolGen A 81.4181.41 84.7684.76 70.0170.01 62.30±0.7062.30± 0.70 59.04±2.0159.04± 2.01 HYSET (BERT) A+R 84.75±0.0984.75± 0.09 88.99±0.1688.99± 0.16 77.55±0.1277.55± 0.12 69.69±0.6169.69± 0.61 66.17±1.9366.17± 1.93 HYSET (Qwen2.5) A+R 88.61±0.1288.61± 0.12 91.07±0.1891.07± 0.18 78.13±0.1678.13± 0.16 71.11±0.5871.11± 0.58 69.92±1.8769.92± 1.87 Table 1: Main results on ToolBench. All values are percentages, best in boldface. Sup.: A denotes annotated tool sets only, while A+R additionally uses the execution reward of Eq. (9). Error bars follow Supplementary Material B.8. Results at a cutoff 3 are reported in Supplementary Table 13. Set-Level Inference. The training procedure yields the learned scoring function F^=Fθ F=F_ θ with estimated parameters θ^=(^,^mm=2M,^) θ=( Z,\ M_m\_m=2^M, P). Given a new query xnew∈x_new , directly solving Eq. (1) requires maximizing F F over ℰME_M. Since ℰME_M contains all subsets of V with cardinality between 11 and M, its size grows combinatorially with |||V|, making exhaustive enumeration infeasible. We therefore approximate Eq. (1) by exploiting the additive decomposition of F F in Eq. (2) to construct a two-stage procedure consisting of per-tool retrieval and set-level reranking (Zheng et al. 2024, 2026). In the first stage, we score each individual tool tj∈t_j against xnewx_new. Substituting the singleton hyperedge E=tjE=\t_j\ into Eq. (2) yields F^(xnew,tj)=(xnew)⊤^^j. F(x_new,\t_j\)=r(x_new) P z_j. (12) Let 0⊆S_0 collect the K1K_1 highest-scoring tools under Eq. (12). We score every remaining tool by its learned complementarity with 0S_0, g(tj)=maxti∈0^j⊤^M^ig(t_j)= _t_i _0 z_j M_M z_i, and write t[1],t[2],…t_[1],t_[2],… for the tools of ∖0V _0 in decreasing order of g. The shortlist is then =0∪t[1],…,t[Kpool−K1],S=S_0\ ∪\ \t_[1],…,t_[K_pool-K_1] \, (13) where M≤Kpool≪||M≤ K_pool |V|, so the second term admits a tool of low individual relevance but high complementarity with 0S_0. In the second stage, we evaluate Eq. (2) on every candidate in the reduced space ℰM()=E⊆:1≤|E|≤ME_M(S)=\E :1≤|E|≤ M\, which approximates Eq. (1) by E^(xnew)=argmaxE∈ℰM()F^(xnew,E). E(x_new)= _E _M(S) F(x_new,E). (14) Eq. (14) compares sets of different cardinality, and since FsetF_set aggregates (m2) m2 pairs while FalignF_align is a convex combination of per-tool scores, the two scale differently in m, which the cardinality-specific mM_m absorbs. The reduced space contains ∑m=1M(Kpoolm) _m=1^M K_poolm hyperedges, independently of |||V|. As E^(xnew) E(x_new) is unordered and of variable size, rank-based metrics require a second output, obtained by greedy marginal gain under the same F F with A0=∅A_0= and Ak=Ak−1∪argmaxt∈∖Ak−1F^(xnew,Ak−1∪t)A_k=A_k-1∪\ _t A_k-1 F(x_new,A_k-1∪\t\)\, so every step after the first scores a candidate jointly with the tools already selected. Rank-based metrics are computed from E^(K)(xnew)=(tπ(1),…,tπ(K)) E^(K)(x_new)=(t_π(1),…,t_π(K)), while E^(xnew) E(x_new) is passed to the agent A. Supplementary Material A.10 reports the predicted cardinality distribution and a calibrated variant, A.11 the inference path in pseudocode, and E.3 a comparison against exhaustive maximization. 3 Experiments 3.1 Experimental Setup Task, datasets and baselines. We evaluate offline retrieval by testing whether the retrieved set covers E⋆E , and end-to-end execution by testing whether the frozen agent solves the query when restricted to that set. Our primary benchmark is ToolBench (Qin et al. 2023), whose official filtering retains 13,86013,860 callable API endpoints forming V and 200,311200,311 instructions with ground-truth API sets. We train on the combined official training portions and evaluate on the six held-out test sets comprising 600600 queries. Since training and evaluation share one library, we also report train-test overlap and results restricted to unseen tool sets. To check that the conclusions are not library-specific, we further evaluate on UltraTool (Huang et al. 2024), whose 2,0322,032 tools across 2222 domains are disjoint from ToolBench and in which only 6.1%6.1\% of the ground-truth sets are singletons, against 20.4%20.4\% on ToolBench. We compare against six baselines spanning the three paradigms: BM25 (Robertson and Zaragoza 2009), Contriever (Izacard et al. 2021), ToolLLaMA-Retriever (Qin et al. 2023), ToolRerank (Zheng et al. 2024), COLT (Qu et al. 2024) and ToolGen (Wang et al. 2024). Supplementary Material B.1 and B.2 give dataset, overlap and baseline details. Evaluation metrics. We report Recall@K and NDCG@K for retrieval quality and COMP@K (Qu et al. 2024) for set completeness with K∈3,5K∈\3,5\, together with GPT-4 and Human Pass Rate under the official ToolEval protocol (Qin et al. 2023). Every method passes at least as many tools to the agent as HYSET does, so the tool budget of the main results is generous to the baselines. Since the rank-based metrics are computed from a length-K ranking rather than from the variable-size set E^(x) E(x) that reaches the agent, Section 3.5 scores E^(x) E(x) directly. Supplementary Material B.3 to B.6 and B.8 give the definitions, the tool-budget rule, the annotation protocol, the significance tests and the error-bar accounting. Implementation details. We instantiate HYSET with two frozen backbones, a BERT-base retriever (Qin et al. 2023) and a tuned Qwen2.5-1.5B retriever (Wang et al. 2024), and run every method under the same frozen ToolLLaMA-2-7B-v2 agent (Qin et al. 2023) and DFSDT protocol. Only θ=(,mm=2M,)θ=(Z,\M_m\_m=2^M,P) is trained, with 13.5913.59M parameters under the BERT configuration at dz=768d_z=768, including 10.6410.64M for Z, compared with 109.5109.5M parameters in the frozen encoder. The query encoder, the tool encoder that initializes Z from API descriptions, and the agent are all frozen. We set M=5M=5, the largest annotated set size, with K1=15K_1=15, Kpool=20K_pool=20 and Kneg=64K_neg=64, so reranking scores 21,69921,699 candidate sets per query. Execution feedback rewards Nsub=5,000N_sub=5,000 training queries and refreshes them every Tref=20,000T_ref=20,000 steps, capping it at 20,00020,000 rollouts and as many judge calls, all served from the cached StableToolBench API mirror rather than from live endpoints. Every configuration uses three seeds and early stopping on validation Recall@5. Supplementary Material B.7, B.9 and B.10 give the remaining hyperparameters, all prompts and the reproducibility details. Variant R@5 C@5 PR HYSET (Full) 84.75±0.0984.75± 0.09 77.55±0.1277.55± 0.12 69.69±0.6169.69± 0.61 (a) Component removal w/o FsetF_set 72.04±0.1872.04± 0.18 67.36±0.0967.36± 0.09 57.97±0.7357.97± 0.73 w/o Exec. fb. 82.14±0.0782.14± 0.07 77.02±0.0777.02± 0.07 65.14±0.6365.14± 0.63 (b) Interaction-matrix design m=M_m=I 75.05±0.1675.05± 0.16 68.68±0.1568.68± 0.15 59.92±0.7159.92± 0.71 Shared M 78.43±0.2078.43± 0.20 73.66±0.0873.66± 0.08 64.37±0.6964.37± 0.69 w/o Reg. 83.17±0.0783.17± 0.07 75.34±0.1375.34± 0.13 68.93±0.6468.93± 0.64 Table 2: Ablation of HYSET (BERT). R@5: Recall@5; C@5: COMP@5; PR: GPT-4 Pass Rate. All values are percentages. 3.2 Main Results Table 1 shows that HYSET outperforms every baseline on every retrieval and end-to-end metric. Retrieval gains over the strongest baseline reach 15.3% relative for the BERT configuration and 17.8% for Qwen, and are largest on COMP, where COMP@5 improves by 10.8% and 11.6% relative over ToolGen. Hyperedge scoring therefore recovers complete tool sets rather than merely reranking individual tools, and the two configurations differ by at most 5.7% relative, so the gains come from set-level modeling rather than from the backbone. Every margin is significant at the 5% level under a paired bootstrap with 10410^4 resamples for the retrieval metrics and the exact McNemar test for Pass Rate. COMP@5 gives p<10−4p<10^-4 with a 95% confidence interval of [4.03,11.05][4.03,11.05] for the difference, while the two Pass Rate margins are weakest at p=1.6×10−3p=1.6× 10^-3 and p=1.5×10−2p=1.5× 10^-2, the human one being the only margin that would not survive a Bonferroni correction over all eight tests at the 1% level (B.6). The Sup. column of Table 1 records that the two HYSET rows additionally use execution feedback while every baseline is trained on annotations alone, so the regimes must be separated before the end-to-end margin is read. Trained on annotations alone, HYSET (BERT) still reaches 77.02% COMP@5, an improvement of 10.0% relative over ToolGen, but its Recall@5 falls to 82.14% and its advantage over ToolGen narrows to 0.9% relative. Retraining the two strongest baselines with the identical reward, judge and 20,00020,000-rollout budget raises ToolGen to 83.12% Recall@5, 70.94% COMP@5 and 66.85% Pass Rate, and ToolLLaMA-Retriever to 66.42% Pass Rate. Execution feedback thus helps them by about as much as it helps HYSET, whose own Pass Rate gains 7.0% relative over its annotation-only value of 65.14%. The GPT-4 Pass Rate margin consequently falls from 8.8% to 4.3% relative while the COMP@5 margin falls only from 10.8% to 9.3%. Supplementary Material C.2 reports every method under both regimes and C.3 the execution budget. 3.3 Ablation Study Table 2 isolates the two design choices supporting our claims. Removing FsetF_set reduces COMP@5 by 13.1% and Pass Rate by 16.8%, showing that joint set scoring drives the gains. Removing execution feedback reduces Pass Rate by 6.5% but COMP@5 by only 0.7%, since ℒselfL_self rewards executable sets rather than annotated ones. Set-level modeling therefore mainly improves completeness, while execution feedback mainly improves downstream success. Replacing cardinality-specific matrices with a learned shared matrix reduces performance, and identity matrices perform worse still. These results confirm that compatibility should vary with set size. Supplementary Material C.4, C.6, C.7 and C.8 report the detailed component results and robustness analyses. To determine whether the gains come from set-level modeling in general or from the cardinality-specific design of HYSET, we compare alternative scorers using the same frozen BERT backbone, shortlist, data and annotation-only objective. Cardinality prediction, maximal marginal relevance and facility-location maximization reach 64.72%, 65.93% and 67.16% COMP@5. DeepSets and the Set Transformer improve to 69.85% and 72.41%, but HYSET remains 6.4% better than the Set Transformer while using fewer parameters and lower latency. A shared cardinality-agnostic M reaches 72.87%, nearly matching the Set Transformer and isolating the cardinality index as the main difference from HYSET. The advantage remains under matched execution feedback, indicating that it comes from the model design rather than additional supervision. Supplementary Material C.11 gives the full results. 3.4 Generalization and Data Efficiency Setting Sup. Overall R@5 C@5 PR In-domain reference ID Full 84.75±0.0984.75± 0.09 77.55±0.1277.55± 0.12 69.69±0.6169.69± 0.61 Zero-shot transfer UT 0-shot 79.48±0.1979.48± 0.19 70.71±0.1170.71± 0.11 63.64±0.7963.64± 0.79 UC 0-shot 75.35±0.0975.35± 0.09 65.28±0.1465.28± 0.14 58.23±0.8558.23± 0.85 CD 0-shot 72.83±0.1772.83± 0.17 61.96±0.1861.96± 0.18 54.85±0.9254.85± 0.92 Few-shot target adaptation (UC) UC 1-shot 76.30±0.0776.30± 0.07 67.23±0.0867.23± 0.08 59.88±0.8859.88± 0.88 UC 5-shot 79.13±0.1079.13± 0.10 71.40±0.1471.40± 0.14 63.93±0.8163.93± 0.81 UC 10-shot 81.58±0.1281.58± 0.12 74.38±0.1074.38± 0.10 66.48±0.7766.48± 0.77 UC Full 83.57±0.1183.57± 0.11 76.60±0.1976.60± 0.19 68.80±0.7468.80± 0.74 Table 3: Generalization and data efficiency of HYSET (BERT). Table 3 evaluates three zero-shot settings: held-out tools (UT), held-out categories (UC), and cross-domain transfer (CD). For UC, we additionally report 1-, 5-, and 10-shot adaptation, where each shot provides one labeled example per target category, together with full target-category supervision. All values are percentages. Performance decreases as the shift grows, yet CD retains 79.9% of in-domain set completeness, and 5-shot UC recovers 93.2% of fully supervised performance. A stricter protocol that also excludes held-out tools from every negative pool produces comparable results. The fixed-library splits isolate query and label shifts rather than tool novelty. When the complete tool set is unseen during training, HYSET improves COMP@5 over the strongest baseline by 13.1%. The improvement rises to 15.9% when an unseen tool pair is also required. On UltraTool, whose library is disjoint from ToolBench, HYSET improves COMP@5 by 11.5% over ToolGen. Direct transfer from ToolBench without target training retains 77.9% of the performance obtained by training on UltraTool. Supplementary Material D.1 to D.4 give the full setup and results. 3.5 The Set Delivered to the Agent The metrics above use rankings of fixed length, whereas the agent receives E^(x) E(x), a set whose size varies, with about half as many tools as a top-5 retriever. Direct evaluation shows that HYSET improves coverage of the complete required set by about 7% over ToolGen and nearly doubles exact match accuracy. It also achieves a slightly higher Pass Rate at its predicted cardinality than when forced to return five tools. The maximizer and greedy ranking agree on nearly 90% of queries, indicating that their main difference is set size. Supplementary Material C.10 gives the full metrics and the breakdown by predicted cardinality. 3.6 Cost, Sensitivity and Scalability Inference costs 12.412.4 ms and 3.213.21 GB per query at ||=13,860|V|=13,860, against 6.56.5 ms and 1.711.71 GB for the single dense pass of ToolLLaMA-Retriever and 378.4378.4 ms and 6.836.83 GB for ToolGen. Training costs 43.243.2 GPU hours and USD 186186 in judge calls on two RTX 4090 GPUs, against 38.638.6 GPU hours for ToolGen. Enlarging |||V| with distractor tools leaves Recall@5 and COMP@5 stable, since the reranking cost depends on KpoolK_pool and M alone. Varying η, λ and KpoolK_pool one at a time leaves HYSET stable, and the matrix expanding the shortlist in Eq. (13) changes COMP@5 by at most 0.40%, compared with the 1.07% loss caused by removing the expansion. Supplementary Material E and F give the sweeps and the runtime analyses. 4 Conclusion We cast LLM tool retrieval as query-conditioned hyperedge prediction and introduced HYSET to score candidate tool sets jointly through cardinality-specific interactions. HYSET consistently improves retrieval and end-to-end success and transfers to held-out tools and categories. Extending the method to dynamically growing tool libraries remains future work. References F. Battiston, G. Cencetti, I. Iacopini, V. Latora, M. Lucas, A. Patania, J. Young, and G. Petri (2020) Networks beyond pairwise interactions: structure and dynamics. Physics reports 874, p. 1–92. External Links: Document Cited by: §2.1. M. Grabisch, J. Marichal, and M. Roubens (2000) Equivalent representations of set functions. Mathematics of Operations Research 25 (2), p. 157–178. External Links: Document Cited by: §2.4. X. Hong, S. Xu, and Z. Yu (2026) HYVINT: intensity-driven hypergraph generation with variational representations. arXiv.org. External Links: Document Cited by: §2.4. S. Huang, W. Zhong, J. Lu, Q. Zhu, J. Gao, W. Liu, Y. Hou, X. Zeng, Y. Wang, L. Shang, et al. (2024) Planning, creation, usage: benchmarking LLMs for comprehensive tool utilization in real-world complex scenarios. In Annual Meeting of the Association for Computational Linguistics, p. 4363–4400. External Links: Document Cited by: §3.1. G. Izacard, M. Caron, L. Hosseini, S. Riedel, P. Bojanowski, A. Joulin, and E. Grave (2021) Unsupervised dense information retrieval with contrastive learning. Trans. Mach. Learn. Res.. Cited by: §1, §3.1. V. Karpukhin, B. Oguz, S. Min, P. Lewis, L. Wu, S. Edunov, D. Chen, and W. Yih (2020) Dense passage retrieval for open-domain question answering. In Conference on Empirical Methods in Natural Language Processing, p. 6769–6781. External Links: Document Cited by: §2.4. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang (2023) Lost in the middle: how language models use long contexts. Transactions of the association for computational linguistics 12, p. 157–173. External Links: Document Cited by: §1. A. v. d. Oord, Y. Li, and O. Vinyals (2018) Representation learning with contrastive predictive coding. arXiv.org. Cited by: §2.4. S. G. Patil, T. Zhang, X. Wang, and J. E. Gonzalez (2023) Gorilla: large language model connected with massive apis. Neural Information Processing Systems 37, p. 126544–126565. External Links: Document Cited by: §1. Y. Qin, S. Liang, Y. Ye, K. Zhu, L. Yan, Y. Lu, Y. Lin, X. Cong, X. Tang, B. Qian, et al. (2023) Toolllm: facilitating large language models to master 16000+ real-world apis. In International Conference on Learning Representations, Vol. 2024, p. 9695–9717. External Links: Document Cited by: §1, §1, §2.4, §3.1, §3.1, §3.1, 6. C. Qu, S. Dai, X. Wei, H. Cai, S. Wang, D. Yin, J. Xu, and J. Wen (2024) Towards completeness-oriented tool retrieval for large language models. In International Conference on Information and Knowledge Management, p. 1930–1940. External Links: Document Cited by: §1, §1, §3.1, §3.1. N. Reimers and I. Gurevych (2019) Sentence-bert: sentence embeddings using siamese bert-networks. In Conference on Empirical Methods in Natural Language Processing, p. 3980–3990. External Links: Document Cited by: §1, §2.4. S. Robertson and H. Zaragoza (2009) The probabilistic relevance framework: bm25 and beyond. Vol. 4, Emerald. External Links: Document Cited by: §1, §3.1. J. Robinson, C. Chuang, S. Sra, and S. Jegelka (2020) Contrastive learning with hard negative samples. International Conference on Learning Representations. Cited by: §2.4. T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom (2023) Toolformer: language models can teach themselves to use tools. Neural Information Processing Systems 36, p. 68539–68551. External Links: Document Cited by: §1. Y. Shen, K. Song, X. Tan, D. Li, W. Lu, and Y. Zhuang (2023) Hugginggpt: solving ai tasks with chatgpt and its friends in hugging face. Neural Information Processing Systems 36, p. 38154–38180. External Links: Document Cited by: §1. Y. Tay, V. Q. Tran, M. Dehghani, J. Ni, D. Bahri, H. Mehta, Z. Qin, K. Hui, Z. Zhao, J. Gupta, et al. (2022) Transformer memory as a differentiable search index. Neural Information Processing Systems 35, p. 21831–21843. External Links: Document Cited by: §1. K. Turnbull, S. Lunag’omez, C. Nemeth, and E. Airoldi (2019) Latent space modeling of hypergraph data. Journal of the American Statistical Association 119 (548), p. 2634–2646. External Links: Document Cited by: §2.4. A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin (2017) Attention is all you need. Neural Information Processing Systems 30. External Links: Document Cited by: §2.4. R. Wang, X. Han, L. Ji, S. Wang, T. Baldwin, and H. Li (2024) Toolgen: unified tool retrieval and calling via generation. In International Conference on Learning Representations, Vol. 2025, p. 73473–73498. External Links: Document Cited by: §1, §3.1, §3.1. S. Wu, G. Xu, and J. Zhu (2024) A general latent embedding approach for modeling non-uniform high-dimensional sparse hypergraphs with multiplicity. arXiv preprint arXiv:2410.12108. Cited by: §2.4. S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao (2022) React: synergizing reasoning and acting in language models. International Conference on Learning Representations. Cited by: §1. Y. Zheng, Z. Zhang, C. Ma, Y. Yu, J. Zhu, Y. Wu, T. Xu, B. Dong, H. Zhu, R. Huang, et al. (2026) Skillrouter: skill routing for llm agents at scale. arXiv.org. External Links: Document Cited by: §2.4. Y. Zheng, P. Li, W. Liu, Y. Liu, J. Luan, and B. Wang (2024) Toolrerank: adaptive and hierarchy-aware reranking for tool retrieval. In International Conference on Language Resources and Evaluation, p. 16263–16273. External Links: Document Cited by: §2.4, §3.1.