Paper deep dive
ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover
Minwoo Kim, Soochang Song, Namyoon Lee, Bang Chul Jung, Yongjune Kim
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 8/16/2026, 2:48:47 AM
Summary
The paper proposes ImpactHO, a framework for multi-user edge LLM handover that optimizes key-value (KV) cache transfer over saturated backhaul links. By ordering KV cache entries by importance (using Fast KVzip) and transmitting only the most informative fraction, the system maximizes average inference accuracy. The accuracy gain is modeled as a sigmoid utility function, allowing the allocation problem to be solved via a closed-form weighted water-filling algorithm that ensures convexity and online scheduling capabilities.
Entities (7)
Relation Signals (6)
KV Cache → transferredby → ImpactHO
confidence 98% · ImpactHO... orders each user’s cache by importance and transmits its most informative entries first
ImpactHO → evaluatedon → RULER
confidence 95% · fits measurements on the RULER benchmark
ImpactHO → optimizes → Backhaul
confidence 95% · We cast the transfer as a multi-user backhaul allocation problem
ImpactHO → solvesusing → Weighted Water-Filling
confidence 93% · The optimum is derived via a closed-form weighted water-filling solution
ImpactHO → uses → Fast KVzip
confidence 92% · we obtain the importance ordering from the gating-network scores of Fast KVzip
ImpactHO → testedwith → Qwen3-8b
confidence 90% · We use Qwen3-8B [28] as the primary LLM
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Edge LLMs must preserve inference continuity when a user hands over between edge nodes, requiring key-value (KV) cache transfer to the target node. However, simultaneous handovers saturate the backhaul, preventing full cache delivery within the mobility-imposed transfer window. Rather than allocating bandwidth as if all cache entries were equally valuable, we order each user's KV cache by importance and transmit only its most informative fraction, turning token-level sparsity into communication savings. We cast the transfer as a multi-user backhaul allocation problem that maximizes average accuracy across users. Each user's partial-cache accuracy serves as its utility: a sigmoid that fits measurements on the RULER benchmark with $R^2>0.99$ across models and context lengths. Because importance ordering front-loads the high-value entries, the concave region of the accuracy curve spans nearly the entire cache. Our proposed allocator keeps served users within this region, making each per-slot allocation problem convex. The optimum is derived via a closed-form weighted water-filling solution that generalizes information-theoretic water-filling and enables online scheduling. The proposed allocator attains over 93.7% average accuracy in a 500ms transfer window, within 0.5pp of the full-cache ceiling, and reaches 98.2-99.5% of a clairvoyant upper bound.
Tags
Links
- Source: https://arxiv.org/abs/2608.10545v1
- Canonical: https://arxiv.org/abs/2608.10545v1
Trouble viewing inline? Open PDF directly →
Full Text
61,948 characters extracted from source content.
Expand or collapse full text
ImpactHO: Importance-Aware KV Cache Transfer for Multi-User Edge LLM Handover Minwoo Kim, Soochang Song, Namyoon Lee, Bang Chul Jung, and Yongjune Kim M. Kim, S. Song, N. Lee, and Y. Kim are with the Department of Electrical Engineering, Pohang University of Science and Technology (POSTECH), Pohang 37673, South Korea (e-mail: minwoo.kim, ssc6351, nylee, yongjune@postech.ac.kr) B. C. Jung is with the Department of Electrical and Computer Engineering, Ajou University, Suwon 16499, South Korea (e-mail: bcjung@ajou.ac.kr). Abstract Edge LLMs must preserve inference continuity when a user hands over between edge nodes, requiring key-value (KV) cache transfer to the target node. However, simultaneous handovers saturate the backhaul, preventing full cache delivery within the mobility-imposed transfer window. Rather than allocating bandwidth as if all cache entries were equally valuable, we order each user’s KV cache by importance and transmit only its most informative fraction, turning token-level sparsity into communication savings. We cast the transfer as a multi-user backhaul allocation problem that maximizes average accuracy across users. Each user’s partial-cache accuracy serves as its utility: a sigmoid that fits measurements on the RULER benchmark with R2>0.99R^2>0.99 across models and context lengths. Because importance ordering front-loads the high-value entries, the concave region of the accuracy curve spans nearly the entire cache. Our proposed allocator keeps served users within this region, making each per-slot allocation problem convex. The optimum is derived via a weighted water-filling solution that generalizes information-theoretic water-filling and enables online scheduling. The proposed allocator attains over 93.7% average accuracy in a 500ms transfer window, within 0.5p of the full-cache ceiling, and reaches 98.2–99.5% of a clairvoyant upper bound. I Introduction Next-generation communication networks are evolving beyond simple data delivery into an infrastructure that supports real-time artificial intelligence (AI) inference for applications such as large language model (LLM) services, autonomous driving, and robotics [25, 14]. These services share two requirements: ultra-low latency and seamless mobility. The first is difficult to meet with remote cloud inference, whose round-trip time alone can exceed the millisecond-scale latency budget of real-time AI [19]. Multi-access edge computing (MEC) addresses this by hosting computation at the network edge, close to users. For AI workloads, this paradigm has taken shape as Edge AI and, more recently, Edge LLM, in which model inference runs directly on edge nodes [15, 8]. However, unlike cloud-based serving, edge inference is directly exposed to user mobility. Whenever a moving user is reassigned to another edge node, inference continuity must be preserved across the handover. Although virtual machine or container migration has been studied for MEC handover [26, 18, 20], modern AI models carry tens to hundreds of gigabytes of weights (e.g., 16 GB16\,GB for 8B-parameter models), making such migration impractical and pushing handover latency to tens of seconds. Since the model itself is already provisioned at each edge node, only the user-specific context needs to move. Transformer-based LLMs store this context as a key-value (KV) cache, and recent work proposes treating the KV cache itself as the transfer target [13]. Unlike transferring the raw text or tokens, which requires re-prefilling at the target node, transferring the KV cache lets the target node resume inference immediately. However, the KV cache is substantially larger than the raw text it encodes, so concurrent long-context handovers still saturate the backhaul. Figure 1: Concept of ImpactHO framework. A natural remedy is to transfer only part of the KV cache. Recent studies on KV cache eviction have shown that not all entries are equally important at inference time: LLMs can retain most of their accuracy using only a small fraction of their cache [27, 29, 16, 10, 9]. These results, however, address a single model’s memory budget, not how a shared backhaul should be divided across users. To address this gap, we propose ImpactHO (Importance-aware KV cache transfer for multi-user handover), a framework that orders each user’s cache by importance and transmits its most informative entries first, as illustrated in Fig. 1. To capture how accuracy grows with the delivered cache, we model each user’s accuracy as a function of its received cache fraction and treat this curve as a per-user utility. The scheduling task then becomes a utility-maximization problem: allocate the limited backhaul bandwidth across users so as to maximize the average accuracy. Importance ordering plays a structural role beyond prioritizing informative cache. Partial-cache accuracy is intrinsically concave once enough context has arrived for the model to become operable. By front-loading the highest-value entries, importance ordering pulls this operable point (i.e., the inflection point of the accuracy curve) down to only a small cache fraction. The concave region therefore spans nearly the entire cache, and every user can be brought into it at low cost. An admission rule then lifts every served user past this point, so each per-slot allocation reduces to a convex subproblem, whose optimum takes a weighted water-filling form with each user’s cache size acting as its weight. We summarize our main contributions as follows: • ImpactHO framework: We formulate importance-ordered partial KV cache transfer as a multi-user backhaul allocation problem for edge LLM handover, repurposing per-entry importance scores from KV cache eviction to set the transmission order. Modeling each user’s partial-cache accuracy as its utility casts the scheduling task as a utility-maximization problem over the shared bandwidth. • Empirical sigmoid characterization of partial-cache accuracy: On the RULER benchmark [5], partial-cache accuracy follows a sigmoid (R2>0.99R^2>0.99) robustly across context lengths, models, and ordering schemes. Importance ordering pulls its inflection point down to about 6.5 %6.5\,\% of the cache, so the concave region spans nearly the entire cache. This validates our utility assumption and provides a reusable parametric foundation for KV cache-aware networking research. • Two-regime allocator with low overhead: We derive the per-slot optimum in closed form as a weighted water-filling solution over a feasible region expanded by importance ordering. It reduces to classical water-filling as a special case and runs in near-linear time per slot. Coupled with an admission rule that suppresses starvation under heavy load, the allocator consistently outperforms baselines and reaches near-full accuracy faster than target-side re-prefill of even a single 8K context. I Related Work The KV cache stores the key and value tensors of previously processed tokens at each Transformer layer and attention head, avoiding their recomputation during autoregressive decoding. However, because its memory footprint grows linearly with the context length, the KV cache has become a major bottleneck for long-context inference. To alleviate this bottleneck, eviction-based methods [27, 29, 16, 10, 9] estimate the importance of cached entries and retain only the most important ones, thereby reducing memory consumption while largely preserving model accuracy. Among these, KVzip [10] derives query-agnostic per-(layer, head, token) importance scores from a context-reconstruction pass, reaching near-lossless accuracy with less than 30 %30\,\% of the cache. Fast KVzip [9] distills these scores into lightweight gating modules offline, achieving comparable accuracy without the inference-time scoring overhead. While prior work uses these importance scores only for memory reduction, we repurpose them to set the transmission order over the backhaul. Since the future query is unavailable at handover time, query-agnostic importance estimation is particularly well suited to our setting; hence, we adopt Fast KVzip. Nevertheless, our framework can accommodate any scoring method that provides a per-entry importance ranking. Beyond the memory pressure addressed by eviction, moving the cache between nodes is itself a bottleneck. In disaggregated datacenter serving, DistServe [30] and Splitwise [22] place prefill and decode on separate instances, requiring the KV cache to be transferred between them. However, these systems are designed for datacenter environments with high-bandwidth interconnects and do not address mobility-driven handovers or contention among multiple migrating users over bandwidth-constrained backhaul links. More closely related to our setting, CacheGen [17] streams a compressed KV cache adaptively, but focuses on loading a single context rather than allocating shared backhaul bandwidth across concurrent handovers. Such KV compression is complementary to our allocation framework: it can be applied before transmission to further reduce the required backhaul traffic. While sharing our motivation to preserve inference continuity under mobility, a recent edge LLM handover scheme ctHO [13] adopts a fundamentally different formulation. It minimizes the maximum handover delay across users, couples resource allocation with target-side computation, and requires the handover timing of all users to be known in advance. In contrast, we transmit only a selected fraction of each KV cache and maximize inference accuracy subject to an online per-slot backhaul budget, thereby decoupling backhaul allocation from target-side compute. ImpactHO supports anytime inference because the target can resume decoding after any completed prefix of the KV cache stream using the partial cache received up to that point. In contrast, re-prefill must complete prefill over the full context, while ctHO must complete its prescribed cache-transfer and target-side re-prefill operations before decoding can resume. Hence, neither baseline supports inference before full context restoration. Table I summarizes these structural differences, and Section VI-D provides a quantitative comparison. TABLE I: Structural comparison of handover strategies. Re-prefill ctHO [13] ImpactHO Target-GPU prefill Full Partial None Scheduling N/A Offline Online Anytime property No No Yes Our importance-ordered KV cache transfer shares the principle of task-oriented and semantic communications, which prioritize information according to its relevance to the given task rather than bit-level fidelity [3, 6]. A recent instantiation of this principle is token communications, in which tokenized multimodal source signals are transmitted and reconstructed at the receiver [24, 23]. Our setting differs in what is transmitted: not source tokens, but the per-token KV cache entries constituting the internal inference state of a deployed LLM. The target node consumes these entries directly to resume inference, without reconstructing the original source. By sending only a high-value fraction in descending order of measured importance, we extend this paradigm to mainstream LLM serving. Our framework also relates to network utility maximization (NUM), which shares a fixed capacity by maximizing the sum of per-user utilities. For elastic flows with concave utilities, NUM admits globally optimal distributed solutions [7]. Real-time inelastic flows are a closer match to our setting: their utilities are sigmoidal, and this shape makes the problem nonconvex, so standard dual algorithms can miss a feasible global optimum. Prior work has addressed this regime in two ways: sub-threshold flows can self-regulate, turning off when a persistently low net utility signals infeasibility [12]; alternatively, link capacity can be provisioned large enough to guarantee convergence of the distributed algorithm [4]. In both approaches, the application determines a fixed utility curve, and the network can only select an operating point along that curve. In our setting, by contrast, the utility is empirically measured and depends not only on the user’s context length but also on the order in which KV cache entries are transmitted. Our approach is therefore to shape the utility itself through importance ordering, so that the concave regime covers nearly the entire operating range. I System Model and Problem Formulation I-A Framework Overview The overall concept of ImpactHO is illustrated in Fig. 1. While serving a user, the source node assigns an importance score to each KV cache entry and maintains the entries in descending order of importance. Consequently, any prefix of the ordered cache contains the highest-ranked entries for the corresponding cache size. When a handover occurs, the source node streams these entries to the target node in importance order over the shared backhaul. If the finite transfer window expires before the entire cache is delivered, this ordering ensures that the target has received the most valuable subset available under the transfer budget and can resume inference using the resulting partial cache. When multiple handovers concurrently contend for the shared backhaul, ImpactHO allocates the per-slot transmission budget among users to maximize their aggregate inference accuracy. The remainder of this section formalizes the KV cache size and importance-induced accuracy utility and formulates the resulting per-slot resource-allocation problem. I-B Multi-user Edge LLM Handover We consider an edge node hosting a shared LLM that concurrently serves multiple users, reflecting a typical edge deployment in which GPU memory constraints favor sharing a single model instance across users. Each user i maintains an individual context stored at the edge node as a KV cache of size Li=2nLnHdhqTibits,L_i=2n_Ln_Hd_hqT_i , (1) where the factor of two accounts for the key and value tensors, nLn_L is the number of transformer layers, nHn_H is the number of KV heads, dhd_h is the head dimension, q is the number of bits used to represent each scalar, and TiT_i is the context length. LiL_i depends on the transformer architecture and grows linearly with the context length TiT_i. For example, the KV cache of Qwen3-8B [28] occupies approximately 1.2 GB1.2\,GB (9.66 Gb9.66\,Gb) for an 8K-token context. Once a handover is triggered, we assume that LiL_i remains fixed throughout the KV cache transfer. An edge inference node may serve users across multiple radio cells; hence, inference-state migration is required only when a user moves beyond the coverage of its current node. To preserve inference continuity, the source node transfers the user’s KV cache to the target node. In AI-RAN architectures for beyond-5G networks, edge inference nodes may be deployed at the distributed unit (DU) or central unit (CU) level and connected through fiber-based transport networks using interfaces such as F1 or X2/Xn [11]. For brevity, we collectively refer to the transport path between the source and target nodes as the backhaul. KV cache transfers share a backhaul bandwidth of B bits/s among the active handover users, where B denotes the bandwidth allocated to KV cache transfer, not the raw physical link capacity. We model backhaul allocation in discrete time slots of duration Δt t. At the beginning of each slot, the scheduler observes the set of active handover users and determines their allocations, which remain fixed throughout the slot. A handover request arriving during a slot becomes eligible for scheduling at the beginning of the next slot. We focus on the given slot in which N users concurrently undergo handover and compete for the available backhaul bandwidth B, i.e., per-slot budget of BΔtB t bits. We denote by tN_t the index set of these active handover users, so that |t|=N|N_t|=N. We consider a single transfer direction, with B the bandwidth provisioned for that direction; the N contending users are those handing over in it, and the reverse direction forms a symmetric instance. I-C Importance-aware KV Cache Ordering We assign an importance score to each individual KV cache entry, defined as the key–value pair associated with one token at a specific layer and KV head, rather than to an entire token. Each entry occupies 2dhq2d_hq bits (0.5 KB0.5\,KB for Qwen3-8B in BF16), and user i’s cache therefore contains nLnHTin_Ln_HT_i entries. The source node orders these entries by decreasing importance and transmits them in that order. Consequently, any delivered prefix is the top-ranked subset of its size: when a fraction xi∈[0,1]x_i∈[0,1] of user i’s cache has arrived, the target node holds the highest-ranked xiLix_iL_i bits. As xix_i increases, newly transmitted entries have progressively lower importance, naturally inducing diminishing returns. This observation motivates the concave accuracy-utility model introduced in Section I-D. In this paper, we obtain the importance ordering from the gating-network scores of Fast KVzip [9], chosen for its near-state-of-the-art retention quality at low computational cost. Importance-ordered transmission does introduce implementation overhead beyond standard cache transfer, which we quantify and discuss in Section VI-E. Our optimization framework is not restricted to Fast KVzip or even to importance-based ordering: any fixed ordering, including random ordering, can be accommodated as long as its induced utility is monotonically increasing and concave over the operating region considered; Section IV examines this condition in detail. I-D Utility Function We define the per-user utility Ai(y)A_i(y) as the inference accuracy for user i when the target node has received a fraction y∈[0,1]y∈[0,1] of the user’s importance-ordered KV cache. Once the target node has received a sufficient fraction, increasing y adds progressively lower-ranked entries to those already delivered, so accuracy is expected to improve with diminishing marginal gains. This motivates the following assumption, on which the optimal-allocation analysis of Section V relies. Assumption 1 (Concave Operating Region) For each user i, there exists a concavity anchor τi∈(0,1) _i∈(0,1) such that Ai(y)A_i(y) is continuously differentiable, strictly increasing, and strictly concave on [τi,1][ _i,1]. We restrict the analysis to y∈[τi,1]y∈[ _i,1]. Operationally, τi _i denotes the minimum cache fraction at which the target enters the concave operating region considered for resource allocation. Below τi _i, the received KV cache entries may be insufficient for reliable task performance, and AiA_i need not be concave. Above τi _i, transmitting additional, progressively lower-ranked entries yields diminishing accuracy gains. Section IV shows that the measured accuracy curves exhibit operating regions consistent with Assumption 1. I-E Optimization Problem At the beginning of a slot, the target node has received a fraction xix_i of user i’s KV cache. The scheduler assigns user i a normalized cache-delivery rate bib_i, in cache fractions per second, so that the received fraction at the end of the slot is yi≜xi+biΔt.y_i x_i+b_i t. (2) Because the full cache contains LiL_i bits, this allocation corresponds to a transmission rate of LibiL_ib_i bits/s. We maximize the aggregate accuracy improvement achieved during the slot: maximizebi *maximize_\b_i\ ∑i=1N[Ai(yi)−Ai(xi)] _i=1^N [\,A_i(y_i)-A_i(x_i)\, ] (3) subject to ∑i=1NLibi≤B, _i=1^NL_i\,b_i≤ B, bi≥0,τi≤yi≤1,∀i∈1,…,N. b_i≥ 0, _i≤ y_i≤ 1, ∀\,i∈\1,…,N\. The first constraint enforces that the aggregate transmission rate does not exceed the available backhaul bandwidth B. The remaining constraints prevent the received cache fraction from decreasing, restrict each user to the concave operating region specified in Assumption 1, and cap the received fraction at the full cache. Because bi≥0b_i≥ 0 is equivalent to yi≥xiy_i≥ x_i, the two lower bounds on yiy_i can be combined as x~i≜maxxi,τi x_i \x_i, _i\, yielding the individual feasible interval yi∈[x~i,1]y_i∈[ x_i,1]. Moreover, xix_i is fixed at the beginning of the slot, so ∑iAi(xi) _iA_i(x_i) is constant and can be omitted from the objective. Using bi=(yi−xi)/Δtb_i=(y_i-x_i)/ t, Problem (3) is equivalent to minimizeyi *minimize_\y_i\ −∑i=1NAi(yi) - _i=1^NA_i(y_i) (4) subject to ∑i=1NLi(yi−xi)Δt≤B, _i=1^N L_i(y_i-x_i) t≤ B, x~i≤yi≤1,∀i∈1,…,N. x_i≤ y_i≤ 1, ∀ i∈\1,…,N\. The objective over the feasible set is convex and all constraints are affine. We solve this convex optimization problem in Section V. The formulation requires every active user to reach its concave operating region by the end of the slot. This is possible only when the available backhaul budget is sufficient to raise every user currently below its concavity anchor τ. Lemma 1 (Feasibility) Problem (4) is feasible if and only if B≥Bmin≜1Δt∑i=1NLi[τi−xi]+,B\;≥\;B_ \; \; 1 t _i=1^NL_i\,[ _i-x_i]^+, (5) where [⋅]+≜max⋅, 0[\,·\,]^+ \·,\,0\. Proof: Any feasible yi\y_i\ satisfies yi≥x~iy_i≥ x_i, hence yi−xi≥maxxi,τi−xi=[τi−xi]+.y_i-x_i≥ \x_i, _i\-x_i=[ _i-x_i]^+. It must therefore transmit at least ∑iLi[τi−xi]+ _iL_i[ _i-x_i]^+ bits during the slot, proving the necessity of (5). Conversely, if (5) holds, choosing yi=x~iy_i= x_i for every user satisfies both the individual bounds and the backhaul constraint. ∎ When B<BminB<B_ , not all active users can reach their concave operating regions within the slot, and we instead invoke the fallback policy described in Section V. IV Utility Function Characterization Recall from Section I-D that Ai(y)A_i(y) denotes the inference accuracy for user i as a function of its received cache fraction y, and that Assumption 1 posits a concavity anchor τi _i beyond which Ai(y)A_i(y) is concave. We hypothesize that Ai(y)A_i(y) exhibits a sigmoidal shape: accuracy remains low when the received cache is insufficient, increases rapidly once enough informative entries have been delivered, and eventually saturates because subsequently delivered entries are progressively lower ranked and provide smaller marginal gains. This section evaluates this hypothesis through systematic measurements. In addition to supporting the analysis in Section V, the resulting characterization provides a compact parametric utility model for importance-ordered KV cache transfer. IV-1 Empirical Setup We test this hypothesis by fitting four canonical sigmoid families—algebraic, logistic, error function (erf), and arctan—to measured LLM accuracy across multiple context lengths. Although prior KV cache eviction studies have reported the qualitative dependence of LLM accuracy on the retained cache fraction [29, 9], our objective is to identify a parametric model that both accurately represents the measured utility and permits tractable resource-allocation analysis. We use Qwen3-8B [28] as the primary LLM, and additionally evaluate Qwen3-14B and Llama-3.1-8B-Instruct [2] to assess robustness across model architectures and sizes. As the benchmark, we use RULER [5], a long-context evaluation suite comprising diverse tasks at multiple context lengths. We compare the importance ordering produced by Fast KVzip with random ordering as a baseline. IV-2 Fitting Results (a) Fast KVzip, 8K context (b) Random ordering, 8K context Figure 2: Algebraic sigmoid fits to the measured RULER accuracy as a function of the received KV cache fraction at 8K context length for Qwen3-8B, Qwen3-14B, and Llama-3.1-8B-Instruct: (a) Fast KVzip and (b) random ordering. Markers indicate measured RULER accuracy, and solid curves show the algebraic sigmoid fits defined in (6). Fig. 2 shows that the algebraic sigmoid fits well for Qwen3-8B at 8K (R2>0.999R^2>0.999, RMSE =0.936=0.936). The algebraic sigmoid remains accurate across the 4K, 8K, and 16K context lengths, achieving R2>0.999R^2>0.999 in every case. Thus, the observed sigmoidal behavior is not specific to a single context length. The remaining three families fit the same measurements equally well, so the sigmoidal shape is a property of the measured accuracy. The fitted inflection points are stable across the four functional families, lying within the narrow range τ∈[0.064,0.067]τ∈[0.064,0.067]. Moreover, the algebraic sigmoid achieves R2>0.99R^2>0.99 for both Qwen3-14B and Llama-3.1-8B-Instruct under both Fast KVzip and random ordering. Collectively, these results support the existence of a concave operating region consistent with Assumption 1 across the considered models, context lengths, and transmission orderings. IV-3 Choice of Functional Form A practical utility model should provide both an accurate empirical fit and a tractable marginal-utility inversion. Although all four fitted families yield optimal solutions, we adopt the algebraic sigmoid because its marginal-utility inverse involves only arithmetic operations and radicals: Ai(y) A_i(y) =Mi2( 1+ki(y−τi)1+ki2(y−τi)2), = M_i2\! (\,1+ k_i(y- _i) 1+k_i^2(y- _i)^2\, ), (6) Ai′(y) A_i (y) =Miki2(1+ki2(y−τi)2)3/2. = M_i\,k_i2\, (1+k_i^2(y- _i)^2 )^3/2. (7) The parameters MiM_i, kik_i, and τi _i represent the upper accuracy asymptote, transition sharpness, and concavity anchor, respectively. These parameters may vary across users because of differences in context length and task characteristics. V Importance-Aware Resource Allocation via Weighted Water-Filling V-A Optimal Weighted Water-Filling Structure The solution to Problem (4) exhibits a weighted water-filling structure, analogous to generalized water-filling solutions for constrained resource allocation [21]. A single dual price coordinates all users, but their water levels differ according to their cache sizes and marginal-utility curves. Each user’s allocation fills the positive gap between its current progress xix_i and its price-dependent water level. Theorem 1 (Weighted Water-Filling) Under Assumption 1, if B≥BminB≥ B_ , the primal optimum is yi⋆=maxxi,Wi(λ⋆),bi⋆=yi⋆−xiΔt,y_i = \x_i,W_i(λ )\, b_i = y_i -x_i t, (8) where Wi(λ)≜1,si(λ)≤Ai′(1),(Ai′)−1(si(λ)),Ai′(1)<si(λ)<Ai′(τi),τi,si(λ)≥Ai′(τi).W_i(λ) cases1,&s_i(λ)≤ A_i (1),\\[2.84526pt] (A_i )^-1 (s_i(λ) ),&A_i (1)<s_i(λ)<A_i ( _i),\\[2.84526pt] _i,&s_i(λ)≥ A_i ( _i). cases (9) Here, we set si(λ)≜λLi/Δts_i(λ) λ L_i/ t, and λ⋆≥0λ ≥ 0 is a dual-optimal variable associated with the backhaul-rate constraint. When the budget binds, λ⋆λ can be chosen to satisfy ∑iLi(yi⋆−xi)Δt=B. _i L_i(y_i -x_i) t=B. (10) Since AiA_i is strictly concave on [τi,1][ _i,1], its derivative is strictly decreasing on this interval, and the inverse in the second branch is well defined. These optimal solutions are derived via the Karush–Kuhn–Tucker (KKT) conditions of Problem (4); a formal proof is provided in Appendix A. Let zi=Li(yi−xi)z_i=L_i(y_i-x_i) denote the number of additional cache bits transmitted for user i. For every user satisfying x~i<yi⋆<1 x_i<y_i <1, dAi(xi+zi/Li)dzi|zi=zi⋆=Ai′(yi⋆)Li=λ⋆Δt, . dA_i(x_i+z_i/L_i)dz_i |_z_i=z_i = A_i (y_i )L_i= λ t, (11) where zi⋆=Li(yi⋆−xi)z_i =L_i(y_i -x_i). Thus, the optimal allocation equalizes the marginal accuracy gain per additional transmitted cache bit across all interior users. Theorem 1 does not require a particular parametric utility family. Specializing the result to a given family requires only evaluating (Ai′)−1(A_i )^-1 for the interior branch, while the boundary cases are handled by (9). We illustrate this using the algebraic and logistic sigmoids characterized in Section IV. Example 1 (Water Levels for Sigmoid Utilities) For the interior branch of (9), the algebraic sigmoid in (6) gives (Ai′)−1(si(λ))=τi+1ki(Miki2si(λ))2/3−1.(A_i )^-1 (s_i(λ) )= _i+ 1k_i ( M_ik_i2s_i(λ) )^2/3-1. (12) For the logistic sigmoid Ai(y)=Mi/(1+e−ki(y−τi))A_i(y)=M_i/ (1+e^-k_i(y- _i) ), it gives (Ai′)−1(si(λ))=τi+1kiarcosh(Miki2si(λ)−1).(A_i )^-1 (s_i(λ) )= _i+ 1k_iarcosh ( M_ik_i2s_i(λ)-1 ). (13) We adopt the algebraic specialization because it combines strong empirical fit with efficient per-user allocation: once the dual price is determined, each user’s optimal allocation can be evaluated using only arithmetic operations and radicals. When B<BminB<B_ , the feasibility condition of Lemma 1 fails and the closed form above no longer applies; Section V-B handles this regime with an explicit admission policy. V-B Fallback Policy and Resource Allocation Algorithm By Lemma 1, Problem (4) is feasible if and only if B≥BminB≥ B_ . When B<BminB<B_ , no allocation can bring every active user into its concave region within the current slot. The scheduler must therefore invoke a fallback admission policy that prioritizes users under the insufficient backhaul budget. The benefit of such an explicit policy is evaluated empirically in Section VI-C (Fig. 5(b)). As our default fallback, we adopt equalized bytes (EB). Let ≜i∈t:xi<τiS \i _t:x_i< _i\ denote the users that have not yet reached their concavity anchors. EB equalizes the number of cache bits delivered during the slot among the users in S. Since all users share the same slot duration, this is equivalent to initially assigning each user i∈i a physical backhaul rate of B/||B/|S|, corresponding to the relative allocation biEB=B||Lib_i^EB= B|S|L_i. Each relative allocation is capped at the rate required to complete the remaining cache transfer within the slot, (1−xi)/Δt(1-x_i)/ t. If a user reaches this cap, the released backhaul rate is redistributed equally among the remaining unfinished users in S. Users outside S receive no bandwidth under the fallback policy. By prioritizing users that have not yet reached their anchors, EB mitigates sub-τ starvation under heavy load. Algorithm 1 Importance-Aware Resource Allocation Input: Active user index set tN_t with user parameters (Li,xi,τi,Ai)i∈t\(L_i,x_i, _i,A_i)\_i _t, total bandwidth B, slot duration Δt t. Output: Bandwidth allocation bi⋆i∈t\b_i \_i _t. 1:←i∈t:xi<τiS←\i _t:x_i< _i\ ⊳ Sub-τ users 2:Bmin←1Δt∑i∈Li(τi−xi)B_ ← 1 t _i L_i( _i-x_i) 3:if B<BminB<B_ then ⊳ Infeasibility 4: return EqualizedBytes(,B,Δt) EqualizedBytes(S,B, t) 5:end if 6:λ⋆←FindWaterLevel(t,B,Δt)λ ← FindWaterLevel(N_t,B, t) 7:for each i∈ti _t do 8: yi⋆←max(xi,Wi(λ⋆))y_i ← (x_i,W_i(λ )) ⊳ Optimal target 9: bi⋆←(yi⋆−xi)/Δtb_i ←(y_i -x_i)/ t ⊳ Bandwidth assignment 10:end for 11:return bi⋆i∈t\b_i \_i _t The complete two-regime allocator is summarized in Algorithm 1. It first evaluates the feasibility threshold BminB_ . If B<BminB<B_ , it invokes the EB fallback. Otherwise, it computes the optimal allocation characterized in Theorem 1 by locating a dual-optimal price λ⋆λ through the bisection subroutine FindWaterLevel. FindWaterLevel exploits the monotonicity of the aggregate demand g(λ)≜∑i=1NLi(yi⋆(λ)−xi)/Δtg(λ) _i=1^NL_i (y_i (λ)-x_i )/ t, which is continuous and non-increasing in λ. If g(0)≤Bg(0)≤ B, the entire remaining cache fits within the slot and λ⋆=0λ =0; otherwise, bisection solves g(λ)=Bg(λ)=B to tolerance ε . Each evaluation of g costs O(|t|)O(|N_t|), and the search range is bounded by λ¯ λ, the price at which g(λ¯)=Bming( λ)=B_ , so the allocator runs in O(|t|log(λ¯/ε))O(|N_t| ( λ/ )) time per slot, i.e., near-linear in the number of active users. V-C Connection to Classical Water-Filling When all users have the same utility function and cache size, i.e., Ai(⋅)=A(⋅)A_i(·)=A(·) and Li=L_i=L, the user-specific water levels Wi(λ)W_i(λ) reduce to a common level W(λ)W(λ), and users differ only in their current progress xix_i. When the backhaul constraint is active, let W⋆≜W(λ⋆)W W(λ ) and ≜i∈t:xi<W⋆A \i _t:x_i<W \ denote the set of users receiving positive allocations. The budget equality then gives W⋆=1||∑i∈xi+BΔt||L,bi⋆=1Δt[W⋆−xi]+.W = 1|A| _i x_i+ B\, t|A|\,L, b_i = 1 t [W -x_i ]^+. (14) This has the same algebraic form as classical water-filling [1, 21], with the channel-dependent ground level replaced by the progress floor xix_i and the total power replaced by the deliverable cache fraction BΔt/LB t/L. Then, a user with greater current progress has a higher ground level and requires less additional bandwidth to reach the common target level. VI Experimental Results VI-A Experiment Settings VI-A1 Simulation environment We follow the slotted system model of Section I. At the beginning of each time slot of duration Δt t, the scheduler observes all active users tN_t and updates the bandwidth allocation, which remains fixed throughout the slot. Handover events arrive according to a Poisson process with mean rate ρ (users per second). Upon each event, the context length of a new user is uniformly sampled from 4K, 8K, 16K\4K,\,8K,\,16K\ tokens to model heterogeneous context lengths. Each user attempts to complete its handover within an allowed transfer window TmaxT_ . Unless otherwise noted, the default settings are B=20 GbpsB=$20\,Gbps$, Δt=100 ms t=$100\,ms$, Tmax=500 msT_ =$500\,ms$, ρ=4 users/sρ=$4\,users/s$, and context lengths drawn uniformly from 4K,8K,16K\4K,8K,16K\, with each point averaged over 100 s100\,s of simulation across 100 Monte-Carlo runs. We adopt B=20 GbpsB=$20\,Gbps$ as a beyond-5G baseline. While over-provisioned networks (≥ 50 Gbps50\,Gbps) make allocation trivial, and severely limited ones reduce the problem to pure admission control, 20 Gbps20\,Gbps serves as a practical yet challenging operating point; its sensitivity is examined in Section VI-B. Similarly, the slot duration Δt= t= 100 ms100\,ms balances the re-allocation period against scheduler overhead. The transfer window Tmax=T_ = 500 ms500\,ms is a mobility-imposed budget for completing the transfer in the background, not an inference-time latency: in soft-handover architectures, the target can begin loading the cache before the user formally migrates. When the window expires, the transfer is truncated, and the user resumes inference on whatever prefix has arrived. We analyze the sensitivity of both Δt t and TmaxT_ in Section VI-C. All experiments use Qwen3-8B [28] in BF16 precision with KV cache importance scored by Fast KVzip [9]. The per-user accuracy utility Ai(⋅)A_i(·) follows the algebraic sigmoid form of Eq. (6), with parameters Mi,ki,τiM_i,k_i, _i fitted independently for each context length 4,8,16\4,8,16\K on the RULER benchmark [5] as detailed in Section IV. Because delivery proceeds in importance order, the received fraction xix_i in the simulation indexes exactly the top-xix_i fraction of Section I-C. Thus, the reported accuracy at each slot is obtained by evaluating this fitted sigmoid at the user’s current xix_i. To further validate that this sigmoid-based simulation tracks real inference under the scheduler-induced distribution of xix_i, we also measure end-to-end accuracy under the same simulation settings. It is obtained by having each user perform inference using the directly delivered top-xix_i fraction of the cache. VI-A2 Baselines and metrics Throughout, Ours refers to the proposed two-regime allocator. When the per-slot budget suffices to lift every sub-τ user to the concavity anchor (i.e., the feasibility regime), it applies the weighted water-filling of Theorem 1. Otherwise, it falls back to an admission policy that distributes the budget equally among the contending sub-τ users. We compare the proposed allocator against three baselines, each of which applies a fixed rule in every slot, irrespective of feasibility. • Equal allocation (EA): The backhaul capacity is distributed uniformly across active users so that each user receives bandwidth Libi=B/|t|L_ib_i=B/|N_t| bps\,bps, independent of importance or current progress. • Winner-take-all (WTA): The greedy heuristic which assigns the slot budget to the user with the largest single-slot marginal accuracy gain. We use the cascading variant throughout: when the chosen user cannot absorb the full slot budget (its remaining cache is smaller than B⋅ΔtB· t), the leftover rolls over to the next-best user within the same slot. The pure WTA, which discards any leftover capacity, is strictly dominated and is therefore omitted from the main comparisons. • Proportional-fair (PF): The unweighted proportional-fair allocation that maximizes ∑ilog(Liyi) _i (L_iy_i) subject to the shared budget, without accessing the accuracy curves [7]. In our single-resource setting, its KKT solution reduces to a common-water-level water-filling on cumulative received bits, which we solve directly. It thus represents a fairness-oriented baseline that is agnostic to the sigmoidal accuracy utility. These baselines span the fairness–throughput spectrum of resource allocation, with EA and WTA at its two extremes and PF occupying the middle ground [7]. All schemes share the same importance ordering, differing only in allocation: among the baselines, only WTA consults the accuracy utility, while EA and PF are utility-agnostic. Additionally, we compare the proposed method against compute-based baselines: a target-side re-prefill strategy and the hybrid token and cache transmission design from [13]. VI-B Main Results (a) Accuracy vs. ρ. (b) Accuracy vs. B. Figure 3: Performance comparison of the proposed allocator against EA, PF, and WTA baselines under edge-LLM handover conditions across (a) varying handover frequency ρ and (b) varying backhaul bandwidth B. Solid curves are computed from the simulation, evaluating each user’s accuracy at its delivered fraction xix_i through the fitted sigmoid utility Ai(xi)A_i(x_i). Markers additionally show end-to-end measured accuracy. Each marker averages the first 40004000 users of the trace, and error bars are 95% confidence intervals over RULER samples. The close agreement between curves and markers confirms that the sigmoid-based simulation reliably predicts real inference accuracy. We first evaluate the performance of the proposed allocator against the baselines along two axes that jointly govern the system load: the handover frequency ρ and the backhaul bandwidth B. In Fig. 3, the solid curves denote the simulation accuracy derived from the fitted sigmoid utility in Fig. 2, whereas the markers with error bars indicate the measured end-to-end accuracy. Focusing on Fig. 3(a), ρ is varied over 18users/s. As ρ increases, the accuracy of all schemes decreases, since the fixed budget B must be shared among more concurrent transfers. The proposed allocator outperforms the baselines throughout, and the gap widens, since heavy loads make importance-aware prioritization essential for maximizing accuracy with scarce resources. At the default ρ=4 users/sρ=$4\,users/s$, it attains 93.7 %93.7\,\% accuracy, within 0.5 p0.5\,pp of the 94.1 %94.1\,\% full-cache ceiling. In Fig. 3(b), we sweep B over 1030Gbps. The result demonstrates that the proposed allocator consistently outperforms all three baselines across the entire operating range. Among the baselines, the two extremes, EA and WTA, exhibit a characteristic crossover. In the bandwidth-scarce regime, WTA outperforms EA by concentrating the limited budget to ensure at least one user reaches the operable region, whereas EA leaves all users starved. Conversely, in the bandwidth-abundant regime, the ordering reverses: EA benefits from serving multiple users in parallel, while WTA wastes capacity by driving its current winner deep into the saturated tail of its utility before advancing to the next. PF weights each share by progress and cache size, yet trails the proposed allocator: blind to the accuracy curve, it over-serves saturated users and under-serves those near τ. VI-C Sensitivity and Ablation Analysis (a) Accuracy vs. Δt t. (b) Accuracy vs. TmaxT_ . Figure 4: Performance comparison of the proposed allocator against baselines across (a) varying slot duration Δt t and (b) varying transfer window TmaxT_ . We next examine robustness to two system configuration parameters: the slot duration Δt t, and the transfer window TmaxT_ . Fig. 4(a) sweeps Δt t over 2020–100100 ms with all other parameters at their defaults. The proposed allocator stays above all three baselines across the entire range. EA and WTA cross near Δt≈ t≈ 90 ms90\,ms: as the slot gets coarser, the better fixed rule switches from one to the other. The proposed allocator, however, keeps its accuracy even at the coarsest slots, so it can re-allocate far less often without losing quality. In Fig. 4(b), the transfer window is swept over 2001000ms, showing the same crossover near 400 ms400\,ms: short windows penalize EA, while longer ones let its parallelism overtake WTA. The proposed allocator again outperforms throughout, so its advantage is not tied to a particular TmaxT_ . (a) Effect of ordering. (b) Admission control policy. Figure 5: The impacts of (a) importance-aware KV cache ordering relative to random ordering and (b) the admission control mechanism. To analyze the sources of the performance improvements, we isolate the two core mechanisms behind our method in Fig. 5: importance-based cache ordering and the admission-control policy. For the latter, we compare the default policy (Ours (EB), which refers to equalized bytes) against the same allocator with cascading WTA as its fallback (Ours (WTA)). Ours (WTA) is distinct from the standalone WTA baseline used above, which applies cascading WTA in every slot. Fig. 5(a) sweeps ρ for the proposed allocator under Fast KVzip and random ordering. Importance ordering keeps average accuracy above 90 %90\,\% across the entire range, whereas random ordering falls sharply from 56.7 %56.7\,\% at ρ=2ρ=2 to 20.2 %20.2\,\% at ρ=8ρ=8. This demonstrates that importance-based ordering strengthens ImpactHO’s anytime inference capability by transmitting higher-ranked cache entries first, thereby improving inference accuracy at intermediate transfer points. Fig. 5(b) sweeps ρ over 412users/s to stress the infeasibility regime. Both variants with admission control are compared against the same allocator with the fallback disabled (No admission). At low ρ, the three curves coincide, as the feasibility regime dominates. As ρ grows, both variants with admission control pull away sharply from the one without it, confirming that admission control is the dominant factor in this regime. We define the sub-τ starvation rate as the fraction of users whose received fraction never reaches τ within the transfer window TmaxT_ . At ρ=12 users/sρ=$12\,users/s$, the sub-τ starvation rate is 20.8 %20.8\, 37 without admission versus 0.65 %0.65\, 37 with EB. At our main slot duration (Δt=100 ms t=$100\,ms$), EB consistently outperforms WTA, whose starvation rate is 15.5 %15.5\, 37 versus EB’s 0.65 %0.65\, 37 . This is because WTA drives one user into the saturated tail beyond τ before advancing the next, whereas EB lets several progress toward operability. VI-D Comparison with Compute-Based Baselines TABLE I: Per-user latency comparison of the compute-based baselines and the proposed method for Qwen3-8B. Context Re-prefill (ms) Hybrid (ms) Ours (ms) NVIDIA RTX PRO 6000 PRO 6000, 20 Gbps20\,Gbps 20 Gbps20\,Gbps 4K 300 211 229 8K 676 396 353 16K 1,566 694 590 Avg 847 434 391 We evaluate the per-user latency performance of the proposed method by comparing it against compute-based re-prefill baselines. In Table I, Re-prefill reconstructs the context directly from the raw tokens, whereas Hybrid transmits both tokens and KV cache to balance the prefill processing time and cache transmission latency, a scheme appropriately modified from [13] for our evaluation. In the hybrid baseline, KV cache importance is not taken into account, and the backhaul bandwidth is allocated entirely in a first-come, first-served (FCFS) manner. Because the data size of the tokens is negligible compared to the KV cache, we do not consider token transmission latency. For comparison, we generate user arrivals following a Poisson process with ρ=4ρ=4 over the time window t∈[0,1]t∈[0,1] s\,s, and repeat this process 1,000 times. For each user, the service latency is measured from the moment of request arrival. Specifically, for the re-prefill and hybrid baselines, latency spans until the context is fully restored; for the proposed method, it lasts until a sufficient amount of the KV cache is transferred to reach 99 %99\, 37 of the full accuracy. Here, the transfer window TmaxT_ is set to ∞, so that no scheme is truncated by the deadline. We assume that both the re-prefill and hybrid baselines are provisioned with sufficient GPUs to process each user request independently, so their latencies are free of compute contention. This idealized assumption establishes a much stricter baseline than a direct comparison with [13]. At a context length of 4K, the proposed method shows comparable performance to the baselines. However, as the context length increases, the proposed method outperforms the baselines, which suffer from severe computational overhead during full context recomputation. Moreover, the proposed method can further reduce latency by leveraging its anytime property, while ensuring graceful performance degradation. VI-E Discussion The reported gains require importance-ordered transmission, which adds two source-side costs and a small wire overhead. (i) Scoring: assigning a Fast KVzip [9] score to every KV entry is a one-time, per-session computation done while the source still serves the user, not at handover. (i) Sorting: reordering by score is an O(nlogn)O(n n) sort over indices, a small fraction of the transfer time. (i) Metadata: because the transfer may be truncated, each entry carries its 3-byte (layer,head,token)(layer,head,token) coordinate against the 0.5 KB0.5\,KB payload of Section I-C, under 1 %1\,\% overhead. The proposed allocator optimizes each slot rather than the horizon. To bound the resulting loss, we compare against a clairvoyant upper bound that observes all future arrivals and maximizes the objective without the concave-region restriction. Solved to certified global optimality via a piecewise-linear MILP with a validated upper correction, this bound never underestimates the true continuous optimum. The proposed allocator attains 98.299.5% of it across all loads (within 0.53 %0.53\,\% at ρ=1ρ=1, 1.77 %1.77\,\% at ρ=8ρ=8). Thus, neither the myopic objective nor the concave-region restriction costs much in practice, though the widening gap at higher load suggests horizon-aware scheduling as a direction for future work. VII Conclusion We presented the ImpactHO framework, an importance-aware framework for maximizing the average inference accuracy across users during edge LLM handover subject to limited backhaul bandwidth. ImpactHO combines three key components: (1) importance-ordered sequential KV cache transfer, which exploits sparsity in token-level cache importance to transmit the most useful entries first; (2) an empirically validated sigmoid characterization of inference accuracy with partially transferred caches; and (3) an optimal weighted water-filling allocator whose homogeneous special case reduces to classical water-filling. In realistic edge regimes, importance-ordered transfer attains near-full-cache accuracy with lower latency than baselines. Moreover, across a range of slot durations, backhaul bandwidths, and concurrent handover loads, the proposed allocator consistently achieves higher average accuracy than the baselines. These results confirm that jointly accounting for KV cache importance and backhaul resource allocation enables accurate and efficient LLM handover at the network edge. Appendix A Proof of Theorem 1 Recall the effective lower bound x~i=max(τi,xi) x_i= ( _i,\,x_i) from Section I, which collapses the box constraint of (4) to the single interval yi∈[x~i,1]y_i∈[ x_i,1]. We work in the feasibility regime B≥BminB≥ B_ of Lemma 1. By Assumption 1, −Ai-A_i is strictly convex on [τi,1][ _i,1] and the constraint set is a polytope, so Problem (4) is convex with a strictly convex objective. The Lagrangian is ℒ(,λ,,)= (y,λ, μ, ν)= −∑i=1NAi(yi)+λ(∑i=1NLi(yi−xi)Δt−B) - _i=1^NA_i(y_i)+λ\! ( _i=1^N L_i(y_i-x_i) t-B ) +∑i=1Nμi(x~i−yi)+∑i=1Nνi(yi−1). + _i=1^N _i( x_i-y_i)+ _i=1^N _i(y_i-1). (15) The KKT conditions for the optimal (⋆,λ⋆,⋆,⋆)(y ,λ , μ , ν ) are as follows: ∑i=1NLi(yi⋆−xi)Δt≤B,λ⋆≥0, _i=1^N L_i(y_i -x_i) t≤ B,\;λ ≥ 0, (16) λ⋆(∑i=1NLi(yi⋆−xi)Δt−B)=0, λ \! ( _i=1^N L_i(y_i -x_i) t-B )=0, (17) x~i≤yi⋆≤1,μi⋆,νi⋆≥0, x_i≤ y_i ≤ 1,\; _i , _i ≥ 0, (18) μi⋆(x~i−yi⋆)=0,νi⋆(yi⋆−1)=0 _i ( x_i-y_i )=0,\; _i (y_i -1)=0 (19) for i∈1,…,Ni∈\1,…,N\. From ∂ℒ/∂yi=0 /∂ y_i=0, the stationarity condition gives Ai′(yi⋆)=λ⋆LiΔt−μi⋆+νi⋆.A_i (y_i )= λ L_i t- _i + _i . (20) By Assumption 1, Ai′A_i is strictly decreasing on [τi,1][ _i,1], hence invertible. Stationarity (20) with complementary slackness (19) gives the three branches of (9): the interior case μi⋆=νi⋆=0 _i = _i =0 yields Ai′(yi⋆)=si(λ⋆)A_i (y_i )=s_i(λ ), while νi⋆>0 _i >0 and μi⋆>0 _i >0 activate the bounds Wi=1W_i=1 and Wi=τiW_i= _i when si(λ⋆)≤Ai′(1)s_i(λ )≤ A_i (1) and si(λ⋆)≥Ai′(τi)s_i(λ )≥ A_i ( _i), respectively. Combining with yi⋆≥xiy_i ≥ x_i from bi⋆≥0b_i ≥ 0 gives yi⋆=maxxi,Wi(λ⋆).y_i \;=\; \x_i,\,W_i(λ )\. (21) It remains to characterize λ⋆λ . Define the aggregate demand g(λ)≜∑i=1NLi(yi⋆(λ)−xi)Δtg(λ)\; \; _i=1^N L_i\,(y_i (λ)-x_i) t (22) with yi⋆(λ)y_i (λ) given by (21). Because Ai′A_i is strictly decreasing on [τi,1][ _i,1], Wi(λ)W_i(λ) is strictly decreasing in λ on its interior region and constant elsewhere; taking maxxi,⋅ \x_i,·\ preserves this monotonicity, so g(λ)g(λ) is continuous and non-increasing on λ≥0λ≥ 0, with boundary values g(0)=∑iLi(1−xi)/Δtg(0)= _iL_i(1-x_i)/ t and g(λ¯)=Bming( λ)=B_ , where λ¯≜maxiAi′(τi)Δt/Li λ _iA_i ( _i) t/L_i. By the intermediate value theorem, a λ⋆≥0λ ≥ 0 with g(λ⋆)=Bg(λ )=B exists for every B∈[Bmin,g(0)]B∈[B_ ,g(0)], and for B>g(0)B>g(0) the budget is slack with λ⋆=0λ =0; λ⋆λ is unique when some user is strictly interior, and otherwise any such λ⋆λ yields the same ⋆y , which is unique by strict convexity. This completes the proof of Theorem 1. References [1] T. M. Cover and J. A. Thomas (2006) Elements of information theory. Second edition, Wiley-Interscience, Hoboken, NJ, USA. Cited by: §V-C. [2] A. Grattafiori et al. (2024-07) The Llama 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: §IV-1. [3] D. Gündüz, Z. Qin, I. E. Aguerri, H. S. Dhillon, Z. Yang, A. Yener, K. K. Wong, and C. Chae (2023-01) Beyond transmitting bits: context, semantics, and task-oriented communications. IEEE J. Sel. Areas Commun. 41 (1), p. 5–41. Cited by: §I. [4] P. Hande, S. Zhang, and M. Chiang (2007-12) Distributed rate allocation for inelastic flows. IEEE/ACM Trans. Netw. 15 (6), p. 1240–1253. External Links: Document Cited by: §I. [5] C. Hsieh, S. Sun, S. Kriman, S. Acharya, D. Rekesh, F. Jia, and B. Ginsburg (2024) RULER: what’s the real context size of your long-context language models?. In Proc. Conf. Lang. Model. (COLM), Cited by: 2nd item, §IV-1, §VI-A1. [6] J. Im, N. Kwon, T. Park, J. Woo, J. Lee, and Y. Kim (2024-11) Attention-aware semantic communications for collaborative inference. IEEE Internet Things J. 11 (22), p. 37008–37020. Cited by: §I. [7] F. P. Kelly, A. K. Maulloo, and D. K. H. Tan (1998) Rate control for communication networks: shadow prices, proportional fairness and stability. J. Oper. Res. Soc. 49 (3), p. 237–252. External Links: Document Cited by: §I, 3rd item, §VI-A2. [8] S. Kholmatov, S. Cho, S. Chong, and K. Lee (2025-08) AoRA: AI-on-RAN for backhaul-free edge inference. In Proc. ACM Conf. SIGCOMM, p. 1263–1265. Cited by: §I. [9] J. Kim, D. Han, and S. Yun (2026-01) Fast KVzip: efficient and accurate LLM inference with gated KV eviction. arXiv preprint arXiv:2601.17668. Cited by: §I, §I, §I, §I-C, §IV-1, §VI-A1, §VI-E. [10] J. Kim, J. Kim, S. Kwon, J. W. Lee, S. Yun, and H. O. Song (2025-10) KVzip: query-agnostic KV cache compression with context reconstruction. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), Cited by: §I, §I, §I. [11] L. Kundu, X. Lin, R. Gadiyar, J. Lacasse, and S. Chowdhury (2026-01) AI-ran: transforming ran with ai-driven computing infrastructure. IEEE Commun. Mag. 64 (1), p. 168–174. Cited by: §I-B. [12] J.-W. Lee, R.R. Mazumdar, and N.B. Shroff (2005-08) Non-convex optimization and rate control for multi-class services in the internet. IEEE/ACM Trans. Netw. 13 (4), p. 827–840. External Links: Document Cited by: §I. [13] S. Lee, J. Park, C. Zheng, and H. Park (2026-03) Low-latency edge LLM handover via joint KV cache transfer and token prefill. arXiv preprint arXiv:2603.28018. Cited by: §I, TABLE I, §I, §VI-A2, §VI-D, §VI-D. [14] K. B. Letaief, W. Chen, Y. Shi, J. Zhang, and Y. A. Zhang (2019-08) The roadmap to 6G: AI empowered wireless networks. IEEE Commun. Mag. 57 (8), p. 84–90. Cited by: §I. [15] E. Li, L. Zeng, Z. Zhou, and X. Chen (2020-01) Edge AI: on-demand accelerating deep neural network inference via edge computing. IEEE Trans. Wireless Commun. 19 (1), p. 447–457. Cited by: §I. [16] Y. Li, Y. Huang, B. Yang, B. Venkitesh, A. Locatelli, H. Ye, T. Cai, P. Lewis, and D. Chen (2024-12) SnapKV: LLM knows what you are looking for before generation. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), Vol. 37, p. 22947–22970. Cited by: §I, §I. [17] Y. Liu, H. Li, Y. Cheng, S. Ray, Y. Huang, Q. Zhang, K. Du, J. Yao, S. Lu, G. Ananthanarayanan, M. Maire, H. Hoffmann, A. Holtzman, and J. Jiang (2024) CacheGen: KV cache compression and streaming for fast large language model serving. In Proc. ACM Conf. SIGCOMM, p. 38–56. External Links: Document Cited by: §I. [18] A. Machen, S. Wang, K. K. Leung, B. J. Ko, and T. Salonidis (2018-02) Live service migration in mobile edge clouds. IEEE Wireless Commun. Mag. 25 (1), p. 140–147. Cited by: §I. [19] Y. Mao, C. You, J. Zhang, K. Huang, and K. B. Letaief (2017) A survey on mobile edge computing: the communication perspective. IEEE Commun. Surveys Tuts. 19 (4), p. 2322–2358. External Links: Document Cited by: §I. [20] M. V. Ngo, T. Luo, H. T. Hoang, and T. Q.S. Quek (2020) Coordinated container migration and base station handover in mobile edge computing. In Proc. IEEE Global Commun. Conf. (GLOBECOM), p. 1–6. External Links: Document Cited by: §I. [21] D. P. Palomar and J. R. Fonollosa (2005-02) Practical algorithms for a family of waterfilling solutions. IEEE Trans. Signal Process. 53 (2), p. 686–695. Cited by: §V-A, §V-C. [22] P. Patel, E. Choukse, C. Zhang, A. Shah, I. Goiri, S. Maleki, and R. Bianchini (2024) Splitwise: efficient generative llm inference using phase splitting. In Proc. ACM/IEEE Annu. Int. Symp. Comput. Archit. (ISCA), p. 118–132. External Links: Document Cited by: §I. [23] L. Qiao, M. B. Mashhadi, Z. Gao, and D. Gündüz (2025) Token-domain multiple access: exploiting semantic orthogonality for collision mitigation. In IEEE INFOCOM 2025 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), p. 1–6. External Links: Document Cited by: §I. [24] L. Qiao, M. B. Mashhadi, Z. Gao, R. Tafazolli, M. Bennis, and D. Niyato (2025-10) Token communications: a large model-driven framework for cross-modal context-aware semantic communications. IEEE Wireless Communications 32 (5), p. 80–88. Cited by: §I. [25] W. Saad, M. Bennis, and M. Chen (2020) A vision of 6G wireless systems: applications, trends, technologies, and open research problems. IEEE Netw. 34 (3), p. 134–142. Cited by: §I. [26] S. Wang, R. Urgaonkar, M. Zafer, T. He, K. Chan, and K. K. Leung (2019-06) Dynamic service migration in mobile edge computing based on Markov decision process. IEEE/ACM Trans. Netw. 27 (3), p. 1272–1288. Cited by: §I. [27] G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis (2024) Efficient streaming language models with attention sinks. In Proc. Int. Conf. Learn. Representations (ICLR), Cited by: §I, §I. [28] A. Yang et al. (2025-05) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §I-B, §IV-1, §VI-A1. [29] Z. Zhang, Y. Sheng, T. Zhou, T. Chen, L. Zheng, R. Cai, Z. Song, Y. Tian, C. Ré, C. Barrett, Z. Wang, and B. Chen (2023) H2O: heavy-hitter oracle for efficient generative inference of large language models. In Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), Vol. 36, p. 34661–34710. Cited by: §I, §I, §IV-1. [30] Y. Zhong, S. Liu, J. Chen, J. Hu, Y. Zhu, X. Liu, X. Jin, and H. Zhang (2024-07) DistServe: disaggregating prefill and decoding for goodput-optimized large language model serving. In Proc. USENIX Symp. Oper. Syst. Des. Implement. (OSDI), p. 193–210. Cited by: §I.