Paper deep dive
Goodput Maximization for Large Language Model Edge Inference: A Two-Phase Maskable PPO Approach
Xiaojing Chen, Qi Zhang, Wei Ni, Shunqing Zhang, Yanzan Sun
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/27/2026, 5:28:27 AM
Summary
This paper proposes a Two-Phase Maskable Proximal Policy Optimization (TP-MPPO) algorithm to maximize system goodput for Large Language Model (LLM) inference in wireless edge networks. The approach jointly optimizes task offloading decisions and bandwidth allocation while adhering to strict Service Level Objectives (SLOs) and GPU memory constraints. The first phase uses Maskable PPO (MPPO) with action masking to determine optimal offloading, avoiding invalid actions. The second phase derives closed-form solutions for uplink bandwidth allocation and employs a greedy algorithm for downlink allocation. Simulations show TP-MPPO improves system reward by 33.3%–87.5% compared to benchmarks.
Entities (9)
Relation Signals (7)
TP-MPPO → maximizes → Goodput
confidence 98% · This paper presents a novel two-phase maskable proximal policy optimization (TP-MPPO) algorithm, which maximizes the system goodput
TP-MPPO → optimizes → Bandwidth Allocation
confidence 95% · In the second phase, closed-form solutions are derived for uplink bandwidth allocation; a greedy algorithm is designed for downlink bandwidth allocation
TP-MPPO → optimizes → Task Offloading
confidence 95% · In the first phase of TP-MPPO, we optimize the task offloading decisions by MPPO with action masking mechanism
MPPO → uses → Action Masking
confidence 95% · we optimize the task offloading decisions by MPPO with action masking mechanism, effectively avoiding exploring invalid actions
TP-MPPO → ensurescompliancewith → SLO
confidence 92% · ...with strict service level objective (SLO) compliance for large language model (LLM) inference services
TP-MPPO → outperforms → Benchmarks
confidence 90% · Simulation results demonstrate that TP-MPPO can improve the system reward by 33.3%--87.5% compared to its benchmarks
Edge Node → hasconstraint → GPU Memory
confidence 85% · preventing out-of-memory (OOM) failures caused by excessive batch sizes... m_j^t <= M_j
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:This paper presents a novel two-phase maskable proximal policy optimization (TP-MPPO) algorithm, which maximizes the system goodput counting request throughput with strict service level objective (SLO) compliance for large language model (LLM) inference services in wireless edge networks. In the first phase of TP-MPPO, we optimize the task offloading decisions by MPPO with action masking mechanism, effectively avoiding exploring invalid actions and reducing the action space. In the second phase, closed-form solutions are derived for uplink bandwidth allocation; a greedy algorithm is designed for downlink bandwidth allocation to provide immediate rewards for the MPPO in the next round. The two stages alternate till convergence. Simulation results demonstrate that TP-MPPO can improve the system reward by 33.3%--87.5% compared to its benchmarks and achieve the highest goodput.
Tags
Links
- Source: https://arxiv.org/abs/2608.25543v1
- Canonical: https://arxiv.org/abs/2608.25543v1
Trouble viewing inline? Open PDF directly →
Full Text
35,393 characters extracted from source content.
Expand or collapse full text
Goodput Maximization for Large Language Model Edge Inference: A Two-Phase Maskable PPO Approach Xiaojing Chen Qi Zhang Wei Ni Shunqing Zhang Yanzan Sun Abstract This paper presents a novel two-phase maskable proximal policy optimization (TP-MPPO) algorithm, which maximizes the system goodput counting request throughput with strict service level objective (SLO) compliance for large language model (LLM) inference services in wireless edge networks. In the first phase of TP-MPPO, we optimize the task offloading decisions by MPPO with action masking mechanism, effectively avoiding exploring invalid actions and reducing the action space. In the second phase, closed-form solutions are derived for uplink bandwidth allocation; a greedy algorithm is designed for downlink bandwidth allocation to provide immediate rewards for the MPPO in the next round. The two stages alternate till convergence. Simulation results demonstrate that TP-MPPO can improve the system reward by 33.3%–87.5% compared to its benchmarks and achieve the highest goodput. Index Terms: Large language model, edge inference, resource allocation, task offloading. 00 0 †Work in the paper was supported by Shanghai Municipal Science and Technology Commission Foundation grants 25DP1500300 and 24DP1501001. (Corresponding author: Yanzan Sun.) X. Chen, Q. Zhang, S. Zhang and Y. Sun are with the Key Laboratory of Specialty Fiber Optics and Optical Access Networks, Shanghai University, Shanghai 200444, China. Emails: jodiechen, zhangqi_\_kawhi2, shunqing, yanzansun@shu.edu.cn. W. Ni is with the School of Engineering, Edith Cowan University, Perth, WA 6027, Australia. Email: wei.ni@ieee.org. I Introduction The rapid proliferation of large language models (LLMs) has surged inference demands, straining traditional cloud-based LLM inference services that struggle with throughput and latency [1]. Edge computing supplements cloud-based LLM inference by leveraging distributed resources to scale service capacity. Furthermore, deployment on nearby edge nodes effectively mitigates propagation latency. Unlike conventional edge scenarios, LLM services are constrained by intensive workloads and GPU scarcity. At the model and system level, efficient inference techniques such as KV cache compression [2] and speculative decoding [3] reduce per-request compute overhead, while selective batching [4] improves server-side utilization. These techniques are complementary to network-level resource management, which requires jointly optimizing task offloading and bandwidth allocation across heterogeneous users and wireless edge nodes [5, 6]. In [5], LLM task offloading, computational, bandwidth and graphics memory resource allocation were optimized to minimize the average latency of all LLM tasks. In [6], LLM task distribution, computational and communication resource allocation were optimized to maximize the system utility of 6G networks. However, the studies in [5] and [6] overlooked throughput, a critical metric for LLM service systems. Although some studies have targeted aggregate throughput [1], they have overlooked the fact that downstream applications have different latency requirements for user experience, leading to dramatically different service level objectives (SLOs) that need to be satisfied. The most widely used SLOs in LLM services are Time-to-First Token (TTFT), Time per Output Token (TPOT) and End-to-End (E2E) latency. Goodput [7], which captures request throughput under SLO attainment reflecting both cost and service quality, can be an effective metric for LLM service systems. To fill the void in goodput-oriented optimization, this paper presents a two-phase maskable proximal policy optimization (TP-MPPO) algorithm, which determines offloading and bandwidth allocation for edge LLM inference with GPU-constrained users, while preventing out-of-memory (OOM) failures caused by excessive batch sizes. The contributions of this paper are collated below. • We model a wireless edge LLM inference system capturing full E2E latency pipeline and GPU memory constraints, enabling a realistic formulation of joint offloading and resource allocation. • We formulate a mixed-integer nonlinear programming (MINLP) problem to maximize goodput, bridging the gap between raw throughput and service quality. The problem jointly optimizes task offloading and bandwidth allocation under SLO and memory capacity constraints. • We propose TP-MPPO, which decomposes the MINLP into an MPPO-based offloading phase and an analytical bandwidth allocation phase with closed-form uplink and greedy downlink solutions. This algorithm provides immediate rewards that drive stable policy convergence, e.g., within 200 episodes. Experiment results demonstrate that TP-MPPO improves the system reward by 33.3%–87.5% compared to its benchmarks and achieves the best goodput performance under diverse configurations of user and node numbers. The rest of this paper is organized as follows. Section I provides the system model. Section I formulates the considered problem. The TP-MPPO algorithm is proposed in Section IV and evaluated numerically in Section V, followed by concluding remarks in Section VI. I System Model Fig. 1: Illustration of the system supporting LLM inference services. We consider an LLM inference system comprising a service controller co-located with an access point (AP), J GPU-enabled edge nodes (=1,…,JJ=\1,…,J\), and N users (=1,…,NN=\1,…,N\); see Fig. 1. These users generate LLM inference requests but cannot perform local inference due to lack of GPU resources; they need to offload these tasks to the edge nodes. The AP acts as a centralized scheduler and controller, collecting requests and making offloading decisions [1]. As depicted in Fig. 2, the AP schedules requests on the basis of time slots =1,…,KT=\1,…,K\. The slot length is denoted as τ, which acts as a macroscopic scheduling window rather than an orthogonal frequency division multiplexing (OFDM) symbol duration. Requests arriving in slot t are buffered at the AP and scheduled at the end of this slot. Each request from user n is denoted by qnt=⟨snin,t,snout,t,ςnt⟩q_n^t= s_n^in,t,s_n^out,t, _n^t , where snin,ts_n^in,t is the input prompt length, snout,ts_n^out,t is the maximum output sequence length, and ςnt _n^t defines the E2E delay bound. The AP decides whether to offload a request to node j or reject it. The offloading decision of request qntq_n^t is an integer variable αnt∈∪0 _n^t ∪\0\, and αnt=0 _n^t=0 indicates that the request is rejected and failed. After scheduling, users transmit their inference prompts to the designated edge nodes, which aggregate the prompts into a single batch, perform inference, and return the output results. We assume each user initiates a new request only after receiving the previous response [4], with “thinking time” following an exponential distribution. The LLM inference process comprises two stages[8]: the prefill~stage, which processes inputs in parallel to initialize the Key-Value (KV) cache and generate the first token; and the decoding~stage, which utilizes the KV cache to auto-regressively generate subsequent tokens. Both stages rely on the Transformer architecture [9], with the key distinction that prefill computes self-attention across all input tokens concurrently, whereas decoding restricts attention to the new token and cached history. Let ont≤snout,to_n^t≤ s_n^out,t denote the total number of output tokens generated for request qntq_n^t. I-A Communication Model Fig. 2: Workflow of LLM inference system: Requests R1–R6 arrive in the first slot; the AP assigns R1–R4 to Node 1 and R5–R6 to Node 2. Subsequently, R7–R9 are allocated to Node 3, since Nodes 1 and 2 remain occupied with ongoing inference. The communication model focuses on prompt transmission and output feedback, with negligible overhead from inference request and scheduling. We consider OFDM, with each edge node operating on non-overlapping frequency bands to avoid interference. Let βnjup,t,βnjdown,t∈[0,1] _nj^up,t, _nj^down,t∈[0,1] denote the bandwidth fractions assigned by edge node j to user n (αnt=j _n^t=j) for uplink and downlink transmissions, respectively. Per time slot t, the uplink transmit rate of user n is given by Rnjup,t=βnjup,tBjuplog2(1+pnjup,thnjup,tψ),R_nj^up,t= _nj^up,tB_j^up _2(1+ p_nj^up,th_nj^up,tψ), (1) where BjupB_j^up is the uplink bandwidth of edge node j; ψ is the background noise power at the edge node; pnjup,tp_nj^up,t denotes the uplink transmit power of user n; and hnjup,th_nj^up,t is the channel gain from user n to edge node j. The wireless channel consists of large-scale path loss and small-scale fading: hnjup,t=ξnjup,th¯njuph_nj^up,t= _nj^up,t h_nj^up, where ξnjup,t _nj^up,t follows an exponential distribution with unit mean corresponding to Rayleigh fading, and h¯njup=(Ad3×1084πfcdnj)de h_nj^up=(A_d 3× 10^84π f_cd_nj)^d_e is the distance-dependent average channel gain, with AdA_d the antenna gain, fcf_c the carrier frequency, ded_e the path loss exponent, and dnjd_nj the distance between user n and edge node j. The channel gain hnjup,th_nj^up,t is treated as slot-level effective average value under a quasi-static fading assumption, as widely adopted in wireless edge computing models [1, 5, 6]. Likewise, the downlink transmit rate is Rnjdown,t=βnjdown,tBjdownlog2(1+pnjdown,thnjdown,tψ′)R_nj^down,t= _nj^down,tB_j^down _2(1+ p_nj^down,th_nj^down,tψ ), where BjdownB_j^down is the downlink bandwidth of edge node j; ψ′ψ is the background noise power at user n; pnjdown,tp_nj^down,t denotes the downlink transmit power of edge node j; and hnjdown,th_nj^down,t is the channel gain from edge node j to user n. I-B Latency and Memory Analysis For request qntq_n^t (if user n generates a request during slot t), let Tnup,tT_n^up,t denote its uplink latency, and Dnin,tD_n^in,t denote the data size of its input prompt. We have Tnup,t=Dnin,t/Rnjup,tT_n^up,t=D_n^in,t/R_nj^up,t. Likewise, the downlink latency of qntq_n^t is Tndown,t=Dnout,t/Rnjdown,tT_n^down,t=D_n^out,t/R_nj^down,t, where Dnout,tD_n^out,t is the data size of its output. Each uplink and downlink transmission is assumed to complete within one scheduling slot, over which the channel gain is treated as quasi-static. Extending the framework to account for transmissions spanning multiple slots is left for future work. The E2E latency of user n’s task is composed of five components. Since requests arriving at the AP are scheduled at the end of each time slot, a waiting time for scheduling Tnwait,tT_n^wait,t is incurred at the AP. Upon finishing the prompt transmission which takes Tnup,tT_n^up,t, user n may need to wait for other users assigned to the same node to complete their prompt transmissions before conducting batch inference, resulting in a batching wait time Tnstay,tT_n^stay,t at the edge node. Let jt=n|αnt=jN_j^t=\n| _n^t=j\ denote the set of requests assigned to edge node j, with batch size bjt=|jt|b_j^t=|N_j^t|. During batch inference at edge node j, all requests assigned to the same batch are processed together, and the batch completion latency is given by Tjinf,t=TjP,t+TjD,tT_j^inf,t=T_j^P,t+T_j^D,t, where TjP,tT_j^P,t and TjD,tT_j^D,t denote the prefill and decoding latency, respectively. Following the analytical latency model for Transformer inference in [1], the prefill latency is TjP,t=Lbjt(6sjth2+(4(sjt)2h+2sjth2)+4sjth)/FjT_j^P,t=Lb_j^t(6s_j^th^2+(4(s_j^t)^2h+2s_j^th^2)+4s_j^th h)/F_j and the decoding latency is TjD,t=Lbjt(ojt−1)(6h2+(4(sjt+ojt2)h+2h2)+4h)/FjT_j^D,t=Lb_j^t(o_j^t-1)(6h^2+(4(s_j^t+ o_j^t2)h+2h^2)+4h h)/F_j. Here, sjts_j^t is the padded input length, ojt=maxn∈jtonto_j^t= _n _j^to_n^t is the batch-level output length, L is the number of Transformer layers, h is the Transformer hidden dimension, h is the feed-forward network (FFN) hidden dimension, and FjF_j is the computational capacity (in FLOPs/s) of edge node j. The inference latency model captures the latency trends observed in practical inference systems, where larger batch sizes, longer sequences, and larger models increase computation latency, while stronger GPU resources reduce latency. Finally, the output feedback consumes Tndown,tT_n^down,t. The total E2E latency of user n is TnE2E,t=Tnwait,t+Tnup,t+Tnstay,t+Tjinf,t+Tndown,tT_n^E2E,t=T_n^wait,t+T_n^up,t+T_n^stay,t+T_j^inf,t+T_n^down,t. For batch inference, the memory footprint of edge node j is mjt=mwts+mjkv,tm_j^t=m^wts+m_j^kv,t, which consists of the memory for model weights, mwtsm^wts, and the KV cache footprint, mjkv,tm_j^kv,t. The KV cache footprint mjkv,t=4hLbjt(sjt+ojt)m_j^kv,t=4hLb_j^t(s_j^t+o_j^t) [1]. I Problem Formulation Let a binary variable nt∈0,1 r_n^t∈\0,1\ denote the completion status of request qntq_n^t, where nt=1 r_n^t=1 indicates that request qntq_n^t is successfully completed, and nt=0 r_n^t=0 indicates otherwise. Request qntq_n^t is considered successfully completed (nt=1 r_n^t=1) if it is assigned to an edge node (i.e., not rejected, αnt≠0 _n^t≠ 0), the batch inference finishes without an OOM failure, and the output is returned to the user. The system throughput can be given as Y=∑n∈∑t∈ntY= _n _t r_n^t. Larger batch sizes enhance throughput; conversely, oversized batch sizes can trigger memory exhaustion, causing the inference to fail. Let MjM_j denote the memory capacity of edge node j, which depends on its GPU resources. We define goodput as the number of completed requests satisfying the E2E latency SLO [7]. For request qntq_n^t, let nt∈0,1 g_n^t∈\0,1\ denote whether request qntq_n^t contributes to the goodput. nt=1 g_n^t=1 only if nt=1 r_n^t=1 and TnE2E,t≤ςntT_n^E2E,t≤ _n^t. Requests that are rejected, fail due to OOM, or are completed but violate the SLO are not counted. The system goodput is defined as Y′=∑n∈∑t∈ntY^ = _n _t g_n^t, a dimensionless metric representing the total number of SLO-compliant completed requests over the entire simulation horizon (i.e., all K time slots). Our goal is to maximize the system goodput by optimizing offloading decisions and bandwidth allocation, accounting for the unique challenges of LLM inference driven by dynamic KV cache footprint and heterogeneous SLO requirements. Let mjtm_j^t represent the memory footprint of edge node j for batch inference. The optimization problem is formulated as: 1: 1: maxt,up,t,down,ttY′ max _ \ subarrayc$ α^t$,$ β^up,t$,$ β^down,t$ subarray \_tY^ (2a) s.t. ∑n∈jtβnjup,t≤1,∑n∈jtβnjdown,t≤1,∀j,t, _n _j^t _nj^up,t≤ 1,~ _n _j^t _nj^down,t≤ 1,~~∀ j,t, (2b) mjt≤Mj,∀j,t. m_j^t≤ M_j, ∀ j,t. (2c) Here, t=αnt,∀n α^t=\ _n^t,∀ n\, up,t=βnjup,t,∀n,j β^up,t=\ _nj^up,t,∀ n,j\, and down,t=βnjdown,t,∀n,j β^down,t=\ _nj^down,t,∀ n,j\. Constraint (2b) limits uplink and downlink bandwidth allocation, while Constraint (2c) ensures that each edge node’s memory footprint remains within its capacity. Solving 1P1 is challenging because it is an NP-hard MINLP problem involving non-convex and discontinuous binary decisions. Moreover, LLM batch inference creates strong coupling among users, where the latency and memory consumption of a batch depend on the specific combination of assigned tasks. IV Proposed TP-MPPO Scheme We present TP-MPPO to address 1P1. In the first phase, MPPO determines the offloading decisions. In the second phase, the remaining problem is decomposed into two subproblems. Closed-form solutions are derived for uplink bandwidth allocation, while downlink bandwidth allocation is solved using a greedy-based algorithm. IV-A PPO Framework with Invalid Action Masking Upon inspecting 1P1, optimizing t α^t first makes the remaining problem over up,t,down,t\ β^up,t, β^down,t\ more tractable. We propose MPPO to tackle offloading decisions t α^t. MPPO extends PPO with invalid action masking [10], enabling agents to avoid infeasible actions under state-dependent constraints and complex action spaces. The MPPO components are designed as follows. IV-A1 State The state space is t=qnt,hnjup,t,hnjdown,t,Ijt,∀n,j s^t=\q_n^t,h_nj^up,t,h_nj^down,t,I_j^t, ∀ n,j\, where Ijt∈0,1I_j^t∈ \0,1 \ indicates whether edge node j is idle, taking 11 if idle and 00 otherwise. IV-A2 Action To prevent invalid node selection, we introduce a binary mask vector nt=vn,jt,∀jv_n^t=\v_n,j^t,∀ j\ for each user. Each element is dynamically generated based on the current environment: vn,jt=1v_n,j^t=1 when request qntq_n^t exists and node j is available at slot t (i.e., Ijt=1I_j^t=1); otherwise, vn,jt=0v_n,j^t=0. During actor network forward propagation, logits of invalid actions (vn,jt=0v_n,j^t=0) are set to −∞-∞. After the Softmax, their selection probability becomes zero. This Action Masking dynamically prunes the action space, ensuring t α^t is always executable. The resulting action space is t=t a^t= \ α^t \. IV-A3 Reward The objective is to maximize the system goodput while penalizing memory overflow failures. The immediate reward rtr^t is formulated as a normalized utility given by rt=νgood∑n∈nt−νoomNoom,tNreq,tr^t= _good _n g_n^t- _oomN^oom,tN^req,t, where Noom,tN^oom,t is the number of requests that failed due to memory overflow, and Nreq,tN^req,t is the total number of requests generated during slot t. νgood _good and νoom _oom are weight coefficients. MPPO selects the action t a^t by pruning invalid choices via the mask t=ntn=1Nv^t= \v_n^t \_n=1^N. For a given t α^t, the subproblems are solved to determine up,t β^up,t and down,t β^down,t. The complete action set t,up,t,down,t\ α^t, β^up,t, β^down,t\ then updates the environment, triggering the update of Ijt\I_j^t\ and the state transition from t s^t to t+1 s^t+1. Specifically, the behavior actor network (with parameters old θ_old) generates action t a^t under the masked policy πold(t|t,t) _ θ_old ( a^t| s^t,v^t ). The critic network (with parameters ω) estimates the state value V(t)V_ ω( s^t). Collected trajectories (t,t,t,rt,t+1)( s^t,v^t, a^t,r^t, s^t+1) are sampled from the replay buffer to update the target actor network θ by minimizing LCLIPt()=^t[min(gt()A^t,clip(gt(),−ϵ,+ϵ)A^t)]L_CLIP^t( θ)\!=\! E_t[ (g^t( θ) A^t,clip(g^t( θ),1\!-\!ε,1\!+\!ε) A^t)], where gt()=π(t|t,t)πold(t|t,t)g^t( θ)= _ θ ( a^t| s^t,v^t ) _ θ_old ( a^t| s^t,v^t ) is the masked-policy probability ratio, and ϵε is the clipping factor. A^t=∑i=0K−t(γ)iδt+i A^t= _i=0^K-t(γ)^iδ^t+i is the advantage function calculated via generalized advantage estimation [11], where γ is the discount factor and δt=rt+γV(t+1)−V(t)δ^t=r^t+γ V_ ω( s^t+1)-V_ ω( s^t) represents the temporal-difference error. The critic network is updated by minimizing Lt()=t[(δt)2]L^t ( ω )=E_t[ (δ^t )^2]. IV-B Closed-Form Solutions to Uplink Bandwidth Allocation Given t α^t, the subset of users offloaded to edge node j is determined. For user n∈jtn _j^t, the prompt transmission time Tnup,tT_n^up,t and waiting time Tnstay,tT_n^stay,t are dictated by the slowest user in the batch, as batch inference requires all prompts uploaded before padding. Thus, the uplink bandwidth allocation problem at node j is to minimize the maximum uplink latency among its assigned users, expressed as Tjup,t=maxn∈jtTnup,tT_j^up,t= _n _j^tT_n^up,t. The subproblem of uplink bandwidth allocation is given by 2:minβnjup,tn∈jtTjup,ts.t. ∑n∈jtβnjup,t≤1,∀j,t. 2: min _\ _nj^up,t\_n _j^tT_j^up,t~~ .t. ~ _n _j^t _nj^up,t≤ 1,~∀ j,t. (3) According to the min-max fairness principle, the optimal strategy equalizes uplink latencies across all users in the batch. Using the auxiliary variable ℜnjup,t=Bjuplog2(1+pnjup,thnjup,tψ) R_nj^up,t=B_j^up _2(1+ p_nj^up,th_nj^up,tψ) and equating Dnin,tβnjup,tℜnjup,t D_n^in,t _nj^up,t R_nj^up,t for all users n∈jtn _j^t, the optimal allocation is (βnjup,t)∗=Dnin,t/ℜnjup,t∑i∈jt(Diin,t/ℜijup,t)( _nj^up,t)^*= D_n^in,t/ R_nj^up,t _i _j^t(D_i^in,t/ R_ij^up,t). Algorithm 1 Optimal Solution to 3P3 1 Initialize satisfied set jt=∅S_j^t= , and cumulative bandwidth usage βj,sumdown,t=0 _j,sum^down,t=0. 2 for each user n∈jtn _j^t do 3 Calculate the minimum required bandwidth fraction βnj,mindown,t _nj,min^down,t. 4 end for 5 Sort users in jtN_j^t based on βnj,mindown,t _nj,min^down,t in ascending order to obtain the sorted index list =k1,…,k|jt|K=\k_1,…,k_|N_j^t|\. 6 for i=1i=1 to |jt||N_j^t| do 7 if βj,sumdown,t+βkij,mindown,t≤1 _j,sum^down,t+ _k_ij,min^down,t≤ 1 then 8 Add user kik_i to satisfied set: jt←jt∪kiS_j^t _j^t∪\k_i\. 9 Allocate minimum requirement: βkijdown,t=βkij,mindown,t _k_ij^down,t= _k_ij,min^down,t. 10 Update usage: βj,sumdown,t←βj,sumdown,t+βkij,mindown,t _j,sum^down,t← _j,sum^down,t+ _k_ij,min^down,t. 11 end if 12 end for 13 Obtain the optimal βnjdown,tn∈jt\ _nj^down,t\_n _j^t. IV-C Optimal Solution to Downlink Bandwidth Allocation After batch inference, user n must satisfy the SLO, with downlink latency Tndown,tT_n^down,t constrained by the residual margin Tnre,t=ςnt−Tnwait,t−Tnup,t−Tnstay,t−Tjinf,tT_n^re,t= _n^t-T_n^wait,t-T_n^up,t-T_n^stay,t-T_j^inf,t. The downlink bandwidth allocation problem at node j seeks to maximize the number of requests meeting the SLO, aligning with system goodput. Let jgood,t=n|n∈jt,Tndown,t≤Tnre,tN_j^good,t=\n|n _j^t,T_n^down,t≤ T_n^re,t\ be the set of requests contributing to goodput. The subproblem of downlink bandwidth allocation is then formulated as 3:maxβnjdown,tn∈jt|jgood,t|s.t. ∑n∈jtβnjdown,t≤1,∀j,t. 3: max _\ _nj^down,t\_n _j^t|N_j^good,t|~~ .t. _n _j^t _nj^down,t≤ 1,~~∀ j,t. (4) Based on the expressions of Rnjdown,tR_nj^down,t and Tndown,tT_n^down,t, we calculate the minimum required bandwidth fraction (subject to Tndown,t≤Tnre,tT_n^down,t≤ T_n^re,t) for user n by setting Tndown,t=Tnre,tT_n^down,t=T_n^re,t: βnj,mindown,t≜Dnout,tTnre,tBjdownlog2(1+pnjdown,thnjdown,tψ′). _nj,min^down,t D_n^out,tT_n^re,tB_j^down _2(1+ p_nj^down,th_nj^down,tψ ). (5) By introducing βnj,mindown,t _nj,min^down,t, we can reformulate problem 3P3 as finding the largest subset jt⊆jtS_j^t _j^t that satisfies the constraint ∑n∈jtβnj,mindown,t≤1 _n _j^t _nj,min^down,t≤ 1. This problem has a special equal-value structure, where all selected requests contribute equally (i.e., one unit) to the goodput. Hence, Algorithm 1 achieves the optimal solution by prioritizing requests with the smallest bandwidth requirements, attaining zero optimality gap. The complexity of Algorithm 1 is dominated by the sorting operation, which is (NlogN)O(N N). V Numerical Results Fig. 3: Reward as the number of episode grows. Fig. 4: Comparisons of throughput, goodput, TTFT, and TPOT. (a) (b) (c) Fig. 5: Goodput ratio with different numbers of nodes and users, and different edge-node GPU provisioning levels (Llama-30B). Consider a system where 3 edge nodes and 15 users are randomly deployed within a 300 m radius circular area. The input sequence length of inference request is randomly generated from 128,256,512\128,256,512\ tokens, with corresponding SLO targets of 3,5,9\3,5,9\ s. The maximum output sequence length snout,ts_n^out,t is randomly generated from 128,256,512,1024\128,256,512,1024\ tokens, and the realized output length is set as ont=λntsnout,to_n^t= _n^ts_n^out,t, where λnt∈0.5,0.75,1 _n^t∈\0.5,0.75,1\ [8, 1]. All these task parameters are independently sampled uniformly from their respective discrete sets. For communication modeling, we assume 32 bits per token, consistent with the INT32 representation commonly used for token IDs. To ensure the simulated edge nodes reflect a realistic deployment, the memory capacity and computational capacity are set according to the specifications of an NVIDIA L2 PCIe GPU with 24 GB memory. We adopt the native Llama-7B hyperparameters: h=4096h=4096 and L=32L=32. We also set fc=3.5f_c=3.5 GHz and de=3d_e=3, representing sub-6 GHz fifth-generation (5G) edge access and urban/suburban non-line-of-sight (NLOS) propagation, respectively, together with τ=1τ=1 s, ψ=10−14ψ=10^-14 W, ψ′=2×10−14ψ =2× 10^-14 W, ϵ=0.2ε=0.2, νgood=1 _good=1, and νoom=0.5 _oom=0.5. We compare TP-MPPO with four benchmarks: 1) MPPO, which learns all variables solely via MPPO. 2) TP-PPO [12], which adopts PPO without action masking for offloading decisions. 3) Rewardless [6], which adopts classical active inference for offloading without reward guidance. 4) Heuristic [13], which employs a weighted scoring mechanism based on channel conditions and memory load for offloading decisions. Fig. 3 shows the rewards of the considered algorithms. TP-MPPO yields the highest reward and converges within 200 episodes. The reward of TP-MPPO is 33.3%, 87.5% and 71.4% higher than those of MPPO, TP-PPO and Rewardless, respectively. Action masking in TP-MPPO and MPPO accelerates convergence by eliminating invalid action explorations. TP-PPO struggles with early-stage cold starts, while Rewardless converges to a low reward, indicating its free-energy-based guidance is insufficient to avoid OOM states. Fig. 4 shows the impact of model size on the throughput–goodput gap and latency. Lighter models achieve lower latency and higher throughput/goodput with a smaller gap, as their lower resource demands reduce SLO violations and OOM failures. In contrast, Llama-7B incurs a larger gap due to its higher memory and computational demands, highlighting the importance of goodput-oriented resource allocation and the trade-off between model capacity and service reliability. Figs. 5(a) and (b) demonstrate how number of edge nodes and users affect the ratio of goodput to total requests. In Fig. 5(a) (with 20 users), TP-MPPO surpasses all benchmarks. With 6 nodes, the goodput ratio of TP-MPPO exceeds MPPO, TP-PPO, Heuristic and Rewardless by 5.4%, 9.5%, 7.1% and 10.2%, respectively. In Fig. 5(b) (with 3 nodes), as the user count scales to 30, the goodput ratio of TP-MPPO exceeds MPPO, TP-PPO, Heuristic and Rewardless by 3%, 7.3%, 6.5% and 7%, respectively. The relatively low goodput ratios in Figs. 5(a) and (b) is caused by the constrained edge deployment scenario, where limited GPU resources and strict SLOs reduce the number of successfully served requests. Fig. 5(c) further evaluates the goodput ratio with the Llama-30B model under varying GPU provisioning levels. As GPU resources increase, TP-MPPO consistently outperforms the baselines, improving the goodput ratio from 42.0% to 60.7% by reducing OOM failures and inference latency. VI Conclusions This paper presented the TP-MPPO framework for edge LLM inference services, where the task offloading decisions were learned by MPPO, uplink bandwidth allocation was given in closed form, and downlink bandwidth allocation was decided by the greedy algorithm. Simulation results have demonstrated that TP-MPPO improves the reward by 33.3%–87.5% and outperforms the benchmarks significantly in goodput performance across various network configurations. References [1] X. Zhang, J. Nie, Y. Huang, G. Xie, Z. Xiong, J. Liu, D. Niyato, and X. Shen (2025) Beyond the cloud: edge inference for generative large language models in wireless networks. IEEE Trans. Wireless Commun. 24 (1), p. 643–658. External Links: Document Cited by: §I, §I, §I-A, §I-B, §I-B, §I, §V. [2] Z. Liu, J. Yuan, H. Jin, S. Zhong, Z. Xu, V. Braverman, B. Chen, and X. Hu (2024) KIVI: a tuning-free asymmetric 2bit quantization for KV cache. In International Conference on Machine Learning (ICML), Cited by: §I. [3] D. Xu, W. Yin, H. Zhang, X. Jin, Y. Zhang, S. Wei, M. Xu, and X. Liu (2025) EdgeLLM: fast on-device LLM inference with speculative decoding. IEEE Transactions on Mobile Computing 24 (4), p. 3256–3273. External Links: Document Cited by: §I. [4] G. Yu, J. S. Jeong, G. Kim, S. Kim, and B. Chun (2022) Orca: a distributed serving system for transformer-based generative models. In Proc. 16th USENIX Symp. Operating Syst. Design Implement. (OSDI), p. 521–538. Cited by: §I, §I. [5] Y. He, J. Fang, F. R. Yu, and V. C. Leung (2024) Large language models (LLMs) inference offloading and resource allocation in cloud-edge computing: an active inference approach. IEEE Trans. Mob. Comput. 23 (12), p. 11253–11264. External Links: Document Cited by: §I, §I-A. [6] X. He, Y. Jiang, X. Xu, H. Cui, Y. Liu, M. Chen, Y. Hong, and J. Zhang (2025) Large language model offloading using active inference in 6G symbiotic IoT. IEEE Internet Things J. (), p. 1–1. External Links: Document Cited by: §I, §I-A, §V. [7] Y. Zhong, S. Liu, J. Chen, J. Hu, Y. Zhu, X. Liu, X. Jin, and H. Zhang (2024) DistServe: disaggregating prefill and decoding for goodput-optimized large language model serving. In Proc. 18th USENIX Symp. Operating Syst. Design Implement. (OSDI), p. 193–210. Cited by: §I, §I. [8] H. Zhou, C. Hu, D. Yuan, Y. Yuan, D. Wu, X. Liu, Z. Han, and J. Zhang (2025) Generative AI as a service in 6G edge-cloud: generation task offloading by in-context learning. IEEE Wireless Commun. Lett. 14 (3), p. 711–715. External Links: Document Cited by: §I, §V. [9] A. Vaswani et al. (2017) Attention is all you need. Adv. Neural Inf. Process. Syst. 30. Cited by: §I. [10] H. Han, Y. Xu, Z. Feng, X. Wang, Y. Xu, W. Li, and F. Zhou (2024) Efficient dynamic spectrum anti-jamming access with large action space: an action space decomposition-based approach. IEEE Wireless Commun. Lett. 13 (10), p. 2667–2671. External Links: Document Cited by: §IV-A. [11] X. Ding, Y. Zhang, B. Chen, D. Ying, T. Zhang, J. Chen, L. Zhang, A. Cerpa, and W. Du (2026) Scalable and efficient reinforcement learning for virtual machine rescheduling in cloud data centers. IEEE Trans. Parallel Distrib. Syst. (), p. 1–18. External Links: Document Cited by: §IV-A3. [12] X. Ye, Y. Sun, D. Wen, G. Pan, and S. Zhang (2023) End-to-end delay minimization based on joint optimization of DNN partitioning and resource allocation for cooperative edge inference. In Proc. IEEE Veh. Technol. Conf. (VTC), Vol. , p. 1–7. External Links: Document Cited by: §V. [13] Z. Liu, Q. Lan, and K. Huang (2023) Resource allocation for multiuser edge inference with batching and early exiting. IEEE J. Sel. Areas Commun. 41 (4), p. 1186–1200. External Links: Document Cited by: §V.