Paper deep dive
Intelligent Multi-UAV Navigation in ITNTNs: A Hierarchical LLM Approach
Zijiang Yan, Hao Zhou, Wael Jaafar, Jianhua Pei, Ping Wang, Halim Yanikomeroglu, Hina Tabassum
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 7/22/2026, 2:51:00 AM
Summary
The paper proposes a Hierarchical LLM-driven control framework for multi-UAV navigation in Integrated Terrestrial and Non-Terrestrial Networks (ITNTNs). It addresses the latency limitations of Large Language Models (LLMs) and the strategic reasoning gaps of Deep Reinforcement Learning (DRL) by employing a dual-timescale architecture. A cloud-based LLM (Qwen3.5-122B) on a High-Altitude Platform Station (HAPS) handles slow-timescale global load balancing, while lightweight edge-LLMs (Qwen3.5-9B) on UAVs perform tactical reasoning to adjust DRL reward weights. A fast-timescale DRL controller executes physical flight kinematics. This agentic architecture aims to reduce collision rates and improve system throughput in 3D aerial highways.
Entities (9)
Relation Signals (7)
Hierarchical LLM-driven control framework → optimizes → ITNTN
confidence 97% · Intelligent Multi-UAV Navigation in ITNTNs... robust coordination of physical flight kinematics and multi-tier network handovers
HAPS → hosts → Qwen3.5-122B
confidence 95% · massive cloud-based LLM deployed on a High-Altitude Platform Station (HAPS)... Qwen3.5-122B
UAV → hosts → Qwen3.5-9B
confidence 95% · lightweight edge-LLMs on individual UAVs... Qwen3.5-9B
Hierarchical LLM-driven control framework → uses → Qwen3.5-9B
confidence 95% · lightweight edge-LLMs on individual UAVs translate local observations... Qwen3.5-9B
Hierarchical LLM-driven control framework → uses → Qwen3.5-122B
confidence 95% · A massive cloud-based LLM deployed on a High-Altitude Platform Station (HAPS) manages slow-timescale global load balancing... Qwen3.5-122B
Qwen3.5-9B → guides → DDQN
confidence 92% · edge-LLMs... translate local observations into tactical sub-goals. These sub-goals guide a fast-timescale physical DRL controller
Hierarchical LLM-driven control framework → solves → H-MO-POMDP
confidence 90% · formulate the problem as a Hierarchical Multi-Objective Partially Observable Markov Decision Process (H-MO-POMDP)... To address this complex hierarchy, we introduce a dual-timescale architecture
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:The deployment of high-speed Uncrewed Aerial Vehicles (UAVs) in 3D aerial highways necessitates robust coordination of physical flight kinematics and multi-tier network handovers. While Deep Reinforcement Learning (DRL) offers rapid tactical control, it lacks the zero-shot strategic reasoning required to quickly adapt to dynamic Integrated Terrestrial and Non-Terrestrial Networks (ITNTNs). Conversely, Large Language Models (LLMs) excel at semantic reasoning but suffer from high inference latency, rendering them unsuitable for real-time aerodynamic control. To bridge this gap, we propose a novel Hierarchical LLM-driven control framework. A massive cloud-based LLM deployed on a High-Altitude Platform Station (HAPS) manages slow-timescale global load balancing, while lightweight edge-LLMs on individual UAVs translate local observations into tactical sub-goals. These sub-goals guide a fast-timescale physical DRL controller to execute collision-free, handover-aware trajectories. Simulation results demonstrate that our agentic architecture significantly reduces collision rates and improves aggregate system throughput compared to existing baselines.
Tags
Links
- Source: https://arxiv.org/abs/2607.18604v1
- Canonical: https://arxiv.org/abs/2607.18604v1
Trouble viewing inline? Open PDF directly →
Full Text
32,589 characters extracted from source content.
Expand or collapse full text
Intelligent Multi-UAV Navigation in ITNTNs: A Hierarchical LLM Approach Zijiang Yan1, Hao Zhou2, Wael Jaafar3, Jianhua Pei1, Ping Wang1, Halim Yanikomeroglu4, and Hina Tabassum1 Emails: zijiang, pingw, hinat@yorku.ca, haozhou029@gmail.com, wael.jaafar@etsmtl.ca, jianhuapei98@gmail.com, halim@sce.carleton.ca. Abstract The deployment of high-speed Uncrewed Aerial Vehicles (UAVs) in 3D aerial highways necessitates robust coordination of physical flight kinematics and multi-tier network handovers. While Deep Reinforcement Learning (DRL) offers rapid tactical control, it lacks the zero-shot strategic reasoning required to quickly adapt to dynamic Integrated Terrestrial and Non-Terrestrial Networks (ITNTNs). Conversely, Large Language Models (LLMs) excel at semantic reasoning but suffer from high inference latency, rendering them unsuitable for real-time aerodynamic control. To bridge this gap, we propose a novel Hierarchical LLM-driven control framework. A massive cloud-based LLM deployed on a High-Altitude Platform Station (HAPS) manages slow-timescale global load balancing, while lightweight edge-LLMs on individual UAVs translate local observations into tactical sub-goals. These sub-goals guide a fast-timescale physical DRL controller to execute collision-free, handover-aware trajectories. Simulation results demonstrate that our agentic architecture significantly reduces collision rates and improves aggregate system throughput compared to existing baselines. I Introduction The rapid proliferation of Uncrewed Aerial Vehicles (UAVs) is driving the conceptualization of 3D aerial highways, dedicated corridors designed to support dense, high-speed autonomous logistics [2]. Operating safely in these environments requires uninterrupted Command and Control (C2) and telemetry links [9]. To satisfy these stringent connectivity requirements, 6G architectures are shifting toward Integrated Terrestrial and Non-Terrestrial Networks (ITNTNs) [7], leveraging High-Altitude Platform Stations (HAPS) to complement the terrestrial base stations’ (TBSs) coverage. However, as UAVs traverse these overlapping coverage tiers at high velocities, they inevitably trigger frequent network handovers (HOs) [13], necessitating the joint optimization of physical flight kinematics and communication reliability. Although Deep Reinforcement Learning (DRL) has been widely adopted for UAV trajectory planning and cell association [3], it inherently relies on trial-and-error exploration [9]. In multi-agent aerial corridors, such exploration often leads to localized and greedy optimization, which can result in the systemic depletion of shared resources. Uncoordinated agents may inadvertently overload specific TBSs or saturate the limited bandwidth of the HAPS, ultimately compromising both the aggregate system capacity and flight safety [6, 10]. Recently, the integration of Generative AI, especially Large Language Models (LLMs), has emerged as a transformative approach for autonomous vehicle control [5]. LLMs possess unparalleled zero-shot reasoning, enabling complex mission adaptation and global strategic planning [11]. Yet, standard LLM inference suffers from significant latency bottlenecks. Relying purely on LLMs is fundamentally incompatible with the fast-timescale execution required for sub-millisecond RF channel handovers and high-frequency rotor control [1, 12]. Bridging the gap between slow-timescale semantic reasoning and fast-timescale physical execution is an open challenge. To overcome these limitations, this paper proposes a novel Dual-Timescale Hierarchical LLM architecture that synergizes the cognitive reasoning of foundation models with the rapid execution of DRL. Indeed, given the potential of Agentic AI and the latency challenges inherent in dynamic aerial networks, we propose here a novel Cloud-Edge framework that jointly optimizes UAV mobility and handover management. In particular, we first formulate the problem as a Hierarchical Multi-Objective Partially Observable Markov Decision Process (H-MO-POMDP), which strictly accounts for high-fidelity 3D rigid-body kinematics and realistic ITNTN capacity constraints. To address this complex hierarchy, we introduce a dual-timescale architecture powered by LLMs. Specifically, a strategic meta-controller (Qwen3.5-122B [15]) deployed on the HAPS operates at a slow timescale to dictate global load-balancing policies. Moreover, unlike previous studies that relied solely on trial-and-error DRL, we integrate here lightweight edge-agents (Qwen3.5-9B [15]) directly on individual UAVs. The latter generate cognitive reflections to dynamically tune the reward functions of a fast-timescale Deep Double Q-Network (DDQN), successfully bridging semantic reasoning with real-time physical execution. Finally, extensive simulations in a high-fidelity 3D physics environment demonstrate that the proposed LLM-guided framework mitigates severe collision penalties and maximizes system throughput, thus outperforming baselines. The remainder of this paper is organized as follows. Section I describes the ITNTN system model, including the high-fidelity 3D UAV kinematics and the multi-tier communication architecture. Section I formulates the joint optimization problem as a H-MO-POMDP. The proposed dual-timescale hierarchical LLM framework and its cognitive reasoning modules are introduced in Section IV. Section V presents the numerical results and performance evaluations. Finally, Section VI concludes the paper. I System Model Figure 1: 3D aerial network model. I-A 3D Aerial Highway and ITNTN Architecture As depicted in Fig. 1, we consider a dynamic 3D aerial highway where a set of M UAVs, denoted by ℳ=1,2,…,MM=\1,2,…,M\, navigate a shared airspace. Each UAV m∈ℳm is assigned an independent transit mission to a unique 3D target waypoint, targetmx_target^m, and must continuously adjust its altitude and heading to bypass obstacles and avoid collisions. To support mission-critical C2 and telemetry, the airspace is covered by an ITNTN. The network comprises a set of B TBSs, denoted as ℬ=1,2,…,BB=\1,2,…,B\, and a one HAPS, denoted by H. A UAV m can connect to any serving node c∈ℬ∪Hc ∪\H\, but high-speed traversal triggers handovers. I-B Communication Model To evaluate the reliability of the C2 links, we formulate the signal-to-interference-plus-noise ratio (SINR)[4]. Let Gtm,cG_t^m,c denote the linear channel power gain between UAV m and serving node c at time t, encapsulating both large-scale path loss and small-scale Rician fading. The received downlink SINR at UAV m is given by SINRtm,c=PcGtm,cN0Bc+It,SINR_t^m,c= P_cG_t^m,cN_0B_c+I_t, (1) where PcP_c is the transmit power of node c, N0N_0 is the noise power spectral density, and BcB_c is the allocated bandwidth. The term It=∑c′≠cPc′Gtm,c′I_t= _c ≠ cP_c G_t^m,c represents the aggregate interference from other active transmitting nodes in the tier. The achievable data rate for UAV m is calculated as Rtm,c=Bclog2(1+SINRtm,c).R_t^m,c=B_c _2 (1+SINR_t^m,c ). (2) To penalize the latency and packet drops associated with frequent network switching, we define the handover-aware weighted data rate as WRtm,c=Rtm,c−γHO,tm,WR_t^m,c=R_t^m,c- _HO,t^m, (3) where HO,tm∈0,1I_HO,t^m∈\0,1\ is a binary indicator that equals 11 if a handover occurs at time t, and γ is the handover penalty coefficient. Also, the HAPS is constrained by a maximum aggregate capacity CmaxHAPSC_ ^HAPS and any node c has a users association capacity QcQ_c, ∀c∈ℬ∪H∀ c ∪\H\. I-C Kinematic Model We adopt a high-fidelity 3D rigid-body kinematics model, where the physical state of UAV m at time t is defined by its 3D position tm∈ℝ3x_t^m ^3 and linear velocity tm∈ℝ3v_t^m ^3. The system evolves at a fast discrete timescale Δt t (e.g., =0.05 s), and the continuous kinematic state updates are governed by [8] t+1m=tm+tmΔt,x_t+1^m=x_t^m+v_t^m t, (4) t+1m=tm+(T,mMu−m)Δt,v_t+1^m=v_t^m+ ( F_T,mM_u-g-d_m ) t, (5) where MuM_u is the UAV mass, g is the gravitational acceleration vector, and md_m is the aerodynamic drag. The collective thrust vector T,mF_T,m is derived from the real-time rotational speeds (RPM) of the four individual rotors. To ensure flight safety, a severe collision penalty is incurred if the 3D Euclidean distance between any two UAVs falls below the physical safety threshold: ‖tm−tj‖2<dsafe\|x_t^m-x_t^j\|_2<d_safe for any m≠jm≠ j. I Problem Formulation To capture the coupled dynamics of flight kinematics and network handovers across different operational timescales, we formulate the system as a H-MO-POMDP consisting of a slow-timescale global meta-controller (HAPS) and fast-timescale tactical edge-agents (UAVs). I-A State and Observation Spaces Due to the limited sensing range of individual UAVs, the global environmental state tS_t is partially observable. At the fast tactical timescale t, each UAV m receives a local observation tm∈Ωo_t^m∈ , defined as tm=[tm,tm,targetm,tj,tjj∈tm,ctm,WRtm,c],o_t^m= [x_t^m,v_t^m,x_target^m,\x_t^j,v_t^j\_j _t^m,c_t^m,WR_t^m,c ], (6) where tmN_t^m is the set of neighboring UAVs within sensing range, ctmc_t^m is the current serving node, and WRtm,cWR_t^m,c is the handover-aware weighted data rate. Conversely, at the slow strategic timescale THAPST_HAPS, the HAPS receives a global meta-observation THAPSo_T^HAPS comprising the aggregate traffic load on each TBS, the available HAPS backhaul capacity, and the spatial distribution of the UAV swarm. I-B Hybrid Action Spaces The action space encompasses both continuous physical controls and discrete network decisions as follows. I-B1 HAPS Strategic Action At each macro-step THAPST_HAPS, the HAPS agent outputs a discrete meta-action THAPS∈Offload,Recall,Idlea_T^HAPS∈\Offload,Recall,Idle\. These directives are broadcast to specific high-density congestion zones to enforce load-balancing before node capacities are breached. I-B2 UAV Tactical Action At each micro-step t, UAV m executes a joint hybrid action tm=[mot,tm,tele,tm]⊤a_t^m=[a_mot,t^m,a_tele,t^m] . The continuous motion action mot,tm∈[−1,1]4a_mot,t^m∈[-1,1]^4 regulates the RPM of the four individual rotors for physical 3D navigation. Concurrently, the discrete telecommunication action tele,tm∈ℬ∪Ha_tele,t^m ∪\H\ selects the target serving node, triggering a handover if atele,tm≠ctma_tele,t^m≠ c_t^m. I-C Multi-Objective Reward Functions To simultaneously optimize mobility and connectivity, we define scalarized reward structures for edge and HAPS levels. I-C1 UAV Edge Reward The objective of UAV m is to safely navigate to its target while maintaining a robust C2 link. The reward is formulated as Rtm=α1Rtran,tm+α2WRtm,c+α3ρcrash,R_t^m= _1R_tran,t^m+ _2WR_t^m,c+ _3 _crash, (7) where Rtran,tm=exp(−‖tm−targetm‖2)−λ‖mot,tm‖22R_tran,t^m= (-\|x^m_t-x_target^m\|_2)-λ\|a_mot,t^m\|_2^2 rewards forward progression while penalizing aggressive, energy-consuming rotor maneuvers. The term WRtm,cWR_t^m,c maximizes the data rate while penalizing handovers. Finally, ρcrash _crash applies a severe negative penalty if the safety distance dsafed_safe is violated. The weights α1,2,3 _\1,2,3\ balance the rewrards/penalties. I-C2 HAPS Reward The objective of the HAPS meta-controller is to maximize aggregate system throughput while enforcing node capacity limits. Its reward is given by RTHAPS=η1∑m∈ℳRtm,c−η2∑c∈ℬ∪Hmax(0,nc,t−Qc),R_T^HAPS= _1 _m R_t^m,c- _2 _c ∪\H\ (0,n_c,t-Q_c), (8) where nc,tn_c,t is the number of UAVs currently associated with node c. The second term heavily penalizes any policy that results in network congestion (i.e., exceeding the capacity QcQ_c), balanced by the weights η1,2 _\1,2\. To capture the coupled dynamics of 3D trajectory control and handover-aware cell association, we define the joint optimization problem P below. The objective is to maximize the aggregate system utility while adhering to physical kinematics and network capacity constraints. :max,[∑t=0Tγt∑m∈ℳℛtm(tm,tm)]s.t.C1:t+1m=f(tm,tm,mot,tm),∀m,t,C2:‖tm−tj‖2≥dsafe,∀m≠j,C3:∑c∈ℬ∪Hutm,c=1,∀m,t,C4:∑m∈ℳutm,c≤Qc,∀c∈ℬ∪H,C5:tm∈Δ3,∑i=13αt,im=1. aligned P: _A, α &E [ _t=0^Tγ^t _m R_t^m(a_t^m, α_t^m) ]\\ s.t. &C1: 10000\ x_t+1^m=f(x_t^m,v_t^m,a_mot,t^m),&&∀ m,t,\\ &C2: 10000\ \|x_t^m-x_t^j\|_2≥ d_safe,&&∀ m≠ j,\\ &C3: 10000\ _c ∪\H\u_t^m,c=1,&&∀ m,t,\\ &C4: 10000\ _m u_t^m,c≤ Q_c,&&∀ c ∪\H\,\\ &C5: 10000\ α_t^m∈ ^3, _i=1^3 _t,i^m=1. aligned (9) In (P), ≜tmA \a_t^m\ and ≜tm α \ α_t^m\ (∀m∈ℳ,∀t∀ m ,∀ t) denote the joint hybrid action policy and the dynamically tuned multi-objective reward weights across all UAVs over the operational horizon, respectively. tm=[mot,tm,atele,tm]⊤a_t^m=[a_mot,t^m,a_tele,t^m] represents the hybrid action vector. C1 ensures the trajectory updates follow the 3D rigid-body physics model defined in (4)-(5). C2 maintains the physical safety separation between any two UAVs. C3 ensures that each UAV m is associated with exactly one serving node (TBS or HAPS) at any time t. C4 restricts the number of associated users per node to its capacity QcQ_c to prevent network saturation. C5 defines the simplex Δ3 ^3 for the cognitive reward weights α1,2,3 _\1,2,3\ tuned by the edge-LLM. Problem P is a highly non-convex mixed-integer non-linear programming problem coupling flight kinematics and aerial communications. To enable real-time control, we decompose this optimization via a two-tier Hierarchical Multi-Objective POMDP framework, delegating specific constraints to distinct LLM cognitive tiers, as described in the next section. IV Proposed Hierarchical LLM Framework Figure 2: Hierarchical control for UAVs in an ITNTN. To bridge the latency gap between semantic reasoning and real-time aerodynamic control, we propose a Cloud-Edge Agentic architecture, as illustrated in Fig. 2. The framework separates decision-making into three distinct timescales: 1) slow strategic meta-timescale (THAPST_HAPS), 2) intermediate tactical reasoning timescale (TLLMT_LLM), and 3) fast physical execution timescale (TfastT_fast). IV-A HAPS Cloud Meta-Controller At the highest level of the hierarchy, the HAPS acts as a global orchestrator. Since global load balancing requires complex spatial reasoning across the entire ITNTN, we deploy a large foundation model, i.e., Qwen3.5-122B, hosted in the cloud computing cluster of HAPS. Operating at THAPS=5.0T_HAPS=5.0 s, the HAPS meta-controller evaluates the global observation THAPSo_T^HAPS. In particular, if the number of users associated with a specific TBS nc,tn_c,t is close to QcQ_c, the LLM generates zero-shot strategic directives (THAPSa_T^HAPS), forcing specific UAV edge-agents to offload their C2 links to neighboring TBSs or the HAPS. IV-B UAV Edge-Agent Each UAV is equipped with a lightweight edge-LLM (i.e., Qwen3.5-9B) that operates at TLLM=1.0T_LLM=1.0 s. The cognitive edge-agent translates the localized observation (tmo_t^m) into a structured textual prompt. Rather than outputting direct motor controls, the edge-LLM performs cognitive reflection, i.e., if the UAV experiences severe handover penalties or is close to a collision threshold (dsafed_safe), the LLM dynamically adjusts the weights α1,2,3 _\1,2,3\ of the multi-objective reward vector. By doing so, this semantic reasoning guides the underlying DRL, shifting its priority between aggressive forward transit and conservative safety. IV-C Fast-Timescale DRL Execution At the lowest level, a DDQN operates at the physics timescale Tfast=0.05T_fast=0.05 s (20 Hz). The DDQN receives the dynamic reward weights generated by the edge-LLM and executes the continuous motor commands (mot,tma_mot,t^m) and discrete handover actions (tele,tma_tele,t^m). By offloading semantic reasoning to the LLM, the DDQN acts purely as a high-speed, reflexive physical controller. The complete execution flow is summarized in Algorithm 1. Initialize: Environment state 0S_0, HAPS LLM (Qwen-122B), UAV LLMs (Qwen-9B), and DDQN networks. for each physics micro-step t=1,2,…,Tmaxt=1,2,…,T_ do if t(modTHAPS)==0t T_HAPS==0 then Extract global meta-observation THAPSo_T^HAPS; HAPS LLM generates strategic meta-action THAPSa_T^HAPS to balance node quotas QcQ_c; for each UAV m∈ℳm do Extract local numerical observation tmo_t^m; if t(modTLLM)==0t T_LLM==0 then Format tmo_t^m into dynamic text prompt tdynp_t^dyn; Edge LLM evaluates safety threshold ρthresh _thresh; Edge LLM outputs cognitive reflection to dynamically tune reward weights α1,2,3 _\1,2,3\; DDQN receives tuned rewards and selects hybrid action tm=[mot,tm,tele,tm]⊤a_t^m=[a_mot,t^m,a_tele,t^m] ; Execute tma_t^m, update kinematics at TfastT_fast, and store transition in memory buffer mD^m; Sample mini-batch from mD^m to train DDQN; Algorithm 1 Dual-Timescale Hierarchical LLM Control IV-D State Discretization and Prompt Engineering To bridge the gap between numerical kinematics and the semantic reasoning of the Qwen3.5-9B edge-LLM, we leverage a structured prompt engineering methodology. Directly feeding high-precision floating-point arrays (e.g., tm=[12.43,−3.21,0.5]v_t^m=[12.43,-3.21,0.5]) often degrades LLM reasoning and inflates inference latency. Therefore, we discretize the continuous observation tmo_t^m into semantic language descriptors, denoted as ℒ(tm)L(o_t^m). Specifically, the 3D relative distance to neighboring UAVs (‖tm−tj‖2\|x_t^m-x_t^j\|_2, ∀j∈ℳ\m∀ j \m\) is categorized into predefined semantic zones: Safe (d>3dsafed>3d_safe), Warning (dsafe≤d≤3dsafed_safe≤ d≤ 3d_safe), and Critical (d<dsafed<d_safe). Similarly, the handover-aware network state WRtm,cWR_t^m,c is mapped to Stable (when WRtm,c≥20WR_t^m,c≥ 20 Mbps, indicating an un-congested link), Degraded (when 0≤WRtm,c<200 _t^m,c<20 Mbps, indicating capacity saturation), or Hand-over (when WRtm,c<0WR_t^m,c<0, triggered by the massive γ penalty during network switching). To enhance the zero-shot generalization of the LLM, we utilize distance-based few-shot example selection. Rather than using static prompts, the edge-agent retrieves a set of K historical states from its local memory buffer mD^m that exhibit the shortest Euclidean distance to the current kinematic state. The successful reward-weight configurations from these retrieved states are appended as in-context learning examples. The final synthesized prompt strictly follows a modular template: [Role], [Current State], [Historical Examples], and [Task]. An illustrative example of the generated prompt is provided below: [Role]: You are an autonomous UAV edge-agent navigating a 3D corridor. Your task is to dynamically tune the multi-objective DRL reward weights (α1,α2,α3)( _1, _2, _3) for Transit Efficiency, Network Reliability, and Safety. [Current State]: Distance to target is Approaching. Neighbor UAV-3 is in the Warning zone (7.2m). Network link to TBS-2 is Stable. [Historical Example]: When Neighbor was Warning and Link was Stable, optimal weights were (0.2,0.1,0.7)(0.2,0.1,0.7) to heavily prioritize collision avoidance over forward transit. [Task]: Based on the current state, output the updated scalarization weights in the exact format [w1, w2, w3]. Do not output conversational text or explanations. V Numerical Results V-A Simulation Setup We evaluate the proposed Cloud-Edge Agentic framework using a high-fidelity 3D multi-rotor physics simulator based on gym-pybullet-drones [8]. The ITNTN environment consists of 4 TBSs and 1 HAPS serving a number of UAVs M∈10,20,30M∈\10,20,30\. The Qwen-based LLMs interact with the physical simulation via structured API calls. The detailed simulation parameters are summarized in Table I. A key challenge in applying generative AI to high-speed UAV networks is LLM inference latency, which is incompatible with the fast reaction time required for aerodynamic stabilization. To address this issue, we adopted a two-level HAPS–UAV architecture with separated cognitive and control timescales. At the UAV edge, we consider a 4-bit quantized Qwen3.5-9B deployed on an edge-AI platform. With an average generation speed of about 2525 tokens/s, a structured prompt and a concise output yield an estimated edge latency of tedge=0.6t_edge=0.6 s. At the HAPS layer, the larger Qwen3.5-122B meta-controller runs on a high-performance GPU cluster, with total delay modeled as tcloud=tinf+ttx=2.55t_cloud=t_inf+t_tx=2.55 s. These values mathematically motivate our proposed timescale hierarchy. The fast aerodynamic loop runs at Tfast=0.05T_fast=0.05 s, making direct LLM inference in the control path physically impractical. We therefore set the UAV-level cognitive reflection interval to TLLM=1.0T_LLM=1.0 s and the HAPS-level coordination interval to THAPS=5.0T_HAPS=5.0 s, strictly satisfying the hardware bounds TLLM>tedgeT_LLM>t_edge and THAPS≫tcloudT_HAPS t_cloud. During one edge-LLM inference interval, the DDQN controller can still execute 20 fast, uninterrupted control steps using the latest validated reward configuration. This asynchronous decoupling avoids computation-induced crashes while enabling robust LLM strategic reasoning. In our simulations, we compare the proposed Hierarchical LLM-DDQN framework against the “DDQN” baseline [13, 16], which relies on trial-and-error exploration without LLM cognitive reflection or HAPS load-balancing, and the “Envelope MORL” method proposed in [14]. Table I: Simulation Parameters Parameter Value Physics & Airspace Parameters Simulation airspace volume 1000×1000×3001000× 1000× 300 m3 Number of UAVs (M) 5,10,15,20,25,30\5,10,15,20,25,30\ UAV Mass (MuM_u) 1.51.5 kg Physical safety separation (dsafed_safe) 5.05.0 m Gravity (g) 9.819.81 m/s2 Control frequency (1/Tfast1/T_fast) 2020 Hz (Δt=0.05 t=0.05 s) ITNTN Communication Parameters Number of TBSs (B) 44 HAPS altitude 2020 km Carrier frequencies 2.02.0 GHz (HAPS), 2.12.1 GHz (TBS) Node bandwidth (BcB_c) 2020 MHz HAPS total capacity (CmaxHAPSC^HAPS_ ) 100100 Mbps Node transmit power (PcP_c) 4040 dBm (10,00010,000 mW) TBS peak antenna gain (GmaxG_ ) 88 dBi Node capacity quota (QcQ_c) 55 concurrent users Path loss exponents ηLoS=2.0 _LoS=2.0, ηNLoS=3.5 _NLoS=3.5 Rician LoS factor 1515 dB Noise power spectral density (N0N_0) −174-174 dBm/Hz Algorithm & Cognitive Parameters HAPS Meta-Controller LLM Qwen3.5-122B UAV Edge-Agent LLM Qwen3.5-9B HAPS strategic timescale (THAPST_HAPS) 5.05.0 s UAV LLM reasoning interval (TLLMT_LLM) 1.01.0 s Edge inference latency (tedget_edge) 0.60.6 s Cloud inference latency (tcloudt_cloud) 2.552.55 s Reflection trigger threshold (ρthresh _thresh) −10.0-10.0 Collision crash penalty (ρcrash _crash) −100.0-100.0 HAPS reward weights (η1,2 _\1,2\) 1.0,50.0\1.0,50.0\ UAV reward weights (α1,2,3 _\1,2,3\) 1.0,0.1,0.2\1.0,0.1,0.2\ Handover penalty coeff. (γ) 5.05.0 Energy penalty coeff. (λ) 0.10.1 Memory retrieval size (K) 55 experiences DDQN learning rate 1×10−41× 10^-4 Discount factor (γd _d) 0.990.99 Target update frequency 500500 steps (a) Transportation Reward (Rtran,tmR_tran,t^m) (b) Survival (Episode Length) (c) Communication Reward (d) HO Probability Figure 3: Training convergence and scalability evaluation of the proposed Cloud-Edge Agentic framework. V-B Simulation Results To evaluate the robustness in congested aerial corridors, Fig. 3 reports key metrics, i.e., transportation reward, survival time, communication reward, and HO probability. Fig. 3(a) and Fig. 3(b) illustrate the convergence of the transportation reward and the survival step count, respectively. As shown in Fig. 3(a), the conventional DDQN baseline learns quickly but saturates at a suboptimal local minimum. This is because standard DRL struggles with the large and sparse exploration space of joint 3D kinematics and network selection. In contrast, the proposed framework exhibits a steeper, more sustained learning curve, overtaking DDQN near episode 1,500 and achieving the highest steady-state reward. This superiority is directly tied to physical safety, as reflected by the survival step count in Fig. 3(b). Specifically, in DDQN, severe collisions cause early termination. Because the latter relies only on trial-and-error, it suffers from frequent early collisions. Conversely, when a UAV in our framework detects a high-risk state, the onboard edge-LLM immediately shifts the scalarization weights via cognitive reflection to prioritize the collision penalty (ρcrash _crash). This zero-shot guidance avoids early terminations and allows the proposed framework to converge to near-maximum episode lengths. Fig. 3(c) evaluates the communication reward as a function of the number of TBSs. Initially, without any TBS, UAVs are connected to the HAPS without any attenuation in the link quality. However, introducing a single or 2 TBSs creates severe interference and uncoordinated handover opportunities between the TBSs and HAPS, causing a sharp drop in the communication reward for all methods. However, as more TBSs are added to provide continuous coverage, the reward recovers and better coordination is achieved. The proposed framework consistently maintains the highest communication reward since the HAPS meta-controller intelligently coordinates associations to minimize cross-tier interference. Fig. 3(d) illustrates the handover probability as the UAV swarm density increases from M=5M=5 to 3030. As the airspace becomes congested, baseline DRL agents greedily chase the strongest instantaneous signal, resulting in erratic, high-frequency handovers. In contrast, the HAPS meta-controller anticipates congestion and issues zero-shot spatial offloading directives. This strategic coordination successfully suppresses unnecessary network switching, granting the proposed framework the lowest and most stable handover probability across all traffic densities. VI Conclusion In this paper, we proposed a Cloud-Edge Agentic AI framework for jointly optimizing 3D UAV mobility and handover management in ITNTNs. By decoupling strategic reasoning and fast physical control across multiple timescales, the framework bridges LLM reasoning with real-time DRL execution. A HAPS-based meta-controller mitigates network congestion, while edge-LLMs improve local safety-aware decision-making. Simulation results show clear gains over conventional DRL in safety, learning efficiency, and system throughput. Future work will investigate lightweight quantization to reduce edge inference latency. References [1] J. Chen et al. (2021) BDFL: a byzantine-fault-tolerance decentralized federated learning method for autonomous vehicle. IEEE Trans. Veh. Technol. 70 (9), p. 8639–8652. Cited by: §I. [2] N. Cherif et al. (2021) 3D aerial highway: the key enabler of the retail industry transformation. IEEE Commun. Mag. 59 (9), p. 65–71. Cited by: §I. [3] N. Cherif et al. (2024) RL-based cargo-UAV trajectory planning and cell association for minimum handoffs, disconnectivity, and energy consumption. IEEE Trans. Veh. Technol. 73 (5), p. 7304–7309. External Links: Document Cited by: §I. [4] N. Cherif (2022) Cellular-connected UAV in next-generation wireless networks. Ph.D. Thesis, Univ. Ottawa. Cited by: §I-B. [5] L. Dong et al. (2026) Aerial Agentic AI: Synergizing LLM and SLM for Low-Altitude Wireless Networks. arXiv preprint arXiv:2603.22866. Cited by: §I. [6] W. Jaafar and H. Yanikomeroglu (2020) Dynamics of laser-charged UAVs: a battery perspective. IEEE Internet Things J. 8 (13), p. 10573–10582. Cited by: §I. [7] W. Jaafar and H. Yanikomeroglu (2022) HAPS-ITS: enabling future its services in trans-continental highways. IEEE Commun. Mag. 60 (10), p. 80–86. External Links: Document Cited by: §I. [8] J. Panerati et al. (2021) Learning to fly—a gym environment with pybullet physics for reinforcement learning of multi-agent quadcopter control. In Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), p. 7512–7519. Cited by: §I-C, §V-A. [9] Z. Yan et al. (2023) Multi-UAV speed control with collision avoidance and handover-aware cell association: DRL with action branching. In Proc. IEEE Global Commun. Conf. (GLOBECOM), p. 5067–5072. Cited by: §I, §I. [10] Z. Yan et al. (2025) CVaR-based variational quantum optimization for user association in handoff-aware vehicular networks. In Proc. IEEE Int. Conf. Commun. (ICC), p. 6088–6093. Cited by: §I. [11] Z. Yan et al. (2025) Hierarchical and Collaborative LLM-Based Control for Multi-UAV Motion and Communication in Integrated Terrestrial and Non-Terrestrial Networks. arXiv preprint arXiv:2506.06532. Cited by: §I. [12] Z. Yan et al. (2025) Hybrid LLM-DDQN-Based Joint Optimization of V2I Communication and Autonomous Driving. IEEE Wireless Commun. Lett. 14 (4), p. 1214–1218. External Links: Document Cited by: §I. [13] Z. Yan and H. Tabassum (2022) Reinforcement learning for joint V2I network selection and autonomous driving policies. In Proc. IEEE Global Commun. Conf. (GLOBECOM), p. 1241–1246. External Links: Document Cited by: §I, §V-A. [14] Z. Yan and H. Tabassum (2025) Generalized multi-objective reinforcement learning with envelope updates in URLLC-enabled vehicular networks. IEEE Trans. Veh. Technol. 74 (11), p. 17666–17682. External Links: Document Cited by: §V-A. [15] A. Yang et al. (2025) Qwen3 technical report. arXiv preprint arXiv:2505.09388. Cited by: §I. [16] K. Zhang et al. (2024) A machine learning-based toolbox for p4 programmable data-planes. IEEE Trans. Netw. Serv. Manag. 21 (4), p. 4450–4465. Cited by: §V-A.