Paper deep dive

Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird's-Eye Maps

Shuning Zhang, Liang Li, Yunheng Wang, Tao Wang, Yihang Kang, Renjing Xu

Year: 2026Venue: arXiv preprintArea: cs.ROType: PreprintEmbeddings: 46

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%

Last extracted: 9/4/2026, 3:18:35 AM

Summary

The paper introduces AGC-VLN, a training-free baseline for air-ground collaborative Vision-and-Language Navigation (VLN). It pairs an Unmanned Aerial Vehicle (UAV) with a global bird's-eye view and an Unmanned Ground Vehicle (UGV) with a local first-person view. The system uses a shared bird's-eye map where the UAV renders the UGV's pose and the VLM-anchored target as markers. The UGV plans a road-following path on this map using a frozen VLM, while the UAV executes a 3D-Spatial Point-and-Fly (3D-SPF) algorithm. In CARLA-Air simulations, AGC-VLN achieves a 77.0% joint success rate, demonstrating a significant collaboration gain over single-agent baselines.

Entities (8)

AGC-VLN · system · 98%UAV · robot · 95%UGV · robot · 95%3D-SPF · algorithm · 92%CARLA-Air · simulation-environment · 90%VLM · model · 90%Town10HD · mapscene · 85%Travel UAV · baseline-method · 80%

Relation Signals (8)

AGC-VLN → achieves → 77.0% joint success rate

confidence 95% · AGC-VLN reaches a 77.0% joint success rate

AGC-VLN → pairs → UAV

confidence 95% · Air-ground collaborative Vision-and-Language Navigation (VLN) pairs an unmanned aerial vehicle (UAV) with a global bird's-eye view

AGC-VLN → pairs → UGV

confidence 95% · and an unmanned ground vehicle (UGV) with a local first-person view

AGC-VLN → uses → 3D-SPF

confidence 95% · in parallel, the UAV runs 3D-SPF... AGC-VLN reaches a 77.0% joint success rate

AGC-VLN → evaluatedin → CARLA-Air

confidence 90% · On 100 closed-loop episodes in CARLA-Air's Town10HD scene

UGV → provides → local first-person view

confidence 90% · UGV with a local first-person view

UAV → provides → global bird's-eye view

confidence 90% · UAV with a global bird's-eye view

AGC-VLN → outperforms → Travel UAV

confidence 85% · exceeds the strongest published single-agent baseline (Travel UAV, 53.0%) by 24.0 points

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Air-ground collaborative Vision-and-Language Navigation (VLN) pairs an unmanned aerial vehicle (UAV) with a global bird's-eye view and an unmanned ground vehicle (UGV) with a local first-person view, yet the setting remains largely unexplored: existing training-free methods solve single-agent tasks but offer no collaboration mechanism, and a recent CARLA-Air evaluation found no stable cooperative behavior across five state-of-the-art VLA models; naive semantic communication or bidirectional coupling even degrades performance. We establish AGC-VLN (Air-Ground Collaborative VLN), the first training-free baseline for air-ground collaborative VLN. The key insight is that training-free methods decompose navigation into VLM-based semantic reasoning and deterministic geometric execution, exposing a collaboration interface: the UAV's global view, over which it renders the UGV's reported pose and the VLM-anchored target as CAR/GOAL markers with distance labels, yielding a shared bird's-eye map. From this map, the UGV acquires global spatial context its first-person view cannot provide, plans a road-following path with a frozen VLM, and executes it under closed-loop control; in parallel, the UAV runs 3D-SPF, a spatial-search upgrade of SPF that localizes the target in the downward view and flies toward it. On 100 closed-loop episodes in CARLA-Air's Town10HD scene, AGC-VLN reaches a 77.0% joint success rate, a collaboration gain of +27.0% over the weaker individual agent (the UAV, 50.0%), and exceeds the strongest published single-agent baseline (Travel UAV, 53.0%) by 24.0 points, stemming from the complementarity of the UAV's global view and the UGV's road-following execution. Project page: this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

42,484 characters extracted from source content.

Expand or collapse full text

Air-Ground Collaborative Vision-and-Language Navigation via Shared Bird’s-Eye Maps Shuning Zhang Liang Li Yunheng Wang Tao Wang Yihang Kang Renjing Xu †thanks: Shuning Zhang and Liang Li have the same contributions.†thanks: Shuning Zhang, Yunheng Wang, Tao Wang, Yihang Kang and Renjing Xu are with the Robotics and Autonomous Systems Thrust, Systems Hub, The Hong Kong University of Science and Technology (Guangzhou), China szhang272@connect.hkust-gz.edu.cn, yunhengwang1214@gmail.com, 19819824568@163.com, KYH2107405729@163.com, renjingxu@hkust-gz.edu.cn†thanks: Liang Li is with the School of Communications and Information Engineering, Nanjing University of Posts and Telecommunications, China lianglee2004@163.com Abstract Air-ground collaborative Vision-and-Language Navigation (VLN) pairs an unmanned aerial vehicle (UAV) with a global bird’s-eye view and an unmanned ground vehicle (UGV) with a local first-person view, yet the setting remains largely unexplored: existing training-free methods solve single-agent tasks but offer no collaboration mechanism, and a recent CARLA-Air evaluation found no stable cooperative behavior across five state-of-the-art VLA models; naive semantic communication or bidirectional coupling even degrades performance. We establish AGC-VLN (Air-Ground Collaborative VLN), the first training-free baseline for air-ground collaborative VLN. The key insight is that training-free methods decompose navigation into VLM-based semantic reasoning and deterministic geometric execution, exposing a collaboration interface: the UAV’s global view, over which it renders the UGV’s reported pose and the VLM-anchored target as CAR/GOAL markers with distance labels, yielding a shared bird’s-eye map. From this map, the UGV acquires global spatial context its first-person view cannot provide, plans a road-following path with a frozen VLM, and executes it under closed-loop control; in parallel, the UAV runs 3D-SPF, a spatial-search upgrade of SPF that localizes the target in the downward view and flies toward it. On 100 closed-loop episodes in CARLA-Air’s Town10HD scene, AGC-VLN reaches a 77.0% joint success rate, a collaboration gain of +27.0% over the weaker individual agent (the UAV, 50.0%), and exceeds the strongest published single-agent baseline (Travel UAV, 53.0%) by 24.0 points, stemming from the complementarity of the UAV’s global view and the UGV’s road-following execution. Project page: https://github.com/ZSN2024/AGC-VLN. †aftertitle: Fig. 1: The UAV turns its “can see” into the UGV’s “can drive”. View asymmetry: the UAV sees the whole block from a global bird’s-eye view while the UGV drives with a local first-person view. Shared map: the UAV renders the UGV pose and the VLM-anchored target as CAR/GOAL markers into a shared bird’s-eye map. Parallel execution: the UAV flies 3D-SPF toward the target while the UGV follows a VLM road path on the map, and either arriving counts as success, a training-free air-ground collaboration. I INTRODUCTION Frozen vision-language models have rewritten the recipe for Vision-and-Language Navigation (VLN). The classic benchmarks (R2R [1], REVERIE [2]) belonged to an era of training a dedicated policy per dataset. See-Point-Fly (SPF) [3] already flies a real drone to 92.7% success by merely asking a frozen VLM to “point” at the image, with no training at all. Uni-LaViRA [4], Fly0 [5], and FineCog-Nav [6] carry the same recipe across embodiments and reasoning styles. One assumption, however, survives intact from the classic era: a single robot navigates alone. Multi-robot collaborative navigation is the natural next step, and heterogeneous air-ground teams are its most representative form. Search-and-rescue, last-mile delivery, and escort need wide-area coverage from the air plus ground-level reachability from a wheeled platform; pairing the two promises a division of labor no single embodiment can replicate. The infrastructure has recently arrived: CARLA-Air [7] unifies CARLA [8] and AirSim [9] in a single Unreal Engine process with zero-latency sensor synchronization, and AirGroundBench [10] provides 115 closed-loop air-ground VLN episodes across 11 environments. The CARLA-Air cooperation study [11] evaluated five aerial VLA models (AerialVLA, OpenFly, OpenUAV, SPF, AerialVLN) on closed-loop air-ground tasks, and the verdict was blunt: none of them could turn single-agent skill into cooperative behavior. Worse, naive communication hurt: UGV text hints degraded most models, bidirectional coupling amplified errors for all, and even oracle geometric cues did not close the gap. The study attributes this to missing partner-state anchoring, low-latency action coordination, and team-level objective alignment, while a rule-based controller showed the tasks are solvable. In short: the platform exists, the benchmark exists, the failure is diagnosed, but no working air-ground collaborative VLN method has been demonstrated. In this paper we establish AGC-VLN (Air-Ground Collaborative VLN), deliberately a training-free baseline. Our key insight is that the very decomposition that makes training-free methods work for a single agent also solves the collaboration problem. SPF-style pipelines separate semantic reasoning (a frozen VLM marks the target in the image) from geometric execution (deterministic projection and control). We keep this decomposition but extend the VLM’s action interface. Beyond the 2D point, the VLM also outputs a discrete height command (descend, hold, or ascend). This gives the UAV active search in the vertical dimension and upgrades SPF’s fixed-altitude planar point-fly into three-dimensional spatial search. This spatial-aware, 3D-action-interface refinement of SPF is our 3D-SPF. The seam between the two stages is a metric quantity: the viewpoint-invariant spatial annotation in the UAV’s bird’s-eye view, where the UAV renders the teammate’s reported pose and the VLM-anchored target (against PgP_g) into a shared map the UGV’s first-person view could never obtain. Collaboration therefore requires no learned cross-agent representation at all: the UAV supplies global spatial context, and the UGV performs road-level planning and execution on top of it, each in its own role. Our contributions are: 1. 3D-SPF algorithm: we upgrade SPF [3] by adding a discrete height command (descend/hold/ascend) to the VLM’s action interface, turning fixed-altitude planar point-fly into three-dimensional spatial search; relative to native UAV-only SPF, this lifts the UAV’s success rate from 20.0% to 60.0%, a 3× gain, on our air-ground search task. 2. AGC-VLN system: we couple 3D-SPF on the UAV with VLM road-path planning on the UGV through a shared bird’s-eye map (the UAV renders the teammate’s reported pose and the VLM-anchored target), requiring no training or learned cross-agent representation. To the best of our knowledge, at the time of writing, no UAV+UGV collaboration VLN system with a positive collaboration gain has been reported in the open literature [11], and AGC-VLN reaches a 77.0% joint success rate with a +27.0% collaboration gain. 3. View-complementary collaboration mechanism: we show that the UAV’s global bird’s-eye map fills the global spatial context missing from the UGV’s first-person view, lifting the UGV’s success rate from 12.0% (UGV-only VLM) to 75.0% (AGC-VLN), enabling road-level planning and closed-loop execution. I RELATED WORK I-A Foundation-Model Navigation Without Training The training-free paradigm replaces task-specific policies with frozen foundation models orchestrated by deterministic glue. SPF [3] is the purest instance: a frozen VLM marks a waypoint pixel and a depth label per frame, and fixed geometry turns the annotation into a control command (no gradient ever flows). Fly0 [5] pushes the same semantic/geometric separation; Uni-LaViRA [4] drives four embodiments with a single language-vision-action layer; FineCog-Nav [6] splits reasoning into seven inspectable modules. The learned VLM-navigation line likewise replaces discrete action spaces with continuous reasoning: NaVILA [12] lifts VLM features into 3D, MapGPT [13] grounds navigation in map-guided prompting, GOAT [14] generalizes goal specification to any object, and StreamVLN [15] streams history tokens for long horizons. Large vision-language-action models (ABot-N1 [16], LongNav-R1 [17], EvolveNav [18]) push toward one general navigation policy, but stay trained and single-agent; none define what two agents should say to each other, the question this paper answers. I-B Cooperation Between Robots Simulation support for UAV+UGV teams arrived with CARLA-Air [7], which runs CARLA [8] and AirSim [9] in one Unreal Engine process with synchronized physics ticks (Δt=0 t=0 ms). The follow-up study [11] asked whether aerial VLA models can cooperate, testing three coupling modes (none, UGV-to-UAV text hints (C1), bidirectional velocity coupling (C2)) and finding that every mode failed to improve (and often degraded) single-agent success across five models; partner-state anchoring was isolated as the root deficit. AirGroundBench [10] supplies the evaluation material: 115 closed-loop episodes plus a four-level VQA taxonomy whose hardest level probes cross-view reasoning. On the method side, collaborative VLN exists only for homogeneous teams: two ground robots in CoNavBench [19] and two UAVs at different altitudes in AeroDuo [20], whose high/low-viewpoint division of labor echoes our view complementarity. OmniVLN [21] spans platforms but needs rotating LiDAR, and JanusVLN [22] decouples semantics from spatiality for cross-platform navigation. The aerial single-agent line has matured, from CityNav [23] on real-world data to CLOSER-VLN [24] and FSD-VLN [25] on long-horizon aerial VLN, but none couple an aerial observer to a ground executor. Complementary benchmarks probe the building blocks: SpatialUAV [26] (low-altitude collaboration), an integrated planning framework [27] (air-ground field deployment), and a survey [28] (aerial VLN). I METHOD We formalize the task (Sec. I-A), describe the UAV’s global perception and map-rendering module (Sec. I-B) and the UGV’s map path-planning and closed-loop execution module (Sec. I-C), and give the two agents’ parallel collaboration loop (Sec. I-D). Fig. 2: Architecture of AGC-VLN. On the UAV side, 3D-SPF queries a frozen VLM in closed loop to localize the target in the downward view from the target photo, returning a JSON point and a height command (descend/hold/ascend) that drive horizontal flight and 10 m-step vertical search; it renders the UGV pose and the VLM-anchored target as CAR/GOAL markers into a shared bird’s-eye map. On the UGV side (right), a frozen VLM plans a road path on the map, which is inverse-projected to world coordinates and tracked by a closed-loop controller with six driving primitives and stuck recovery. I-A Problem Formulation An episode hands a two-robot team a static target (a truck), specified by a target photo PgP_g and a natural-language description L. The team consists of a UAV hovering at altitude hUh_U whose RGB camera faces straight down (intrinsics UK_U), and a UGV whose RGB camera faces forward (intrinsics GK_G). The episode succeeds if, before the time budget TmaxT_ expires, any team member arrives within ϵε of the target without collision: success⇔min⁡‖U−g‖2,‖G−g‖2≤ϵ,success \\|x_U-x_g\|_2,\ \|x_G-x_g\|_2 \\;≤\;ε, (1) where U,Gx_U,x_G are the UAV and UGV positions and gx_g the target position. Both agents receive synchronized poses from the simulator; no map, no UAV depth sensor, and no training data are assumed. In this implementation ϵ=5ε=5 m and Tmax=180T_ =180 s. I-B UAV Module: Global Perception and Map Rendering At each decision step (interval ∼ 3 s, about 0.33 Hz) the UAV performs two functions: rendering the shared bird’s-eye map for the UGV, and running 3D-SPF for itself. I-B1 Bird’s-Eye Annotation (Shared Map Rendering) The UAV reads the UGV’s reported pose Gx_G from the shared state and projects Gx_G onto the current downward image IUtI_U^t; the target object is anchored in IUtI_U^t by a frozen VLM against the target photo PgP_g (sharing the same localization as 3D-SPF). The UAV then renders the annotations (Fig. 2): • a blue CAR marker: the UGV’s current position; • a red GOAL marker: the target truck’s position; • a green cross: the UAV itself (image center); • a yellow line: the CAR→ straight line (for reference only, not drivable); • distance labels: UAV→ and CAR→ . The fully annotated image constitutes a shared bird’s-eye map, transmitted to the UGV through shared memory together with the UAV pose at annotation time (used for deterministic pixel→ inverse projection). I-B2 3D-SPF (Three-Dimensional SPF) In parallel, the UAV runs 3D-SPF to fly toward the target itself: a frozen VLM receives the downward view IUtI_U^t and the target photo PgP_g, and returns structured JSON containing the target truck’s 2D pixel position (u,v)(u,v) in the downward view and a discrete height command hc∈−1,0,1h_c∈\-1,0,1\ (descend/hold/ascend). The pixel position is converted to a world coordinate wp^w via flat-ground ray casting under a planar-ground assumption, parameterized by the downward field of view θ and the altitude h: w ^w =U+⁡(ψ)htan⁡θ2[−2(vH−12)2(uW−12)], =c_U+R(ψ)\,h\, \! θ2 bmatrix-2 ( vH- 12 )\\[2.0pt] 2 ( uW- 12 ) bmatrix, (2) ⁡(ψ) (ψ) =[cos⁡ψ−sin⁡ψsin⁡ψcos⁡ψ], = bmatrix ψ&- ψ\\ ψ& ψ bmatrix, where ψ is the yaw and (W,H)(W,H) the image size. The UAV flies horizontally toward wp^w with proportional velocity =min⁡(vmax,kp‖w−U‖)w−U‖w−U‖,v= \! (v_ ,\;k_p\,\|p^w-c_U\| )\; p^w-c_U\|p^w-c_U\|, (3) then, once stabilized, descends/ascends 10 m according to hch_c (with a safety floor), then hovers awaiting the next decision. I-B3 Coordinate Transformation The UAV and UGV live in different coordinate systems: AirSim uses NED (z pointing down), CARLA uses a world frame (z pointing up). The two are converted through a constant offset o obtained by one-time calibration: NED _NED =+CARLA,=diag(1,1,−1), =o+S\,x_CARLA, =diag(1,1,-1), (4) =(apx−dlx,apy−dly,apz+dlz)⊤, = (ap_x-dl_x,\;ap_y-dl_y,\;ap_z+dl_z ) , where apap and dldl denote the drone position reported by AirSim and CARLA, respectively, at calibration. Pixel↔ conversion is done by flat-ground ray casting (Eq. 2 and its inverse), echoing cross-view geo-localization, which aligns drone and overhead views [29, 30]. I-C UGV Module: Path Planning and Closed-Loop Execution Upon receiving the annotated bird’s-eye map, the UGV executes three steps: I-C1 Road Path Planning A frozen VLM combines the target photo PgP_g, the pixel coordinates of CAR/GOAL in the map, and the image size to plan a path of 10 pixel waypoints ii=110\w_i\_i=1^10 along visible roads. The first point must equal its own position and the last must equal the target position. The intermediate points follow the road direction, turn at intersections, and never cut through buildings or leave the road. Before returning, the path is sanity-checked, discarding hallucinated paths whose first/last points clearly deviate from CAR/GOAL. I-C2 Pixel Path → World Path Using the UAV pose at annotation time, each pixel waypoint is inverse-projected to NED coordinates, then converted through the offset o to CARLA world coordinates, yielding the world path iw\w_i^w\. I-C3 Closed-Loop Path Tracking The UGV tracks the world path point by point with a closed-loop controller: at each tick it computes the wrapped heading error to the current waypoint and a distance-scaled throttle, eψ=wrapπ⁡(ψwp−ψ),τ=clip⁡(d10, 0.3, 1),e_ψ=wrap_π\! ( _wp-ψ ), τ=clip\! ( d10,\;0.3,\;1 ), (5) where ψ is the UGV heading, ψwp _wp the heading toward the waypoint, and d the remaining distance. The error eψe_ψ is discretized into six driving primitives (forward, left, right, reverse, reverse-left, reverse-right) by angle thresholds. When the vehicle stalls beyond a threshold (throttle applied but near-zero displacement), a reverse escape is triggered. After reaching each waypoint, it checks whether it has entered the ϵε range of the target. Algorithm 1 AGC-VLN: Training-Free Air-Ground Collaborative VLN (Bird’s-Eye Map Sharing) 0: Target photo PgP_g, budget TmaxT_ 0: Episode outcome ∈ success, failure 1: Start UAV and UGV threads in parallel, sharing state S; clear both success flags 2: while t<Tmaxt<T_ and not both agents succeeded do 3: UAV thread: read .ugv_posS.ugv\_pos; write its own pose 4: (u,v,hc)←VLM.Locate(IUt,Pg)(u,v,h_c) .Locate(I_U^t,P_g) // target pixel (u,v)(u,v); height hc∈−1,0,1h_c∈\-1,0,1\ 5: Mt←Annotate(IUt,G,w)M^t (I_U^t,x_G,p^w) // CAR + GOAL markers; w=Project(u,v)p^w=Project(u,v) is the VLM-anchored target 6: .map←(Mt,poset)S.map←(M^t,pose_t) // shared bird’s-eye map + pose at annotation time 7: fly toward wp^w; adjust altitude by hch_c // 3D-SPF 8: if UAV within ϵε of target: mark UAV success (keep annotating) 9: UGV thread: read .mapS.map 10: ii=110←VLM.PlanPath(Mt,Pg)\w_i\_i=1^10 .PlanPath(M^t,P_g) // road path planning 11: iw←InverseProject(i,poset)\w_i^w\ (\w_i\,pose_t) 12: FollowPath(iw)FollowPath(\w_i^w\) // closed-loop tracking + stuck recovery 13: if UGV within ϵε of target: mark UGV success (keep reporting pose) 14: end while 15: return success if either agent marked success, else failure I-D Collaboration Mechanism Algorithm 1 summarizes the complete parallel loop. The two agents collaborate asynchronously through shared memory, exchanging two kinds of information: • UGV→ (pose): the UGV continuously reports its CARLA world coordinate, for the UAV to render the CAR marker; • UAV→ (shared bird’s-eye map): the UAV continuously outputs the annotated bird’s-eye image and pose at annotation time, for the UGV to plan a path. The key to the collaboration is view complementarity. The UAV has a global bird’s-eye view but cannot drive along roads, while the UGV can drive along roads but has only a first-person local view. The annotated map delivers the UAV’s global spatial context (relative positions of teammate and target, road topology) in an image form the UGV’s VLM can consume directly, thereby filling the UGV’s blind spot. IV EXPERIMENTS We design experiments to answer three research questions: 1. RQ1 (Feasibility): Can training-free single-agent methods be composed into a working air-ground collaborative VLN system, where trained VLA models failed [11]? 2. RQ2 (Collaboration gain): Does the UAV-UGV team outperform each agent operating alone, and by how much? 3. RQ3 (Failures): Where does the baseline still fail, and what do the failures imply for future learned components? IV-A Experimental Setup Testbed: simulation experiments run inside CARLA-Air [7], whose single-process design guarantees that UAV and UGV observations sample the same physics tick. The quadrotor carries a downward 1080p RGB camera (FOV 108∘), hovering at 60 m; the UGV carries a forward camera; both read poses from the synchronized pose stream. Episodes: 100 closed-loop episodes across 50 spawn points (2 runs each) in the Town10HD scene, each specified by a target-truck photo PgP_g and a natural-language description L; a Mini Cooper (UGV) is spawned at the start and an HGV truck at the goal. Metrics: we report the UAV success rate (SRUAVSR_UAV, the UAV arrives within ϵε of the target), the UGV success rate (SRUGVSR_UGV), and the joint success rate (SRjointSR_joint, either member arrives), with ϵ=5ε=5 m. We also report Success weighted by Path Length (SPL), Navigation Error (NE), and the collaboration gain, CG=SRjoint−min⁡(SRUAV,SRUGV)CG=SR_joint- (SR_UAV,SR_UGV), which measures how much the team exceeds the weaker single agent. Configuration: gemini-3.7-flash [31] (temperature 0.1) serves as the frozen VLM on both platforms; the UAV/UGV decision interval is 3 s (∼ 0.33 Hz); the time budget is 180 s. IV-B Baselines (1) Published single-agent aerial VLN methods: OpenFly [32], FineCog-Nav [6], 3DG-VLN [33], and Travel UAV [34], reproduced under the same single-agent evaluation protocol for comparison. (2) Single-agent (collaboration lower bound): UAV-only SPF, UAV-only 3D-SPF, and UGV-only VLM, each solving the whole episode alone. IV-C Main Results (RQ1, RQ2) TABLE I: Main results of air-ground collaborative VLN (100 episodes). See the note below the table for column definitions. Method SRUGV_UGV ↑ SRUAV_UAV ↑ SRjoint_joint ↑ SPL ↑ NE (m) ↓ CG ↑ Time (s) ↓ VLM calls ↓ PathUAV_UAV (m) ↓ PathUGV_UGV (m) ↓ Published single-agent VLN baselines OpenFly [32] — 0.0% 0.0% 0.0% 78.4± 18.7 0.0% — 34.1± 7.3 56.4± 16.0 — FineCog-Nav [6] — 13.0% 13.0% 10.0% 28.4± 12.9 0.0% 177.2± 35.4 58.0± 12.4 19.9± 10.4 — 3DG-VLN [33] — 33.0% 33.0% 26.0% 49.6± 32.1 0.0% 142.1± 60.4 11.7± 5.2 49.2± 21.7 — Travel UAV [34] — 53.0% 53.0% 45.0% 30.3± 24.3 0.0% 115.8± 63.4 21.0± 11.5 42.24± 27.70 — Single-agent ablations (our components) UGV-only VLM 12.0% — 12.0% 11.6% 115.2± 65.6 0.0% 28.2± 3.0 12.1± 4.0 — 49.4± 4.0 UAV-only SPF [3] — 20.0% 20.0% 18.0% 155.5± 94.9 0.0% 58.5± 9.6 12.9± 6.5 119.1± 74.1 — UAV-only 3D-SPF — 60.0% 60.0% 46.0% 30.0± 40.9 0.0% 104.0± 8.0 13.0± 7.1 84.3± 36.7 — Ours: Air-Ground Collaborative VLN AGC-VLN (ours) 75.0% 50.0% 77.0% 62.0% 10.9± 16.2 +27.0% 82.2± 27.0 12.1± 2.9 65.1± 22.2 43.5± 14.0 Note: ↑ /↓ = higher/lower is better. SRUAV_UAV/SRUGV_UGV/SRjoint_joint: UAV/UGV/joint success rate (goal within 5 m). SPL: success weighted by path length; NE: navigation error; CG: collaboration gain =SRjoint−min⁡(SRUAV,SRUGV)=SR_joint- (SR_UAV,SR_UGV); Time: mean arrival time over successful episodes; VLM calls: inference count; PathUAV_UAV/PathUGV_UGV: distance traveled. SR/SPL/CG are percentages; NE/VLM calls/path are mean± over all episodes; Time is mean± over successful episodes; “—” = not applicable/reported. Table I reports the main results. We measure the collaboration gain as the amount by which the joint success rate exceeds the weaker individual agent: CG=SRjoint−min⁡(SRUAV,SRUGV).CG=SR_joint- \! (SR_UAV,\,SR_UGV ). (6) Two findings follow: (1) positive collaboration gain. AGC-VLN attains SRjoint=77.0%SR_joint=77.0\% (SRUGV=75.0%SR_UGV=75.0\%, SRUAV=50.0%SR_UAV=50.0\%), a collaboration gain of CG=+27.0%CG=+27.0\% over the weaker agent, together with an SPL of 62.0%62.0\% and an NE of 10.910.9 m; the joint rate thereby exceeds the strongest published single-agent baseline (Travel UAV, 53.0%53.0\%) by 24.024.0 points, unlike the VLA baselines whose coupling degraded performance; (2) success means either member arrives. The task is judged successful if either agent reaches the target, so SRjointSR_joint is the union of the per-agent rates, with the first arrival waiting for the other. IV-D Ablation: VLM Backbone Following SPF’s cross-VLM study, we swap the frozen VLM on both platforms across four backbones (gpt-5.6-luna, gemini-2.5-flash, gemini-3.7-flash, and qwen-vl-max [31]) and report success rates and costs (Table I). gemini-3.7-flash attains the highest joint success rate (77.0%) as the most balanced backbone, keeping both agents strong (SRUGV=75.0%SR_UGV=75.0\%, SRUAV=50.0%SR_UAV=50.0\%); the alternatives leave the UAV weaker (SRUAV=7.0%SR_UAV=7.0\%–47.0%47.0\%), so their joint rates fall short (67.0%67.0\%–73.0%73.0\%). TABLE I: VLM Backbone Ablation. VLM SRUGV_UGV ↑ SRUAV_UAV ↑ SRjoint_joint ↑ CG ↑ Time (s) ↓ qwen-vl-max 73.0% 47.0% 73.0% +26.0% 75.23± 68.80 gpt-5.6-luna 67.0% 13.0% 67.0% +54.0% 102.97± 63.10 gemini-2.5-flash 67.0% 7.0% 73.0% +66.0% 91.91± 63.36 gemini-3.7-flash 75.0% 50.0% 77.0% +27.0% 82.2± 27.0 IV-E Ablation: UAV Altitude To test 3D-SPF’s sensitivity to the UAV’s initial altitude, we vary it across 30/60/90/120 m while keeping the UGV module and the height command (descend/hold/ascend) unchanged (Table I). 3D-SPF is most effective at 60 m, where both agents contribute: the UAV’s success rate peaks at 50.0% and the UGV’s road-following reaches 75.0%, yielding the best joint rate of 77.0%. At 30 m the narrowed field of view drops the UAV to 20.0% and the UGV to 14.0%, collapsing the joint rate to 28.0%; at 90/120 m the truck shrinks to a handful of pixels and UAV localization fails entirely (0.0%), so the high joint rate (77.0%/72.0%) is delivered by the UGV alone, with no aerial contribution. TABLE I: 3D-SPF Altitude Ablation Initial altitude SRUGV_UGV ↑ SRUAV_UAV ↑ SRjoint_joint ↑ CG ↑ Time (s) ↓ 30 m 14.0% 20.0% 28.0% +14.0% 141.3± 29.7 60 m (Ours) 75.0% 50.0% 77.0% +27.0% 82.2± 27.0 90 m 77.0% 0.0% 77.0% +77.0% 66.1± 29.5 120 m 72.0% 0.0% 72.0% +72.0% 70.6± 35.0 IV-F Ablation: Map Annotation Richness To test the necessity of the shared map’s annotation content, we degrade the bird’s-eye annotation stepwise through five levels of richness: full annotation (CAR+GOAL+distance labels+reference line), CAR+GOAL+reference line, CAR+GOAL, GOAL only, and finally a completely unannotated raw bird’s-eye image (Table IV). Adding the CAR marker lifts the UGV’s success rate from 60.0% (GOAL only) to 70.0% (CAR+GOAL), the reference line further to 75.0%, and full annotation (with distance labels) reaches 77.0% joint success. Notably, GOAL-only (60.0%) underperforms the unannotated raw image (65.0%), indicating that a lone GOAL marker without the teammate’s CAR anchor misleads the UGV’s path planner. Figure 3 summarizes the three ablation axes. TABLE IV: Map Annotation Richness Ablation (bird’s-eye map sharing) Map annotation SRUGV_UGV ↑ SRUAV_UAV ↑ SRjoint_joint ↑ CG ↑ Time (s) ↓ None (raw) 60.0% 15.0% 65.0% +50.0% 118.8± 53.4 GOAL only 60.0% 15.0% 60.0% +45.0% 116.0± 55.4 CAR+GOAL 70.0% 20.0% 70.0% +50.0% 106.2± 51.8 CAR+GOAL+line 75.0% 40.0% 75.0% +35.0% 95.6± 50.6 Full (ours) 75.0% 50.0% 77.0% +27.0% 82.2± 27.0 Fig. 3: Ablation across the three design axes. (a) VLM backbone (Qwen-vl-max, GPT-5.6 Luna, Gemini 2.5-flash, and Gemini 3.7-flash). (b) UAV initial altitude (30/60/90/120 m). (c) Map annotation richness (raw, goal-only, CAR+GOAL, CAR+GOAL+line, and full). Each panel reports SRjoint_joint (green line) together with the per-agent success rates SRUGV_UGV and SRUAV_UAV (bars), with percentage value labels and the adopted configuration (“ours”) highlighted. IV-G Failure Source Analysis (RQ3) Fig. 4: Representative episodes. Success: the annotated map drives a VLM road path that the UGV tracks to the goal, with the UAV and UGV both arriving. Mis-anchoring failure: the UAV’s VLM anchors onto a similar but wrong truck and flies toward the wrong object. Path-planning failure: the UGV’s VLM plans a path that collides with a tree, stalling the UGV. Figure 4 shows representative episodes. We attribute each failed episode to the first pipeline stage that deviated from ground truth (Table V): global localization (the UAV’s VLM marks a wrong target region), path planning (an undrivable path), or local execution (failing to close the final meters). TABLE V: Failure Attribution by Pipeline Stage Stage Share Typical case Global localization 17% mis-anchoring 4% locks onto a wrong target unstable tracking 13% points jump across frames Path planning 61% undrivable path; vehicle crashes and stalls Local execution 22% final-approach dead end or timeout Total 100% 23 jointly-failed episodes Measured across the 2323 jointly-failed episodes, path planning is the dominant stage (14/23=61%14/23=61\%; Table V): the UGV’s VLM emits a path that collapses onto a building, tree, or the map border, and the vehicle crashes and stalls 1616–7373 m from the goal. Local execution (5/23=22%5/23=22\%) closes within 1515 m but hits the 180180 s limit. Global localization is the residual UAV-side failure (4/23=17%4/23=17\%), split between unstable tracking (3/23=13%3/23=13\%), whose points jump across frames, and mis-anchoring (1/23=4%1/23=4\%), the reference-binding failure we had hypothesized, in which the UAV locks onto a similar but wrong truck and flies toward it. Map rendering/projection contributed no failures and is omitted. Per-agent, the UAV fails from global localization (76%76\%) and the UGV from path planning (60%60\%); the UGV rescues 2727 of the UAV’s 5050 failures versus 22 the other way. Cross-view target re-identification is the highest-leverage learned component [10]. V REAL-ROBOT EXPERIMENT CASE Beyond the simulation, we deploy the same training-free pipeline on real hardware to verify its feasibility once decoupled from the simulator’s synchronized pose stream. A UAV (downward RGB camera) and a UGV (forward camera) form the team. The UAV runs 3D-SPF, and the UGV runs road-level path planning on the shared bird’s-eye map. The frozen VLM remains gemini-3.7-flash, and the decision cadence matches the simulation. In the deployment the UAV pose is provided by LiDAR odometry, while the stages (annotation, projection, closed-loop execution) are identical to the simulation. The physical deployment confirms that the training-free pipeline drives both agents to the target, as shown in Figure 5, which depicts the real-world air-ground collaborative VLN hardware and scene. The UAV, running Linux for VLN collaboration, carries a Mid360 LiDAR and a RealSense D435i, while the UGV, likewise running Linux for control and data processing, carries a forward camera. The scene spans six time steps in both top-down and third-person views: the UAV first climbs to altitude, calls the VLM to annotate the bird’s-eye view, and sends it to the UGV, which calls the VLM to plan a road path (the green point set); the UAV then approaches the target with the proposed 3D-SPF, while the UGV automatically steers along the planned path and moves toward the target. Fig. 5: Real-robot case: the same training-free pipeline (3D-SPF + shared bird’s-eye map road planning) running on a physical UAV+UGV team. Six time steps in top-down and third-person views show the quadrotor (Mid360 LiDAR + RealSense D435i) annotating the bird’s-eye map and flying 3D-SPF toward the target, while the omnidirectional robot follows its VLM-planned road path. VI LESSONS AND LIMITATIONS VI-A Lessons for Air-Ground Collaboration Lesson 1: put the interface at the global view. The UAV’s bird’s-eye view naturally carries the global relative positions of teammate and target, spatial relationships that mean the same thing in both agents’ frames, so choosing it as the communication interface (rather than text, features, or control signals) is what turns collaboration from hurting into helping relative to [11]. Lesson 2: view complementarity, not message richness. The gain comes not from transmitting richer information but from supplying the viewpoint the receiver lacks: the global spatial context invisible in the UGV’s first-person view, which lets the UGV’s VLM turn the UAV’s “can see” into the UGV’s “can drive”. Lesson 3: determinism confines error propagation. Because projection and inverse projection contain no learned parts, a VLM mistake on one platform cannot propagate to the other, structurally the opposite of the C2 velocity coupling that amplified errors in [11]. VI-B Limitations The baseline adopts three deliberate simplifications. It anchors the target with a frozen VLM against the target photo PgP_g rather than a ground-truth coordinate, which keeps the setup lightweight but inherits the VLM’s mis-anchoring risk. The 3 s VLM latency confines the agents to quasi-static scenes, leaving fast-moving targets out of reach. And CARLA-Air’s synchronized poses stand in for the GPS/SLAM a real field deployment would need. Each simplification is intentional: a minimal baseline keeps the remaining gaps measurable. VII CONCLUSION We presented AGC-VLN (Air-Ground Collaborative VLN), the first training-free baseline for air-ground collaborative VLN, which couples 3D-SPF on the UAV with VLM road-path planning on the UGV through a shared bird’s-eye map. On 100 closed-loop episodes in Town10HD it achieves a clear positive collaboration gain, in contrast to trained VLA models whose coupling degraded single-agent performance. Residual failures concentrate on global localization, chiefly mis-anchoring onto a wrong truck (reference binding), which delineates a concrete agenda: robust cross-view target re-identification, learned verification, and latency-robust coordination. We release the full system as a reproducible starting point for the field. References [1] P. Anderson, Q. Wu, D. Teney, J. Bruce, M. Johnson, N. Sünderhauf, I. Reid, S. Gould, and A. van den Hengel (2018) Vision-and-language navigation: interpreting visually-grounded navigation instructions in real environments. In CVPR, Cited by: §I. [2] Y. Qi, Q. Wu, P. Anderson, X. Wang, W. Y. Wang, C. Shen, and A. van den Hengel (2020) REVERIE: remote embodied visual referring expression in real indoor environments. In CVPR, Cited by: §I. [3] C. Hu, Y. Lin, Y. Lee, C. Su, J. Lee, S. Tsai, C. Lin, K. Chen, T. Ke, and Y. Liu (2025) See, point, fly: a learning-free vlm framework for universal unmanned aerial navigation. In Proceedings of the 9th Conference on Robot Learning (CoRL), p. 4697–4708. Cited by: item 1, §I, §I-A, TABLE I. [4] Ding, Zhang, Xu, et al. (2025) Uni-lavira: language-vision-robot actions translation for unified embodied navigation. arXiv preprint arXiv:2605.27582. Cited by: §I, §I-A. [5] Z. Xu, B. Lu, W. Bao, Z. Zhu, J. Zhang, H. Yan, W. Lu, and J. Wang (2025) Fly0: decoupling semantic grounding from geometric planning for zero-shot aerial navigation. arXiv preprint arXiv:2602.15875. Cited by: §I, §I-A. [6] D. Shao, Z. Xu, P. Wang, L. Liu, Y. Wang, J. Shi, and J. Huo (2026) FineCog-nav: integrating fine-grained cognitive modules for zero-shot multimodal uav navigation. In CVPR 2026 Findings, Cited by: §I, §I-A, §IV-B, TABLE I. [7] T. Zeng, Y. Wen, H. Chen, and H. Zhang (2025) CARLA-air: fly drones inside a carla world – a unified infrastructure for air-ground embodied intelligence. arXiv preprint arXiv:2603.28032. Cited by: §I, §I-B, §IV-A. [8] A. Dosovitskiy, G. Ros, F. Codevilla, A. Lopez, and V. Koltun (2017) CARLA: an open urban driving simulator. Cited by: §I, §I-B. [9] S. Shah, D. Dey, C. Lovett, and A. Kapoor (2018) AirSim: high-fidelity visual and physical simulation for autonomous vehicles. Cited by: §I, §I-B. [10] H. Li, Y. Wang, L. Wang, et al. (2026) AirGroundBench: probing spatial intelligence in multimodal large models under heterogeneous multi-view embodied collaboration. arXiv preprint arXiv:2606.28049. Cited by: §I, §I-B, §IV-G. [11] T. Zeng, Y. Wen, X. Yu, and H. Zhang (2026) Can aerial vla models cooperate? evaluating closed-loop air-ground coordination with carla-air. arXiv preprint arXiv:2605.31066. Cited by: item 2, §I, §I-B, item 1, §VI-A, §VI-A. [12] Anonymous (2025) NaVILA: towards efficient and generalizable vision-language navigation with 3d-aware vlms. In CoRL, Cited by: §I-A. [13] J. Chen, B. Lin, R. Xu, Z. Chai, X. Liang, and K. K. Wong (2024) MapGPT: map-guided prompting with adaptive path planning for vision-and-language navigation. In Annual Meeting of the Association for Computational Linguistics (ACL), Cited by: §I-A. [14] M. Chang, T. Gervet, M. Khanna, S. Yenamandra, D. Shah, S. Y. Min, K. Shah, C. Paxton, S. Gupta, D. Batra, R. Mottaghi, J. Malik, and D. S. Chaplot (2024) GOAT: go to any thing. In Robotics: Science and Systems (RSS), Cited by: §I-A. [15] Anonymous (2025) StreamVLN: streaming vision-and-language navigation with historical tokens. arXiv preprint. Cited by: §I-A. [16] Gong Guo et al. (2026) ABot-n1: toward a general visual language navigation foundation model. arXiv preprint arXiv:2607.10383. Cited by: §I-A. [17] Y. Hu, A. Xi, Q. Xiao, S. Isaacson, H. X. Liu, R. Vasudevan, and M. Ghaffari (2026) LongNav-r1: horizon-adaptive multi-turn rl for long-horizon vla navigation. arXiv preprint arXiv:2602.12351. Cited by: §I-A. [18] B. Lin, Y. Nie, K. L. Zai, Z. Wei, M. Han, R. Xu, M. Niu, J. Han, H. Zhang, L. Lin, B. Chen, C. Lu, and X. Liang (2026) EvolveNav: empowering llm-based vision-language navigation via self-improving embodied reasoning. IEEE Transactions on Pattern Analysis and Machine Intelligence (TPAMI). Cited by: §I-A. [19] T. Wang, X. Li, F. Lu, T. Gong, J. Dong, W. Xue, S. Qu, C. Bai, and G. Chen (2026) CoNavBench: collaborative long-horizon vision-language navigation benchmark. In International Conference on Learning Representations (ICLR), Cited by: §I-B. [20] S. Liu et al. (2025) AeroDuo: aerial duo for uav-based vision and language navigation. In ACM Multimedia (M), Cited by: §I-B. [21] Z. Liu, M. He, S. Yu, et al. (2026) OmniVLN: omnidirectional 3d perception and token-efficient llm reasoning for visual-language navigation across air and ground platforms. arXiv preprint arXiv:2603.17351. Cited by: §I-B. [22] Amap and MIV-XJTU (2026) JanusVLN: decoupling semantics and spatiality with dual implicit memory for vision-language navigation. In International Conference on Learning Representations (ICLR), Cited by: §I-B. [23] J. Lee, T. Miyanishi, S. Kurita, K. Sakamoto, D. Azuma, Y. Matsuo, and N. Inoue (2025) CityNav: a large-scale dataset for real-world aerial navigation. In International Conference on Computer Vision (ICCV), Cited by: §I-B. [24] S. Li, X. Dong, X. Ma, J. Chen, H. Zhao, and Y. Zhou (2026) CLOSER-vln: closed-loop self-verified retrieval-augmented reasoning for aerial vision-language navigation. arXiv preprint arXiv:2606.28397. Cited by: §I-B. [25] X. Zhu, Q. Meng, L. Yu, W. Zhang, Z. Ma, H. Zhou, and Y. Tian (2026) FSD-vln: fast-slow dual-system modeling for aerial long-horizon vision-language navigation. arXiv preprint arXiv:2607.08359. Cited by: §I-B. [26] H. Zhang, M. Liu, Q. Xiang, K. Wang, Y. Wang, and L. Nie (2026) SpatialUAV: benchmarking spatial intelligence for low-altitude uav perception, collaboration, and motion. arXiv preprint arXiv:2606.27876. Cited by: §I-B. [27] M. S. Mondal, L. Russo, J. D. Humann, J. M. Dotterweich, and P. Bhounsule (2026) Towards reliable aerial ground vehicle collaboration: an integrated planning and autonomy framework for field deployment. arXiv preprint arXiv:2607.07350. Cited by: §I-B. [28] H. Chen, J. Zheng, S. Yang, T. Zeng, S. Feng, S. Cheng, R. Ren, H. Guo, S. Yuan, X. Wang, K. Wang, and J. Pei (2026) Vision-and-language navigation for uavs: progress, challenges, and a research roadmap. arXiv preprint arXiv:2604.13654. Cited by: §I-B. [29] Zhang Zhang et al. (2026) Geo2^2: geometry-guided cross-view geo-localization and image synthesis. In CVPR, Cited by: §I-B3. [30] Anonymous (2026) GenGeo: robust cross-view geo-localization via foundation model and dynamic feature aggregation. Remote Sensing 18 (8), p. 1116. Cited by: §I-B3. [31] G. Gemini Team (2023) Gemini: a family of highly capable multimodal models. arXiv preprint arXiv:2312.11805. Cited by: §IV-A, §IV-D. [32] Y. Gao, C. Li, Z. You, J. Liu, Z. Li, P. Chen, Q. Chen, Z. Tang, L. Wang, P. Yang, Y. Tang, Y. Tang, S. Liang, S. Zhu, Z. Xiong, Y. Su, X. Ye, J. Li, Y. Ding, D. Wang, X. Li, Z. Wang, and B. Zhao (2026) OpenFly: a comprehensive platform for aerial vision-language navigation. In International Conference on Learning Representations (ICLR), Cited by: §IV-B, TABLE I. [33] F. Xue, E. Yu, Y. Shen, Z. Hu, H. Wang, Y. Yang, X. Wang, and J. Sun (2026) See-and-reach: precise vision-language navigation for uavs within the field of view. arXiv preprint arXiv:2606.20045. Cited by: §IV-B, TABLE I. [34] X. Wang, D. Yang, Z. Wang, H. Kwan, J. Chen, W. Wu, H. Li, Y. Liao, and S. Liu (2025) Towards realistic uav vision-language navigation: platform, benchmark, and methodology. In International Conference on Learning Representations (ICLR), Cited by: §IV-B, TABLE I.

Instant research discovery