Paper deep dive
Language-Guided Terrain-Adaptive Neural MPC for Autonomous Traversal of Articulated Tracked Robots
Zhenfeng Gan, Yanbo Chen, Lirong Che, Yongyi Ma, Rongkai Zhu, Xueqian Wang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 9/17/2026, 3:25:20 AM
Summary
The paper introduces ASTRIL-MPC, a language-guided neural Model Predictive Control (MPC) framework for the autonomous traversal of Articulated Tracked Robots (ATRs) in contact-rich environments. The system integrates a terrain-conditioned neural kinematics model for short-horizon state prediction, a constrained NMPC for optimization-based planning, and a Large Language Model (LLM) that acts as a high-level regulator to adapt MPC weights and bounds via natural language instructions. This approach improves traversal quality significantly over non-adaptive NMPC and PPO baselines while ensuring safety through bounded, safety-checked parameter updates.
Entities (7)
Relation Signals (6)
ASTRIL-MPC → designedfor → Articulated Tracked Robots
confidence 95% · framework for autonomous traversal of Articulated Tracked Robots
ASTRIL-MPC → uses → Neural Kinematics Model
confidence 95% · A learned kinematics model predicts short-horizon task-state increments... ASTRIL-MPC... framework
ASTRIL-MPC → uses → Large Language Model
confidence 95% · a large language model (LLM) proposes bounded updates to selected weights and bounds
ASTRIL-MPC → uses → Model Predictive Control
confidence 95% · neural MPC plans with multi-objective costs and strict feasibility constraints
Large Language Model → modifies → Model Predictive Control
confidence 90% · LLM proposes bounded updates to selected weights and bounds... of MPC
ASTRIL-MPC → outperforms → PPO
confidence 90% · improves an aggregate traversal-quality score by up to 71% over a non-adaptive NMPC and by 67% over a PPO baseline
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:In urban search and rescue, articulated tracked robots (ATRs) must traverse structured but contact-rich environments such as stairwells and cluttered building interiors. Reliable autonomy remains challenging because robot-terrain interaction (RTI) is hybrid and discontinuous, and effective flipper-track coordination is difficult to model analytically. We present ASTRIL-MPC, a language-guided neural kinematics model predictive control (MPC) framework for autonomous traversal. A learned kinematics model predicts short-horizon task-state increments from a height sequence and recent trajectories; NMPC plans with multi-objective costs and strict feasibility constraints; and a large language model (LLM) proposes bounded updates to selected weights and bounds through a safety-checked interface with range clipping, rate limiting, and consistency checks. The compiled predictor enables a full control cycle within 100 ms. Across three traversal tasks and a multi-height generalization setting, ASTRIL-MPC improves an aggregate traversal-quality score by up to 71% over a non-adaptive NMPC and by 67% over a PPO baseline, while eliminating measurable collision impacts during descent. These results indicate that combining terrain-conditioned neural kinematics, optimization-based planning, and language-guided adaptation yields data-efficient and robust autonomy for articulated tracked robots. Real-robot trials over four indoor obstacles further demonstrate transfer to contact-rich physical traversal.
Tags
Links
- Source: https://arxiv.org/abs/2609.13083v3
- Canonical: https://arxiv.org/abs/2609.13083v3
Trouble viewing inline? Open PDF directly →
Full Text
48,513 characters extracted from source content.
Expand or collapse full text
Language-Guided Terrain-Adaptive Neural MPC for Autonomous Traversal of Articulated Tracked Robots Zhenfeng Gan* Yanbo Chen* Lirong Che Yongyi Ma Rongkai Zhu Xueqian Wang† Abstract In urban search and rescue, articulated tracked robots (ATRs) must traverse structured but contact-rich environments such as stairwells and cluttered building interiors. Reliable autonomy remains challenging because robot–terrain interaction (RTI) is hybrid and discontinuous, and effective flipper–track coordination is difficult to model analytically. We present a language-guided terrain-adaptive neural model predictive control (MPC) framework for autonomous traversal. A terrain-conditioned neural kinematics model predicts short-horizon task-state increments from a local height sequence and recent trajectories; neural MPC plans with multi-objective costs and strict feasibility constraints; and a large language model (LLM) enables terrain adaptation by proposing bounded updates to selected weights and bounds through a safety-checked interface with range clipping, rate limiting, and consistency checks. The compiled predictor enables a full control cycle within 100 ms. Across three traversal tasks and a multi-height generalization setting, the proposed terrain-adaptive neural MPC improves an aggregate traversal-quality score by up to 71% over non-adaptive neural MPC and by 67% over a PPO baseline, while eliminating measurable collision impacts during descent. These results indicate that combining terrain-conditioned neural kinematics, optimization-based planning, and language-guided adaptation yields data-efficient and robust autonomy for articulated tracked robots. Real-robot trials over four indoor obstacles further demonstrate transfer to contact-rich physical traversal. I Introduction Recent vision-language-action (VLA) models show that large-scale visual and linguistic priors can be transferred to robotic control by mapping observations and language instructions directly to actions [1, 2]. This paradigm is promising for semantic generalization, but it is not sufficient by itself for contact-rich autonomy. In many deployments, the action generator is executed as a feedforward policy conditioned on the current observation; it does not explicitly optimize over future contact events, actuator limits, body-attitude bounds, or task-specific safety constraints. Consequently, when the robot encounters a new terrain geometry or a changed traversal preference, the observation can become out of distribution and the policy may produce actions that are semantically plausible but dynamically infeasible. This limitation is especially pronounced in urban search and rescue, where articulated tracked robots (ATRs) must traverse stairwells, landings, and cluttered building interiors [3, 4, 5]. Their articulated flippers enlarge the contact patch, improve traction, and facilitate obstacle traversal [6, 7] (Fig. 1), but reliable motion requires coordinated chassis–flipper behavior under hybrid robot–terrain interaction (RTI). Frequent support-mode switches and discontinuous contacts make explicit modeling difficult, while small geometric variations can change the desired trade-off between stability, impact, and speed. These factors motivate a controller that can retain hard feasibility constraints while adapting its objective to different environments and operator preferences. Fig. 1: Articulated tracked robot studied in this work. Classical geometry-based planners ensure basic traversability but rely on pre-defined robot configurations, which limits optimality and often produces non-smooth trajectories on irregular terrain [8, 9, 10]. Reinforcement learning avoids explicit modeling yet commonly exhibits jerky switching in discrete action spaces, oscillations in continuous action spaces, and requires terrain-specific reward shaping [11, 12, 13, 14, 15, 16, 17]. Vision-language policies improve semantic grounding, but direct action prediction still lacks an explicit mechanism for enforcing traversal constraints and retuning objective trade-offs under out-of-distribution terrain. Model predictive control (MPC) offers a complementary route: it enforces hard constraints, coordinates multiple objectives via tunable cost weights, and yields smooth behavior through receding-horizon optimization. However, practical MPC deployment on ATRs faces two obstacles. First, accurate modeling under hybrid contacts is difficult; non-smooth contact transitions and support changes make real-time numerical optimization challenging. Second, tuning is labor-intensive: weights and constraints in a multi-objective formulation often require scenario-specific adjustment by experts. To address these challenges, we present a language-guided terrain-adaptive neural MPC framework for ATRs. Prior work has shown that neural NMPC can be deployed in safety-critical domains such as automated driving under unknown friction conditions [18]. We augment MPC with a terrain-conditioned neural kinematic predictor to bypass explicit contact modeling and introduce a regulation layer powered by a large language model (LLM). Rather than generating actions directly, the LLM acts as a high-level optimizer over a restricted parameter set, translating natural-language guidance and recent terrain-dependent behavior into bounded edits of MPC weights and bounds, in the spirit of recent LLM-based optimization methods [19, 20] and safety-guarded parameter tuning in autonomous systems [21]. The low-level neural MPC then closes the loop with hard feasibility constraints, while the language-guided regulator enables fast adaptation across terrain geometries and task preferences. In summary, our full contributions are as follows: • Terrain-conditioned neural kinematics. We develop a map-attached neural kinematics model that fuses terrain and recent motion history to capture hybrid-contact effects without explicit contact-mode enumeration. This yields accurate yet lightweight motion prediction as the foundation for real-time planning. • Constrained neural MPC. We embed the learned kinematics in NMPC as the state-transition constraint and solve the full problem in 100ms100\,ms, enabling receding-horizon control with explicit state and input constraints. • Language-guided terrain adaptation. We use an LLM to adapt cost weights, input bounds, and flipper PD gains from high-level guidance and recent terrain-dependent behavior, while a safety gate enforces editable-set restrictions, clipping, rate limits, smoothing, and feasibility checks. This improves traversal quality by 67%67\% over PPO and enables terrain-dependent generalization without retraining the predictor or changing the solver. I Related Work Classical planners for articulated tracked robots with flippers typically rely on geometric reasoning and pre-defined motion primitives. Early systems combined terrain scanning with semi-autonomous flipper control and safety constraints, achieving reliable but conservative behaviors under limited perception and modeling assumptions [8, 14]. Recent geometry-based planners explicitly encode robot–terrain contact relationships to generate real-time flipper motions on rough terrain [22]. Hybrid sampling/optimization methods further improve feasibility on discontinuous, contact-rich surfaces [23, 24], yet the reliance on hand-crafted templates and mode switching restricts expressiveness and can yield sub-optimal trajectories and unnecessary energy expenditure. An alternative line employs RL. Template-based RL selects among a small set of discrete postures [12], while continuous-action policies learn end-to-end commands [15, 5, 17, 16]. Despite promising results, RL for flipper robots often requires terrain-specific reward shaping and long training cycles; instability and divergence are common failure modes. The recent FTR-Bench study standardizes tasks and metrics for flipper-track control and highlights the sensitivity of RL performance to reward design and training stability [13]. Moreover, discrete posture switching introduces additional, sometimes pointless, energy consumption. To balance optimality and data efficiency, learning-augmented MPC integrates learned models into constrained optimization. Real-time neural MPC demonstrates that neural kinematics models can be embedded in real-time control loops for agile platforms [25]. Tooling such as Learning for CasADi facilitates differentiable PyTorch models inside numerical optimization, enabling neural NMPC pipelines with principled constraints and warm-starts [26, 27, 28, 29]. These advances motivate our formulation, which preserves the structure and safety of NMPC while leveraging data-driven components [30, 31, 32, 33, 34]. Recent work uses language to condition robot controllers. General VLA models such as RT-2 and OpenVLA learn to map visual observations and language commands to robot actions, improving semantic generalization from large-scale pretraining [1, 2]. However, direct action generation does not by itself impose hard state/input constraints or solve a terrain-dependent optimal control problem. For manipulation, VLMPC parses visual–language instructions and exposes them to an MPC layer that enforces goal and constraint satisfaction [35]. On the learning side, language-to-reward pipelines show that LLMs can synthesize rewards or curricula that steer policy training efficiently and improve transfer [36, 37]. LLMs have also been explored as optimizers, either by iteratively proposing candidate solutions in natural language or by formulating solver-ready optimization models [19, 20]. Closer to our setting, Liang et al. [38] couple language feedback with MPC for data-efficient adaptation, while fielded studies deploy compact, on-board LLMs to perform low-rate, safety-guarded retuning of MPC weights and constraints without disturbing the fast control loop [21]. Our approach follows this pattern: the LLM provides low-frequency, bounded updates to NMPC objectives/constraints, whereas the neural NMPC backbone enforces constraints and closes the high-rate loop. I Methodology I-A System Overview Fig. 2: Overview of the proposed language-guided terrain-adaptive neural MPC system, which integrates terrain-conditioned neural kinematics, guarded parameter adaptation, and constrained motion planning. The proposed language-guided terrain-adaptive neural MPC architecture (Fig. 2) comprises three tightly coupled layers. First, the front end converts incoming 3D point clouds into a local height map and buffers a short history of recent robot states and control commands. Second, a compiled neural kinematics consumes the height map together with this history to provide short-horizon state increments in a map-attached frame (Sec. I-B). Third, this kinematics is embedded as the model constraint within an NMPC, which solves a finite-horizon optimal control problem with multi-objective costs and hard feasibility constraints, applies the first control input, and then shifts the horizon as new measurements arrive (Sec. I-C). An in-the-loop LLM serves as a terrain-adaptation regulator: given language guidance, a compact summary of the recent terrain-dependent state–input history, and the current parameter vector, it proposes edits to a restricted set of weights, bounds, and flipper PD gains; a guarded interface enforces value ranges, change-rate limits, and basic consistency before any update is applied (Sec. I-D). The full control loop implementing these interactions is detailed in Algorithm 1. I-B Modeling I-B1 State Representation Explicitly first principles modeling the discontinuous contacts and support-mode switches is too complex to run in real time. We therefore adopt a lightweight neural kinematics model conditioned on a local height sequence sampled from the global map. The state is restricted to the 2D sagittal (x–z) plane, which is sufficient for capturing the dominant motion of articulated tracked robots (cf. Fig. 3). This restriction is also consistent with our platform: the coaxial flipper pair cannot actively control roll, so roll stability must be handled primarily by selecting a globally roll-favorable path. Rather than addressing this global planning, our work serves as a downstream module, focusing on generating the appropriate articulated motions to track predefined trajectories and negotiate obstacles. At each control cycle t, we anchor at the robot’s current longitudinal position xtmx_t^m, corresponding to the x-coordinate of its center of mass (COM). We take nmn_m samples ahead and nmn_m samples behind, spaced by dmd_m, to form an elevation sequence MtM_t that is fixed over the prediction horizon (H): Mt=h(xtm+idm)|i=−nm,…,0,…,nm, M_t= \h(x_t^m+id_m)\ |\ i=-n_m,…,0,…,n_m \, (1a) Mt+j≡Mt,j=0,…,H. M_t+j≡ M_t, j=0,…,H. (1b) Here Mt∈ℝ2nm+1M_t\!∈\!R^2n_m+1 is the sampled height sequence (cf. (1)), where each entry is the ground height returned by h(⋅)h(·) at a longitudinal coordinate. To represent the robot’s position relative to this terrain profile within a prediction horizon, we define a local longitudinal coordinate x~t+j:=xt+j−xmt,j=0,…,H. x_t+j:=x_t+j-x^m_t, j=0,…,H. (2) Based on these coordinates, we define two types of vectors: (1) a task vector sts_t that describes the robot’s absolute state, and (2) a state vector s~t s_t that describes the robot’s state relative to the sampled map, using local longitudinal offsets (anchored at x~t=0 x_t=0), together with their associated control inputs: t+j _t+j =[xt+j,zt+j,θp,t+j,θt+jf,θt+jr]⊤, = [x_t+j,\,z_t+j,\, _p,t+j,\,θ^f_t+j,\,θ^r_t+j ]^\! , (3) ~t+j s_t+j =[x~t+j,zt+j,θp,t+j,θt+jf,θt+jr]⊤, = [ x_t+j,\,z_t+j,\, _p,t+j,\,θ^f_t+j,\,θ^r_t+j ]^\! , (4) t+j _t+j =[vt+j,ωt+jf,ωt+jr]⊤. = [v_t+j,\,ω^f_t+j,\,ω^r_t+j ]^\! . (5) for j=0,…,Hj=0,…,H. To provide the learned kinematics and the NMPC with short-term motion context, we maintain a length-k buffer of recent states and inputs: Bt:=s~t−k+1:t,t−k+1:t,B_t:=\ s_t-k+1:t,\,u_t-k+1:t\, (6) where k is the buffer size (number of most recent time steps). Fig. 3: (a) Planar configuration of the articulated tracked robot in the sagittal x–z plane. The task variables are the COM position (x,z)(x,\,z), the pitch angle θp _p, and the front and rear flipper angles θf,θrθ^f,\,θ^r, with motion variables x˙,θ˙f,θ˙r x,\, θ^f,\, θ^r. (b) Map-attached representation for prediction and control. At control step t, the elevation sequence MtM_t is anchored at the longitudinal COM position xtx_t, yielding a fixed terrain profile Mt+j≡MtM_t+j≡ M_t over the horizon. I-B2 Neural Kinematics Model We employ a learned discrete-time kinematic model fθf_θ that predicts the task-space increment at each stage. The model is queried with: (i) the local terrain sequence MtM_t held fixed over the prediction horizon (1); (i) a short history buffer BtB_t capturing recent motion context; and (i) the stage control t+ju_t+j. The per-stage prediction and update are Δ^t+j s_t+j =fθ(Mt,Bt,t+j), =f_θ\! (M_t,\,B_t,\,u_t+j ), (7a) ^t+j+1 s_t+j+1 =t+j+Δ^t+j,j=0,…,H−1. =s_t+j+ s_t+j, j=0,…,H-1. (7b) where each ^t+j s_t+j has the same physical components as ~t+j s_t+j (x~,z,θp,θf,θr x,\,z,\, _p,\,θ^f,\,θ^r). To guide the network toward stable and control-aware predictions, we design a composite loss that balances accuracy, smoothness, and robustness. The details are summarized in Table I. We use a diagonal positive weight matrix W_fit. The positive part is defined as (a)+=max(a,0).(a)_+= (a,0). (8) Elementwise clipping is clip(z,δ)=sign(z)min(|z|,δ).clip(z;δ)=sign(z)\, (|z|,δ). (9) The short-horizon forward-motion trend is Δx¯i=12(Δxi+Δxi−1), x_i= 12 ( x_i+ x_i-1 ), (10) where Δxi x_i denotes the x–component of Δi _i. Fit (ℒfitL_fit) is a weighted MSE on the predicted increments, letting critical channels receive larger weights. Overshoot (ℒoverL_over) penalizes only positive forward-direction bias (Δx^−Δx)+2( x- x)_+^2, discouraging overly aggressive advancement. Smoothness (ℒsmL_sm) aligns the predicted forward increment Δx x with the local trend Δx¯ x to reduce jitter across steps. Magnitude (ℒmagL_mag) penalizes clipped predicted increments to keep outputs within plausible ranges and improve robustness to outliers. Regularization (ℒregL_reg) applies ℓ2 _2 weight decay on θ to curb overfitting. TABLE I: Control-aware training loss terms. Sums over i run across the training mini-batch. Term Definition ℒfitL_fit ∑i‖Δi+1−Δ^i+1‖2 _i\| _i+1- s_i+1\|_W_fit^2 ℒoverL_over λover∑i(Δx^i+1−Δxi+1)+2 _over\! _i( x_i+1- x_i+1)_+^2 ℒsmL_sm λsm∑i‖Δx^i+1−Δx¯i‖22 _sm\! _i\| x_i+1- x_i\|_2^2 ℒmagL_mag λmag∑i‖clip(Δ^i+1,δ)‖22 _mag\! _i\|clip( s_i+1;δ)\|_2^2 ℒregL_reg λreg‖θ‖22 _reg\|θ\|_2^2 I-C Model Predicted Control At each control step t, the NMPC optimizes over the task-space states t+js_t+j and control inputs t+ju_t+j defined in Sec. I-B. It uses the learned kinematics (7a–7b) together with the fixed height sequence MtM_t (1) to solve a finite-horizon problem with multi-objective costs and strict feasibility constraints. Input penalties are centered around a PD feedforward term f,t+ju_f,t+j. mins^t:t+Ht:t+H subarrayc s_t:t+H\\ u_t:t+H subarray \; ∑j=0H−1(‖s^t+j−t+jref‖2+‖t+j−f,t+j‖2) _j=0^H-1\! (\| s_t+j-s^ref_t+j\|_W^2+\|u_t+j-u_f,t+j\|_R^2 ) +‖s^t+H−t+Href‖e2, \;+\| s_t+H-s^ref_t+H\|_W_e^2, (11a) s.t. =tini, _t=B_t^ini, (11b) s^t+j+1 s_t+j+1 =s^t+j+fθ(Mt,Bt^,t+j), = s_t+j+f_θ\! (M_t, B_t,u_t+j ), (11c) for j=0,…,H−1. j=0,…,H-1. Bt^:=s^t−k+1:t,t−k+1:t, B_t:=\ s_t-k+1:t,\,u_t-k+1:t\, (11d) lb ^lb ≤t+j≤ub, _t+j ^ub, (11e) lb ^lb ≤s^t+j≤ub. ≤ s_t+j ^ub. (11f) I-C1 Cost and Reference We use diagonal state weights W (terminal eW_e) on the five task variables and diagonal input penalties R. The input term is centered at a feedforward sequence f,t+j=[uf,t+jff,ur,t+jff]⊤u_f,t+j=[\,u^f_f,t+j,\,u^f_r,t+j\,] aligned to the NMPC grid. For the flippers, fu_f is generated by per-joint PD laws: uf,t+jff u^f_f,t+j =KPf(θt+jf,ref−θtf)+KDf(ωt+jf,ref−ωtf), =K^f_P (θ^f,ref_t+j-θ^f_t )+K^f_D (ω^f,ref_t+j-ω^f_t ), (12a) ur,t+jff u^f_r,t+j =KPr(θt+jr,ref−θtr)+KDr(ωt+jr,ref−ωtr). =K^r_P (θ^r,ref_t+j-θ^r_t )+K^r_D (ω^r,ref_t+j-ω^r_t ). (12b) Here (θtf,ωtf)(θ^f_t,ω^f_t) and (θtr,ωtr)(θ^r_t,ω^r_t) are the measured joint positions and velocities at time t, while (θt+jf,ref,ωt+jf,ref)(θ^f,ref_t+j,ω^f,ref_t+j) and (θt+jr,ref,ωt+jr,ref)(θ^r,ref_t+j,ω^r,ref_t+j) are the stage-t+jt+j references. To suppress first-stage jerk on the chassis, the forward-velocity reference is ramped within the horizon: vt+jref=v0+jH(vref−v0),j=0,…,H−1.v^ref_t+j\;=\;v_0+ jH (v_ref-v_0 ), j=0,…,H-1. (13) with v0v_0 the measured speed at time t. Before entering the cost, each f,t+ju_f,t+j is clipped to the actuator box in (11e) and subjected to the same rate limit Δumax u_ (with smoothing factor αsm _sm) as the applied control, which keeps the penalty center feasible and reduces early-stage optimizer effort. I-C2 Constraints We enforce feasibility by clamping the initial state to the current measurement ^t s_t and constraining all control inputs within a bounded feasible set. At each stage, t+j=[vt+j,ωt+jf,ωt+jr]⊤u_t+j=[\,v_t+j,\,ω^f_t+j,\,ω^r_t+j\,] must lie in =∣lb≤ubU=\u ^lb ^ub\, which includes explicit forward-speed bounds vmin≤vt+j≤vmaxv_ ≤ v_t+j≤ v_ . These bounds are part of the editable parameter set (Table I) and can be adjusted online under safety gating. I-D Language-Guided Terrain Adaptation via LLM Although neural MPC provides a principled way to enforce constraints and balance multiple objectives, its performance on ATRs is sensitive to cost weights and constraint bounds. This sensitivity becomes more severe under out-of-distribution terrain, where a fixed objective may no longer express the desired trade-off among speed, stability, and impact. Instead of using an LLM as an open-loop action policy, we use it as a structured terrain-adaptation assistant: following the idea that LLMs can propose and refine optimization variables or solver-ready formulations [19, 20], the model only suggests parameter edits for an existing constrained neural MPC controller. This yields a rapid and interpretable language-guided adaptation layer without altering the MPC formulation or bypassing its feasibility constraints (Fig. 4). Fig. 4: Language guidance and recent terrain-dependent history are formatted by the Prompt Builder and sent to the LLM. The LLM returns a JSON of neural MPC parameter updates (tracking weights, input penalties, flipper PD gains, smoothing/rate limits, and speed reference/bounds), which overwrite the current set. Neural MPC applies these updates only to costs and constraints (see Eqs. 11a–11f), solves the horizon, and sends control commands; parameters persist until new guidance arrives. When a new operator instruction arrives, the controller assembles a compact prompt QtQ_t from: (i) the current measurement (~t,t)( s_t,\,u_t); (i) the length-k history buffer BtB_t; (i) the active parameter vector t−1 λ_t-1 that instantiates the NMPC weights (,e,)(W,W_e,R) and bounds (vmin,vmax)(v_ ,v_ ); and (iv) a concise parameter dictionary and optimization template (objective and constraints in (11a)–(11f)), followed by the operator instruction instrtinstr_t. The prompt therefore asks the LLM to solve a small, semantic parameter-selection problem rather than the continuous NMPC itself. The LLM is treated as a conditional generator of parameter updates; its output is restricted to a predefined subset and passed through a safety gate that enforces bounds, per-update rate limits, and smoothing before any change takes effect. TABLE I: Parameters adapted online by the LLM. Category Parameters State tracking weights Wfwd,Wz,Wθp,Wθf,WθrW_fwd,\,W_z,\,W_ _p,\,W_ _f,\,W_ _r Control penalties Rv,Rωf,RωrR_v,\,R_ω^f,\,R_ω^r Feedforward shaping KPf,KDf,KPr,KDrK^f_P,\,K^f_D,\,K^r_P,\,K^r_D Smoothing and limits Δumax,αsm u_ ,\, _sm Velocity reference/bounds vref,vmin,vmaxv_ref,\,v_ ,\,v_ Prompt construction and parameter extraction Using the above quantities, prompt construction and parameter extraction are written as Qt Q_t =PromptBuild(Bt,Mt,t−1,instrt), =PromptBuild\! (B_t,\,M_t,\, λ_t-1,\,instr_t), (14a) ~t λ_t =ParamExtract(LLM(Qt))|. =ParamExtract\! (LLM(Q_t) ) |_S. (14b) where S is the editable set in Table I (state-tracking weights, input penalties, flipper PD gains, smoothing factors, and velocity bounds). All other components remain fixed. Guarded update The suggested edits are passed through a safety gate that enforces validity, magnitude limits, and temporal smoothness: ¯t λ_t =clip(~t,lb,ub), =clip\! ( λ_t;\ λ^lb,\, λ^ub ), (15a) ^t λ_t =proj∥⋅−t−1∥∞≤Δumax(¯t), =proj_\|·- λ_t-1\|_∞≤ u_ \! ( λ_t ), (15b) t λ_t =(1−αsm)t−1+αsm^t. =(1- _sm)\, λ_t-1+ _sm\, λ_t. (15c) with elementwise bounds (lb,ub)( λ^lb, λ^ub), a per-update ℓ∞ _∞ cap Δumax u_ , and smoothing factor αsm∈(0,1] _sm\!∈\!(0,1]. Feasibility checks (e.g., vmin<vmaxv_ <v_ and ,e,≻0W,W_e,R 0) are verified before applying (15c). Only λ is updated; the perception stack, the learned kinematic model fθf_θ, and the NMPC solver remain unchanged. This mechanism enables fast, language-guided terrain adaptation while preserving the underlying control pipeline. Algorithm 1 Language-guided terrain-adaptive neural MPC pipeline 1. Offline: M←ExtractHeightMap()M (P); t+jrefj=0H←GenerateReference(M)\s^ref_t+j\_j=0^H (M) 2. Init: buffer BtB_t, model fθf_θ, params 0 λ_0, horizon H 3. while robot active do 4. Update BtB_t and local map slice MtM_t 5. if new instruction then 6. Qt←BuildPrompt(instrt,Mt,Bt,t−1)Q_t (instr_t,M_t,B_t, λ_t-1) 7. apply guarded update (15a–15c) 8. end if 9. Build PD feedforward and ramp vrefv^ref 10. for j=0j=0 to H−1H-1 do 11. ^t+j+1←^t+j+fθ(Mt,B^t,t+j) s_t+j+1← s_t+j+f_θ(M_t, B_t,u_t+j) 12. B^t←^t−k+1:t,t−k+1:t B_t←\ s_t-k+1:t,\,u_t-k+1:t\ 13. end for 14. Solve NMPC (11a–11f); apply smoothing; send tu_t 15. end while Runtime loop. Algorithm 1 summarizes the closed-loop procedure. Offline, a local height map and reference profiles are prepared (Step 1–2). Online, each cycle updates the history buffer BtB_t and the fixed terrain slice MtM_t (Step 4). If an operator instruction arrives, the controller freezes actuation, builds a prompt QtQ_t (Step 6), parses the LLM output and applies the guarded update (Step 7). It then constructs the flipper PD feedforward fu_f (Step 9), rolls the learned kinematics over the horizon (Step 10–12), and solves the NMPC (Step 14). IV Experiments and Results IV-A Experimental Setup IV-A1 Platform and Simulation Setup We use the tracked rescue platform from [39] (Fig. 1), featuring four independently actuated flippers and a symmetric chassis. Simulation scenes follow [22]; the robot traverses terrains represented as height maps with a sampling interval dm=0.0625d_m=0.0625 m (1a) and the full NMPC loop runs within 100100 ms per cycle. IV-A2 Hardware and Software All components run on ROS Noetic, Ubuntu 20.04. The NMPC is modeled in CasADi [27]; the learned kinematic model is compiled to solver-compatible code via L4CasADi [25, 26]; and the resulting nonlinear program is solved online with ACADOS [28]. Experiments run on an Intel i5 CPU with an NVIDIA RTX 2080Ti GPU. IV-A3 Default Parameters Unless otherwise stated, we use horizon H=12H=12 with Δt=0.1 t=0.1 s; state–tracking weights Wfwd=3.0,Wz=0.5,Wθp=15.0,Wθf=5.0,Wθr=5.0W_fwd=3.0,\;W_z=0.5,\;W_ _p=15.0,\;W_ _f=5.0,\;W_ _r=5.0; input penalties Rv=10−3,Rωf=2×10−2,Rωr=2×10−2R_v=10^-3,\;R_ω^f=2×10^-2,\;R_ω^r=2×10^-2; flipper PD gains KPf=KPr=2.0,KDf=KDr=0.3K^f_P=K^r_P=2.0,\;K^f_D=K^r_D=0.3; rate/smoothing Δumax=0.18,αsm=0.8 u_ =0.18,\; _sm=0.8; and speed targets vref=0.30v_ref=0.30 m/s with initial command v0=0.12v_0=0.12 m/s. These categories correspond to the tunable entries in Table I. IV-A4 Language Model Language-guided terrain adaptation is realized by the DashScope Qwen-Plus LLM [40], which generates bounded updates to neural MPC weights and soft constraints from sliding-window summaries of terrain-dependent behavior and high-level task instructions. IV-B Data Collection and Training Details We train a next-step kinematic predictor as supervised regression that maps a history-augmented input to the 5-dimensional task state at t+1t+1. The input concatenates a 64-bin local height sequence, recent longitudinal positions, recent commands, and short-horizon robot states (Sec. I-B); samples are recorded at 10 Hz and post-processed into the compact representation. The dataset comprises 5k simulated samples from stair ascent/descent, split 95:5 for training/testing. The predictor is a feedforward MLP with residual connections and a linear 5D output head, trained with Adam (lr 1×10−41× 10^-4, batch 2048) using the control-aware loss in Table I; generalization is evaluated separately in Sec. IV-E. IV-C Evaluation and Analysis of Neural Kinematics To isolate the effect of the loss (symbols as in Table I), we compare three configurations: Type 1 (ℒfitL_fit only), Type 2 (ℒfit+ℒover+ℒmag+ℒregL_fit+L_over+L_mag+L_reg), and Type 3 (Type 2 +ℒsm+L_sm). Here ℒfitL_fit is the fitting term, ℒoverL_over penalizes forward overshoot, ℒmagL_mag constrains prediction magnitude, ℒregL_reg is the L2L_2 regularizer, and ℒsmL_sm encourages temporal smoothness. All models are trained and evaluated under identical data and rollout settings. IV-C1 Evaluation Protocol and RMSE Aggregation Let the prediction horizon be H=10H=10. For each evaluation trajectory of length T, we split it into S=⌊T/H⌋S= T/H non-overlapping windows of length H. At the start of each window we freeze the height sequence and history buffer, run the model open-loop for j=1,…,Hj=1,…,H, and measure step-ahead errors on the task-vector components c∈x,z,θp,θf,θrc∈\x,z, _p, _f, _r\. For each step j, we compute the RMSE across the S windows between the j-step predictions and ground truth for each component. We compare the per-step RMSE across the x direction, the z direction, the pitch angle, and the front/rear flipper angles. Fig. 5: Visualizing prediction vs. ground truth: from one continuous prediction rollout, we randomly sample three discrete time steps and show the predicted states (blue) against the ground-truth trajectory (red) for each loss design (Type 1/2/3). Fig. 6: Per-step RMSE (j=1…10j=1… 10) under three loss designs: Type 1 (ℒfitL_fit), Type 2 (Type 1 +ℒover+ℒmag+ℒreg+L_over+L_mag+L_reg), and Type 3 (Type 2 +ℒsm+L_sm). Subplots show (a) x-direction error, (b) z-direction error, (c) pitch-angle error, and (d) front/rear flipper angle errors. Curves show RMSE ± standard deviation. IV-C2 Results and Analysis As shown in Fig. 5 and Fig. 6, Type 1 exhibits horizon-wise drift, Type 2 reduces this drift but still shows noticeable errors, and the full loss (Type 3) yields the most accurate and stable rollouts across all state components. Meanwhile, the trajectory visualization shows the same trend: Type 1/2 diverge from the ground truth as the horizon unfolds, whereas Type 3 stays closely aligned throughout the rollout, supporting the effectiveness of our final loss design. The RMSE curves and the visualization are consistent across all variables, indicating that the errors are not dominated by a single state. Fig. 7: Generalization experiment in a single environment with multiple obstacle heights. The controller maintains feasible configurations and consistent traversal behaviors across height variations. IV-D Comparative Experiment We compare against a reinforcement learning baseline implemented with Proximal Policy Optimization (PPO) [13]. IV-D1 Evaluation Tasks We evaluate three scenarios: single-block crossing, stair ascent, and stair descent. Each scenario is executed three times per controller, with success defined as reaching the goal within 6060 s without tipping. IV-D2 Performance Metrics To evaluate traversal quality, we record the following six raw indicators over each run: ϕ,θpφ, _p are the roll and pitch angles (in rad), ϕ˙,θp˙ φ, _p their angular velocities (in rad/s), and ax,aza^x,a^z are the linear accelerations of the robot center of mass along the forward and vertical axes (in m/s2m/s^2). For clarity, we group them into three paired terms that summarize stability, smoothness, and impact: Stability Stability =12(norm(|ϕ|max)+norm(|θp|max)), = 12 (norm(|φ|_ )+norm(| _p|_ ) ), (16a) Swing Swing =12(norm(|ϕ˙|max)+norm(|θ˙p|max)), = 12 (norm(| φ|_ )+norm(| θ_p|_ ) ), (16b) Shock -27.74982ptShock =12(norm(|ax|max)+norm(|az|max)). = 12 (norm(|a^x|_ )+norm(|a^z|_ ) ). (16c) Each pair of indicators is normalized to [0,1][0,1] and combined into three dimensionless penalty terms (lower is better). These penalty terms are broadly applicable to most prior studies on ATRs traversal, as they capture stability, smoothness, and impact in a task-agnostic manner. All experiments are repeated three times per scenario, and the reported values are the means across repetitions. IV-D3 Results Table I reports the three normalized penalties and their aggregate mean across terrains. Terrain-adaptive neural MPC improves the overall mean by 42% over fixed-parameter neural MPC and by 67% over PPO, including gains of 33% on ascent and 71% on descent over fixed-parameter neural MPC. Although the fixed controller has a slightly better single-block mean, adaptation reduces its Shock penalty from 3.0×10−33.0×10^-3 to 1.4×10−41.4×10^-4. The largest benefits therefore occur on the more dynamic stair scenarios. IV-E Generative Experiment We further test multiple obstacle heights in a single environment (Fig. 7). Without task-specific retraining and using the same initial parameters, the controller maintains feasible configurations and bounded chassis pitch by coordinating front-flipper probing and rear-flipper bracing. Stable bounded LLM updates across the height variations indicate transfer to unseen geometric conditions. TABLE I: Comparative results across terrains. Terrain Method Stability Swing Shock Single-Block Terrain-adaptive neural MPC (ours) 0.39 0.63 1.40×−1.40×10^-4 Neural MPC (ours) 0.50 0.33 3.04×10−3¯ 3.04×10^-3 PPO 0.50 0.50 1.00 Stair Ascent Terrain-adaptive neural MPC (ours) 0.37 0.08 2.96×10−4¯ 2.96×10^-4 Neural MPC (ours) 0.66 0.00 0.00 PPO 0.50 1.00 1.00 Stair Descent Terrain-adaptive neural MPC (ours) 0.49 0.12 0.00 Neural MPC (ours) 0.56 0.98 0.57 PPO 0.50 0.50 0.76 Note: Each column corresponds to one penalty term defined in (16). bold denotes the best result and underlined denotes the second best. IV-F Real-Robot Validation We validate on the physical ATR over four indoor obstacles (Fig. 8). Both signals come from the Ouster lidar IMU: the upper plots show the raw and 0.5 s-smoothed acceleration-magnitude residual ‖2−1\| a\|_2-1, which indicates vibration and contact impacts, while the lower plots show an attitude-compensated, IMU-integrated three-dimensional speed magnitude. Because the latter can drift, it is used only as a qualitative motion indicator. The trials demonstrate contact-rich traversal across different obstacle geometries. Fig. 8: Real-robot traversal over four indoor obstacles. Red: Ouster-IMU acceleration-magnitude residual; blue: IMU-integrated speed magnitude used as a qualitative motion indicator. V Conclusion We proposed a language-guided terrain-adaptive neural MPC framework that brings the strengths of constrained predictive control to ATR traversal while addressing practical modeling and terrain-dependent tuning challenges. By combining terrain-conditioned neural kinematics with a language-guided LLM regulator, our method reduces the burden of scenario-specific parameter tuning while preserving feasibility and real-time performance. Furthermore, compiling the neural kinematics into the solver accelerates optimization, enabling real-time control on ATRs operating in structured, contact-rich environments. Our experiments indicate that the learned kinematics and composite loss improve multi-step prediction quality, the loss design stabilizes rollouts by curbing overshoot and encouraging smoothness, and language-guided terrain adaptation yields more stable traversal across varied terrains. Real-robot trials on four indoor obstacles further demonstrate contact-rich physical traversal across different obstacle geometries. In future work, we plan to extend the terrain-adaptive neural MPC framework toward handling deformable or variable-geometry objects and incorporating energy-aware objectives to further improve efficiency in long-duration missions. We also aim to validate the framework on additional real-world platforms and expand the range of terrains and contact conditions. References [1] A. Brohan et al., “RT-2: Vision-language-action models transfer web knowledge to robotic control,” arXiv preprint arXiv:2307.15818, 2023. [2] M. J. Kim et al., “OpenVLA: An open-source vision-language-action model,” arXiv preprint arXiv:2406.09246, 2024. [3] T. Klamt, D. Rodriguez, L. Baccelliere, X. Chen, D. Chiaradia, T. Cichon et al., “Flexible disaster response of tomorrow: Final presentation and evaluation of the Centauro system,” IEEE Robotics & Automation Magazine, vol. 26, no. 4, p. 59–72, 2019. [4] N. Michael, S. Shen, K. Mohta, Y. Mulgaonkar, V. Kumar, K. Nagatani et al., “Collaborative mapping of an earthquake-damaged building via ground and aerial robots,” Journal of Field Robotics, vol. 29, no. 5, p. 832–841, 2012. [5] F. Niroui, K. Zhang, Z. Kashino, and G. Nejat, “Deep reinforcement learning robot for search and rescue applications: Exploration in unknown cluttered environments,” IEEE Robotics and Automation Letters, vol. 4, no. 2, p. 610–617, 2019. [6] J. Delmerico, S. Mintchev, A. Giusti, B. Gromov, K. Melo, T. Horvat et al., “The current state and future outlook of rescue robotics,” Journal of Field Robotics, vol. 36, no. 7, p. 1171–1191, 2019. [7] A. Ugenti, R. Galati, G. Mantriota, and G. Reina, “Analysis of an all-terrain tracked robot with innovative suspension system,” Mechanism and Machine Theory, vol. 182, p. 105237, 2023. [8] Y. Okada, K. Nagatani, and K. Yoshida, “Semi-autonomous operation of tracked vehicles on rough terrain using autonomous control of active flippers,” in 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, 2009, p. 2815–2820. [9] Y. Okada, K. Nagatani, K. Yoshida, T. Yoshida, and E. Koyanagi, “Shared autonomy system for tracked vehicles to traverse rough terrain based on continuous three-dimensional terrain scanning,” p. 357–362, 2010. [10] M. Gianni, F. Ferri, M. Menna, and F. Pirri, “Adaptive robust three-dimensional trajectory tracking for actively articulated tracked vehicles,” Journal of Field Robotics, vol. 33, no. 7, p. 901–930, 2016. [11] Z. Xu, Y. Chen, Z. Jian, J. Tan, X. Wang, and B. L. Liang, “Hybrid trajectory optimization for autonomous terrain traversal of articulated tracked robots,” IEEE Robotics and Automation Letters, vol. 9, no. 1, p. 755–762, 2024. [12] A. Mitriakov, P. Papadakis, J. Kerdreux, and S. Garlatti, “Reinforcement learning based, staircase negotiation learning: Simulation and transfer to reality for articulated tracked robots,” IEEE Robotics & Automation Magazine, vol. 28, no. 4, p. 10–20, 2021. [13] H. Zhang, J. Ren, J. Xiao, H. Pan, H. Lu, and X. Xu, “Ftr‐bench: Benchmarking deep reinforcement learning for flipper‐track robot control,” Journal of Field Robotics, 02 2025. [14] M. Pecka, V. Šalanský, K. Zimmermann, and T. Svoboda, “Autonomous flipper control with safety constraints,” in 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), 2016, p. 2889–2894. [15] M. Pecka, K. Zimmermann, M. Petrlík, and T. Svoboda, “Data-driven policy transfer with imprecise perception simulation,” IEEE Robotics and Automation Letters, vol. 3, no. 4, p. 3916–3921, 2018. [16] H. Nguyen and H. La, “Review of deep reinforcement learning for robot manipulation,” in 2019 Third IEEE International Conference on Robotic Computing (IRC), 2019, p. 590–595. [17] D. Kalashnikov, A. Irpan, P. Pastor, J. Ibarz, A. Herzog, E. Jang, D. Quillen, E. Holly, M. Kalakrishnan, V. Vanhoucke, and S. Levine, “Scalable deep reinforcement learning for vision-based robotic manipulation,” in Proceedings of The 2nd Conference on Robot Learning, ser. Proceedings of Machine Learning Research, A. Billard, A. Dragan, J. Peters, and J. Morimoto, Eds., vol. 87. PMLR, 2018, p. 651–673. [18] N. A. Spielberg, M. Brown, and J. C. Gerdes, “Neural network model predictive motion control applied to automated driving with unknown friction,” IEEE Transactions on Control Systems Technology, vol. 30, no. 5, p. 1934–1945, 2022. [19] C. Yang et al., “Large language models as optimizers,” arXiv preprint arXiv:2309.03409, 2023. [20] A. AhmadiTeshnizi et al., “OptiMUS: Optimization modeling using MIP solvers and large language models,” arXiv preprint arXiv:2310.06116, 2023. [21] N. Baumann, C. Hu, P. Sivasothilingam, H. Qin, L. Xie, M. Magno, and L. Benini, “Enhancing autonomous driving systems with on-board deployed large language models,” arXiv preprint arXiv:2504.11514, 2025. [22] B. Chen, K. Huang, H. Pan, H. Ren, X. Chen, J. Xiao, W. Wu, and H. Lu, “Geometry-based flipper motion planning for articulated tracked robots traversing rough terrain in real-time,” Journal of Field Robotics, vol. 40, no. 8, p. 2010–2029, 2023. [23] H. Li and P. M. Wensing, “Hybrid systems differential dynamic programming for whole-body motion planning of legged robots,” IEEE Robotics and Automation Letters, vol. 5, no. 4, p. 5448–5455, 2020. [24] Y. Ding, M. Zhang, C. Li, H.-W. Park, and K. Hauser, “Hybrid sampling/optimization-based planning for agile jumping robots on challenging terrains,” in 2021 IEEE International Conference on Robotics and Automation (ICRA), May 2021, p. 2839–2845. [25] T. Salzmann, E. Kaufmann, J. Arrizabalaga, M. Pavone, D. Scaramuzza, and M. Ryll, “Real-time neural-mpc: Deep learning model predictive control for quadrotors and agile robotic platforms,” IEEE Robotics and Automation Letters, 2023. [26] T. Salzmann, J. Arrizabalaga, J. Andersson, M. Pavone, and M. Ryll, “Learning for casadi: Data-driven models in numerical optimization,” in Learning for Dynamics and Control Conference (L4DC), 2024. [27] J. A. Andersson, J. Gillis, G. Horn, J. B. Rawlings, and M. Diehl, “Casadi: a software framework for nonlinear optimization and optimal control,” Mathematical Programming Computation, vol. 11, p. 1–36, 2019. [28] R. Verschueren, G. Frison, D. Kouzoupis, J. Frey, N. van Duijkeren, A. Zanelli, B. Novoselnik, T. Albin, R. Quirynen, and M. Diehl, “acados – a modular open-source framework for fast embedded optimal control,” Mathematical Programming Computation, 2021. [29] G. Frison and M. Diehl, “Hpipm: a high-performance quadratic programming framework for model predictive control,” IFAC-PapersOnLine, vol. 53, no. 2, p. 6563–6569, 2020. [30] K. Y. Chee, T. Z. Jiahao, and M. A. Hsieh, “Knode-mpc: A knowledge-based data-driven predictive control framework for aerial robots,” IEEE Robotics and Automation Letters, vol. 7, no. 2, p. 2819–2826, 2022. [31] J. Ren, J. Xiang, H. Gao, J. Zhang, Y. Ren, Y. Ma, Y. Wu, R. Yang, and W. Li, “Npc: Neural predictive control for fuel-efficient autonomous trucks,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, p. 14 251–14 257. [32] H. Xue, E. L. Zhu, J. M. Dolan, and F. Borrelli, “Learning model predictive control with error dynamics regression for autonomous racing,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, p. 13 250–13 256. [33] X. Zeng, H. Zhang, L. Yue, Z. Song, L. Zhang, and Y.-H. Liu, “Adaptive model predictive control with data-driven error model for quadrupedal locomotion,” in 2024 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2024, p. 5731–5737. [34] Y. Sun, W. L. Ubellacker, W.-L. Ma, X. Zhang, C. Wang, N. V. Csomay-Shanklin, M. Tomizuka, K. Sreenath, and A. D. Ames, “Online learning of unknown dynamics for model-based controllers in legged locomotion,” IEEE Robotics and Automation Letters, vol. 6, no. 4, p. 8442–8449, 2021. [35] W. Zhao, J. Chen, Z. Meng, D. Mao, R. Song, and W. Zhang, “Vlmpc: Vision-language model predictive control for robotic manipulation,” in Proceedings of Robotics: Science and Systems (RSS), 2024, arXiv:2407.09829. [36] W. Yu, N. Gileadi, C. Fu, S. Kirmani, K.-H. Lee, M. Gonzalez Arenas, H.-T. L. Chiang, T. Erez, L. Hasenclever, J. Humplik et al., “Language to rewards for robotic skill synthesis,” arXiv preprint arXiv:2306.08647, 2023. [37] Y. J. Ma, W. Liang, G. Wang, D.-A. Huang, O. Bastani, D. Jayaraman, Y. Zhu, L. Fan, and A. Anandkumar, “Eureka: Human-level reward design via coding large language models,” arXiv preprint arXiv:2310.12931, 2023. [38] J. Liang, F. Xia, W. Yu, A. Zeng, and et al., “Learning to learn faster from human feedback with language model predictive control,” arXiv preprint arXiv:2402.11450, 2024. [39] NuBot Team, “Nubot_rescue_gazebo: Gazebo simulation environment for rescue robotics,” https://github.com/nubot-nudt/NuBot_Rescue_Gazebo, gitHub repository, accessed 2025-08-24. [40] J. Bai, S. Bai, Y. Chu, and et al., “Qwen technical report,” arXiv preprint arXiv:2309.16609, 2023.