Paper deep dive
SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control
Ruihua Han, Rui Gao, Zhe Liu, Xinyi Wang, Chang Chen, Shuai Wang, Qi Hao, Jia Pan, Hengshuang Zhao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/24/2026, 6:17:43 AM
Summary
The paper introduces SRL-MPC, a shape-aware reinforcement learned model predictive control framework for safe navigation in dense crowds with heterogeneous robot shapes. It formulates high-order control barrier function (HOCBF) constraints using geometric separation features (GSFs) derived from support function transformations. A reinforcement learning policy adapts MPC parameters in real-time based on GSFs, allowing the system to balance safety and efficiency without manual tuning or geometry simplification.
Entities (7)
Relation Signals (6)
SRL-MPC → uses → HOCBF
confidence 95% · formulate high-order control barrier function (HOCBF) constraints from geometric separation features
SRL-MPC → uses → GSF
confidence 95% · formulate high-order control barrier function (HOCBF) constraints from geometric separation features (GSFs)
SRL-MPC → uses → Reinforcement Learning
confidence 95% · A reinforcement learning (RL) framework then learns a neural policy that reads GSFs and outputs real-time MPC parameter updates
SRL-MPC → uses → MPC
confidence 95% · Shape-Aware Reinforcement Learned Model Predictive Control (SRL-MPC)
GSF → derivedfrom → Support Function
confidence 90% · geometric separation features (GSFs) based on support function transformation
GSF → computedby → GEOS
confidence 85% · geometry-based shortest-line computation implemented by the Geometry Engine Open Source (GEOS) library
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Safe and efficient shape-aware navigation in heterogeneous crowds and robot fleets remains challenging. Traditional approaches often assume homogeneous robots, sparse workspaces, simplified geometry, offline computation, or handcrafted parameters to make the problem tractable, which limits their deployment in dense crowd scenarios. Toward this end, we propose Shape-Aware Reinforcement Learned Model Predictive Control (SRL-MPC), a method for safe, efficient, and adaptive navigation in crowds with heterogeneous shapes without geometry simplification. To encode shape-aware safety, we formulate high-order control barrier function (HOCBF) constraints from geometric separation features (GSFs) based on support function transformation. A reinforcement learning (RL) framework then learns a neural policy that reads GSFs and outputs real-time MPC parameter updates, enabling the MPC solver to adapt to neighboring crowd geometries. The key advantage of SRL-MPC is that it preserves the safety structure and generalizability of MPC while integrating the adaptability and intelligence of RL. Experiments in randomized crowd scenarios with arbitrary shaped robot fleets demonstrate the effectiveness, scalability, and robustness of SRL-MPC. The results show that SRL-MPC substantially outperforms representative baselines in safety and adaptability. Project website: this https URL
Tags
Links
- Source: https://arxiv.org/abs/2608.21175v1
- Canonical: https://arxiv.org/abs/2608.21175v1
Trouble viewing inline? Open PDF directly →
Full Text
67,179 characters extracted from source content.
Expand or collapse full text
SRL-MPC: Shape-Aware Reinforcement Learned Model Predictive Control Ruihua Han Affiliation: The University of Hong Kong Rui Gao Affiliation: Southern University of Science and Technology Zhe Liu Affiliation: The University of Hong Kong Xinyi Wang Affiliation: University of Michigan Chang Chen Affiliation: The University of Hong Kong Shuai Wang Affiliation: Shenzhen Institutes of Advanced Technology Qi Hao Affiliation: Southern University of Science and Technology Jia Pan Affiliation: The University of Hong Kong Hengshuang Zhao Affiliation: The University of Hong Kong Abstract Safe and efficient shape-aware navigation in heterogeneous crowds and robot fleets remains challenging. Traditional approaches often assume homogeneous robots, sparse workspaces, simplified geometry, offline computation, or handcrafted parameters to make the problem tractable, which limits their deployment in dense crowd scenarios. Toward this end, we propose Shape-Aware Reinforcement Learned Model Predictive Control (SRL-MPC), a method for safe, efficient, and adaptive navigation in crowds with heterogeneous shapes without geometry simplification. To encode shape-aware safety, we formulate high-order control barrier function (HOCBF) constraints from geometric separation features (GSFs) based on support function transformation. A reinforcement learning (RL) framework then learns a neural policy that reads GSFs and outputs real-time MPC parameter updates, enabling the MPC solver to adapt to neighboring crowd geometries. The key advantage of SRL-MPC is that it preserves the safety structure and generalizability of MPC while integrating the adaptability and intelligence of RL. Experiments in randomized crowd scenarios with arbitrary shaped robot fleets demonstrate the effectiveness, scalability, and robustness of SRL-MPC. The results show that SRL-MPC substantially outperforms representative baselines in safety and adaptability. Project website: https://hanruihua.github.io/srl_mpc_project/ 1 Introduction Safe navigation in crowds with heterogeneous shapes is a central capability for robots operating in shared spaces with other robots and humans [9]. The problem becomes difficult when a large number of robots must satisfy safety, efficiency, and kinematic constraints simultaneously. Existing approaches, including velocity obstacle (VO)-based methods [32, 11, 25], optimization-based methods [40, 24, 21, 22], and reinforcement learning (RL)-based methods [4, 13, 33, 37], have shown promising results, but they often rely on assumptions such as homogeneous robots, sparse workspaces, or simplified circular shape representations [32, 4, 14, 31], as well as offline computation or repeatedly tuned parameters for specific scenarios [40, 22, 20]. These assumptions make the problem tractable, but they also limit deployment in dense and dynamic crowds, where robots may get stuck or collide if the geometric information is not considered explicitly. The core difficulty is to avoid collisions with dense, dynamic, and arbitrarily shaped objects in real time. RL-based methods can learn strong neural policies for specific scenarios, but they are prone to overfitting and may not generalize to arbitrary shapes or scale to large crowds [31]. Optimization-based approaches have the ability to handle static shaped obstacles by explicitly formulating constraints by compact sets [40, 17] or building a prior grid map [28]. Recent works highlight the promise of combining learning with optimization to integrate these advantages, such as embedding differentiable model predictive control (MPC) into actor-critic learning [27], and using learned dynamics inside sampling based MPC for agile adaptive control [36]. However, these methods mainly focus on single robot agile control, and their safety properties typically depend on the accuracy of learned costs or dynamics rather than explicit shape-aware safety terms. To this end, this paper proposes SRL-MPC, a distributed shape-aware reinforcement learned Model Predictive Control (MPC) framework for robot navigation in crowds with arbitrary shaped obstacles or other robots. The key idea is to formulate a safe set based on geometric separation features (GSFs) as high-order control barrier function (HOCBF) constraints in the primal optimization problem, while using RL to adjust MPC parameters based on the neighboring agent GSFs. After problem decomposition, the local MPC subproblem handles the HOCBF condition through a soft shape-aware residual penalty. In this way, RL does not replace the model-based planner, instead, it adapts the parameters of an explicit shape-aware HOCBF-MPC problem. This approach has several advantages. First, unlike RL-based approaches that are sensitive to carefully designed reward functions and may suffer from limited generalization [14, 37], the proposed method learns parameter adaptation for an explicit shape-aware HOCBF-MPC problem rather than an unconstrained end-to-end policy. Second, the HOCBF-MPC parameters are adapted from neighboring GSFs, avoiding repeated manual weight tuning for different scenarios as required by many optimization-based approaches [22]. Third, the proposed method directly handles explicit convex geometric shape representations, and nonconvex objects can be represented as unions of convex components. This representation is more accurate and generalizable than approximated circular shape models. Finally, following the problem decomposition technique in [16], the deployed local HOCBF-MPC subproblem is simple enough to solve in real time for each robot, resulting in a practical solution for dense dynamic crowd navigation. To highlight the effectiveness of SRL-MPC, we evaluate it in highly randomized scenarios consisting of multiple differential-driven robots, where the positions, goals, and shapes are all randomly generated. This is quite challenging for existing methods, while results show that SRL-MPC outperforms the baselines in terms of task completion, safety, and robustness, especially as crowd density increases. 2 Related Work Traditional Approaches. Traditional collision avoidance approaches often rely on geometric or optimization-based formulations. Recent VO-based methods such as Adaptive Optimal Collision Avoidance Driven by Opinion (AVOCADO) estimate an agent’s cooperation level online through nonlinear opinion dynamics, improving collision avoidance in mixed crowds without communication [25]. MPC is a popular optimization framework that optimizes controls over a receding horizon with various constraints [39, 6]. To improve shape-aware collision avoidance, prior methods introduce disk primitives [41], polytopic velocity obstacles [19], sequential convex optimization [29], dual optimization-based collision avoidance (OBCA) constraints for convex sets [40], and accelerated optimization by problem decomposition and edge computation [17, 21]. These methods improve geometric fidelity, but their exact constraints can grow with the number of object surfaces. SRL-MPC follows this optimization line by using GSFs derived from support function representation, a compact fixed-dimensional representation to encode convex objects without scaling with the number of surfaces. Reinforcement Learning Approaches. RL approaches learn navigation policies from interaction data and have become an important category in socially aware and crowd-aware robot navigation [23, 8, 31]. Early socially aware deep reinforcement learning (DRL) methods learn collision-avoidance policies from local observations and social rewards [5, 7, 34]. Typical methods such as socially aware reinforcement learning (SARL) use attention mechanisms to encode human-robot interactions in a crowd-level representation [4]. More recent methods use richer sequence, occupancy-map, or transformer-based representations to reason about dynamic environments [33, 37]. These methods are flexible and adaptive in uncertain crowds, but safety and generalization remain difficult under unseen densities, agent behaviors, and body shapes. Safe RL methods add model-based lookahead or shielding to reduce violations [1], but they do not directly provide explicit shape-aware safety terms. Consequently, few RL-based methods explicitly address arbitrary shaped objects. Hybrid Methods. Hybrid methods aim to combine the adaptability of learning techniques with the structure of model-based collision avoidance [26]. For example, reinforcement learning reciprocal velocity obstacle (RL-RVO) uses reciprocal velocity obstacle shaped rewards to guide distributed multi-robot policy learning [14]. Another line couples learned decision making with MPC controller execution in social navigation [3]. Recent Deep Residual MPC (DR-MPC) blends MPC path tracking with model-free DRL for real-world navigation and uses out-of-distribution detection with a heuristic safety check to reduce unsafe exploration [12]. More generally, learning-based MPC can learn dynamics models, costs, constraints, or terminal value approximations, and can also use MPC as a safety layer around RL policies [18, 38, 36, 27]. Unlike these methods, SRL-MPC does not learn the entire navigation policy or use optimization only as a shield for unsafe actions. Instead, RL adapts the HOCBF-MPC parameters from local GSFs, while the executed control is computed by the explicit shape-aware MPC optimization problem. 3 Problem Statement Consider robot i navigating in a local crowd of N objects, including other robots, pedestrians, and static or dynamic obstacles. At each control cycle, robot i plans a state sequence i=i,0,…,i,TS_i=\s_i,0,…,s_i,T\ and a control sequence i=i,0,…,i,T−1U_i=\u_i,0,…,u_i,T-1\ over an MPC horizon T, where k denotes the prediction-step index. The state i,ks_i,k contains the planar position i,k=[xi,k,yi,k]⊤p_i,k=[x_i,k,y_i,k] and heading θi,k _i,k, and i,ku_i,k is the control input. The feasible set ℱiF_i collects the kinematic model, input bounds, input-increment bounds, and initial condition. The world-frame occupied set of robot i at step k is denoted by ℤi(i,k)Z_i(s_i,k), which is obtained from a body-frame convex set ℂiC_i by state transformation. Detailed kinematic and geometric definitions are given in Appendix A.1. Let iN_i be the neighbor set considered by robot i, and let the bar notation denote nominal or predicted neighbor quantities, e.g., ¯j,k s_j,k and ¯j,k p_j,k for neighbor j. The navigation objective is to follow a reference path while maintaining a desired control profile. Let irefS_i^ref and irefU_i^ref denote the reference state and control sequences. Following the path-tracking objective used in [16], the cost function is Ci(i,i) C_i(S_i,U_i) =wp∥(i)−(iref)∥22+wθ∥θ(i)−θ(iref)∥22+wu∥i−iref∥22, =w_p (S_i)-p(S_i^ref) _2^2+w_θ θ(S_i)-θ(S_i^ref) _2^2+w_u _i-U_i^ref _2^2, (1) where (i)p(S_i) and θ(i)θ(S_i) denote the stacked positions and headings extracted from iS_i, respectively, and wpw_p, wθw_θ, and wuw_u balance position tracking, heading tracking, and control effort. This cost encourages the robot to progress toward the goal while maintaining the desired control profile. Collision avoidance is imposed through the minimum distance between occupied sets. For each neighbor j∈ij _i and step k, define Dij,k=mini∈ℤi(i,k),j∈ℤj(¯j,k)∥i−j∥2,D_ij,k= _x_i _i(s_i,k),\,x_j _j( s_j,k) _i-x_j _2, (2) where ix_i and jx_j are world-frame points on the two occupied sets, and Dij,kD_ij,k is the minimum Euclidean distance between robot i and neighbor j. The required safety margin is denoted by dsafed_safe. The resulting local planning problem is mini,iCi(i,i)s.t.(i,i)∈ℱi,Dij,k≥dsafe,j∈i,k=1,…,T. _S_i,U_iC_i(S_i,U_i) .t. (S_i,U_i) _i,\;D_ij,k≥ d_safe,\;j _i,\;k=1,…,T. (3) where (i,i)(S_i,U_i) are the decision variables, (i,i)∈ℱi(S_i,U_i) _i enforces kinematic feasibility, and Dij,k≥dsafeD_ij,k≥ d_safe enforces pairwise shape-aware separation along the horizon. This coupled problem is nonconvex and hard to solve directly. The next section introduces SRL-MPC, which separates geometry updates from local motion optimization and adapts the key MPC parameters online. Figure 1: Overview of SRL-MPC. Key component: 1. Shape aware HOCBF condition is formulated based on GSFs and work as residual penalty in the optimization to improve safety; 2. the reinforcement learned policy adapts MPC parameters based on neighboring GSFs to handle dense scenarios. 4 Method The framework of the proposed method is illustrated in Figure 1. The method has two key components. First, it starts from the support function form of the pairwise minimum-distance problem and expresses the shape-aware safe set as degree-2 HOCBF constraints in the coupled primal problem. After problem decomposition, the GSFs are updated by a geometric solver, and the HOCBF condition is represented as a residual in the local MPC problem. Second, the reinforcement learning framework maps GSFs to MPC parameter updates, adapting tracking weights, desired speed weights, and the HOCBF safety distance to autonomously balance task completion efficiency and safety. 4.1 Shape-Aware HOCBF Constraint Few methods consider shape-aware HOCBF constraints because the exact conic constraints grow with the number of shape edges. Here we introduce the support function transformation [2] to formulate shape-aware HOCBF constraints based on fixed-dimensional GSFs. For a pair (i,j)(i,j) and prediction step k, let ¯j,k s_j,k be the nominal state of neighbor j. Under the strong-duality conditions used in [40, 17], the minimum-distance computation in (2), equivalently the body-frame program in (18), can be rewritten in dual form. To avoid the conic constraints, we further transform it into a support function form by: Dij,k=max∥μ∥∗≤1Φij,k(μ),Φij,k(μ)=μ⊤(i,k−¯j,k)−σℂi(−i,k⊤μ)−σℂj(¯j,k⊤μ), D_ij,k= _ μ _*≤ 1 _ij,k(μ), _ij,k(μ)=μ (p_i,k- p_j,k)- _C_i(-R_i,k μ)- _C_j( R_j,k μ), (4) where μ⊤μ represents the separating hyperplane between two convex occupied sets, μ is the minimum-distance direction, and ∥⋅∥∗ · _* denotes the dual norm. We define the geometric separation feature as GSFij,k≜(μij,k,Dij,k)GSF_ij,k ( _ij,k,D_ij,k), where μij,k∈argmax∥μ∥∗≤1Φij,k(μ) _ij,k∈ _ μ _*≤ 1 _ij,k(μ) and Dij,kD_ij,k is the corresponding minimum distance. For a generic body-frame occupied set ℂ⊂ℝ2C ^2 and a query direction ∈ℝ2 ξ ^2, the support function σℂ()=sup⊤∣∈ℂ _C( ξ)= \\, ξ z \,\ gives the maximum projection of ℂC along ξ. Equation (4) therefore expresses the shape-aware minimum distance through the maximum projection induced by the separating hyperplane μ⊤μ . Subtracting the required safety margin dsafed_safe gives: hij,k(μ)=Φij,k(μ)−dsafe.h_ij,k(μ)= _ij,k(μ)-d_safe. For a selected μij,k _ij,k, the collision avoidance condition at step k is written as hij,k(μij,k)≥0h_ij,k( _ij,k)≥ 0. Thus, the shape-aware safe set at prediction step k is ij,k _ij,k =(i,k,¯j,k)∣Hij,k≥0,Hij,k≜hij,k(μij,k). =\(s_i,k, s_j,k) H_ij,k≥ 0\, H_ij,k h_ij,k( _ij,k). (5) A first-order discrete control barrier function (CBF) only imposes a one-step condition on the barrier value, which is weak for dense dynamic interactions with high speed profile. We therefore use a degree-2 discrete HOCBF [35]. For α1,α2∈(0,1] _1, _2∈(0,1], the degree-2 HOCBF left-hand side is denoted as ηij,k _ij,k =Hij,k+2−(2−α1−α2)Hij,k+1+(1−α1)(1−α2)Hij,k. =H_ij,k+2-(2- _1- _2)H_ij,k+1+(1- _1)(1- _2)H_ij,k. (6) In the primal problem, the HOCBF condition is imposed as the hard safety constraint ηij,k≥0 _ij,k≥ 0. With α1=α2=γ _1= _2=γ and γ∈(0,1]γ∈(0,1], this constraint becomes Hij,k+2−2(1−γ)Hij,k+1+(1−γ)2Hij,k≥0,j∈icbf,k=1,…,Hc.H_ij,k+2-2(1-γ)H_ij,k+1+(1-γ)^2H_ij,k≥ 0, j _i^cbf,\;k=1,…,H_c. (7) where icbf⊆iN_i^cbf _i is the selected nearest-neighbor subset used for HOCBF terms, and Hc≤T−2H_c≤ T-2 is the HOCBF horizon. Problem Decomposition: The minimum-distance line direction μ and the MPC trajectory p are coupled in the HOCBF constraint, leading to a bi-convex optimization problem. Following the block decomposition idea used in [17, 16], SRL-MPC separates the coupled problem into a GSF update subproblem Q1Q_1 and a local HOCBF-MPC subproblem Q2Q_2. The first block fixes nominal trajectories and computes nominal GSFs, while the second block fixes μ and computes the action sequence. After this decomposition, the HOCBF condition is handled through a soft residual penalty in Q2Q_2. By iteratively solving Q1Q_1 and Q2Q_2, the geometric features and local trajectory are updated consistently, providing stronger optimization guidance than a static distance-margin penalty. Q1Q_1: GSF computation subproblem. Given nominal trajectories ¯i S_i and ¯j S_j, Q1Q_1 computes the nominal GSFs between two transformed convex occupied sets. Instead of solving the optimization program, which is computationally expensive, we use a geometry-based shortest-line computation implemented by the Geometry Engine Open Source (GEOS) library [10], which is efficient for geometries: GSF¯ij,k≜(μ¯ij,k,d¯ij,k)=ShortestLine(ℤi(¯i,k),ℤj(¯j,k)), GSF_ij,k ( μ_ij,k, d_ij,k)=ShortestLine (Z_i( s_i,k),Z_j( s_j,k) ), (8) where ShortestLine(⋅)ShortestLine(·) denotes the geometric shortest-line algorithm that returns the unit direction from the neighbor set to the ego set and the corresponding minimum distance. Specifically, for a circular set, μ¯ij,k μ_ij,k can be obtained in closed form from the normalized center difference and the radius. Neighbors considered in the local MPC problem are ranked by the minimum distance over a short distance horizon, and only the closest K neighbors are retained in iN_i, where K is a user-specified neighbor budget. Q2Q_2: local HOCBF-MPC subproblem. Given the GSFs produced by Q1Q_1, the local HOCBF-MPC subproblem Q2Q_2 is formulated as Q2:mini,i Q_2: _S_i,U_i Ci(i,i)+ρobs2∑j∈icbf∑k=1Hc[neg(η¯ij,k)]2, C_i(S_i,U_i)+ _obs2 _j _i^cbf _k=1^H_c[neg( η_ij,k)]^2, (9) s.t. .t. (i,i)∈ℱi. (S_i,U_i) _i. where ρobs>0 _obs>0 is the penalty weight for HOCBF violation and is selected as a large value (e.g., 100). neg(x)=max(0,−x)neg(x)= (0,-x) is the negative-part operator, and η¯ij,k η_ij,k is the fixed-geometry counterpart of (6) after substituting the GSFs produced by Q1Q_1. The penalty is zero when the decomposed high-order condition is satisfied and positive only when the predicted trajectory violates it. The soft penalty form avoids infeasibility that can occur when high-order conditions are imposed as hard local constraints in dense crowds, while keeping the local feasible set ℱiF_i unchanged. With the geometric terms fixed by Q1Q_1 and the kinematics represented by affine linearization, Q2Q_2 is a convex local HOCBF-MPC subproblem. At each control cycle, the solver alternates these two subproblems for a small number of iterations. Nominal trajectories are first propagated from the current states. Then, for each robot, Q1Q_1 updates the GSFs for the selected neighbors, Q2Q_2 solves the local HOCBF-MPC problem, and the resulting trajectory becomes the nominal trajectory for the next iteration. The first control in the optimized sequence is applied to the controlled object. 4.2 Reinforcement Learned MPC Figure 2: Neural network architecture. The GSFs are encoded by CNNs and then concatenated with the self-observation to form a fused feature, which is shared by the actor and critic. The solution of Q2Q_2 depends strongly on the parameters in the MPC cost and safety terms, including wpw_p, wθw_θ, wuw_u, and dsafed_safe. These weights and safety parameters determine how aggressively or conservatively a robot tracks the reference path (i.e., wpw_p), how strongly it penalizes control effort (i.e., wuw_u), and how much safety margin it requests from nearby agents (i.e., dsafed_safe). These parameters usually require repeated manual tuning for each scenario. For example, in open space, larger wuw_u and wpw_p can guide the robot to track the reference path toward the goal quickly, while in dense scenarios, smaller wpw_p or dsafed_safe may be needed to avoid collisions or getting stuck. This work proposes an RL-based parameter adaptation method to automatically tune these parameters, as illustrated in Figure 2, while preserving the safety and generalizability of the model-based optimization structure. The learned policy observes local GSFs and current parameter values, outputs bounded parameter increments, and leaves control synthesis to MPC. Because the GSF tensor stacks the closest neighbors over the horizon, the policy can adapt the MPC behavior to local crowd density and shape geometry. Additionally, the adaptability provided by RL allows the two decomposed subproblems to be solved with a single iteration, dramatically reducing computational cost. Action space: The adaptive parameters are wpw_p, wuw_u, and dsafed_safe, while wθw_θ is fixed to a small value, e.g., 0.010.01, to encourage robots to turn during collision avoidance. To make the parameters change smoothly over time, the policy outputs parameter increments rather than absolute parameter values. Thus, for robot i at time step t, the policy output is Δλit=[Δwp,itΔwu,itΔdsafe,it]⊤. _i^t= bmatrix w_p,i^t& w_u,i^t& d_safe,i^t bmatrix . The parameter vector is updated as λit+1=λit+Δλit∈Λ,Λ=[0.01,1.0]×[0.1,10.0]×[0.2,1.0]. _i^t+1= _i^t+ _i^t∈ , =[0.01,1.0]×[0.1,10.0]×[0.2,1.0]. (10) The increment is bounded by Δλit∈[−0.5,0.5]×[−0.5,0.5]×[−0.1,0.1] _i^t∈[-0.5,0.5]×[-0.5,0.5]×[-0.1,0.1], where the third component is measured in meters. The absolute parameter values are clipped to the admissible set Λ , where wp∈[0.01,1.0]w_p∈[0.01,1.0], wu∈[0.1,10.0]w_u∈[0.1,10.0], and dsafe∈[0.2,1.0]d_safe∈[0.2,1.0]. A larger range is assigned to wuw_u to allow the policy to adjust the control effort strongly in dense scenarios. Observation space: The robot observation ito_i^t has two parts: the self-observation i,selfto_i,self^t and the neighbor observation i,neighborto_i,neighbor^t: i,selft=[vitwp,itwu,itdsafe,it]⊤,ij,k,neighbort=[μ¯ij,k,xtμ¯ij,k,ytd¯ij,kt]⊤. gatheredo_i,self^t= bmatrixv_i^t&w_p,i^t&w_u,i^t&d_safe,i^t bmatrix , _ij,k,neighbor^t= bmatrix μ_ij,k,x^t& μ_ij,k,y^t& d_ij,k^t bmatrix . gathered (11) Here ij,k,neighborto_ij,k,neighbor^t stores GSFs, i.e., (μ¯ij,kt,d¯ij,kt)( μ_ij,k^t, d_ij,k^t) for neighbor j at horizon step k. Thus, i,neighbort∈ℝ3×K×To_i,neighbor^t ^3× K× T stacks these geometric features for the K closest neighbors over the MPC horizon. The neighbors are selected by the minimum predicted distance over a short distance horizon, and the tensor is zero-padded when fewer than K neighbors are active. The self-observation includes the current MPC parameters wp,itw_p,i^t, wu,itw_u,i^t, dsafe,itd_safe,i^t, and the forward speed vitv_i^t. Compared with general RL-based approaches [4, 14, 37], this observation space is compact and does not require explicit pose or goal-direction features. This compactness is possible because goal tracking and collision avoidance are handled by HOCBF-MPC; RL only learns how to adapt the MPC parameters to changing obstacle distributions, avoiding the generalizability and scalability issues that end-to-end policies often face. Neural network design. The neural network architecture is shown in Figure 2. The neighbor observation i,neighborto_i,neighbor^t is treated as a three-channel image over the neighbor-horizon grid. The encoder applies two padded 3×33× 3 convolutional layers, using a rectified linear unit (ReLU) after the first convolution and an exponential linear unit (ELU) after the second. This gives each output cell access to the full neighbor-horizon grid before pooling. An adaptive max-pooling layer keeps the strongest local interaction response, and a linear projection followed by layer normalization produces a fixed-size geometric feature. The encoded geometric feature is concatenated with the 4-dimensional self-observation to form a shared feature. This shared encoder is used by both the actor and the critic. The actor outputs a 6-dimensional vector that is split into a 3-dimensional location vector and a 3-dimensional log-scale vector for the parameter increment Δλit _i^t. The resulting squashed Gaussian distribution samples bounded parameter increments. The critic uses the same shared encoder and a parallel linear head with LeakyReLU and layer normalization to output the scalar value estimate for proximal policy optimization (PPO) advantage computation. Layer normalization is used because the observation contains raw physical units, including velocity, MPC weights, and safety distance. Reward design: The reward function is designed to align policy learning with task completion and safety. It uses a one-shot arrival bonus, a collision penalty, a small per-step time penalty, and a safety log-barrier penalty: rit=rarriarrives−rcolicollides−rsafeℓsafe,it−rstep, r_i^t=r_arr1\i\ arrives\-r_col1\i\ collides\-r_safe _safe,i^t-r_step, (12) ℓsafe,it=di,neart<dsafe,itclip[0,1](log(dsafe,it/max(di,neart,ϵd))log(dsafemax/dsafemin)), _safe,i^t=1\d_i,near^t<d_safe,i^t\clip_[0,1] ( \! (d_safe,i^t/ (d_i,near^t, _d) ) \! (d_safe /d_safe ) ), Here ⋅1\·\ is a binary indicator, di,neartd_i,near^t is the current minimum distance from robot i to its nearest neighbor, dsafe,itd_safe,i^t is the policy-adapted safety distance, and ϵd _d is a small numerical floor. The safety term is active only when the current minimum distance is smaller than the selected safety distance. The arrival bonus is assigned once when the robot reaches its goal and then parks at the goal, while the collision penalty is assigned when a collision is detected. The collision penalty is larger than the arrival bonus, encouraging the policy to choose MPC parameters that reach the goal without relying on unsafe behaviors. The numerical reward constants are listed in Table 5 of Appendix A.5. Training: The policy is trained with PPO [30] using decentralized execution and shared parameters. All robots share the same network parameters, and each robot reads the observation in (11) to generate its own MPC parameter increment. The training scenario is generated by IR-SIM [15] and shown in Figure 3: each episode samples 1515 robots in a 10m×10m10\,m× 10\,m workspace with random start-goal pairs and convex polygon robot footprints. Each robot stops at its goal after arrival and then acts as a static obstacle for the other robots. A robot is reset when it collides or times out. Compared with other RL-based approaches trained with circular robots, sparse dynamic obstacles, or a fixed workspace, the training scenario is challenging because it combines randomized geometry, dense crowds, and both static and dynamic obstacles. The training process takes about 44 hours to achieve stable performance. Figure 3: Training scenario examples. Each episode randomly samples start positions, goal positions, and convex polygon footprints; arrived robots park as static obstacles. 5 Experiments Evaluation: We evaluate SRL-MPC in the same IR-SIM randomized scenario family used for training, but with held-out random seeds, so no evaluation episode is reused during policy training. To test scalability and density robustness within this randomized shape-aware setting, the robot count is swept over N∈10,15,20,25N∈\10,15,20,25\ to represent different crowd densities, while using the same model trained only in 15-robot scenarios. Figure 4 shows a rollout with 2525 randomly generated polygonal robots and qualitative held-out challenging scenarios. The qualitative scenarios include cross geometry, nonconvex-union obstacles represented by convex components, circular polygon layouts, and through-traffic interactions, covering different geometric and crowd-flow patterns beyond the main random-polygon density sweep. Guided by SRL-MPC, each robot adapts its parameters to different neighbor geometries and reaches its goal successfully. Even in congested situations caused by parked robots, the ego robot adjusts its parameters to detour around the obstacles. All quantitative results use 100100 episodes per robot count, a maximum episode length of 500500 control steps, and the same random seeds across compared methods. An episode is counted as successful only when all robots reach their goals without collision or timeout. Navigation time and path length are averaged over arrived robots, while speed is averaged over active robot frames. All ± values denote standard deviations over the corresponding evaluated samples. Experiments are conducted on a MacBook Pro with an Apple M4 Pro CPU. The detailed controller, training, and evaluation parameters are summarized in Tables 4–6 of Appendix A.5. Step 01 Step 35 Step 70 Step 109 Step 188 Random Cross Nonconvex Circular Traffic Figure 4: Evaluation rollout with 2525 randomly generated polygonal robots and qualitative held-out challenging scenarios, including cross geometry, nonconvex union, circular polygon, and through traffic. Full rollouts are shown in Appendix A.4. Figure 5: Episode outcomes for baseline methods in the random convex polygon scenario. Comparison with Baselines: We compare SRL-MPC with five representative baselines: ORCA, a classic reciprocal velocity obstacle method [32]; AVOCADO, an adaptive optimal collision avoidance method with opinion dynamics [25]; VO-polytope, a polygon-precise velocity obstacle method [19]; SARL, a socially aware reinforcement learning policy that uses attention to encode crowd interactions [4]; and RL-RVO, a pretrained reinforcement learning policy based on reciprocal velocity obstacle features [14]. These baselines cover reactive, geometry-aware, learning-based, and adaptive collision avoidance methods commonly used in crowd navigation. Figure 6 shows the real-world experiment setup. Table 1: Baseline comparison in the random convex polygon scenario. Robots Method Success↑ Collision↓ Timeout↓ Time (s)↓ Speed (m/s)↑ Path (m)↓ 10 ORCA [32] 86.0 8.0 6.0 5.09±2.585.09± 2.58 0.878±0.1370.878± 0.137 4.14±2.254.14± 2.25 AVOCADO [25] 95.0 0.0 5.0 5.61±2.635.61± 2.63 0.785±0.1370.785± 0.137 4.58±2.284.58± 2.28 VO-polytope [19] 65.0 35.0 0.0 7.46±3.917.46± 3.91 0.536±0.0670.536± 0.067 4.14±2.254.14± 2.25 SARL [4] 89.0 8.0 3.0 8.90±5.248.90± 5.24 0.664±0.1640.664± 0.164 5.78±3.115.78± 3.11 RL-RVO [14] 79.0 14.0 7.0 6.99±4.366.99± 4.36 0.780±0.1360.780± 0.136 5.42±2.965.42± 2.96 SRL-MPC (ours) 100.0 0.0 0.0 6.11±3.656.11± 3.65 0.981±0.0910.981± 0.091 6.02±3.556.02± 3.55 15 ORCA 64.0 20.0 16.0 5.22±2.815.22± 2.81 0.850±0.1510.850± 0.151 3.94±2.243.94± 2.24 AVOCADO 72.0 0.0 28.0 6.29±4.416.29± 4.41 0.727±0.1910.727± 0.191 4.64±2.424.64± 2.42 VO-polytope 52.0 47.0 1.0 7.16±3.907.16± 3.90 0.531±0.0650.531± 0.065 3.94±2.243.94± 2.24 SARL 65.0 32.0 3.0 8.38±5.648.38± 5.64 0.663±0.1660.663± 0.166 5.44±3.155.44± 3.15 RL-RVO 43.0 41.0 16.0 7.34±4.867.34± 4.86 0.734±0.1440.734± 0.144 5.43±3.315.43± 3.31 SRL-MPC (ours) 98.0 1.0 1.0 6.58±4.006.58± 4.00 0.970±0.0930.970± 0.093 6.39±3.806.39± 3.80 20 ORCA 29.0 41.0 30.0 5.35±3.355.35± 3.35 0.816±0.1730.816± 0.173 3.51±2.203.51± 2.20 AVOCADO 37.0 0.0 63.0 6.69±3.986.69± 3.98 0.684±0.2090.684± 0.209 4.76±2.414.76± 2.41 VO-polytope 21.0 78.0 1.0 6.52±3.996.52± 3.99 0.511±0.0770.511± 0.077 3.51±2.203.51± 2.20 SARL 38.0 62.0 0.0 8.01±5.058.01± 5.05 0.653±0.1620.653± 0.162 5.11±2.885.11± 2.88 RL-RVO 12.0 71.0 17.0 7.53±5.797.53± 5.79 0.685±0.1550.685± 0.155 5.32±3.595.32± 3.59 SRL-MPC (ours) 93.0 6.0 1.0 7.26±4.447.26± 4.44 0.960±0.0800.960± 0.080 6.93±4.126.93± 4.12 25 ORCA 7.0 58.0 35.0 5.38±3.495.38± 3.49 0.780±0.1910.780± 0.191 2.84±2.002.84± 2.00 AVOCADO 11.0 0.0 89.0 7.64±5.517.64± 5.51 0.638±0.2280.638± 0.228 4.91±2.524.91± 2.52 VO-polytope 8.0 91.0 1.0 5.38±3.685.38± 3.68 0.496±0.0860.496± 0.086 2.84±2.002.84± 2.00 SARL 21.0 79.0 0.0 7.38±4.707.38± 4.70 0.639±0.1680.639± 0.168 4.75±2.814.75± 2.81 RL-RVO 1.0 90.0 9.0 6.71±5.956.71± 5.95 0.657±0.1620.657± 0.162 4.74±4.034.74± 4.03 SRL-MPC (ours) 92.0 6.0 2.0 7.94±5.007.94± 5.00 0.943±0.0930.943± 0.093 7.41±4.417.41± 4.41 Table 1 shows that SRL-MPC achieves the highest success rate across all robot counts. The gap becomes larger in dense scenes: with 2525 robots, SRL-MPC reaches 92.0%92.0\% success, while ORCA, AVOCADO, VO-polytope, SARL, and RL-RVO achieve 7.0%7.0\%, 11.0%11.0\%, 8.0%8.0\%, 21.0%21.0\%, and 1.0%1.0\%, respectively. The improvement over the strongest external baseline is 55.055.0 percentage points at 2020 robots and 71.071.0 percentage points at 2525 robots. ORCA often has the shortest navigation time and path length among arrived robots, but these averages exclude many failed robots in dense scenes. AVOCADO avoids collisions but produces many timeout episodes in dense settings. Although VO-polytope models polygonal geometry more explicitly than circular VO methods, it remains a reactive pairwise velocity-obstacle method. In dense multi-robot scenes, the polygonal velocity cones become tight and overlapping, and the method lacks receding-horizon optimization or adaptive parameter tuning to resolve multi-way conflicts; therefore, its failure mode becomes collision-heavy as density increases. Together, Table 1 and Figure 5 show that SRL-MPC improves the task-completion and safety tradeoff, rather than only changing the speed profile of successful trajectories. Ablation Study: To validate the functionality of the proposed components, we also run an ablation study with dist_mpc, manual_mpc, rule_mpc, and SRL-MPC. dist_mpc uses neither RL nor HOCBF; it replaces the HOCBF term with a static distance-margin penalty over the horizon, Jdist=(ρobs/2)∑j∈i∑k=1T[neg(H¯ij,k|k)]2J_dist=( _obs/2) _j _i _k=1^T[neg( H_ij,k k)]^2, where H¯ij,k|k H_ij,k k is the fixed-geometry distance barrier obtained from the GSFs at step k. manual_mpc uses HOCBF without RL by solving the same HOCBF-MPC problem with fixed handcrafted MPC parameters. rule_mpc also uses HOCBF without RL, but replaces learned adaptation with a hand coded stuck rule. It uses the default parameters (wp,wu,dsafe)=(0.01,10.0,0.30)(w_p,w_u,d_safe)=(0.01,10.0,0.30) and switches to (1.0,0.1,0.20)(1.0,0.1,0.20) when the robot satisfies |vi|<0.10m/s|v_i|<0.10\,m/s for 1010 consecutive control steps. SRL-MPC uses both HOCBF and RL based continuous parameter adaptation. Table 2: Ablation study in the random convex polygon scenario. Robots Method Success↑ Collision↓ Timeout↓ Time (s)↓ Speed (m/s)↑ Path (m)↓ 10 dist_mpc (w/o HOCBF+RL) 77.0 21.0 2.0 4.99±2.494.99± 2.49 0.997±0.1070.997± 0.107 5.00±2.515.00± 2.51 10 manual_mpc (w/o RL) 96.0 1.0 3.0 6.69±5.326.69± 5.32 0.983±0.0910.983± 0.091 6.60±5.246.60± 5.24 10 rule_mpc (rule, w/o RL) 94.0 0.0 6.0 6.87±5.556.87± 5.55 0.980±0.0940.980± 0.094 6.76±5.436.76± 5.43 10 SRL-MPC (ours, with HOCBF+RL) 100.0 0.0 0.0 6.11±3.656.11± 3.65 0.981±0.0910.981± 0.091 6.02±3.556.02± 3.55 15 dist_mpc (w/o HOCBF+RL) 51.0 46.0 3.0 4.81±2.734.81± 2.73 0.997±0.1060.997± 0.106 4.81±2.744.81± 2.74 15 manual_mpc (w/o RL) 93.0 2.0 5.0 6.85±4.746.85± 4.74 0.969±0.1010.969± 0.101 6.64±4.536.64± 4.53 15 rule_mpc (rule, w/o RL) 92.0 1.0 7.0 7.21±5.277.21± 5.27 0.965±0.0980.965± 0.098 6.92±4.906.92± 4.90 15 SRL-MPC (ours, with HOCBF+RL) 98.0 1.0 1.0 6.58±4.006.58± 4.00 0.970±0.0930.970± 0.093 6.39±3.806.39± 3.80 20 dist_mpc (w/o HOCBF+RL) 15.0 84.0 1.0 3.91±2.673.91± 2.67 1.003±0.0971.003± 0.097 3.90±2.603.90± 2.60 20 manual_mpc (w/o RL) 75.0 5.0 20.0 7.43±5.127.43± 5.12 0.959±0.1020.959± 0.102 7.13±4.857.13± 4.85 20 rule_mpc (rule, w/o RL) 85.0 1.0 14.0 7.93±5.307.93± 5.30 0.950±0.0900.950± 0.090 7.43±4.777.43± 4.77 20 SRL-MPC (ours, with HOCBF+RL) 93.0 6.0 1.0 7.26±4.447.26± 4.44 0.960±0.0800.960± 0.080 6.93±4.126.93± 4.12 25 dist_mpc (w/o HOCBF+RL) 7.0 92.0 1.0 3.57±2.703.57± 2.70 1.002±0.1061.002± 0.106 3.55±2.683.55± 2.68 25 manual_mpc (w/o RL) 68.0 13.0 19.0 7.72±5.057.72± 5.05 0.945±0.1100.945± 0.110 7.27±4.627.27± 4.62 25 rule_mpc (rule, w/o RL) 75.0 2.0 23.0 8.67±5.978.67± 5.97 0.931±0.1030.931± 0.103 7.94±5.227.94± 5.22 25 SRL-MPC (ours, with HOCBF+RL) 92.0 6.0 2.0 7.94±5.007.94± 5.00 0.943±0.0930.943± 0.093 7.41±4.417.41± 4.41 Table 2 shows the superior performance of SRL-MPC over the ablation baselines. manual_mpc is competitive at low and moderate densities, but drops to 75.0%75.0\% and 68.0%68.0\% success at 2020 and 2525 robots, whereas SRL-MPC reaches 93.0%93.0\% and 92.0%92.0\%. rule_mpc improves over manual_mpc at high density by using a binary stuck rule: at 2020 and 2525 robots, success rates increase from 75.0%75.0\% to 85.0%85.0\% and from 68.0%68.0\% to 75.0%75.0\%, respectively. However, rule_mpc still depends on hand tuned thresholds and only switches between two parameter settings. Its wider default safety distance also makes it timeout leaning, with 23.0%23.0\% timeout at 2525 robots. In contrast, SRL-MPC improves over rule_mpc by 6.06.0, 6.06.0, 8.08.0, and 17.017.0 percentage points for 1010, 1515, 2020, and 2525 robots, respectively, by adapting parameters continuously from local GSFs. At 2525 robots, rule_mpc remains collision conservative but timeout heavy, whereas SRL-MPC reaches 92.0%92.0\% success by reducing timeout to 2.0%2.0\% while keeping collision to 6.0%6.0\%. dist_mpc is fast among the robots that arrive, but its collision rate increases to 84.0%84.0\% at 2020 robots and 92.0%92.0\% at 2525 robots, confirming that the static distance margin penalty is insufficient as the primary safety mechanism in dense polygonal crowds. Unlike the static distance margin penalty, the HOCBF residual couples barrier values across consecutive predicted steps, so it penalizes not only instantaneous distance violations but also unsafe trends along the horizon. This provides more informative optimization guidance in dense interactions. The comparison isolates the benefits of the HOCBF residual, rule based adaptation, and learned continuous parameter adaptation in sequence. Robustness Analysis: We further evaluate SRL-MPC under perception noise, action delay, and parameter delay in the N=15N=15 random polygon setting, using the same scenario, evaluation seed, and max-step setting as the main evaluation. The success rate remains 100.0%100.0\% with Gaussian neighbor-position noise of σ=0.02mσ=0.02\,m and 83.0%83.0\% at σ=0.05mσ=0.05\,m, and reaches 99.0%99.0\% with one-step action delay (0.1s0.1\,s). The method is also insensitive to delayed RL parameter updates: with a five-step parameter delay (0.5s0.5\,s), success remains 100.0%100.0\%. Larger perception noise or action delay causes a clear collision increase, indicating that accurate short-horizon state estimation and low-latency actuation remain important. The detailed sweep is reported in Table 3. We also deploy SRL-MPC on real-world robot platforms to validate practical effectiveness and real-time execution, as shown in Figure 6. Table 3: Robustness analysis under perception noise and delay perturbations for N=15N=15. Perception noise Action delay Parameter delay σ (m) Succ. Coll. T.out K (step) Succ. Coll. T.out K (step) Succ. Coll. T.out 0.00 98.0 1.0 1.0 0 98.0 1.0 1.0 0 98.0 1.0 1.0 0.02 100.0 0.0 0.0 1 99.0 1.0 0.0 1 98.0 2.0 0.0 0.05 83.0 17.0 0.0 2 82.0 17.0 1.0 2 100.0 0.0 0.0 0.10 40.0 60.0 0.0 3 2.0 98.0 0.0 3 99.0 1.0 0.0 0.20 1.0 99.0 0.0 5 0.0 100.0 0.0 5 100.0 0.0 0.0 6 Conclusion Figure 6: Real-world platform deployment. This paper presents SRL-MPC, a shape-aware reinforcement learned MPC framework for collision avoidance in crowded dynamic environments with multiple robots and obstacles. It represents geometric constraints through GSFs based on support function transformation, formulates degree-2 HOCBF constraints in the coupled primal problem, and decomposes online planning into a GSFs update and a local HOCBF-MPC subproblem where the decomposed HOCBF residual is softly penalized. Reinforcement learning adapts MPC parameters from neighboring shape-aware geometric features, while the executed control remains the solution of an explicit MPC optimization problem. Experiments in random polygon scenarios with density sweeps show that SRL-MPC achieves the highest success rate among representative baselines, especially in dense scenes where the baselines degrade sharply. Ablations show that the decomposed soft HOCBF penalty is more reliable than a static distance-margin penalty and that learned parameter adaptation improves robustness over fixed handcrafted settings. Future work includes uncertainty-aware neighbor prediction, richer nonconvex body decompositions, and real-world deployment with onboard sensing and computation. References [1] M. Alshiekh, R. Bloem, R. Ehlers, B. Könighofer, S. Niekum, and U. Topcu (2018) Safe reinforcement learning via shielding. In Proceedings of the AAAI conference on artificial intelligence, Vol. 32, p. 2669–2678. External Links: Document Cited by: §2. [2] S. P. Boyd and L. Vandenberghe (2004) Convex optimization. Cambridge university press. Cited by: §4.1. [3] B. Brito, M. Everett, J. P. How, and J. Alonso-Mora (2021) Where to go next: learning a subgoal recommendation policy for navigation in dynamic environments. IEEE Robotics and Automation Letters 6 (3), p. 4616–4623. Cited by: §2. [4] C. Chen, Y. Liu, S. Kreiss, and A. Alahi (2019) Crowd-robot interaction: crowd-aware robot navigation with attention-based deep reinforcement learning. In 2019 international conference on robotics and automation (ICRA), p. 6015–6022. Cited by: §1, §2, §4.2, Table 1, §5. [5] Y. F. Chen, M. Everett, M. Liu, and J. P. How (2017) Socially aware motion planning with deep reinforcement learning. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 1343–1350. Cited by: §2. [6] Y. Chen, C. Wang, M. Guo, and Z. Li (2023) Multi-robot trajectory planning with feasibility guarantee and deadlock resolution: an obstacle-dense environment. IEEE Robotics and Automation Letters 8 (4), p. 2197–2204. Cited by: §2. [7] M. Everett, Y. F. Chen, and J. P. How (2021) Collision avoidance in pedestrian-rich environments with deep reinforcement learning. IEEE Access 9, p. 10357–10377. External Links: Document Cited by: §2. [8] T. Fan, P. Long, W. Liu, and J. Pan (2020) Distributed multi-robot collision avoidance via deep reinforcement learning for navigation in complex scenarios. The International Journal of Robotics Research 39 (7), p. 856–892. Cited by: §2. [9] A. Francis, C. Perez-D’Arpino, C. Li, F. Xia, A. Alahi, et al. (2025) Principles and guidelines for evaluating social robot navigation algorithms. ACM Transactions on Human-Robot Interaction 14 (2), p. 1–65. External Links: Document Cited by: §1. [10] GEOS contributors (2025) GEOS computational geometry library. Open Source Geospatial Foundation. External Links: Link, Document Cited by: §4.1. [11] K. Guo, D. Wang, T. Fan, and J. Pan (2021) VR-orca: variable responsibility optimal reciprocal collision avoidance. IEEE Robotics and Automation Letters 6 (3), p. 4520–4527. Cited by: §1. [12] J. R. Han, H. Thomas, J. Zhang, N. Rhinehart, and T. D. Barfoot (2025) DR-MPC: deep residual model predictive control for real-world social navigation. IEEE Robotics and Automation Letters 10 (4), p. 4029–4036. External Links: 2410.10646, Document Cited by: §2. [13] R. Han, S. Chen, and Q. Hao (2020) Cooperative multi-robot navigation in dynamic environment with deep reinforcement learning. In 2020 IEEE International Conference on Robotics and Automation (ICRA), p. 448–454. Cited by: §1. [14] R. Han, S. Chen, S. Wang, Z. Zhang, R. Gao, Q. Hao, and J. Pan (2022) Reinforcement learned distributed multi-robot navigation with reciprocal velocity obstacle shaped rewards. IEEE Robotics and Automation Letters 7 (3), p. 5896–5903. External Links: Document Cited by: §1, §1, §2, §4.2, Table 1, §5. [15] R. Han, S. Wang, C. Li, R. Gao, X. Wang, Z. Liu, G. Li, Y. Lu, Q. Hao, J. Pan, and H. Zhao (2026) IR-SIM: a lightweight skill-native simulator for navigation, learning, and benchmarking. arXiv preprint arXiv:2606.08729. External Links: Document, Link, 2606.08729 Cited by: §4.2. [16] R. Han, S. Wang, S. Wang, Z. Zhang, J. Chen, S. Lin, C. Li, C. Xu, Y. C. Eldar, Q. Hao, et al. (2025) NeuPAN: direct point robot navigation with end-to-end model-based learning. IEEE Transactions on Robotics 41, p. 2804–2824. External Links: Document Cited by: §A.1, §1, §3, §4.1. [17] R. Han, S. Wang, S. Wang, Z. Zhang, Q. Zhang, Y. C. Eldar, Q. Hao, and J. Pan (2023) RDA: an accelerated collision free motion planner for autonomous navigation in cluttered environments. IEEE Robotics and Automation Letters 8 (3), p. 1715–1722. External Links: Document Cited by: §A.1, §1, §2, §4.1, §4.1. [18] L. Hewing, K. P. Wabersich, M. Menner, and M. N. Zeilinger (2020) Learning-based model predictive control: toward safe learning in control. Annual Review of Control, Robotics, and Autonomous Systems 3, p. 269–296. External Links: Document Cited by: §2. [19] J. Huang, J. Zeng, X. Chi, K. Sreenath, Z. Liu, and H. Su (2023) Velocity obstacle for polytopic collision avoidance for distributed multi-robot systems. IEEE Robotics and Automation Letters 8 (6), p. 3502–3509. Cited by: §2, Table 1, §5. [20] T. Kim, R. I. Kee, and D. Panagou (2025) Learning to refine input constrained control barrier functions via uncertainty-aware online parameter adaptation. In 2025 IEEE International Conference on Robotics and Automation (ICRA), Vol. , p. 3868–3875. External Links: Document Cited by: §1. [21] G. Li, R. Han, S. Wang, F. Gao, Y. C. Eldar, and C. Xu (2025) Edge accelerated robot navigation with collaborative motion planning. IEEE/ASME Transactions on Mechatronics 30 (2), p. 1166–1178. External Links: Document Cited by: §1, §2. [22] Y. Li, L. Yin, Y. Cai, J. Liu, F. Zhu, M. Ma, S. Liang, H. Li, and F. Zhang (2025) Efficient swept volume-based trajectory generation for arbitrary-shaped ground robot navigation. In 2025 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 2076–2083. External Links: Document Cited by: §1, §1. [23] P. Long, T. Fan, X. Liao, W. Liu, H. Zhang, and J. Pan (2018) Towards optimally decentralized multi-robot collision avoidance via deep reinforcement learning. In 2018 IEEE international conference on robotics and automation (ICRA), p. 6252–6259. Cited by: §2. [24] W. Luo, W. Sun, and A. Kapoor (2020) Multi-robot collision avoidance under uncertainty with probabilistic safety barrier certificates. In Advances in Neural Information Processing Systems, Vol. 33, p. 372–383. Cited by: §1. [25] D. Martinez-Baselga, E. Sebastián, E. Montijano, L. Riazuelo, C. Sagüés, and L. Montano (2025) AVOCADO: adaptive optimal collision avoidance driven by opinion. IEEE Transactions on Robotics 41, p. 2495–2511. External Links: Document Cited by: §1, §2, Table 1, §5. [26] J. Qin, J. Qin, J. Qiu, Q. Liu, M. Li, and Q. Ma (2024) SRL-ORCA: a socially aware multi-agent mapless navigation algorithm in complex dynamic scenes. IEEE Robotics and Automation Letters 9 (1), p. 143–150. External Links: Document Cited by: §2. [27] A. Romero, E. Aljalbout, Y. Song, and D. Scaramuzza (2026) Actor–critic model predictive control: differentiable optimization meets reinforcement learning for agile flight. IEEE Transactions on Robotics 42, p. 673–692. External Links: Document Cited by: §1, §2. [28] C. Rösmann, F. Hoffmann, and T. Bertram (2017) Kinodynamic trajectory optimization and control for car-like robots. In 2017 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 5681–5686. Cited by: §1. [29] J. Schulman, Y. Duan, J. Ho, A. Lee, I. Awwal, H. Bradlow, J. Pan, S. Patil, K. Goldberg, and P. Abbeel (2014) Motion planning with sequential convex optimization and convex collision checking. The International Journal of Robotics Research 33 (9), p. 1251–1270. Cited by: §2. [30] J. Schulman, F. Wolski, P. Dhariwal, A. Radford, and O. Klimov (2017) Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347. Cited by: §4.2. [31] P. T. Singamaneni, P. Bachiller-Burgos, L. J. Manso, A. Garrell, A. Sanfeliu, A. Spalanzani, and R. Alami (2024) A survey on socially aware robot navigation: taxonomy and future challenges. The International Journal of Robotics Research 43 (10), p. 1533–1572. External Links: Document Cited by: §1, §1, §2. [32] J. Van Den Berg, S. J. Guy, M. Lin, and D. Manocha (2011) Reciprocal n-body collision avoidance. In Robotics Research: The 14th International Symposium ISRR, p. 3–19. Cited by: §1, Table 1, §5. [33] H. Wang, A. H. Tan, and G. Nejat (2024) NavFormer: a transformer architecture for robot target-driven navigation in unknown and dynamic environments. IEEE Robotics and Automation Letters 9 (8), p. 6808–6815. Cited by: §1, §2. [34] S. Wang, R. Gao, R. Han, S. Chen, C. Li, and Q. Hao (2022) Adaptive environment modeling based reinforcement learning for collision avoidance in complex scenes. In 2022 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 9011–9018. Cited by: §2. [35] W. Xiao and C. Belta (2022) High-order control barrier functions. IEEE Transactions on Automatic Control 67 (7), p. 3655–3662. External Links: Document Cited by: §4.1. [36] W. Xiao, H. Xue, T. Tao, D. Kalaria, J. M. Dolan, and G. Shi (2025) AnyCar to anywhere: learning universal dynamics model for agile and adaptive mobility. In 2025 IEEE International Conference on Robotics and Automation (ICRA), p. 8819–8825. External Links: Document Cited by: §1, §2. [37] Z. Xu, X. Han, H. Shen, H. Jin, and K. Shimada (2025) NavRL: learning safe flight in dynamic environments. IEEE Robotics and Automation Letters 10 (4), p. 3668–3675. External Links: Document Cited by: §1, §1, §2, §4.2. [38] M. Zanon and S. Gros (2020) Safe reinforcement learning using robust mpc. IEEE Transactions on Automatic Control 66 (8), p. 3638–3652. Cited by: §2. [39] J. Zeng, B. Zhang, and K. Sreenath (2021) Safety-critical model predictive control with discrete-time control barrier function. In 2021 American Control Conference (ACC), p. 3882–3889. Cited by: §2. [40] X. Zhang, A. Liniger, and F. Borrelli (2020) Optimization-based collision avoidance. IEEE Transactions on Control Systems Technology 29 (3), p. 972–983. Cited by: §A.1, §1, §1, §2, §4.1. [41] J. Ziegler and C. Stiller (2010) Fast collision checking for intelligent vehicle motion planning. In 2010 IEEE intelligent vehicles symposium, p. 518–522. Cited by: §2. Appendix A Technical Appendices A.1 Kinematic and Geometric Model This appendix provides the detailed definitions of the feasible set ℱiF_i and the occupied set ℤi(i,k)Z_i(s_i,k) used in Section 3. The state and control satisfy i,k+1=fi(i,k,i,k),k=0,…,T−1,s_i,k+1=f_i(s_i,k,u_i,k), k=0,…,T-1, (13) where fi:i×i→ℝnf_i:X_i×V_i ^n is the discrete-time robot dynamics, iX_i is the state space, and iV_i is the admissible input set. In planar navigation, n=3n=3 and i,k=[xi,k,yi,k,θi,k]⊤s_i,k=[x_i,k,y_i,k, _i,k] . The implementation uses a differential-drive model with i,k=[vi,k,ωi,k]⊤u_i,k=[v_i,k, _i,k] and sampling time Δt t: xi,k+1 x_i,k+1 =xi,k+Δtvi,kcosθi,k,yi,k+1=yi,k+Δtvi,ksinθi,k,θi,k+1=θi,k+Δtωi,k. =x_i,k+ t\,v_i,k _i,k, y_i,k+1=y_i,k+ t\,v_i,k _i,k, _i,k+1= _i,k+ t\, _i,k. (14) The nonlinear kinematics are linearized around the nominal trajectory as in [17]. The input and input increment satisfy i,min≤i,k≤i,maxu_i, _i,k _i, and Δi,min≤Δi,k≤Δi,max _i, ≤ _i,k≤ _i, for k=0,…,T−1k=0,…,T-1, where Δi,k=i,k−i,k−1 _i,k=u_i,k-u_i,k-1 and i,−1u_i,-1 is the previously applied control. Thus, ℱiF_i is the set of all (i,i)(S_i,U_i) satisfying (13), the input constraints, and the initial condition i,0=icurs_i,0=s_i^cur. Following the geometric convention in [40, 16], the convex body-frame occupied set of object i and its world-frame transformation at prediction step k are ℂi _i =∈ℝ2∣i⪯i,ℤi(i,k)=i,k+i,k∣∈ℂi. =\\,z ^2 _iz _K_ih_i\,\, _i(s_i,k)=\\,R_i,kz+p_i,k _i\,\. (15) Here, z is a body-frame point, iG_i and ih_i define the conic inequalities, iK_i is a proper cone, and ⪯i _K_i denotes the partial order induced by iK_i. The matrix i,k≜(θi,k)∈SO(2)R_i,k ( _i,k)∈ SO(2) is the planar rotation matrix, where (θ)=[cosθ−sinθsinθcosθ].R(θ)= bmatrix θ&- θ\\ θ& θ bmatrix. (16) For a nominal neighbor state ¯j,k s_j,k, the corresponding position and rotation are denoted by ¯j,k p_j,k and ¯j,k R_j,k. The support function used in (4) is defined as σℂ()=sup⊤∣∈ℂ,σℂcir()=r∥2,σℂpoly()=maxa=1,…,nv(a)⊤, _C( ξ)= \\, ξ z \,\, _C_cir( ξ)=r ξ _2, _C_poly( ξ)= _a=1,…,n_v(v^a) ξ, (17) where ℂcirC_cir is a body-centered circle with radius r, and ℂpolyC_poly is a polygon with body-frame vertices aa=1nv\v^a\_a=1^n_v. Substituting (15) into (2) gives the equivalent body-frame distance program Dij,k D_ij,k =mini,j∥i,ki+i,k−(¯j,kj+¯j,k)∥2 = _z_i,z_j _i,kz_i+p_i,k- ( R_j,kz_j+ p_j,k ) _2 (18) s.t. .t. i⪯i,j⪯j. _iz_i _K_ih_i, _jz_j _K_jh_j. A.2 Limitations The main limitation is the computational cost of solving the local HOCBF-MPC subproblem. As shown in Table 7, SRL-MPC is heavier than lightweight reactive baselines because each control step solves a convex MPC problem after the geometric feature update. Nevertheless, the measured mean per-robot controller time remains at the 10ms10\,ms level in the tested dense scenarios: 5.795.79, 6.736.73, 9.379.37, and 10.34ms10.34\,ms for 1010, 1515, 2020, and 2525 robots, respectively, supporting real-time per-robot control in the platform experiment shown in Figure 6. The evaluation focuses on randomized convex polygon scenarios with held-out seeds; broader tests with full perception pipelines, heterogeneous dynamics, nonconvex decomposed objects, and stronger centralized shape-aware baselines such as OBCA remain future work. The method also requires sufficient onboard computation and assumes reliable state estimates, short-horizon nominal neighbor predictions, and convex or convex-decomposed footprints. A.3 Broad Impact This work aims to improve the safety and reliability of autonomous robot navigation in shared spaces, including warehouses, service-robot environments, and heterogeneous robot fleets. By keeping the executed control inside an explicit MPC optimization problem and using RL to adapt interpretable parameters, the framework may reduce the risk of opaque end-to-end policy behavior in dense navigation tasks. Potential negative impacts mainly come from premature deployment: inaccurate state estimation, model mismatch, computation delays, or unmodeled pedestrian behavior could still lead to unsafe motion. A.4 Additional Challenging Scenarios Figure 7 shows additional held-out scenarios, including cross geometry, nonconvex union, circular polygon, and through traffic layouts. The snapshots are sampled from one episode per scenario from the beginning to the final step, illustrating that SRL-MPC can guide polygonal robots through different geometry layouts while adapting the MPC parameters during the rollout. Section A.4.1 further discusses a reactive-pair example. Cross geometry Step 0 Step 22 Step 45 Step 67 Step 90 Nonconvex union Step 0 Step 24 Step 48 Step 72 Step 96 Circular polygon Step 0 Step 62 Step 125 Step 188 Step 250 Through traffic Step 0 Step 42 Step 85 Step 127 Step 170 Figure 7: Additional challenging scenario rollouts. Rows show cross geometry, nonconvex union, circular polygon, and through traffic scenarios sampled from held-out seeds. A.4.1 Reactive Parameter Adaptation Figure 8 visualizes a close interaction between two robots and the corresponding learned parameter changes. As the neighboring geometry changes during the interaction, the policy reacts by updating wpw_p, wuw_u, and dsafed_safe online. This example validates that the learned adaptation is not a fixed parameter schedule, but responds to the current GSFs induced by nearby robot shapes and distances. Figure 8: Reactive pair example in the random polygon scenario. The upper row shows the interaction between R6 and R16 at representative time steps, and the lower row shows the corresponding changes in inter-pair distance, wpw_p, wuw_u, and dsafed_safe, indicating that the learned policy adapts the MPC parameters to the current neighbor geometry. A.5 Implementation and Experiment Parameters This subsection summarizes the main implementation settings used for the reported experiments. Table 4 lists the controller and scenario parameters, Table 5 lists the reinforcement learning and neural-network parameters, and Table 6 lists the evaluation and baseline settings. The notation follows the main text: T is the MPC horizon, K is the number of neighbor GSFs in i,neighborto_i,neighbor^t, HcH_c is the HOCBF horizon, and λit=[wp,it,wu,it,dsafe,it]⊤ _i^t=[w_p,i^t,w_u,i^t,d_safe,i^t] is the adaptive parameter vector. Table 4: Controller and scenario parameters used by SRL-MPC. Group Symbol or item Value Prediction model T, Δt t 55, 0.1s0.1\,s HOCBF-MPC K, |icbf||N_i^cbf|, HcH_c, distance horizon, γ, ρobs _obs 55, 33, 33, 11, 0.90.9, 100100 Adaptive parameters λit _i^t Initial: (1.0,1.0,0.2m)(1.0,1.0,0.2\,m); Λ=[0.01,1.0]×[0.1,10.0]×[0.2,1.0] =[0.01,1.0]×[0.1,10.0]×[0.2,1.0]; wθ=0.01w_θ=0.01 fixed Solver and geometry Q1Q_1, Q2Q_2 Shapely, Splitting Conic Solver (SCS), one outer iteration; distance cost disabled for SRL-MPC Training world NtrainN_train, workspace, ℂiC_i 1515 robots; 10m×10m10\,m× 10\,m; convex polygon radius [0.1,0.4]m[0.1,0.4]\,m; irregularity [0.1,1.0][0.1,1.0]; goal threshold 0.3m0.3\,m Table 5: Reinforcement learning and neural-network parameters. Group Symbol or item Value Observation ito_i^t, i,neighborto_i,neighbor^t 79=4+3×5×579=4+3× 5× 5 Action Δλit _i^t [−0.5,0.5][-0.5,0.5], [−0.5,0.5][-0.5,0.5], [−0.1,0.1]m[-0.1,0.1]\,m; absolute values clipped to Λ Network CNN, actor, critic Convolution →323\!→\!16\!→\!32; actor 36→64→636→ 64→ 6; critic 36→64→136→ 64→ 1; log-scale clamp [−5,0][-5,0] PPO schedule Frames, batch, mini-batch, epochs 10710^7, 10001000, 500500, 33 PPO and generalized advantage estimation (GAE) parameters γPPO _PPO, λGAE _GAE 0.990.99, 0.950.95; clip 0.20.2; entropy 0.010.01; learning rate 3×10−43× 10^-4; max grad norm 1.01.0 Training reward rarrr_arr, rcolr_col, rsafer_safe, rstepr_step 4040, 8080, 11, 0.050.05 Safety reward dsafemind_safe , dsafemaxd_safe , ϵd _d 0.2m0.2\,m, 1.0m1.0\,m, 0.01m0.01\,m Table 6: Evaluation and baseline settings. Group Symbol or item Value Evaluation sweep N, nepn_ep, EmaxE_ N∈10,15,20,25N∈\10,15,20,25\; nep=100n_ep=100; Emax=500E_ =500 steps (50s50\,s) Randomization Train/test seeds Training seed 00; evaluation seed 100100; disjoint scenario sequences Scenario random_polygon Random start-goal pairs; random convex polygon footprints; collision mode stop Ablation methods dist_mpc, manual_mpc, rule_mpc Fixed wp=0.01w_p=0.01, wθ=0.01w_θ=0.01, wu=10.0w_u=10.0; rule_mpc uses dsafe=0.30d_safe=0.30 by default and switches to (wp,wu,dsafe)=(1.0,0.1,0.20)(w_p,w_u,d_safe)=(1.0,0.1,0.20) after 1010 stuck steps VO baselines ORCA, AVOCADO, VO-polytope Neighbor distance 15m15\,m; time horizon 0.5s0.5\,s; safety buffer 0.1m0.1\,m; VO-polytope neighbor distance 3m3\,m RL baselines RL-RVO, SARL Pretrained policies; desired speed 1.0m/s1.0\,m/s; Δt=0.1s t=0.1\,s A.6 Compute-Cost Profile Using the controller, learning, and evaluation settings summarized in Tables 4, 5, and 6, we profile the per-robot per-step controller time in the random_polygon scenario using three episodes per cell, a maximum of 5050 steps per episode, and seed 100100. Each sample is the wall-clock duration of one method decision call divided by the number of robots. Table 7 reports mean and standard deviation over timed steps for the robot counts used in the main evaluation. Table 7: Per-robot per-step controller compute time in milliseconds. Method N=10N=10 N=15N=15 N=20N=20 N=25N=25 manual_mpc 5.482±1.5855.482± 1.585 6.586±1.7156.586± 1.715 8.488±1.4188.488± 1.418 9.955±1.8039.955± 1.803 dist_mpc 5.202±1.4495.202± 1.449 7.313±1.1257.313± 1.125 8.693±1.6888.693± 1.688 11.260±1.35811.260± 1.358 ORCA 0.021±0.0030.021± 0.003 0.022±0.0010.022± 0.001 0.024±0.0020.024± 0.002 0.026±0.0020.026± 0.002 AVOCADO 0.022±0.0020.022± 0.002 0.026±0.0020.026± 0.002 0.028±0.0020.028± 0.002 0.028±0.0010.028± 0.001 SARL 15.612±0.90415.612± 0.904 16.353±1.81016.353± 1.810 17.346±1.67017.346± 1.670 18.561±1.73718.561± 1.737 RL-RVO 0.357±0.0480.357± 0.048 0.366±0.0370.366± 0.037 0.370±0.0330.370± 0.033 0.377±0.0410.377± 0.041 SRL-MPC (ours) 5.786±2.0235.786± 2.023 6.729±1.6696.729± 1.669 9.365±1.6799.365± 1.679 10.340±1.80110.340± 1.801