Paper deep dive
RideGym: A Standardized Interface for Real-World Large-Scale Ride-Sharing System
Zijian Zhao, Yulong Hu, Sen Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 90%
Last extracted: 7/14/2026, 2:34:18 AM
Summary
RideGym is an open-source, standardized Gym-style interface designed for Multi-Agent Reinforcement Learning (MARL)-based order dispatch in real-world ride-sharing systems. It decouples the simulation environment from dispatch algorithms, enabling reproducible, large-scale city-level simulations on real road networks. The paper highlights that exploration noise significantly impacts MARL performance and ranking, addressing reproducibility and engineering barriers in existing ride-sharing simulators.
Entities (13)
Relation Signals (9)
Zijian Zhao → affiliatedwith → The Hong Kong University of Science and Technology
confidence 95% · Zijian Zhao The Hong Kong University of Science and Technology
RideGym → developedby → Zijian Zhao
confidence 95% · RideGym: A Standardized Interface for Real-World Large-Scale Ride-Sharing System Zijian Zhao The Hong Kong University of Science and Technology
RideGym → developedby → Yulong Hu
confidence 95% · Yulong Hu The Hong Kong University of Science and Technology
RideGym → developedby → Sen Li
confidence 95% · Sen Li The Hong Kong University of Science and Technology
RideGym → implements → Multi-Agent Reinforcement Learning (MARL)
confidence 95% · RideGym is the first open-source, standardized Gym-style interface tailored to Multi-Agent Reinforcement Learning (MARL)-based order dispatch
RideGym → decouples → Environment
confidence 92% · By fully decoupling the environment from the dispatch algorithm, RideGym enables diverse learning-based and model-based methods to be developed and compared
RideGym → usesdatasource → OpenStreetMap
confidence 90% · Given a real-world road network extracted from OpenStreetMap (Haklay and Weber, 2008)
Exploration Noise → affects →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Ride-sharing has become an essential component of modern urban transportation and has attracted significant attention across computer science, transportation, and management science. While the field spans a broad range of problems, such as driver relocation, dynamic pricing, and vehicle charging or fueling dispatch, the core challenge remains order assignment and trip bundling, which directly affect urban traffic efficiency and carbon emissions. Despite its importance, existing simulation platforms are typically tailored to specific operational studies or tightly coupled to a particular dispatch algorithm, and rarely expose a standardized, learning-friendly interface. As a result, most researchers still build customized environments from scratch, raising serious concerns about reproducibility and fair comparison, and incurring substantial redundant effort. To address this gap, we present RideGym, the first open-source, standardized Gym-style interface tailored to MARL-based order dispatch in real-world ride-sharing systems. By fully decoupling the environment from the dispatch algorithm, RideGym enables diverse learning-based and model-based methods to be developed and compared under identical, fully specified conditions. It supports efficient, large-scale city-level simulations on real road networks, and offers flexible configurations for vehicle attributes, order specifications, and automatic shortest-path routing. We validate RideGym by reproducing several baselines, and demonstrate its high efficiency, with a one-hour simulation involving thousands of vehicles and tens of thousands of orders completed within one minute across all methods. Moreover, we reveal that the choice of exploration noise can significantly affect both the performance and the relative ranking of MARL solutions, an aspect often overlooked in prior work.
Tags
Links
- Source: https://arxiv.org/abs/2607.10173v1
- Canonical: https://arxiv.org/abs/2607.10173v1
Trouble viewing inline? Open PDF directly →
Full Text
77,183 characters extracted from source content.
Expand or collapse full text
RideGym: A Standardized Interface for Real-World Large-Scale Ride-Sharing System Zijian Zhao The Hong Kong University of Science and TechnologyHong KongChina zzhaock@connect.ust.hk , Yulong Hu The Hong Kong University of Science and TechnologyHong KongChina yhucm@connect.ust.hk and Sen Li The Hong Kong University of Science and TechnologyHong KongChina cesli@ust.hk (2018) Abstract. Ride-sharing has become an essential component of modern urban transportation and has attracted significant attention across computer science, transportation, and management science. While the field spans a broad range of problems, such as driver relocation, dynamic pricing, and vehicle charging or fueling dispatch, the core challenge remains order assignment and trip bundling, which directly affect urban traffic efficiency and carbon emissions. Despite its importance, existing simulation platforms are typically tailored to specific operational studies or tightly coupled to a particular dispatch algorithm, and rarely expose a standardized, learning-friendly interface. As a result, most researchers still build customized environments from scratch, raising serious concerns about reproducibility and fair comparison, and incurring substantial redundant effort. To address this gap, we present RideGym, the first open-source, standardized Gym-style interface tailored to Multi-Agent Reinforcement Learning (MARL)-based order dispatch in real-world ride-sharing systems. By fully decoupling the environment from the dispatch algorithm, RideGym enables diverse learning-based and model-based methods to be developed and compared under identical, fully specified conditions. It supports efficient, large-scale city-level simulations on real road networks, and offers flexible configurations for vehicle attributes (e.g., personalized speeds and capacities), order specifications (e.g., multiple passengers per order), and automatic shortest-path routing. We validate RideGym by reproducing several baselines, and demonstrate its high efficiency, with a one-hour simulation involving thousands of vehicles and tens of thousands of orders completed within one minute across all methods. Moreover, we reveal that the choice of exploration noise can significantly affect both the performance and the relative ranking of MARL solutions, an aspect often overlooked in prior work. Our code is available at https://github.com/RS2002/RideGym, and the Python package can be installed via pip install ride-gym. Ride Sharing, Simulation Gym, Standardized Interface, Multi-Agent Reinforcement Learning (MARL) †copyright: acmlicensed†journalyear: 2018†doi: X.X†conference: Make sure to enter the correct conference title from your rights confirmation email; June 03–05, 2018; Woodstock, NY†isbn: 978-1-4503-X-X/2018/06†ccs: Applied computing Transportation†ccs: Computing methodologies Multi-agent planning 1. Introduction Ride-sharing services have fundamentally reshaped urban transportation over the past decade. Platforms such as Uber, Lyft, and DiDi provide on-demand mobility that offers travelers greater convenience, reduced waiting times, and improved accessibility compared to traditional taxis or public transit. For drivers, these platforms create flexible income opportunities, while for cities they influence traffic patterns, vehicle utilization rates, and overall carbon emissions. By enabling more efficient matching between supply and demand, ride-sharing has become an indispensable component of modern smart-city ecosystems, affecting daily commutes, economic productivity, and environmental sustainability at a massive scale (Jin et al., 2018; Shaheen and Cohen, 2019). Within ride-sharing systems, a variety of operational tasks have attracted substantial research attention, including driver relocation (Sun et al., 2022), dynamic pricing (Ge et al., 2025), and vehicle charging/fueling dispatch (Yang et al., 2026). Among these tasks, order assignment and bundling form the core operational engine. They determine which available driver should serve each dynamically arriving passenger request and whether multiple orders can be combined into a single trip. As a result, they directly affect key performance metrics such as passenger waiting time, vehicle utilization, total vehicle miles traveled, traffic congestion, and greenhouse gas emissions. Effective order dispatch therefore lies at the heart of both platform profitability and broader societal impact (Alonso-Mora et al., 2017a). However, ride-sharing order dispatch presents formidable challenges. From an operations research perspective, it is a highly dynamic matching problem under uncertainty: decisions must account for Origin-Destination (OD) relationships, the complex interactions between newly assigned orders and en-route orders (whose routes and capacities are already partially committed), and the stochastic nature of future demand and traffic conditions. Technically, the problem is exacerbated by an enormous joint state space, encompassing vehicle locations, passenger requests, road network states, and driver statuses, and an exponentially large action space, where each action may involve assigning individual orders, forming bundles, or choosing not to serve certain requests. These characteristics render exact solutions intractable at city scale and make real-time decision-making particularly demanding. Despite these advances, progress in the field is held back less by a shortage of algorithms than by the absence of a common environment on which to develop and compare them. Several ride-sharing simulators do exist, yet none offers a standardized, learning-friendly testbed for order dispatch, and they broadly fall into two families, each missing a critical property. Operations-oriented simulators (Chen et al., 2025b; Engelhardt et al., 2022; Kucharski and Cats, 2022) faithfully model real road networks and high-capacity pooling, but are designed to evaluate operational strategies such as pricing, matching, and repositioning; they expose no standardized reset/step interface and are tightly coupled to a specific control logic, making it cumbersome to plug in and fairly compare (MA)RL algorithms. Learning-oriented simulators, in contrast, are typically bundled with one particular algorithm behind a bespoke interface, and several abstract the city into a grid or hexagonal world rather than a real road network (Lin et al., 2018), limiting both fidelity and reusability. Consequently, most researchers still build custom simulators from scratch, leading to inconsistent experimental setups, unfair comparisons, and substantial redundant engineering effort, which ultimately undermines reproducibility and slows the community’s progress. (A detailed review of existing works are deferred to Appendix A.) These observations point to a specific missing ingredient: a standardized, Gym-style interface (Brockman et al., 2016). The reset/step abstraction that catalyzed progress in single-agent RL, and more recently in transportation MARL such as traffic-signal control, confers two properties that ride-sharing dispatch urgently needs. First, it decouples the environment from the decision algorithm: researchers can swap dispatch policies, whether model-based or (MA)RL, behind an identical interface, so that observed performance differences reflect the algorithms themselves rather than idiosyncrasies of independently re-implemented simulators. Second, it lowers the engineering barrier and enforces fully specified, reproducible conditions, allowing the community to accumulate directly comparable results instead of repeatedly rebuilding environments. Such standardization is arguably even more valuable, and harder to attain, in ride-sharing than in conventional single-agent settings: dispatch actions are coupled across agents (an order cannot be assigned to two vehicles simultaneously) and unfold on a real road network with dynamically committed en-route trips. This is precisely the gap RideGym is designed to fill. To fill this gap, we present RideGym, the first open-source, standardized Gym-style interface (Brockman et al., 2016) tailored to (MA)RL-based order dispatch in real-world ride-sharing systems. By fully decoupling the environment from the dispatch algorithm, RideGym enables model-based and learning-based methods alike to be developed and compared under identical, fully specified conditions. It supports efficient, large-scale city-level simulations (e.g., Manhattan) with over one thousand vehicles operating on real road networks, and offers flexible vehicle configurations (e.g., personalized speeds and capacities), order specifications with multiple passengers per order, and automatic shortest-path routing with realistic travel-time estimation. We validate the environment by reproducing several classical model-based and competitive MARL-based baselines. Results demonstrate both the effectiveness of existing methods and the exceptional computational efficiency of our simulator, where one-hour simulations with thousands of vehicles and tens of thousands of orders complete within one minute across all tested methods. Furthermore, our benchmarking study reveals that the choice of exploration noise can significantly affect both the performance and the relative ranking of MARL-based solutions, an aspect often overlooked in prior work. 2. Preliminary: Problem Formulation In our simulator, we consider a ride-sharing system comprising a large number of vehicles and orders (i.e., travel requests) arriving dynamically over time. Each vehicle has a fixed maximum capacity and an average moving speed. At each decision step, a vehicle may be assigned zero or more orders, subject to its capacity constraint, and subsequently optimizes its route by minimizing the total travel time across both en-route and newly assigned orders. For orders, passengers submit requests to the system at arbitrary times, and the system adds them to a pooling buffer for dispatch at the next decision step. Unconfirmed orders remain in the pool and await future decisions; however, passengers are impatient, and any order that remains unconfirmed beyond a waiting-time threshold will be withdrawn, incurring a potential loss for the platform. We model each vehicle as an agent and formulate the problem as a Multi-Agent Markov Decision Process (MAMDP) (Littman, 1994), denoted by <n,S,U,,R,γ,O,T><n,S,U,P,R,γ,O,T>, where the components represent the number of agents (vehicles), joint state, joint action, joint state transition function, joint reward function, discount factor, joint observation, and time horizon, respectively. Although each agent corresponds to a vehicle, the agents collectively serve the platform’s objective rather than individual drivers. Consequently, the system is fully centralized, and every agent has access to the complete global state, i.e., the joint observation O is equivalent to the joint state S. The detailed formulation is as follows: 1) State S: At time step t, the joint state consists of the individual vehicle states si,tvs^v_i,t, the order-pooling state stos^o_t (containing unassigned orders), and the current timestamp t, expressed as: (1) St=[sto,s1,tv,s2,tv,…,sn,tv,t]. S_t=[s^o_t,\,s^v_1,t,\,s^v_2,t,\,…,\,s^v_n,t,\,t]. The order-pooling state stos^o_t is further defined as: (2) sto=[s1,to,s2,to,…,smt,to], s^o_t=[s^o_1,t,\,s^o_2,t,\,…,\,s^o_m_t,t], where mtm_t is the number of orders pending assignment at time t. The state of vehicle i, denoted si,tvs^v_i,t, consists of: (3) si,tv=[li,tp,cim,ci,tr,vi,si,te], s^v_i,t=[l^p_i,t,\,c_i^m,\,c_i,t^r,\,v_i,\,s^e_i,t], where li,tpl^p_i,t is the two-dimensional current position, cimc_i^m and ci,trc_i,t^r are the maximum and remaining capacities, viv_i is the average speed, and si,tes^e_i,t represents the en-route orders, given by: (4) si,te=[si,1,te,si,2,te,…,si,ki,t,te], s^e_i,t=[s^e_i,1,t,\,s^e_i,2,t,\,…,\,s^e_i,k_i,t,t], with ki,tk_i,t denoting the number of en-route orders for vehicle i at time t. Each order o, whether in stos^o_t or si,tes^e_i,t, is represented by: (5) o=[lo,ld,τa,τr,h], o=[l^o,\,l^d,\,τ^a,\,τ^r,\,h], where lol^o and ldl^d are the two-dimensional origin and destination coordinates, τaτ^a is the request time, τrτ^r is the expected remaining travel time (set as 0 for orders in stos^o_t), and h is the number of passengers for this order. 2) Action U: At time t, the joint action UtU_t consists of individual actions: (6) Ut=[u1,t,u2,t,…,un,t]∈0,1n×(mt+1), U_t=[u_1,t,\,u_2,t,\,…,\,u_n,t]∈\0,1\^n×(m_t+1), where ui,t∈0,1mt+1u_i,t∈\0,1\^m_t+1. Specifically, ui,j,t=1u_i,j,t=1 for j≤mtj≤ m_t indicates that the j-th order in the pool is assigned to vehicle i, while ui,mt+1,t=1u_i,m_t+1,t=1 indicates that vehicle i receives no order at this step. Unlike standard MAMDPs where agents act independently, the joint action UtU_t not only has a time-varying dimensionality but must also satisfy the following constraints: (7a) ∑i∈ℐui,j,t _i u_i,j,t ≤1, ≤ 1, ∀j ∀ j ∈t, _t, (7b) ∑j∈t∪mt+1ui,j,t _j _t∪\m_t+1\u_i,j,t ≥1, ≥ 1, ∀i ∀ i ∈ℐ, , (7c) ui,mt+1,t∑j∈tui,j,t u_i,m_t+1,t _j _tu_i,j,t =0, =0, ∀i ∀ i ∈ℐ, , (7d) ∑j∈tui,j,thj,t _j _tu_i,j,t\,h_j,t ≤ci,tr, ≤ c_i,t^r, ∀i ∀ i ∈ℐ, , (7e) ui,j u_i,j ∈0,1, ∈\0,1\, ∀i ∀ i ∈ℐ,∀j∈t∪mt+1, ,\ ∀ j _t∪\m_t+1\, where ℐ=1,…,nI=\1,…,n\ and t=1,…,mtJ_t=\1,…,m_t\ are the sets of vehicle and order indices, respectively, and hj,th_j,t denotes the passenger count of order j in the pool at time t. Eq. (7a) ensures that each order is assigned to at most one vehicle. Eq. (7b)–(7c) guarantee that each vehicle either takes no order or receives at least one, while Eq. (7d) enforces that the total number of assigned passengers does not exceed the remaining capacity. 3) State Transition Function (⋅|⋅,⋅)P(·|·,·): The transition function (⋅|St,Ut)P(·|S_t,U_t) specifies the probability distribution over next states. In our simulator, vehicle state transitions are deterministic: each vehicle moves along the planned shortest route. The only source of stochasticity is the order-pooling state st+1os^o_t+1, into which new orders arrive randomly. For simplicity and alignment with most existing work, we currently do not model complex transportation-network dynamics such as traffic congestion or accidents. 4) Reward Function R(⋅,⋅)R(·,·): Since the ride-sharing order dispatch task is fully cooperative, the global reward is the sum of individual vehicle rewards: (8) R(St,Ut)=∑i∈ℐr(si,tv,ui,t), (S_t,U_t)= _i r(s^v_i,t,u_i,t), where r(⋅,⋅)r(·,·) is an individual reward function that can be customized according to user preferences over criteria such as customer waiting time, service rate, or detour time. A default reward function, designed to reflect real-world ride-sharing market dynamics, is provided in our simulator: (9) r(si,tv,ui,t)= (s^v_i,t,u_i,t)= ∑o∈(ui,t)(β1+β2⋅Dis(o[lo],o[ld])⋅o[h] _o (u_i,t) ( _1+ _2·Dis(o[l^o],o[l^d])· o[h] −β3⋅ExpectedTime(o;si,tv,(ui,t))) - _3·ExpectedTime(o;s^v_i,t,O(u_i,t)) ) −∑o∈si,teβ4⋅ExtraTime(o;si,tv,(ui,t)), - _o∈ s^e_i,t _4·ExtraTime(o;s^v_i,t,O(u_i,t)), where β1 _1 to β4 _4 are non-negative hyperparameters, (ui,t)O(u_i,t) is the set of orders assigned by action ui,tu_i,t, and o[lo],o[ld],o[h]o[l^o],o[l^d],o[h] are the origin, destination, and passenger count as defined in Eq. (5). The function Dis(⋅,⋅)Dis(·,·) computes the distance between origin and destination; ExpectedTime(⋅;⋅,⋅)ExpectedTime(·;·,·) gives the expected travel time for the assigned order bundle under shortest-path routing (i.e., the time from assignment to drop-off minus the request time); and ExtraTime(⋅;⋅,⋅)ExtraTime(·;·,·) measures the additional detour time incurred for all en-route orders due to route updates when inserting the new bundle. This reward structure reflects real-world considerations: β1+β2⋅Dis(o[lo],o[ld])⋅o[h] _1+ _2·Dis(o[l^o],o[l^d])· o[h] represents platform revenue, proportional to trip distance and passenger count; β3⋅ExpectedTime(⋅;⋅,⋅) _3·ExpectedTime(·;·,·) and β4⋅ExtraTime(⋅;⋅,⋅) _4·ExtraTime(·;·,·) capture passenger satisfaction, which may influence future platform retention. Typically, β4>β3 _4> _3, as passengers are more sensitive to uncertain and dynamically increasing travel times. Notably, ExtraTimeExtraTime can be negative for some orders due to path optimization when new orders are inserted. Vehicles receiving no new orders at a given step receive zero reward, as defined in Eq. (9). 5) Objective J(⋅)J(·): For a trajectory =S1,U1,S2,U2,…,ST,UTT=\S_1,U_1,S_2,U_2,…,S_T,U_T\, the return is defined as: (10) G()=∑t=1Tγt−1R(Si,Ui), (T)= _t=1^Tγ^t-1R(S_i,U_i), and the objective of a joint policy Π is: (11) J(Π)=∼π[G()], ( )=E_T π [G(T) ], with the goal of finding an optimal policy Π∗ ^* that maximizes J(Π)J( ). 3. RideGym: Library Construction Figure 1. Workflow of the proposed simulation framework. As shown in Fig. 1, the workflow of our simulator can be summarized as the following steps: Step 1: The simulator initializes the system state according to user-specified configurations, including vehicle attributes (e.g., personalized speed, capacity, and initial location), the order dataset (e.g., order origin-destination, arrival time, and passenger count), the operational region for road network construction, and the reward function. Unspecified parameters are set to random or default values. Step 2.1: At each time step, the order pool is updated: newly arrived orders are added, overdue unconfirmed orders are removed, and the status of en-route and completed orders is refreshed. Step 2.2: The user can then specify the order dispatch decisions through a standardized interface. Step 2.3: Based on the chosen actions, vehicle states are updated in the following order: conflicts between en-route and newly assigned orders are resolved, the shortest path for the updated bundle is re-optimized, and vehicles move along the road network following the planned routes. Step 2.4: The reward function computes the reward based on the intermediate outcomes from Step 2.3. Steps 2.1 through 2.4 repeat until the end of the time horizon, with the option for users to integrate training procedures into this iterative process. In this section, we provide a detailed description of the implementation of the core components and the utilization of our standard Application Programming Interface (API). 3.1. Environment Implementation Our simulator is organized as a set of loosely coupled, interface-driven components, enabling independent replacement of each module without modifying the core simulation loop. This design is essential for rapidly constructing personalized scenarios in future research. The package comprises five core modules: (i) a set of lightweight domain entities (order and vehicle) that encapsulate the full lifecycle state of each request and vehicle; (i) an order generator interface that supplies the demand stream for an episode; (i) a road network interface that handles distance and shortest-path queries; (iv) a route planner interface that sequences each vehicle’s pickups and drop-offs; and (v) a reward function interface that maps a per-step event log to the individual reward. These components are assembled by a central environment that owns all mutable state and enforces the strict event ordering illustrated in Fig. 1. Each interface is implemented as an abstract base class with a fast default implementation, allowing users to inject custom demand models, road backends, planners, or reward functions simply by passing an alternative object at construction time; the simulation loop remains agnostic to the concrete implementation used. Concretely, the environment realizes the MAMDP formulated in Section 2 for a fleet of n vehicles ℐ=1,…,nI=\1,…,n\ over a horizon of T=⌈H/Δt⌉T= H/ t decision steps, where H is the total time horizon and Δt t is the decision interval. At each step, it materializes the joint state StS_t from Eq. (1), receives the joint action UtU_t, applies the deterministic vehicle transition alongside the stochastic arrival of new orders, and emits the reward R(St,Ut)R(S_t,U_t). Below, we detail the two core components of our simulator: the precomputed road network and the precedence-aware route planner, and then outline the state-transition procedure. 1) Precomputed Road Network: The road network is a performance-critical component: the distance query Dis(⋅,⋅)Dis(·,·) is invoked on the order of millions of times per episode, as it underpins dispatch matching, route planning, and vehicle movement. Therefore, we aim to minimize the computational cost of this operation. In contrast to prior approaches (Hu et al., 2025b; Wang et al., 2025) that issue per-query calls to external routing services such as OSRM (Luxen and Vetter, 2011), our design avoids several inherent limitations: limited platform support (e.g., the official OSRM repository 111https://github.com/Project-OSRM/osrm-backend supports Linux but not Windows), high query latency, and the risk of packet loss or communication bottlenecks under parallel querying. To address these issues, we adopt an ahead-of-time all-pairs shortest-path scheme. Given a real-world road network extracted from OpenStreetMap (Haklay and Weber, 2008), we take its largest strongly connected component, relabel the nodes to contiguous indices 1,…,K\1,…,K\, and precompute once the dense all-pairs shortest-path distance matrix ∈ℝK×KD ^K× K along with a predecessor matrix ∈ℤK×KP ^K× K, via K length-weighted Dijkstra passes: (12) ab=minπ:a↝b∑(u,w)∈πℓ(u,w), _ab= _π:\,a b _(u,w)∈π (u,w), ab=predecessor of b on the shortest a↝b path, _ab=predecessor of b on the shortest a b path, where π denotes a path in the road network, and ℓ(u,w) (u,w) is the physical length of directed edge (u,w)(u,w). Both matrices are cached to disk keyed by the node ordering, so that repeated runs load them in sub-second time, and any change to the graph automatically invalidates the cache. Any continuous query point, such as a vehicle position li,tpl^p_i,t or an order endpoint lo,ldl^o,l^d, is mapped to its nearest graph node via snap(l)=argmink‖l−χk‖22snap(l)= _k\|l- _k\|_2^2, where χk _k are the node coordinates; this mapping is memoized, as query coordinates recur frequently. (The time to travel to the nearest node is accounted for through linear interpolation.) Consequently, the network distance reduces to two cached snap operations and a single matrix lookup: (13) Dis(a,b)=snap(a),snap(b),Dis(a,b)=D_snap(a),\,snap(b), and a full node path is reconstructed in O(|π|)O(|π|) from P without per-query Dijkstra. This design offers two key advantages over external routing services: it is faster (constant-time lookups instead of network round-trips and repeated search), and it is self-contained and system-independent, requiring no external service, daemon, or platform-specific binary. Travel time is computed as Dis(a,b)/viDis(a,b)/v_i minutes, decoupling distance topology from the vehicle speed model. For lightweight or purely abstract experiments, we additionally provide closed-form Euclidean and Manhattan backends behind the same interface, bypassing the graph entirely. 2) Shortest-Path Route Planning under Precedence Constraints: When vehicle i is assigned the order bundle (ui,t)O(u_i,t), the environment must sequence the pickup and drop-off stops for both newly assigned and en-route orders si,tes^e_i,t into a single coherent route. Let i,tS_i,t denote the resulting multiset of remaining stops, each either a pickup or a drop-off, subject to the ride-pooling precedence constraint that every order must be picked up before it is dropped off. This modeling choice enhances the fidelity of our simulator in reflecting real-world operations, in contrast to prior works (Hu et al., 2025b; Zhao and Li, 2025a) that prohibit vehicles from accepting new orders while en route to a pickup, a restriction often adopted for implementation simplicity. Denoting by σ a visiting permutation of i,tS_i,t and by posσ(⋅)pos_σ(·) the position of a stop under σ, the planner solves (14) minσ[Dis(li,tp,σ1)+∑k=1|i,t|−1Dis(σk,σk+1)] _σ [Dis(l^p_i,t, _1)+ _k=1^|S_i,t|-1Dis( _k, _k+1) ] s.t. posσ(picko)<posσ(dropo),∀o, _σ(pick_o)<pos_σ(drop_o), ∀ o, i.e., it minimizes the total route travel distance from the vehicle’s current position li,tpl^p_i,t, while respecting pickup-before-drop-off for every order. The default planner solves Eq.(14) via a nearest-feasible-stop heuristic: starting from li,tpl^p_i,t, it iteratively appends the nearest feasible stop, where a drop-off becomes feasible only after its corresponding pickup has been placed, thereby guaranteeing precedence by construction (Hurkens and Woeginger, 2004). This heuristic choice is adopted in place of an exact precedence-constrained Traveling Salesman Problem (TSP) solver because the planner resides on the hot path (invoked millions of times per episode) and the stop set size is bounded by the vehicle capacity cimc^m_i. Consequently, the heuristic delivers near-optimal performance at a fraction of the cost, while the precedence constraint remains strictly enforced through the feasible-set restriction at each step. The same pass accumulates travel time for each leg, enabling the planner to return, at no additional cost, the predicted arrival times at every pickup and drop-off. As the planner is exposed through a generic interface, users may substitute an exact constrained-TSP solver while preserving the same precedence guarantee. 3) State Transition: Given the joint action UtU_t, the environment executes the transition (⋅∣St,Ut)P(· S_t,U_t) in the strict order illustrated in Fig. 1: it (i) resolves conflicts to enforce constraint Eq. (7a) and Eq. (7d), ensuring that no order is assigned to more than one vehicle and that assigned orders do not exceed the remaining capacity; (i) admits each accepted bundle (ui,t)O(u_i,t) and re-solves Eq.(14) for shortest-path route planning; and (i) advances each vehicle i along its planned route by a distance budget viΔtv_i t. On a real graph, a vehicle walks the reconstructed node path edge by edge and may halt part-way along an edge, with its continuous position li,tpl^p_i,t obtained by linear interpolation between the edge endpoints; a long edge is thus consumed across several steps, ensuring deadlock-free progress whenever viΔt>0v_i t>0. Arrivals trigger pickup and drop-off updates that advance each order through its lifecycle and decrement its expected remaining travel time τrτ^r. The vehicle transition is deterministic; the only stochasticity is the arrival of new requests into st+1os^o_t+1, while pending orders whose waiting time exceeds the threshold θ are withdrawn before the next decision. (iv) Finally, the reward function maps the resulting per-vehicle events to the individual reward r(si,tv,ui,t)r(s^v_i,t,u_i,t). 3.2. Standardized Interface The environment exposes a decentralized, multi-agent, Gym-like API that is not dependent on the OpenAI Gym library (Brockman et al., 2016). This section walks through a typical order of use, illustrated by the code a user would actually write in Fig. 2. 1) Preparing the Demand Dataset: The order stream is supplied through an order generator. Users may supply their own dataset (e.g., real historical trips) as a table with one row per order and the columns listed below, corresponding to the fields in Eq. (5) (lo,ld,τa,hl^o,l^d,τ^a,h; the remaining travel time τrτ^r is initialized to 0 internally). A procedural generator (supporting uniform, Poisson, or peaked arrivals) is also provided for synthetic experiments. 2) Building the road network. For real-map experiments, the road network is built once, offline, and cached to disk (the all-pairs matrices ,D,P of Section 3.1); every subsequent run reloads the cache and answers each Dis(⋅,⋅)Dis(·,·) query in O(1)O(1). Abstract experiments instead use the closed-form Euclidean or Manhattan backends. 3) Initializing the Environment: The environment is constructed from the components above together with the fleet configuration. Per-vehicle capacities cimc^m_i and speeds viv_i may be provided as length-n lists for a heterogeneous fleet, or as scalars for a homogeneous one; the planner and reward function r(⋅,⋅)r(·,·) are injected here and may be swapped freely. 4) The reset and step Interface: Two methods drive the entire simulation: (O1,info) (O_1,\,info) =env.reset(seed), = env.reset(seed), (Ot+1,r(si,tv,ui,t)i∈ℐ,done,info) (O_t+1,\,\r(s^v_i,t,u_i,t)\_i ,\,done,\,info) =env.step(Ut). = env.step(U_t). Both the observation and the reward are dictionaries keyed by vehicle index i∈ℐi . Since the system is fully centralized, the returned observation coincides with the joint state, Ot≡StO_t≡ S_t (Eq. (1)): for each vehicle, it contains the vehicle state si,tvs^v_i,t (Eq. (3)) and the shared order-pool state stos^o_t (Eq. (2)). The input UtU_t is the joint action of Eq. (6), supplied as a dictionary of per-vehicle actions ui,tu_i,t; each ui,tu_i,t is given compactly as the set of pool indices the vehicle bids on, and an empty set encodes the no-order action ui,mt+1,t=1u_i,m_t+1,t=1. A single step advances the simulation by Δt t and automatically enforces the feasibility constraints (7a)-(7d). 5) A Minimal Control Loop. Any policy that maps a vehicle state si,tvs^v_i,t and the order pool stos^o_t to the action schema above can be rolled out as Fig. 2 (4). A training procedure may be interleaved directly between successive step calls, matching the iterative workflow of Fig. 1 and optimizing the objective J(Π)J( ) of Eq. (11). (1) Preparing the Demand Dataset ⬇ from ride_gym.order_generator import DataFrameOrderGenerator # Required columns: origin_x, origin_y -> l^o # dest_x, dest_y -> l^d # request_time -> tau^a (minutes from episode start) # num_passengers -> h order_gen = DataFrameOrderGenerator(dataframe=my_orders_df) (2) Building the Road Network ⬇ from ride_gym.osmnx_network import OSMnxNetwork # One-off offline build for a region, then reused via the on-disk cache: # python -m ride_gym.build_network --place "Manhattan, NY" --out data/manhattan.gpickle network = OSMnxNetwork(graph_path="data/manhattan.gpickle") (3) Initializing the Environment ⬇ from ride_gym.env import RidePoolEnv env = RidePoolEnv( num_drivers=1000, # n driver_capacities=[3, 4, ...], # per-vehicle c^m_i (or driver_capacity=4) driver_speeds=[1.0, 0.9, ...], # per-vehicle v_i (or None -> network speed) dt=1.0, horizon=60.0, # decision interval Delta t, horizon H order_timeout=3.0, # impatience threshold order_generator=order_gen, road_network=network, ) (4) Minimal Control Loop ⬇ obs, info = env.reset(seed=0) # initial joint observation O_1 == S_1 done = False while not done: U = # joint action i: u_i,t for i, s_v in obs.items(): pool = s_v["pending_orders"] # shared order-pool state s^o_t u_i = my_policy(s_v, pool) # user policy -> list of bid order ids U[i] = "orders": u_i # empty list encodes the no-order action obs, rewards, dones, info = env.step(U) # rewards[i] = r(s^v_i,t, u_i,t) done = dones["__all__"] # True once step t reaches horizon T (5) Visualization ⬇ from ride_gym.visualize import render_frame, TrajectoryRecorder, render_animation # (a) Single frame -> static image (PNG or PDF). env.render(mode="human", save_path="frame.pdf") # (b) Whole episode -> animation. Snapshot each step, then export to GIF/MP4. rec = TrajectoryRecorder() obs, _ = env.reset(seed=0) done = False while not done: obs, rewards, dones, info = env.step(my_policy(obs)) rec.snapshot(env) # lightweight per-step capture done = dones["__all__"] render_animation(rec, out_path="episode.gif") # or "episode.mp4" Figure 2. Typical Python usage workflow. 3.3. Visualization To make simulation dynamics interpretable, our environment ships with a built-in visualization toolkit. It can render either a single frame (a static snapshot of the system state at one decision step) or an entire episode as an animation, and exports to standard formats: a static image (PNG/PDF) for a frame, and an animated GIF or MP4 for a rollout. Rendering reads only the public state already exposed by the environment, so it is fully decoupled from the simulation loop and adds no overhead when disabled. A typical use is a one-line call after (or during) a rollout, shown in Fig. 2 (5). A detailed example is provided at Appendix B. Beyond the spatial view, our toolkit also provides a set of aggregate, metric-oriented visualizations, including demand and service-rate heatmaps, supply-demand gap maps, system-load time series, and passenger waiting-time distributions, for quantitative analysis of a scenario or a policy. As these are auxiliary to the core environment, we omit their details here and refer the interested reader to our repository for the full set of tools and usage examples. 4. Benchmarking Experiment 4.1. Benchmark Approaches To validate the effectiveness of our proposed simulator, we evaluate a set of benchmark methods in our environment, covering both classical model-based approaches and recent MARL-based solutions. For model-based methods, we select widely used strategies from online ride-hailing platforms such as Didi (Chen, 2023), including Random dispatch, Greedy (Kalyanasundaram and Pruhs, 1993), Kuhn–Munkres (KM) (Kuhn, 1955), and Gale–Shapley (GS) (Gale and Shapley, 1962; Yue et al., 2024). For MARL-based solutions, we choose REDA (Holder et al., 2025), BMG-Q (Hu et al., 2025b), MF-DDQN (Li et al., 2019), Assignment-Net (Zhao and Li, 2025b, 2026a), and CV-Net (Tang et al., 2019), covering a range of innovations in both algorithmic design and network architecture. Detailed introductions to these methods are provided in Appendix C.1. Specifically, we emphasis the Random is not meaningless since the matching is happened in a given radius, so it can be viewed as another type of Greedy. For all MARL-based methods, we introduce a dummy order that represents the option of taking no order at the current step. This design allows agents to actively reject low-value orders—for instance, when an order is not aligned with the en-route trajectories of nearby vehicles and would therefore negatively impact the system. This mechanism is not implemented in the original papers of these methods. 4.2. Experiment Configurations The experiments are conducted using the public ride-hailing dataset from Manhattan, New York, provided by the TLC Trip Record Data (Taxi and Commission, 2024). We set the fleet size to 1,000 vehicles, each with a capacity of 4 passengers and an average speed of 35 km/h, reflecting typical urban driving conditions. Since the TLC data provides only coarse-grained zone-level OD information, we assign each OD to the central coordinate of its corresponding zone, with a random perturbation drawn uniformly from a circle of radius 0.5 km, to avoid unrealistic concentration of orders at identical points. The passenger count per order is sampled uniformly between 1 and 4, capturing the heterogeneity of real-world ride-sharing demand, which is an aspect often overlooked in prior work. Specifically, our simulator also supports a mixture of pooling and non-pooling orders, reflecting the practical scenario where passengers may opt out of ride-sharing by selecting a passenger count of 4 and paying a higher fare. The maximum waiting time for order confirmation is set to 3 minutes; any order not assigned to a vehicle within this window is considered canceled by the passenger. Each episode spans a 60-minute horizon with a decision interval of 1 minute, which is a common setup applied by many papers (Hu et al., 2025b; Zhao and Li, 2025b; Al-Abbasi et al., 2019; Enders et al., 2023; Hoppe et al., 2024). For MARL-based methods, we adopt the Adam optimizer with a learning rate of 5×10−45× 10^-4. The hyper-parameters gammagamma and β1 _1t o β4 _4 are set as 0.99,1.0,0.01,0.04,0.080.99,1.0,0.01,0.04,0.08. The replay buffer size is set to 6,000 (equivalent to 100 episodes), with a batch size of 8, where each sample corresponds to the joint transition of all agents at a single time step. The exploration strategy is detailed in the experiment results section. Each method is trained for 500 episodes. We use data from 8:00 to 20:00 between April 6 and April 12, 2026, as the training set, April 13, 2026, as the validation set (optional, for tracking training progress), and April 14, 2026, as the testing set. Specifically, we report results on an off-peak scenario (10:00–11:00, with 6,863 orders) and an on-peak scenario (18:00–19:00, with 11,219 orders) from April 14, 2026. All data are sourced from the High Volume FHV Trips Data within the TLC Trip Record Data. All training and evaluation are conducted on a workstation running Windows 11, equipped with an Intel(R) Core(TM) i7-14700KF processor and an NVIDIA RTX 4080 graphics card. 4.3. Evaluation Results Table 1. Benchmark performance under off-peak and on-peak settings. ↑ / ↓ denote whether higher or lower is better. All time metrics are in minutes. Best and second-best results per column are marked in bold and underline, respectively. Metric definitions are given in Appendix C.2. All results are averaged over three independent runs. Method Off-Peak (10:00-11:00 with 6,863 orders) On-Peak (18:00-19:00 with 11,219 orders) Service Completion Wait Detour Simulation Service Completion Wait Detour Simulation Rate (%) ↑ Rate (%) ↑ Time ↓ Time ↓ Time ↓ Rate (%) ↑ Rate (%) ↑ Time ↓ Time ↓ Time ↓ Model-based methods Random 91.21±0.19 77.63±0.20 1.84±0.02 1.66±0.01 0.13±0.05 68.87±0.58 55.75±0.61 2.11±0.02 2.61±0.03 0.21±0.06 Greedy 93.98±0.20 80.94±0.28 1.50±0.03 1.61±0.04 0.13±0.05 78.41±0.29 65.12±0.28 2.22±0.04 2.27±0.06 0.18±0.11 KM 93.89±0.34 80.64±0.31 1.44±0.02 1.71±0.05 0.13±0.06 74.13±0.39 60.37±0.34 1.87±0.01 2.38±0.04 0.18±0.09 GS 93.37±0.58 79.98±0.56 1.49±0.02 1.69±0.03 0.13±0.01 63.37±0.23 50.58±0.19 2.01±0.01 2.45±0.04 0.22±0.10 MARL-based methods (INF noise) REDA 91.04±0.38 79.24±0.27 1.14±0.01 1.13±0.01 0.08±0.02 74.97±0.55 61.79±0.41 1.28±0.01 2.33±0.04 0.10±0.05 BMG-Q 94.40±0.28 82.11±0.34 1.35±0.00 1.21±0.02 0.44±0.21 76.22±0.24 62.68±0.09 1.40±0.00 2.39±0.05 0.45±0.03 MF-DDQN 83.64±0.83 71.81±0.80 1.15±0.01 1.29±0.01 0.45±0.013 71.58±0.36 58.86±0.27 1.28±0.00 2.47±0.01 0.50±0.16 Assignment-Net 75.39±0.15 65.69±0.22 1.44±0.03 0.19±0.01 0.08±0.03 73.69±0.29 64.76±0.43 1.42±0.00 0.22±0.01 0.10±0.02 CV-Net 87.88±0.29 76.00±0.40 1.29±0.01 1.23±0.03 0.08±0.03 75.12±0.66 61.68±0.70 1.48±0.02 2.19±0.01 0.10±0.02 MARL-based methods (STD noise) REDA 96.50±0.25 85.68±0.18 1.40±0.02 0.69±0.02 0.08±0.02 78.96±0.41 64.70±0.48 1.58±0.02 2.20±0.01 0.09±0.04 BMG-Q 97.26±0.09 84.64±0.14 1.48±0.01 1.10±0.01 0.44±0.12 78.71±0.25 64.90±0.17 1.50±0.01 2.20±0.03 0.45±0.04 MF-DDQN 97.49±0.11 85.56±0.07 1.56±0.02 0.80±0.03 0.44±0.18 80.16±0.09 66.15±0.07 1.61±0.01 2.18±0.01 0.50±0.17 Assignment-Net 95.83±0.09 85.68±0.14 1.44±0.01 0.34±0.02 0.08±0.02 87.18±0.28 77.30±0.31 1.80±0.01 0.53±0.01 0.10±0.03 CV-Net 96.23±0.11 83.74±0.05 1.50±0.02 0.74±0.02 0.09±0.03 79.42±0.40 65.37±0.40 1.69±0.02 2.16±0.05 0.27±0.03 In this paper, we identify a critical factor that hinders the reproducibility of MARL-based ride-sharing methods: the design of exploration noise. Unlike standard MDP settings, actions in ride-sharing are not independent across agents, as an order cannot be assigned to multiple vehicles simultaneously. Consequently, common exploration techniques such as ϵε-greedy or Boltzmann exploration are not directly applicable. Current methods typically adopt a paradigm in which noise is first added to the Q-values of each vehicle-order pair, followed by bipartite matching to maximize the global Q-value. Unfortunately, many prior works do not clearly specify this implementation detail. To systematically investigate its impact, we reproduce the INF noise from (Hu et al., 2025b), where the Q-value of each vehicle-order pair is set to +∞+∞ with probability ϵε (the exploration rate, which decays from 1 to 0 over time), thereby forcing that pair to be selected. We further propose a simple yet effective STD noise, which adds a zero-mean Gaussian perturbation scaled by the current Q-value volatility: (15) ζ∼(0,(ϵσ)2), ζ (0,(εσ)^2), where σ is the standard deviation of the Q-values across all vehicle-order pairs. The experimental results in Table 1 reveal two striking insights. First, STD noise consistently outperforms INF noise across all MARL baselines in terms of service rate and completion rate. While serving more orders inevitably leads to slightly higher wait and detour times (within 0.5 minutes), the substantial gains in service rate (ranging from 2% to 20%) far outweigh these marginal increases. We attribute this advantage to the adaptive scaling of STD noise, which aligns perturbation magnitude with the underlying Q-value distribution. In contrast, INF noise often induces conflicting simultaneous selections (e.g., multiple vehicles forced to the same order) that are later pruned by the matching step, resulting in insufficient and biased exploration. Second, and more critically, the choice of noise fundamentally alters the relative ranking of algorithms. For example, under INF noise, Assignment-Net and MF-DDQN underperform relative to other MARL baselines; however, under STD noise, they emerge as the strongest contenders. This suggests that conclusions drawn under arbitrary noise configurations may not be universally valid. A closer examination of Table 1 further disentangles algorithmic strengths. Since STD noise yields consistently better overall performance, we consider it a more reliable setting for assessing converged performance and use it as the basis for our analysis. During off-peak periods, MF-DDQN and BMG-Q achieve marginally higher service rates than others, suggesting that neighborhood information aggregation via mean-field or graph attention is beneficial when demand is sparse and competition among vehicles is low. However, during on-peak periods, their advantages become less pronounced, likely because the abundance of orders reduces the risk of conflicting competition, thereby limiting the added value of neighbor-aware aggregation. Moreover, we observe that the per-episode simulation time for MF-DDQN and BMG-Q (0.44–0.50 min) is approximately five times higher than that of Assignment-Net and CV-Net (0.08–0.10 min). Although all methods comfortably complete a one-hour simulation within one minute, this overhead accumulates substantially during training: over 500 episodes, the extra cost amounts to over three hours of wall-clock time. In practice, when extensive hyper-parameter tuning is required, this computational gap becomes a non-negligible factor, and researchers must carefully balance the marginal performance gains of neighborhood aggregation against its significant training overhead. Furthermore, we observe that network architecture design is also a critical factor in ride-sharing performance. Assignment-Net and CV-Net, with their carefully engineered structures, show superior detour performance, especially in on-peak scenarios. Specifically, Assignment-Net achieves exceptionally low detour times (0.34 min off-peak and 0.53 min on-peak), which are roughly two to four times lower than other methods (0.69–1.10 min off-peak and 2.16–2.20 min on-peak). We posit that this dramatic improvement stems from Assignment-Net’s LSTM module, which explicitly preserves the sequential order of en-route pickups and drop-offs. In contrast, MLP- or GAT-based methods typically fuse sequential stop features into a fixed-length vector, discarding temporal order. This effect is amplified during peak hours, when the average number of en-route orders per vehicle increases, making order-preserving representations critical for efficient pooling. Finally, the high computational efficiency of our simulator is validated by the fact that every benchmark method completes a full episode in under one minute across both scenarios, confirming that the environment itself will not become a bottleneck in large-scale MARL training or hyper-parameter search. 5. Conclusion In this paper, we introduce RideGym, the first open-source and standardized Gym-style interface tailored to (MA)RL-based order dispatch in large-scale real-world ride-sharing systems. By fully decoupling the environment from the dispatch algorithm, RideGym provides a fair and efficient benchmarking platform for evaluating diverse methods under consistent, fully specified conditions, while also offering a rich set of modular APIs that serve as a flexible foundation for customized research, significantly reducing the need to build complex simulation systems from scratch. The simulator supports a wide range of configurable features, including personalized vehicle speeds, capacities, and multiple passengers per order, enabling realistic and configurable emulation of ride-sharing operations. We validate RideGym through extensive experiments with classical and competitive baselines, demonstrating its high computational efficiency and fast simulation speed. Notably, during the reproduction process, we uncover that the type of exploration noise can substantially affect both the absolute performance and the relative ranking of MARL-based methods—an important yet often overlooked factor in prior work. We believe RideGym will foster improved reproducibility, fairer comparisons, and reduced redundant engineering efforts in the intelligent transportation research community. More discussions are left at Appendix D. References A. O. Al-Abbasi, A. Ghosh, and V. Aggarwal (2019) Deeppool: distributed model-free algorithm for ride-sharing using deep reinforcement learning. IEEE Transactions on Intelligent Transportation Systems 20 (12), p. 4714–4727. Cited by: §A.2, §4.2. J. Alonso-Mora, S. Samaranayake, A. Wallar, E. Frazzoli, and D. Rus (2017a) On-demand high-capacity ride-sharing via dynamic trip-vehicle assignment. Proceedings of the National Academy of Sciences 114 (3), p. 462–467. Cited by: §A.1, §1. J. Alonso-Mora, A. Wallar, and D. Rus (2017b) Predictive routing for autonomous mobility-on-demand systems with ride-sharing. In IEEE/RSJ International Conference on Intelligent Robots and Systems, p. 3583–3590. Cited by: §A.1. M. Behrisch, L. Bieker, J. Erdmann, and D. Krajzewicz (2011) SUMO–simulation of urban mobility: an overview. In Proceedings of SIMUL 2011, the third international conference on advances in system simulation, Cited by: Appendix D. F. Berto, C. Hua, J. Park, L. Luttmann, Y. Ma, F. Bu, J. Wang, H. Ye, M. Kim, S. Choi, et al. (2025) Rl4co: an extensive reinforcement learning for combinatorial optimization benchmark. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, p. 5278–5289. Cited by: §A.3. G. Brockman, V. Cheung, L. Pettersson, J. Schneider, J. Schulman, J. Tang, and W. Zaremba (2016) Openai gym. arXiv preprint arXiv:1606.01540. Cited by: §A.3, §1, §1, §3.2. T. Chen, Y. Wang, H. Chen, Z. Zhao, X. Li, N. Piovesan, G. Zhu, and Q. Shi (2025a) Modelling the 5g energy consumption using real-world data: energy fingerprint is all you need. In 2025 IEEE Globecom Workshops (GC Wkshps), Vol. , p. 1675–1680. External Links: Document Cited by: 4th item. W. Chen, H. Shi, and J. Ke (2025b) HRSim: an agent-based simulation platform for high-capacity ride-sharing services. arXiv preprint arXiv:2505.17758. Cited by: §A.3, Table 2, §1. Z. Chen (2023) Understanding of didi’s trading strategy: driver order matching. External Links: Link Cited by: §4.1. J. Cordeau and G. Laporte (2007) The dial-a-ride problem: models and algorithms. Annals of Operations Research 153 (1), p. 29–46. Cited by: §A.1. T. Enders, J. Harrison, M. Pavone, and M. Schiffer (2023) Hybrid multi-agent deep reinforcement learning for autonomous mobility on demand systems. In Learning for Dynamics and Control Conference, p. 1284–1296. Cited by: §A.2, Appendix D, §4.2. R. Engelhardt, F. Dandl, A. Syed, Y. Zhang, F. Fehn, F. Wolf, and K. Bogenberger (2022) Fleetpy: a modular open-source simulation tool for mobility on-demand services. arXiv preprint arXiv:2207.14246. Cited by: §A.3, Table 2, §1. S. Feng, T. Chen, Y. Zhang, J. Ke, Z. Zheng, and H. Yang (2024) A multi-functional simulation platform for on-demand ride service operations. Communications in Transportation Research 4, p. 100141. Cited by: §A.3, Table 2. D. Gale and L. S. Shapley (1962) College admissions and the stability of marriage. The American mathematical monthly 69 (1), p. 9–15. Cited by: 4th item, §4.1. S. Ge, X. Zhou, and T. Qiu (2025) Marl-based pricing strategy via mutual attention for mod systems with ridesharing and repositioning. In IEEE INFOCOM 2025-IEEE Conference on Computer Communications, p. 1–10. Cited by: §1. M. Haklay and P. Weber (2008) Openstreetmap: user-generated street maps. IEEE Pervasive computing 7 (4), p. 12–18. Cited by: §3.1. J. Hao and P. Varakantham (2022) Hierarchical value decomposition for effective on-demand ride-pooling. In Proceedings of the 21st International Conference on Autonomous Agents and Multiagent Systems, p. 580–587. Cited by: §A.2, Appendix D. J. Holder, N. Jaques, and M. Mesbahi (2025) Multi agent reinforcement learning for sequential satellite assignment problems. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 39, p. 26516–26524. Cited by: 1st item, §4.1. H. Hoppe, T. Enders, Q. Cappart, and M. Schiffer (2024) Global rewards in multi-agent deep reinforcement learning for autonomous mobility on demand systems. In 6th Annual Learning for Dynamics & Control Conference, p. 260–272. Cited by: §A.2, §4.2. Y. Hu, T. Dong, and S. Li (2025a) Coordinating ride-pooling with public transit using reward-guided conservative q-learning: an offline training and online fine-tuning reinforcement learning framework. Transportation Research Part C: Emerging Technologies 174, p. 105051. Cited by: Appendix D. Y. Hu, S. Feng, and S. Li (2025b) Bmg-q: localized bipartite match graph attention q-learning for ride-pooling order dispatch. IEEE Transactions on Intelligent Transportation Systems. Cited by: §A.2, 2nd item, Appendix D, §3.1, §3.1, §4.1, §4.2, §4.3. C. A. Hurkens and G. J. Woeginger (2004) On the nearest neighbor rule for the traveling salesman problem. Operations Research Letters 32 (1), p. 1–4. Cited by: §3.1. S. T. Jin, H. Kong, R. Wu, and D. Z. Sui (2018) Ridesourcing, the sharing economy, and the future of cities. Cities 76, p. 96–104. Cited by: §1. W. Jin, H. Du, B. Zhao, X. Tian, B. Shi, and G. Yang (2025) A comprehensive survey on multi-agent cooperative decision-making: scenarios, approaches, challenges and perspectives. arXiv preprint arXiv:2503.13415. Cited by: §A.2. B. Kalyanasundaram and K. Pruhs (1993) Online weighted matching. Journal of Algorithms 14 (3), p. 478–488. Cited by: 2nd item, §4.1. R. Kucharski and O. Cats (2022) Simulating two-sided mobility platforms with maassim. Plos one 17 (6), p. e0269682. Cited by: §A.3, Table 2, §1. H. W. Kuhn (1955) The hungarian method for the assignment problem. Naval research logistics quarterly 2 (1-2), p. 83–97. Cited by: 3rd item, §4.1. M. Laupichler, R. Andre, K. Kandler, P. Sanders, and P. Vortisch (2026) Advancing dynamic ride-pooling simulation–a highly scalable dispatcher. arXiv preprint arXiv:2605.11798. Cited by: §A.3, Table 2. M. Li, Z. Qin, Y. Jiao, Y. Yang, J. Wang, C. Wang, G. Wu, and J. Ye (2019) Efficient ridesharing order dispatching with mean field multi-agent reinforcement learning. In The World Wide Web Conference, p. 983–994. Cited by: §A.2, 3rd item, §4.1. K. Lin, R. Zhao, Z. Xu, and J. Zhou (2018) Efficient large-scale fleet management via multi-agent deep reinforcement learning. In Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, p. 1774–1783. Cited by: §A.3, Table 2, §1. M. L. Littman (1994) Markov games as a framework for multi-agent reinforcement learning. In Machine learning proceedings 1994, p. 157–163. Cited by: §2. Y. Liu and S. Li (2025) Piggyback on idle ride-sourcing drivers for integrated on-demand and flexible intracity parcel delivery services. Transportation Science 59 (3), p. 494–517. Cited by: Appendix D. Y. Liu, Y. Shang, and S. Li (2025) Joint infrastructure planning and order assignment for on-demand food-delivery services with coordinated drones and human couriers. arXiv preprint arXiv:2501.14325. Cited by: Appendix D. P. A. Lopez, M. Behrisch, L. Bieker-Walz, J. Erdmann, Y. Flötteröd, R. Hilbrich, L. Lücken, J. Rummel, P. Wagner, and E. Wießner (2018) Microscopic traffic simulation using sumo. In 2018 21st international conference on intelligent transportation systems (ITSC), p. 2575–2582. Cited by: §A.3. D. Luxen and C. Vetter (2011) Real-time routing with openstreetmap data. In Proceedings of the 19th ACM SIGSPATIAL International Conference on Advances in Geographic Information Systems, GIS ’11, New York, NY, USA, p. 513–516. External Links: ISBN 978-1-4503-1031-4, Link, Document Cited by: §3.1. T. Oda and C. Joe-Wong (2018) MOVI: a model-free approach to dynamic fleet management. In IEEE INFOCOM 2018-IEEE Conference on Computer Communications, p. 2708–2716. Cited by: §A.3, Table 2. Z. Qin, X. Tang, Y. Jiao, F. Zhang, Z. Xu, H. Zhu, and J. Ye (2020) Ride-hailing order dispatching at didi via reinforcement learning. INFORMS Journal on Applied Analytics 50 (5), p. 272–286. Cited by: §A.2. C. Riley, P. Van Hentenryck, and E. Yuan (2021) Real-time dispatching of large-scale ride-sharing systems: integrating optimization, machine learning, and model predictive control. In Proceedings of the Twenty-Ninth International Conference on International Joint Conferences on Artificial Intelligence, p. 4417–4423. Cited by: §A.1. S. Shaheen and A. Cohen (2019) Shared ride services in north america: definitions, impacts, and the future of pooling. Transport reviews 39 (4), p. 427–442. Cited by: §1. A. Simonetto, J. Monteil, and C. Gambella (2019) Real-time city-scale ridesharing via linear assignment problems. Transportation Research Part C: Emerging Technologies 101, p. 208–232. Cited by: §A.1. J. Sun, H. Jin, Z. Yang, L. Su, and X. Wang (2022) Optimizing long-term efficiency and fairness in ride-hailing via joint order dispatching and driver repositioning. In Proceedings of the 28th ACM SIGKDD conference on knowledge discovery and data mining, p. 3950–3960. Cited by: §1. S. Y. Tan, H. Fibrianto, and L. Lin (2025) DispatchGym: grab’s reinforcement learning research framework. Note: https://engineering.grab.com/techblog_-dispatchgym Cited by: §A.3. X. Tang, Z. Qin, F. Zhang, Z. Wang, Z. Xu, Y. Ma, H. Zhu, and J. Ye (2019) A deep value-network based approach for multi-driver order dispatching. In Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, p. 1780–1790. Cited by: 5th item, §4.1. N. Y. C. Taxi and L. Commission (2024) Nyc taxi and limousine commission-trip record data nyc.. External Links: Link Cited by: §4.2. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Lio, and Y. Bengio (2017) Graph attention networks. arXiv preprint arXiv:1710.10903. Cited by: 2nd item. C. Wang, Y. Ding, and J. Jiang (2025) On-demand dynamic intercity shared-taxi system with re-optimization. IEEE Transactions on Intelligent Transportation Systems. Cited by: §3.1. Z. Xu, Z. Li, Q. Guan, D. Zhang, Q. Li, J. Nan, C. Liu, W. Bian, and J. Ye (2018) Large-scale order dispatch in on-demand ride-hailing platforms: a learning and planning approach. In Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, p. 905–913. Cited by: §A.2, §C.1. S. Yan, C. Chen, and Y. Lin (2011) A model with a heuristic algorithm for solving the long-term many-to-many car pooling problem. IEEE Transactions on Intelligent Transportation Systems 12 (4), p. 1362–1373. Cited by: §A.1. G. Yang, H. Zhang, and L. Qiu (2026) Graph-based multi-agent reinforcement learning with an enriched environment for joint ride-sharing and charging optimization. Applied Energy 405, p. 127220. Cited by: §1. X. Yue, Y. Liu, F. Shi, S. Luo, C. Zhong, M. Lu, and Z. Xu (2024) An end-to-end reinforcement learning based approach for micro-view order-dispatching in ride-hailing. In Proceedings of the 33rd ACM international conference on information and knowledge management, p. 5054–5061. Cited by: 4th item, §4.1. Z. Zhao, T. Chen, Z. Cai, X. Li, H. Li, Q. Chen, and G. Zhu (2025) Crossfi: a cross domain wi-fi sensing framework based on siamese network. IEEE Internet of Things Journal. Cited by: 4th item. Z. Zhao and S. Li (2025a) One step is enough: multi-agent reinforcement learning based on one-step policy optimization for order dispatch on ride-sharing platforms. arXiv preprint arXiv:2507.15351. Cited by: Table 2, §3.1. Z. Zhao and S. Li (2025b) The impacts of data privacy regulations on food-delivery platforms. Transportation Research Part C: Emerging Technologies 181, p. 105364. Cited by: Table 2, 4th item, §4.1, §4.2. Z. Zhao and S. Li (2026a) Discriminatory order assignment and payment-setting of on-demand food-delivery platforms: a multi-action and multi-agent reinforcement learning framework. Transportation Research Part E: Logistics and Transportation Review 208, p. 104653. Cited by: Table 2, 4th item, §4.1. Z. Zhao and S. Li (2026b) Triple-BERT: do we really need MARL for order dispatch on ride-sharing platforms?. In The Fourteenth International Conference on Learning Representations, Cited by: Table 2, §C.1. Appendix A Literature Reviews A.1. Model-Based Order Dispatch Order dispatch has traditionally been cast as a combinatorial optimization problem and solved with model-based methods. Early formulations assumed that requests were fully known ahead of time and matched trips to vehicles by solving linear assignment programs (Yan et al., 2011; Cordeau and Laporte, 2007). An influential line of work is the Request-Trip-Vehicle (RTV) framework of Alonso-Mora et al. (2017a), which enumerates feasible order bundles, links them to compatible vehicles with cost-weighted edges, and obtains a cost-minimizing assignment by solving a bipartite matching problem, together with a demand-driven vehicle rebalancing step. Because constructing and solving the RTV program is expensive at scale, subsequent studies traded optimality for speed, e.g., restricting each vehicle to at most one new request per epoch and relying on implicit bundling of en-route and incoming orders (Simonetto et al., 2019). To counter the myopia of one-shot matching, later methods incorporated future information like demand forecasts appended to the assignment graph (Alonso-Mora et al., 2017b), rolling-horizon and model-predictive control for joint relocation and dispatch (Riley et al., 2021), and stochastic-programming formulations under demand uncertainty. A.2. Reinforcement Learning for Order Dispatch The complexity of modeling stochastic mobility systems has motivated model-free reinforcement learning (RL), which learns dispatch policies directly from interaction and implicitly captures uncertainty and long-horizon effects. Xu et al. (2018) first scaled RL to ride-hailing by learning a per-vehicle value function and recovering the assignment through global bipartite matching on Q-value-weighted edges; many works inherit this “learn-a-value-then-match” paradigm (Qin et al., 2020; Hu et al., 2025b). Owing to the high-dimensional joint state-action space, most methods adopt a MARL formulation, commonly grouped into Decentralized Training with Decentralized Execution (DTDE), Centralized Training with Decentralized Execution (CTDE), and Centralized Training with Centralized Execution (CTCE) (Jin et al., 2025). DTDE methods treat every vehicle as an independent learner (Al-Abbasi et al., 2019), which is simple but suffers from non-stationarity and weak coordination; neighbor-aware encoders such as graph attention (Hu et al., 2025b) and mean-field approximations (Li et al., 2019) partially alleviate this. CTDE and CTCE methods instead pursue stronger cooperation via centralized critics, global rewards, and value decomposition, adapting architectures such as MASAC (Enders et al., 2023), COMA (Hoppe et al., 2024), and QMIX (Hao and Varakantham, 2022) to the dispatch setting. A.3. Simulation Platforms Unlike domains such as traffic signal control or combinatorial optimization, where shared environments and benchmarks have accelerated progress (e.g., Gym-style RL interfaces (Brockman et al., 2016), microscopic traffic simulators (Lopez et al., 2018), and unified routing benchmarks such as RL4CO (Berto et al., 2025)), ride-sharing order dispatch still lacks a comparably standardized evaluation substrate. This is not for want of simulators: a number of open-source platforms exist, but each misses at least one property required to serve as a standardized (MA)RL dispatch benchmark, as summarized in Table 2. Broadly, these platforms fall into two families. Operations-oriented simulators are built to evaluate system-level strategies such as pricing, matching, and repositioning. HRSim (Chen et al., 2025b) models high-capacity pooling on real road networks with a focus on pricing and emission studies; FleetPy (Engelhardt et al., 2022) provides a mature modular pipeline with heterogeneous fleets and ILP-based pooling; MaaSSim (Kucharski and Cats, 2022) targets two-sided market and behavioral dynamics; and the dispatcher of Laupichler et al. (2026) pushes raw scalability to tens of thousands of vehicles. These platforms are faithful and mature, but they expose no standardized reset/step interface and are tightly coupled to a specific control pipeline, so plugging in and fairly comparing (MA)RL policies requires substantial re-engineering. Learning-oriented simulators, in contrast, are usually released as a byproduct of a particular algorithm: MOVI (Oda and Joe-Wong, 2018) and the widely used DiDi environment (Lin et al., 2018) are each tied to a specific method behind a bespoke interface, and the latter further abstracts the city into a grid/hex world rather than a real road network, limiting fidelity and reusability. The closest platform in spirit is the multi-functional simulator of Feng et al. (2024), which exposes RL “portals” for ride-sourcing operations, yet it offers bespoke interfaces rather than a standardized, algorithm-agnostic Gym API and targets single-occupancy ride-sourcing rather than order bundling. Finally, Grab has reported a Gym-style dispatch environment (Tan et al., 2025), but it remains closed-source, with no code or interface released more than a year after the initial report. These observations expose a precise gap: the field has many dispatch algorithms and several faithful simulators, yet none offers a standardized, reusable, algorithm-agnostic environment on which decentralized and centralized methods can be trained and compared under identical conditions. RideGym fills exactly this gap. It (i) supports both abstract (Euclidean/Manhattan) and real OpenStreetMap-based road networks with precomputed shortest paths for fast, system-independent distance queries; (i) models heterogeneous fleets (per-vehicle speeds and capacities), multiple passengers per order, user-supplied demand datasets, and configurable rewards; and (i) exposes a standardized, algorithm-agnostic, Gym-like reset/step interface, so that model-based and (MA)RL methods alike can be benchmarked reproducibly. As Table 2 shows, RideGym is the only environment that satisfies all of these requirements simultaneously. Table 2. Comparison of open-source ride-sharing / ride-hailing simulators against the requirements for a standardized (MA)RL order-dispatch benchmark. ✓: supported; ×: not supported; ∼ : partial. Simulator Primary purpose Real road network Hetero. fleet Ride pooling Multi-pax per order Gym-style API Algorithm -agnostic DiDi env. (Lin et al., 2018) Fleet rebalancing × × × × × × MOVI (Oda and Joe-Wong, 2018) Dispatch & repositioning ✓ × × × × × MaaSSim (Kucharski and Cats, 2022) Two-sided market ✓ ∼ ✓ × × × FleetPy (Engelhardt et al., 2022) MoD operations ✓ ✓ ✓ × ∼ × Laupichler et al. (2026) Large-scale dispatch ✓ × ✓ × × × HRSim (Chen et al., 2025b) Ride-sharing ops (matching, pricing) ✓ ∼ ✓ × × × Feng et al. (2024) Ride-sourcing RL testbed ✓ × × × × ∼ Triple-BERT (Zhao and Li, 2026b) RL order dispatch ✓ × ✓ × × × OSPO (Zhao and Li, 2025a) MARL order dispatch ✓ × ✓ × × × Zhao et al. (Zhao and Li, 2025b, 2026a) Food delivery dispatch & payment ✓ × ✓ × × × RideGym (ours) MARL dispatch benchmark ✓ ✓ ✓ ✓ ✓ ✓ Appendix B Visualization Example Figure 3. A single-frame visualization of three focused vehicles under the KM dispatch baseline. Each vehicle is drawn in its own color, the disc marks its current location, the solid line its planned route along the real road network, and the up/down triangles the origin (lol^o) and destination (ldl^d) of each order it is currently serving, joined by a dashed origin-destination link. Fig. 3 shows a representative single frame, focusing on three vehicles served by the KM dispatch baseline. To keep the picture legible, each vehicle and its associated elements share a unique color, while the marker shape encodes semantics: a filled disc denotes the vehicle’s current location li,tpl^p_i,t, an up-triangle an order origin lol^o, and a down-triangle an order destination ldl^d. For every vehicle, a solid line traces its planned route along the real road network, computed by the route planner of Section 3.1 under the pickup-before-drop-off constraint, so one can directly read off the sequence of stops it will visit. For each order the vehicle is currently carrying or heading to pick up, a dashed line links that order’s origin and destination, making the demand each vehicle is committed to immediately apparent. Together, color (which vehicle), shape (vehicle, origin, or destination), and the two line styles (planned route vs. origin-destination link) allow a reader to quickly disentangle which vehicle is serving which orders and along what path. Appendix C Experiment Details C.1. Benchmark Methods Introduction For model-based methods, we select the following baselines: • Random: The simplest baseline, which randomly assigns orders to vehicles located within the same region. In our experiments, the region is defined as a circle with a radius of 1 km. • Greedy (Kalyanasundaram and Pruhs, 1993): A fast yet effective heuristic that sequentially assigns each order to the nearest available vehicle. • Kuhn–Munkres (KM) (Kuhn, 1955): A bipartite matching-based dispatch method that minimizes the total matching distance between vehicles and orders. • Gale–Shapley (GS) (Gale and Shapley, 1962; Yue et al., 2024): An online matching algorithm used in Didi, where driver preferences over orders are based on order price (proportional to distance and passenger count), and order preferences over drivers are based on current distance (related to waiting time). For MARL-based methods, we select the following order dispatch approached: • REDA (Holder et al., 2025): A representative IDDQN-based baseline adapted for order dispatch, which employs an MLP to estimate the Q-value for each vehicle-order pair and subsequently applies bipartite matching to maximize the total Q-value under the independent learning assumption. • BMG-Q (Hu et al., 2025b): Building on IDDQN, BMG-Q incorporates neighborhood information via a modified Graph Attention Network (GAT) (Veličković et al., 2017). Following the original paper, we set the neighborhood size to 30. • MF-DDQN (Li et al., 2019): Similar to BMG-Q, MF-DDQN captures neighborhood behavior using a mean-field action. We follow the original implementation by setting the neighborhood size to 30 and using the average order information as the mean-field action. • Assignment-Net (Zhao and Li, 2025b, 2026a): A network architecture specifically designed for order dispatch, which uses an LSTM to encode sequential information and an MLP with ARL (Chen et al., 2025a) enhancement to encode non-sequential features. A QK-Attention (Zhao et al., 2025) module is then employed to compute the Q-value for each vehicle-order pair via a mutual attention mechanism. • CV-Net (Tang et al., 2019): Designed for large-scale ride-hailing tasks, CV-Net introduces a cerebellar embedding module to learn informative region representations across multiple scales, in contrast to directly encoding raw coordinate values. Specifically, to align with our state space, we use three cerebellar embeddings for location encoding at different scales, and employ a linear layer to encode other information. In our implementation, both Assignment-Net and CV-Net adopt the same training paradigm as IDDQN, with only the network architecture being modified. For these two methods, our primary goal is to examine the impact of architectural design on performance; therefore, we omit the Lipschitz normalization loss originally proposed in CV-Net, ensuring that all compared benchmarks share an identical loss function for a fair comparison. We note that some recent SOTA methods and classical algorithms are omitted from our benchmark for the following reasons: (i) The primary contribution of this paper is the design of a standardized Gym-like environment, rather than an exhaustive benchmark evaluation. (i) Certain classical methods, such as (Xu et al., 2018), rely on policy iteration and require frequent reward function evaluations. While this is feasible in ride-hailing settings, it becomes computationally expensive in ride-sharing scenarios, where the reward function is significantly more complex due to the joint consideration of immediate income and future detour costs. (i) Recent SOTA methods, such as Triple-BERT (Zhao and Li, 2026b), challenge the conventional MARL paradigm by proposing a centralized Single-Agent RL (SARL) solution. However, as the mainstream paradigm in this domain remains MARL-based, we defer the support for SARL algorithms to future work. C.2. Evaluation Metrics We evaluate all methods using the following four metrics: • Service Rate: The proportion of orders that are successfully assigned to a vehicle, computed as the number of dispatched orders divided by the total number of requested orders. A higher service rate indicates that more passenger demand is being served. • Completion Rate: The proportion of orders that are not only assigned but also finished within the episode, computed as the number of completed trips divided by the total number of requested orders. It reflects the platform’s end-to-end ability to actually fulfill demand rather than merely matching it. • Wait Time: The average time elapsed between an order being requested and the assigned vehicle arriving at the pickup location. A shorter wait time corresponds to a better passenger pickup experience. • Detour Time: The average extra travel time incurred relative to the shortest path from origin to destination, caused by pooling detours. A shorter detour time indicates more efficient trip execution and lower passenger inconvenience. • Simulation Time: The average runtime per episode, capturing the computational cost of each method. Appendix D Discussions As the first open-source, standardized Gym-style simulator tailored to (MA)RL-based ride-sharing order dispatch, RideGym strives to support a wide range of functionalities—such as personalized vehicle settings and multiple passengers per order—aligning with the experimental setups of most existing works (Hu et al., 2025b; Enders et al., 2023; Hao and Varakantham, 2022). Nevertheless, several promising directions remain for future extensions. First, to enhance realism, the simulator could incorporate dynamic traffic conditions (e.g., congestion) and emergency scenarios, possibly by integrating with microscopic traffic simulators such as SUMO (Behrisch et al., 2011). Second, the framework can be extended to jointly address other operational tasks beyond order dispatch, including vehicle relocation, charging/fueling scheduling, dynamic decision windows, and pricing (to passengers) and payment (to drivers) mechanisms. Furthermore, multi-modal transportation options, such as taxi-subway integration (Hu et al., 2025a) and land-air mobility (Liu et al., 2025), could also be considered. Given the structural similarity between ride-sharing and logistics or food delivery systems, our simulator could potentially be adapted to support these tasks or even hybrid human-goods transportation scenarios (Liu and Li, 2025). From the perspective of academic community development, we strongly encourage researchers to open-source their code and report detailed experimental configurations. In our benchmark study, we observe that many methods are sensitive to hyper-parameters, like the neighborhood size in BMG-Q and MF-DDQN, and the cerebellar resolution in CV-Net, making it impractical for a single research group to thoroughly reproduce and tune all such parameters across multiple baselines. Moreover, our experimental findings reveal that the choice of exploration noise can significantly affect the performance of MARL-based solutions, a factor often overlooked in prior work. This raises important questions: How can we systematically design suitable exploration noise rather than rely on heuristic choices? Is the optimal noise type consistent across different methods? More critically, different noise types can alter the relative performance ordering of methods, confounding the effect of algorithmic innovations and complicating fair comparison. These observations highlight the need for standardized evaluation protocols that mitigate the confounding influence of noise design, and we hope our work serves as a step toward more reproducible and equitable benchmarking practices.