Paper deep dive
Deadline-Aware Multi-Agent Reinforcement Learning for TSN-Based Vehicular Edge Networks
Bernardo A. C. Pereira, Marcos Carvalho, Fatih Temiz, Shavbo Salehi, Melike Erol-Kantarci, Andreas Gavrielides, Johann M. Marquez-Barja, Daniel F. Macedo
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 10/11/2026, 3:23:10 AM
Summary
This paper proposes a Multi-Agent Proximal Policy Optimization (MAPPO) framework for queue-level scheduling in Time-Sensitive Networking (TSN)-enabled Vehicular Edge Computing (VEC) networks. The approach assigns an autonomous agent to each TSN queue to jointly learn service order and time-slot duration, aiming to minimize deadline misses for latency-sensitive Extended Reality (XR) applications under dynamic vehicle speeds. The method utilizes Centralized Training with Decentralized Execution (CTDE) to handle inter-queue dependencies. Evaluation demonstrates that MAPPO significantly reduces service latency and improves reliability compared to single-agent and heuristic baselines.
Entities (10)
Relation Signals (9)
MAPPO → usedfor → TSN Scheduling
confidence 95% · we propose a multi-agent reinforcement learning (MARL) approach for queue-level scheduling in TSN-enabled VEC... We employ multi-agent proximal policy optimization (MAPPO)
MAPPO → employs → CTDE
confidence 94% · MAPPO [9], which employs centralized training with decentralized execution (CTDE).
MAPPO → improves → Reliability
confidence 93% · and improves reliability by up to 271.8%.
MAPPO → reduces → service latency
confidence 93% · Compared with centralized single-agent methods, it reduces service latency by up to 66.2%
MAPPO → outperforms → single-agent methods
confidence 92% · Compared with centralized single-agent methods, it reduces service latency by up to 66.2%
TSN → provides → bounded-latency communication
confidence 92% · While time-sensitive networking (TSN) provides bounded-latency communication
VEC → enables → latency-sensitive applications
confidence 91% · Vehicular edge computing (VEC) enables latency-sensitive applications by bringing computing and networking resources closer to vehicles.
SeAR → hastighterdeadlinethan → AR
confidence 90% · For SeAR applications, we consider a tighter base deadline of 2 ms... Du,fSeAR<Du,fAR
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Vehicular edge computing (VEC) enables latency-sensitive applications by bringing computing and networking resources closer to vehicles. However, existing approaches often overlook network contention among co-located services with heterogeneous and dynamic latency requirements. While time-sensitive networking (TSN) provides bounded-latency communication, conventional and reinforcement learning-based schedulers struggle to adapt to highly dynamic vehicular environments and inter-queue dependencies. To address these limitations, we propose a multi-agent reinforcement learning (MARL) approach for queue-level scheduling in TSN-enabled VEC. Each TSN queue is assigned an autonomous agent that jointly learns the queue service order and time-slot duration to minimize deadline misses under speed-dependent latency requirements. We employ multi-agent proximal policy optimization (MAPPO) to enable coordinated yet autonomous scheduling decisions. Evaluation against single-agent, multi-agent, and non-learning-based baselines shows that MAPPO provides robust performance across different traffic profiles. Compared with centralized single-agent methods, it reduces service latency by up to 66.2% and improves reliability by up to 271.8%. Furthermore, unlike urgency-based heuristics, MAPPO ensures balanced scheduling while achieving lower inference times compared to other MARL methods.
Tags
Links
- Source: https://arxiv.org/abs/2610.09870v1
- Canonical: https://arxiv.org/abs/2610.09870v1
Trouble viewing inline? Open PDF directly →
Full Text
35,383 characters extracted from source content.
Expand or collapse full text
Deadline-Aware Multi-Agent Reinforcement Learning for TSN-Based Vehicular Edge Networks Bernardo A. C. Pereira1, Marcos Carvalho1, Fatih Temiz2, Shavbo Salehi2, Melike Erol-Kantarci2, Fellow, IEEE, Andreas Gavrielides3, Johann M. Marquez-Barja3, Daniel F. Macedo1 Affiliation: 1Universidade Federal de Minas Gerais, Brazil 2School of Electrical Engineering and Computer Science, University of Ottawa, Ottawa, Canada 3University of Antwerp - imec, IDLab - Faculty of Applied Engineering, Belgium E-mails: bernardoalvescosta10@gmail.com,marcoscarvalho, damacedo@dcc.ufmg.br Affiliation: ftemi033, ssale038, melike.erolkantarci@uottawa.ca andreas.gavrielides, johann.marquez-barja@imec.be Abstract Vehicular edge computing (VEC) enables latency-sensitive applications by bringing computing and networking resources closer to vehicles. However, existing approaches often overlook network contention among co-located services with heterogeneous and dynamic latency requirements. While time-sensitive networking (TSN) provides bounded-latency communication, conventional and reinforcement learning-based schedulers struggle to adapt to highly dynamic vehicular environments and inter-queue dependencies. To address these limitations, we propose a multi-agent reinforcement learning (MARL) approach for queue-level scheduling in TSN-enabled VEC. Each TSN queue is assigned an autonomous agent that jointly learns the queue service order and time-slot duration to minimize deadline misses under speed-dependent latency requirements. We employ multi-agent proximal policy optimization (MAPPO) to enable coordinated yet autonomous scheduling decisions. Evaluation against single-agent, multi-agent, and non-learning-based baselines shows that MAPPO provides robust performance across different traffic profiles. Compared with centralized single-agent methods, it reduces service latency by up to 66.2% and improves reliability by up to 271.8%. Furthermore, unlike urgency-based heuristics, MAPPO ensures balanced scheduling while achieving lower inference times compared to other MARL methods. Index Terms: Extended Reality, Multi-agent Reinforcement Learning, Time-Sensitive Networking, Vehicular Edge Computing I Introduction Vehicular edge computing (VEC) has emerged as a key infrastructure for enabling a wide range of vehicular applications, including autonomous driving support and immersive applications such as augmented and virtual reality (AR/VR) [1]. These emerging applications introduce stringent quality of service requirements, demanding ultra-low latency and high reliability to ensure responsive operation. To meet these requirements, VEC deploys computing and networking resources at roadside units (RSUs), bringing them closer to vehicles [2]. In this architecture, RSUs can cache and deliver application content locally, improving service responsiveness. While existing VEC research primarily focuses on managing computational resources at edge servers, it generally assumes that edge servers operate under a best-effort Ethernet communication model. Time-sensitive networking (TSN) has become a key technology for supporting latency-sensitive services over Ethernet networks. Developed by the IEEE 802.1 working group, TSN comprises a set of standards that provide bounded latency and high reliability for time-critical services [3]. Among these mechanisms, the IEEE 802.1Qbv Time-Aware Shaper (TAS) enables time-based traffic scheduling, allowing the management of co-located latency-sensitive flows [4]. Nevertheless, enabling the management of mixed deadline-critical traffic in highly dynamic edge environments remains an open challenge [5]. This challenge becomes even more complex in vehicular networks due to the high mobility of vehicles. To mitigate this issue, we argue that the latency budget for critical services at the edge server must be tightened as vehicle speeds increase, ensuring that packets can be successfully delivered before connectivity degradation occurs. In this context, conventional TSN schedulers based on optimization techniques, such as satisfiability modulo theories and integer linear programming, face challenges in rapidly adapting schedules to highly dynamic environments [6]. Conversely, reinforcement learning (RL) has been used for TSN scheduling by enabling adaptive decision-making through continuous interaction with the environment. Nevertheless, existing studies primarily optimize intermediate network devices while overlooking the traffic sources. Furthermore, existing RL-based approaches rely on a centralized scheduling policy that determines the transmission schedule for all queues, rather than allowing each queue to make autonomous scheduling decisions [7]. As a result, they are limited in capturing inter-queue dependencies in dynamic environments, particularly when latency deadlines vary over time. Figure 1: Overall system model. To address these limitations, inspired by the approach in [8], we propose a multi-agent reinforcement learning (MARL) approach in which each TSN queue is assigned an autonomous agent. The agents learn scheduling policies that jointly determine the service order and time-slot duration of each queue to minimize deadline misses. We also leverage multi-agent proximal policy optimization (MAPPO) [9], which employs centralized training with decentralized execution (CTDE). During training, the centralized critic exploits global state information to produce more informative policy updates, encouraging coordinated scheduling decisions while enabling decentralized execution. We evaluate the effectiveness of the proposed MARL approach against centralized single-agent and non-learning-based baselines. The results demonstrate that MAPPO outperforms the considered approaches, reducing service latency by up to 66.2% and improving reliability by up to 271.8%, while achieving lower inference times than the other MARL methods. I Related Works The authors in [10] employ proximal policy optimization (PPO) to dynamically optimize scheduling parameters, thereby reducing end-to-end latency in industrial network environments. Despite its effectiveness in ensuring bounded latency, the proposed approach is evaluated in a limited scenario with only two queues. This limitation may hinder its scalability and effectiveness in more complex network environments, where scheduling decisions for one queue can directly influence the performance of others. Conversely, the authors in [11] introduce a deep deterministic policy gradient (DDPG)-based method for adapting time-slot durations across multiple queues, aiming to mitigate the impact of transmission failures caused by external factors (i.e., time-synchronization errors) on application latency. Although the approach ensures bounded latency for critical applications, it does not explicitly capture the interdependence among queues, which may limit its ability to coordinate resource allocation in highly dynamic network environments with mixed and heterogeneous latency requirements. Recent studies have explored MARL solutions with different agent designs to address the complexity of TSN scheduling. For instance, [12] divides the scheduling problem into dedicated routing and timing agents, enabling adaptive traffic management at the device level. Likewise, [13] introduces a multi-agent architecture within a centralized control framework, in which routing and scheduling agents collaborate to minimize latency in dynamic network environments. However, these approaches focus on network-level coordination rather than cooperative optimization of individual host-level queues. Our work learns coordinated scheduling policies that account for queue states, heterogeneous latency requirements, and dynamic edge traffic. I System Model and Problem Formulation As illustrated in Figure 1, we consider RSUs co-located with MEC servers and deployed along a highway segment. Let =u1,u2,…,uNU=\u_1,u_2,…,u_N\ denote the set of N vehicles within the coverage range of the RSU, and let =v1,v2,…,vNV=\v_1,v_2,…,v_N\ denote their respective speeds. The MEC server hosts two classes of applications: traditional AR and semantic AR (SeAR) applications [14]. Unlike traditional AR, which transmits the entire video content, SeAR transmits compact data units containing masks that capture the semantic content of the video. This allows SeAR to convey the relevant semantic information while reducing the amount of data transmitted over the network. Furthermore, to ensure strict temporal isolation, the server is equipped with a TSN-capable network interface card (NIC) with a set of queues, denoted as =q0,q1,…,qMQ=\q_0,q_1,…,q_M\, where each XR application is mapped to a dedicated queue. Since our focus is on queue-level scheduling at the MEC layer, we model only the downstream traffic generated by the considered applications at the edge server. I-A XR Traffic, Latency and Dynamic Deadline Model XR Traffic We adopt the statistical traffic model for XR applications presented in [14] and incorporate the protocol data unit (PDU)-set concept as in [15]. The traffic stream for each vehicle un∈u_n is modeled as a sequence of PDU-sets, where each PDU-set is indexed by f∈ℱunf _u_n and consists of a group of closely related PDUs that carry a single application-level payload. The individual PDUs within a given PDU-set are indexed by j∈un,fj _u_n,f. The temporal characteristics of the XR traffic are determined by the arrival times of PDU-sets and their constituent PDUs. Let tun,f,qmt_u_n,f,q_m denote the arrival timestamp of PDU-set f at TSN queue qmq_m of vehicle unu_n. Each PDU-set consists of NIPfN_IP^f PDUs, where tun,f,j,qmt_u_n,f_,j,q_m denotes the arrival timestamp of PDU j within PDU-set f at the queue qmq_m. The PDUs are enqueued at the TSN queue according to their arrival times and served following a first-in, first-out (FIFO) policy during each transmission opportunity. Latency Model PDUs experience queueing latency while waiting for a transmission opportunity. Let tun,f,j,qmst^s_u_n,f,j,q_m denote the service start time of PDU j belonging to PDU-set f of vehicle unu_n at queue qmq_m. To capture the worst-case waiting time at each queue, we define the age of the oldest PDU (AOPDU), representing the waiting time of the oldest PDU currently in queue qmq_m at observation time step t, denoted by gm(t)g_m(t). Let ℬm(t)B_m(t) denote the set of PDUs currently waiting in queue qmq_m. The AOPDU is given by gm(t)=maxj∈ℬm(t)(t−tun,f,j,qm),if ℬm(t)≠∅,0,otherwise.g_m(t)= cases _j _m(t) (t-t_u_n,f,j,q_m ),&if B_m(t)≠ ,\\[6.0pt] 0,&otherwise. cases (1) Due to the interdependencies among the PDUs within a PDU-set, the delayed delivery of a single PDU may prevent the timely reconstruction of an XR video frame or semantic mask. The total latency experienced by PDU-set f, denoted by Γun,f,qm _u_n,f,q_m, is defined as the time elapsed between the arrival of its first PDU at the queue qmq_m and the completion of its last PDU, expressed as Γun,f,qm=tun,f,qmc−tun,f,1,qm _u_n,f,q_m=t^c_u_n,f,q_m-t_u_n,f,1,q_m, where tun,f,qmct^c_u_n,f,q_m denotes the completion time of the last PDU in PDU-set f, and tun,f,1,qmt_u_n,f,1,q_m denotes the arrival time of its first PDU at queue qmq_m. Dynamic PDU-Set Deadline Model The sequence of PDU-sets ℱunF_u_n requested by vehicle unu_n is mapped to a tuple Φun,f=⟨srcun,dstun,NIPf,qm,Dun,f⟩ _u_n,f= src_u_n,dst_u_n,N_IP^f,q_m,D_u_n,f , where srcunsrc_u_n and dstun=undst_u_n=u_n denote the source RSU and destination vehicle, respectively, NIPfN_IP^f is the number of PDUs in PDU-set f, qmq_m is the associated TSN queue, and Dun,fD_u_n,f is its dynamic deadline. The deadline is defined as Dun,f=αDbaseD_u_n,f=α D_base, where DbaseD_base is the fixed base deadline of the corresponding XR application. For traditional AR applications, DbaseD_base is determined by the generation rate; for example, a 90 Hz application has a base deadline of 11.1111.11 ms. For SeAR applications, we consider a tighter base deadline of 22 ms due to the importance of timely semantic information. The speed-dependent scaling factor α is given by α=αmax−vavg−vavgminvavgmax−vavgmin(αmax−αmin).α= _ - v_avg-v_avg v_avg -v_avg ( _ - _ ). (2) where αmax _ and αmin _ are the maximum and minimum deadline adjustment factors, respectively, vavgv_avg denotes the average speed of U, and vavgminv_avg and vavgmaxv_avg denote the minimum and maximum average speeds considered. Thus, α dynamically adjusts the application deadline according to vehicle speed. As speed increases, the vehicle remains within the RSU coverage area for less time, reducing the time available for transmission. To satisfy the XR applications’ requirements, the total latency of each PDU-set must not exceed its corresponding dynamic deadline Dun,fD_u_n,f. Accordingly, we define the indicator of a successful on-time transmission of PDU-set f from vehicle unu_n as un,f=1,if Γun,f,qm≤Dun,f,0,otherwise.I_u_n,f= cases1,&if _u_n,f,q_m≤ D_u_n,f,\\ 0,&otherwise. cases (3) I-B TSN Schedule Model The network execution time is divided into fixed-duration scheduling cycles, indexed by c, with Δtcycle=1 t_cycle=1 ms. At the beginning of each cycle, a schedule ScS_c partitions the cycle into M time slots, where wc,mw_c,m denotes the duration (in μ ) allocated to queue qmq_m. The allocated durations satisfy ∑m=0Mwc,m=Δtcycle _m=0^Mw_c,m= t_cycle. The second step consists of determining the service order of the queues within the cycle. To govern this, we introduce a continuous priority score pc,k∈[0,1]p_c,k∈[0,1]. Queues are scheduled in descending order of their assigned priority score pc,mp_c,m. Furthermore, two or more queues may be assigned the same score, indicating equal urgency. In such cases, ties are resolved by granting higher precedence to queues with larger indices (i.e., qM>qM−1>⋯>q0q_M>q_M-1>…>q_0), reflecting standard hardware driver-level queue prioritization. Consequently, the resulting service order is defined as a permutation πc:1,…,M→0,…,M−1 _c:\1,…,M\→\0,…,M-1\, where πc(k) _c(k) denotes the queue index placed in the k-th position of the schedule. This is obtained by sorting the queues in descending order of pc,mp_c,m, and breaking ties by selecting the largest index m, as πc=argsortm∈0,…,M(−pc,m,−m) _c=argsort_m∈\0,…,M\(-p_c,m,-m). Given the formalization above, the final schedule ScS_c is defined as an ordered sequence of tuples, given by Sc=⟨(wc,πc(1),πc(1)),…,(wc,πc(M),πc(M))⟩S_c= (w_c, _c(1), _c(1)),…,(w_c, _c(M), _c(M)) , where wc,πc(m)w_c, _c(m) is the corresponding assigned time slice duration and πc(k) _c(k) denotes the index of the queue scheduled in the k-th position of the cycle. I-C Problem Formulation The primary objective of the scheduler is to maximize the aggregate number of on-time PDU-sets across all co-located, mixed-criticality XR applications. Let W denote the set of time slot duration decisions and P denote the set of priority scores. The scheduling problem is formulated as max, _W,P ∑u∈∑f∈ℱu,f _u _f _uI_u,f (4) s.t. ∑m=1Mwc,m=Δtcycle,∀c, _m=1^Mw_c,m= t_cycle, ∀ c, (5) Du,f≤Dbase,∀u,∀f, D_u,f≤ D_base, ∀ u,∀ f, (6) Du,fSeAR<Du,fAR,∀u. D_u,f^SeAR<D_u,f^AR, ∀ u. (7) Constraint (5) guarantees that the sum of all allocated time slots equals the predefined cycle duration Δtcycle t_cycle. Constraint (6) dictates that the dynamic deadline Dun,fD_u_n,f of any application cannot exceed its base deadline. Finally, constraint (7) strictly enforces the tighter delay requirements for SeAR applications compared to traditional AR applications. I-D MDP Formulation We formulate the TSN scheduling problem as a multi-agent partially observable Markov decision process. Since each agent controls exactly one queue, we index agents by their queue index m∈0,…,Mm∈\0,…,M\. Below, we describe the three essential components of each agent. • Observation Space: At the start of scheduling cycle c, each queue agent m receives a local observation oc,mo_c,m that captures the traffic state and queue dynamics, given by oc,m=⟨Bc,m,Mc,m,gc,m,dc,m⟩o_c,m= B_c,m,M_c,m,g_c,m,d_c,m , where Bc,mB_c,m represents the backlog, defined as the total number of PDUs currently waiting for transmission in queue qmq_m. Mc,mM_c,m denotes the number of missed PDU-sets waiting in queue qmq_m. gc,mg_c,m represents the AOPDU waiting time in queue qmq_m, while dc,md_c,m denotes the urgency rate, defined as dc,m=gc,mDun,fd_c,m= g_c,mD_u_n,f. • Action Space: At each scheduling cycle c, each agent m generates a two-dimensional action vector. Specifically, the action for agent m is defined as ac,m=⟨ac,mdur,ac,mord⟩a_c,m= a_c,m^dur,a_c,m^ord , where ac,mdur∈[0,1]a_c,m^dur∈[0,1] represents the agent’s desired fraction of the time slice duration. The actual time slice duration wc,mw_c,m (in μ ) allocated to queue qmq_m is computed as wc,m=ac,mdur∑m=0M−1ac,mdur⋅Δtcyclew_c,m= a_c,m^dur _m=0^M-1a_c,m^dur· t_cycle. In contrast, ac,mord∈[0,1]a_c,m^ord∈[0,1] determines the service order πc(m) _c(m) of the queue within the scheduling cycle. • Reward Function: The reward is driven by the urgency rate dc,md_c,m, which measures the AOPDU relative to its dynamic deadline, and by PDU-set deadline satisfaction. While eq. 4 directly captures the objective of maximizing PDU-set deadline satisfaction through the binary indicator in eq. 3, this indicator provides limited intermediate feedback for RL-based scheduling. Therefore, Eq. (8) uses a dense, bottleneck-aware surrogate reward based on queue urgency and accumulated deadline violations, providing continuous feedback to guide the policy toward higher PDU-set deadline satisfaction: Ψc,m=dc,m,if dc,m≤11+λ⋅Mc,m⋅(dc,m−1),if dc,m>1 _c,m= casesd_c,m,&if d_c,m≤ 1\\ 1+λ· M_c,m·(d_c,m-1),&if d_c,m>1 cases (8) where λ is a scaling factor. When a queue operates safely by transmitting each PDU within its dynamic deadline (dc,m≤1d_c,m≤ 1), the penalty simply reflects its current urgency rate. However, once a deadline is violated (dc,m>1d_c,m>1), the penalty is amplified proportionally to both the violation magnitude and the number of missed PDU-sets Mc,mM_c,m. The global team reward RcR_c is determined by the bottleneck queue, defined as the negative maximum penalty across all TSN queues, given by Rc=−maxm∈0,…,MΨc,mR_c=- _m∈\0,…,M\ _c,m. IV MAPPO for TSN-Based VEC IV-A MAPPO Framework To efficiently address the optimization problem, we employ the MAPPO algorithm [9]. As illustrated in fig. 1, M cooperative agents control an individual TSN queue. MAPPO extends the single-agent PPO framework to the multi-agent setting by adopting a CTDE paradigm. Specifically, each agent is equipped with an actor policy that determines its local scheduling action, while a centralized critic estimates the value of the global system state during training. The critic network has access to global information, including the observations and actions of all agents, allowing it to capture the inter-queue dependencies and coupling among the queues. In contrast, each actor receives only its corresponding local observation and learns a policy for selecting its action. The centralized critic network provides a more informative value estimate for computing the policy gradient and advantage function, thereby reducing the non-stationarity and variance that arise when multiple agents simultaneously adapt their policies. Consequently, the actor can learn scheduling decisions that implicitly account for the impact of its actions on the overall system performance. IV-B Training and Inference Procedure In our model, each TSN queue has a local actor policy πθm _ _m, while a centralized critic VϕV_φ is used during training. During data collection, each agent observes its local queue state oc,mto_c,m^t and selects an action according to its policy πθi(ac,m|oc,mt) _ _i(a_c,m|o_c,m^t). The environment then executes the joint actions of all agents, determining the corresponding TSN scheduling decisions, and returns a global team reward RcR_c and the next system state st+1s_t+1. These transitions are stored in the buffer D for subsequent policy updates. After trajectory collection, the centralized critic uses the global system information to estimate the value function, which is used to compute the advantages (i.e., how good each action was relative to expectations) and returns for policy optimization. The actor policies are then updated using the PPO objective, while the centralized critic is updated by minimizing the value-function loss. During inference, the centralized critic is no longer required, and each queue independently selects its scheduling action using only its local observation and the corresponding trained actor policy. V Performance Evaluation V-A Experimental Setup We use vehicular mobility data collected on the Smart Highway testbed [16], providing vehicle counts and average speeds for different highway segments. The TSN network has eight queues supporting five XR applications, categorized as AR or SeAR, with traffic modeled using Johnson’s SUS_U distribution [14]. As shown in Table I, queues q0q_0, q1q_1, q3q_3, q4q_4, and q7q_7 handle traditional AR traffic, while the others support different SeAR resolutions with a 2 ms base deadline. All deadlines are scaled by α∈[0.50,0.90]α∈[0.50,0.90], reducing the MEC delay budget from 90% at low speeds (OPENvavgmin=10)v_avg =10) to 50% at high speeds (vavgmax=120v_avg =120) to account for speed-dependent latency deadlines. Table I also reports the MAPPO hyperparameters. Each episode contains 150 steps, with the number of connected vehicles and their average speed sampled from the dataset at its start. Average speed and vehicle count are clipped to 10–120 and 1–100, respectively. Clients are then distributed among queues using a uniform multinomial distribution, with at least one active user per queue. We set the reward penalty weight to λ=1.0λ=1.0. Results are reported as the mean and standard error across three random seeds. Table I: VEC and MAPPO parameters VEC Value MAPPO parameter Value Network speed (Gbps) 10 Learning rate 10−410^-4 Max packet size (B) 1428 gamma 0.99 Number of vehicles 1–100 Episode length 150 Average speed (km/h) 10–120 PPO epochs 5 AR setting/Base deadline SeAR setting/Base deadline q0, q3: 3840×1920@90 Hz / 11.11 ms q2: 1920×720@60 Hz / 2 ms q1, q4, q7: 3840×1920@72 Hz / 13.80 ms q5: 1280×480@60 Hz / 2 ms q6: 2560×960@60 Hz / 2 ms Baselines: To evaluate the performance of our MAPPO approach, we compare it against a set of baselines, including alternative MARL architectures, single-agent algorithms, and a heuristic-based scheduling policy, as described below: • MARL: Multi-agent transformer (MAT) [17] and heterogeneous-agent proximal policy optimization (HAPPO) [18], where we also model each queue as an independent agent. • Single-agent RL: PPO and advantage actor-critic (A2C) for centralized control, where a single agent has control over all queues. • Urgency-based heuristic: A non-learning policy that prioritizes queues approaching their deadlines based on the urgency rate. The time slot allocation is given by ti=dc,i∑m=0M−1dc,m⋅Δtcyclet_i= d_c,i _m=0^M-1d_c,m· t_cycle. V-B Experimental Results Learning performance: As shown in fig. 2, MARL methods converge faster than single-agent methods, reaching stable reward levels in fewer training steps, whereas PPO and A2C require longer training to converge. Figure 2: Average cumulative reward. By learning from multiple agents, MARL methods can better capture the impact of queue-level decisions on overall network performance, potentially improving convergence over single-agent methods. Nevertheless, MAPPO converges slightly faster than MAT and HAPPO, likely because it jointly updates all agents using a shared objective, whereas MAT and HAPPO rely on sequential mechanisms that introduce additional dependencies during learning. Despite faster MARL convergence, PPO achieves the highest and most stable final reward. However, higher training rewards do not necessarily imply better network performance under heterogeneous queue deadlines, as shown next. Service Latency Analysis: Fig. 3 presents the average and p99p99 PDU-level service latency for each queue. As observed, MARL methods achieve lower latency across all queues, reducing the average PDU latency by at least 19.5% compared with A2C and 14.9% compared with PPO for the AR queues. Similarly, MARL methods reduce worst-case (p99p99) latency, with reductions of up to 38.6% compared with the single-agent methods. This performance gap is even more pronounced for SeAR queues, where MARL methods reduce average and p99p99 latency by up to 66.2% and 68.3%, respectively, compared with single-agent methods. Figure 3: Average (solid fill) and 99th-percentile (transparent fill) PDU-level service latency. The results suggest that single-agent policies do not fully capture the interactions and resource trade-offs among queues with heterogeneous requirements, which may explain why PPO achieves higher training rewards than the MARL methods. In particular, PPO and A2C appear to prioritize AR queues, whose higher traffic intensity contributes more frequently to the reward, while postponing service for lower-demand SeAR queues. Although this behavior can increase the number of satisfied AR deadlines and thus the training reward, it may degrade overall network performance by increasing SeAR delay. In contrast, MAPPO, HAPPO, and MAT explicitly model the scheduling problem in a multi-agent setting, enabling more coordinated resource allocation across queues. Consequently, they achieve better execution performance despite lower cumulative training rewards. Nevertheless, MAPPO and MAT slightly outperform HAPPO, exhibiting lower variance across the queues, as indicated by the standard deviation. This difference may be attributed to the distinct coordination mechanisms employed by each method. HAPPO updates the agents sequentially using a randomly shuffled update order, which can introduce variability in the learning process. In addition, urgency-based heuristics generally outperform all learning-based methods for SeAR queues, reducing latency by at least 29.1%. However, this gain comes at the cost of higher latency in AR queues, with average latency increasing by up to 315% and worst-case latency reaching 80 ms, corresponding to an 373% increase compared with learning-based methods. PDU-Set Reliability: Fig. 4 shows the PDU-sets deadline satisfaction rate (i.e., reliability). To achieve high reliability, scheduling policies must ensure that all PDUs belonging to a PDU-set are delivered within its dynamic deadline, as designed by eq. 3. Accordingly, the reliability results are consistent with the latency analysis, with MARL methods achieving higher reliability across the AR queues, exceeding 70% compared with approximately 60% for single-agent methods, corresponding to a 16.66% improvement. As observed previously, the urgency-based method tends to prioritize SeAR queues due to their stricter deadlines. This urgency-driven approach explains its higher reliability for SeAR traffic, exceeding 90% for SeAR queues, particularly for q2q_2 and q5q_5. Despite this improvement, they perform poorly across the AR queues, achieving reliability of only around 30%. Figure 4: PDU-set reliability (deadline satisfaction). Finally, Table I presents the inference time of the learning-based methods on an RTX 4060 Ti. MAPPO and the single-agent methods exhibit lower inference times, whereas HAPPO and MAT exceed the scheduling cycle by more than 5 and 10 times, respectively. This gap results from differences in action generation: MAPPO and single-agent methods generate actions in a single forward pass, while MAT generates them sequentially through an autoregressive policy. HAPPO also incurs higher inference time due to its separate actor network for each agent. Table I: Inference Time (ms) Method Inference Time Method Inference Time MAPPO 0.69 ± 0.18 MAT 10.90 ± 1.29 HAPPO 5.48 ± 0.61 PPO 0.78 ± 0.22 A2C 0.79 ± 0.27 Therefore, although HAPPO and MAT achieve competitive performance, their higher inference latency may limit real-time deployment under a 1 ms control cycle. In contrast, MAPPO achieves comparable performance with significantly lower inference latency. VI Conclusion This paper addresses co-located XR traffic scheduling in VEC-TSN integration by considering vehicle speed-dependent latency deadlines. We leverage MAPPO to model queue-level interdependencies under dynamic deadlines, using a centralized critic to learn cooperative scheduling policies. Simulation results show that MAPPO outperforms the considered baselines by reducing latency, improving PDU-set reliability, and maintaining low inference time, making it suitable for real-time scheduling. In future work, we intend to evaluate the impact of wireless networks on the proposed framework. Acknowledgment This work was financed by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior - Brasil (CAPES) - Finance Code 001, CNPq (funding agency from the Brazilian federal government), FAPEMIG (Minas Gerais State Funding Agency), and São Paulo Research Foundation (FAPESP) with Brazilian Internet Steering Committee (CGI.br), grants 2018/23097-3. References [1] J. Feng and J. Zhao (2023) Resource allocation for augmented reality empowered vehicular edge metaverse. IEEE Transactions on Communications 73 (3), p. 1987–2001. Cited by: §I. [2] A. A. Barakabitze, A. Ahmad, R. Mijumbi, and A. Hines (2020) 5G Network Slicing Using SDN and NFV: A Survey of Taxonomy, Architectures and Future Challenges. Computer Networks 167, p. 106984. External Links: Document Cited by: §I. [3] J. Farkas, B. Varga, G. Miklós, and J. Sachs (2019) 5G-TSN Integration Meets Networking Requirements for Industrial Automation. Ericsson Technology Review 96 (7), p. 1–8. Cited by: §I. [4] M. Carvalho and D. F. Macedo (2025) Performance analysis of the integration of dynamic cloud computing environments and tsn networks. In 2025 IEEE 11th International Conference on Network Softwarization (NetSoft), p. 406–414. Cited by: §I. [5] V. Gavriluţ and P. Pop (2020) Traffic-type assignment for tsn-based mixed-criticality cyber-physical systems. ACM Transactions on Cyber-Physical Systems 4 (2), p. 1–27. Cited by: §I. [6] T. Stüber, L. Osswald, S. Lindner, and M. Menth (2023) A survey of scheduling algorithms for the time-aware shaper in time-sensitive networking (tsn). Ieee Access 11, p. 61192–61233. Cited by: §I. [7] N. Zhou, Y. Yao, X. Liang, S. Yao, J. Wan, C. Yang, W. Gui, and W. Gao (2026) Convergence of reinforcement learning and time-sensitive networking for future industrial ai agent communication: fundamentals, challenges, and opportunities. IEEE Communications Surveys & Tutorials. Cited by: §I. [8] M. Carvalho, F. Temiz, S. Salehi, M. Erol-Kantarci, and D. F. Macedo (2026) Multi-agent reinforcement learning for online traffic scheduling in time-sensitive application. arXiv preprint arXiv:2608.05346. Cited by: §I. [9] C. Yu, A. Velu, E. Vinitsky, J. Gao, Y. Wang, A. Bayen, and Y. Wu (2022) The surprising effectiveness of ppo in cooperative multi-agent games. Advances in neural information processing systems 35, p. 24611–24624. Cited by: §I, §IV-A. [10] A. Roberty, Q. Besnard, S. B. H. Said, F. Ridouard, H. Bauer, and A. Geniet (2024) Configuring the ieee 802.1 q time-aware shaper with deep reinforcement learning. In NOMS 2024-2024 IEEE Network Operations and Management Symposium, p. 1–7. Cited by: §I. [11] B. Zhou and L. Cheng (2021) Mitigation of scheduling violations in time-sensitive networking using deep deterministic policy gradient. In Proceedings of the 4th FlexNets Workshop on Flexible Networks Artificial Intelligence Supported Network Flexibility and Agility, p. 32–37. Cited by: §I. [12] M. Guo, S. He, C. Gu, X. Guo, J. Chen, T. Gao, and T. Wang (2024) Towards distributed flow scheduling in ieee 802.1 qbv time-sensitive networks. ACM Transactions on Sensor Networks 20 (5), p. 1–30. Cited by: §I. [13] S. Garcia-Cantón, C. Ruiz de Mendoza, C. Cervelló-Pastor, and S. Sallent (2025) Multi-agent reinforcement learning-based routing and scheduling models in time-sensitive networking for internet of vehicles communications between transportation field cabinets. Applied Sciences 15 (3), p. 1122. Cited by: §I. [14] D. G. Morín, D. Medda, A. Iossifides, P. Chatzimisios, A. G. Armada, A. Villegas, and P. Peréz (2023) An extended reality offloading ip traffic dataset and models. IEEE Transactions on Mobile Computing 23 (6), p. 6820–6834. Cited by: §I-A, §I, §V-A. [15] P. Paymard, S. Paris, A. Amiri, T. E. Kolding, F. S. Moya, and K. I. Pedersen (2024) PDU-set scheduling algorithm for xr traffic in multi-service 5g-advanced networks. In ICC 2024 - IEEE International Conference on Communications, Vol. , p. 758–763. External Links: Document Cited by: §I-A. [16] R. C. Bello, M. Camelo, J. Marquez-Barja, and N. Slamnik-Kriještorac (2026) Towards sustainable 6g compute continuum: energy-aware zero-touch network and service management used for dynamic vehicular service deployments. In 2026 IEEE 23rd Consumer Communications & Networking Conference (CCNC), p. 1–6. Cited by: §V-A. [17] M. Wen, J. Kuba, R. Lin, W. Zhang, Y. Wen, J. Wang, and Y. Yang (2022) Multi-agent reinforcement learning is a sequence modeling problem. Advances in Neural Information Processing Systems 35, p. 16509–16521. Cited by: 1st item. [18] J. G. Kuba, R. Chen, M. Wen, Y. Wen, F. Sun, J. Wang, and Y. Yang (2021) Trust region policy optimisation in multi-agent reinforcement learning. arXiv preprint arXiv:2109.11251. Cited by: 1st item.