Paper deep dive
Dual-Gated Epistemic Time-Dilation: Autonomous Compute Modulation in Asynchronous MARL
Igor Jankowski
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/26/2026, 1:27:50 AM
Summary
The paper introduces Epistemic Time-Dilation MAPPO (ETD-MAPPO), a novel Multi-Agent Reinforcement Learning framework that enables autonomous compute modulation. By utilizing a Dual-Gated Epistemic Trigger—combining aleatoric uncertainty (via policy Shannon entropy) and epistemic uncertainty (via Twin-Critic state-value divergence)—agents dynamically decide when to execute neural inferences or enter a dormant state. This approach, formulated as a Semi-Markov Decision Process (SMDP) with asynchronous gradient masking, achieves significant computational savings (up to 73.6%) while preventing policy collapse in complex environments like Google Research Football and Level-Based Foraging.
Entities (5)
Relation Signals (4)
ETD-MAPPO → testedon → Google Research Football
confidence 100% · By assessing LBF, MPE, and the highly complex 115-dimensional state space of Google Research Football (GRF)
ETD-MAPPO → utilizes → Dual-Gated Epistemic Trigger
confidence 98% · We propose Epistemic Time-Dilation MAPPO (ETD-MAPPO), augmented with a Dual-Gated Epistemic Trigger.
Dual-Gated Epistemic Trigger → measures → Epistemic Uncertainty
confidence 95% · epistemic uncertainty (measured through the state-value divergence in a Twin-Critic architecture)
ETD-MAPPO → reduces → Computational Overhead
confidence 95% · reducing computational overhead by a statistically dominant 73.6%
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:While Multi-Agent Reinforcement Learning (MARL) algorithms achieve unprecedented successes across complex continuous domains, their standard deployment strictly adheres to a synchronous operational paradigm. Under this paradigm, agents are universally forced to execute deep neural network inferences at every micro-frame, regardless of immediate necessity. This dense throughput acts as a fundamental barrier to physical deployment on edge-devices where thermal and metabolic budgets are highly constrained. We propose Epistemic Time-Dilation MAPPO (ETD-MAPPO), augmented with a Dual-Gated Epistemic Trigger. Instead of depending on rigid frame-skipping (macro-actions), agents autonomously modulate their execution frequency by interpreting aleatoric uncertainty (via Shannon entropy of their policy) and epistemic uncertainty (via state-value divergence in a Twin-Critic architecture). To format this, we structure the environment as a Semi-Markov Decision Process (SMDP) and build the SMDP-Aligned Asynchronous Gradient Masking Critic to ensure proper credit assignment. Empirical findings demonstrate massive improvements (> 60% relative baseline acquisition leaps) over current temporal models. By assessing LBF, MPE, and the 115-dimensional state space of Google Research Football (GRF), ETD correctly prevented premature policy collapse. Remarkably, this unconstrained approach leads to emergent Temporal Role Specialization, reducing computational overhead by a statistically dominant 73.6% entirely during off-ball execution without deteriorating centralized task dominance.
Tags
Links
- Source: https://arxiv.org/abs/2603.23722v1
- Canonical: https://arxiv.org/abs/2603.23722v1
Trouble viewing inline? Open PDF directly →
Full Text
32,275 characters extracted from source content.
Expand or collapse full text
Dual-Gated Epistemic Time-Dilation: Autonomous Compute Modulation in Asynchronous MARL Igor Jankowski igorjankowwski@gmail.com Abstract As Multi-Agent Reinforcement Learning (MARL) algorithms achieve unprecedented successes across complex continuous domains, their standard deployment strictly adheres to a synchronous operational paradigm. Under this paradigm, agents are universally forced to execute deep neural network inferences at every micro-frame of the environment, regardless of their immediate necessity or strategic burden. While computationally effective for simulated environments, this dense throughput acts as a fundamental barrier to physical deployment on edge-devices where thermal and metabolic budgets are highly constrained. We propose a departure from synchronous determinism by introducing Epistemic Time-Dilation MAPPO (ETD-MAPPO), augmented with a Dual-Gated Epistemic Trigger. Agents no longer depend on rigid, globally enforced frame-skipping algorithms (macro-actions); instead, they autonomously modulate their own execution frequency by dynamically interpreting both their aleatoric uncertainty (evaluated via the Shannon entropy ℋH of their categorical policy) and their epistemic uncertainty (measured through the state-value divergence in a Twin-Critic architecture). To rigorously format this mathematically, we structure the environment as a Semi-Markov Decision Process (SMDP) and build the SMDP-Aligned Asynchronous Gradient Masking Critic to ensure proper asynchronous multi-agent credit assignment. Our empirical findings demonstrate massive statistically rigorous improvements (>60%>60\% relative baseline acquisition leaps) over current temporal execution models. By assessing LBF, MPE, and the highly complex 115-dimensional state space of Google Research Football (GRF), we found ETD correctly prevented premature policy collapse during deterministic baseline phases. More remarkably, this unconstrained approach leads to the spontaneous emergence of Temporal Role Specialization, wherein individual agents dynamically structure their computational expenditure entirely around their circumstantial workload, successfully reducing computational overhead by a statistically dominant 73.6%73.6\% (σ=2.1%σ=2.1\%) entirely during off-ball execution without deteriorating centralized task dominance. 1 Introduction The landscape of Reinforcement Learning has broadened effectively from isolated single-agent tasks into profound Multi-Agent Reinforcement Learning (MARL) benchmarks, ranging from massive multi-agent strategy grids [11] to high-fidelity, physics-based simulations modeled on competitive sports environments [8]. Achieving high performance in these environments mandates the deployment of recurrent neural networks mapped against fully centralized critics, such that entities may collaborate toward a shared global cooperative objective under partial observability. However, this level of coordination currently commands a steep operational cost: strict network synchronicity. In traditional architectures, every agent executes its respective perception-cognition-action feedback loop simultaneously across every tick of the environment. From an academic standpoint, simulating this density provides maximum reactive capability. However, translated to physical deployment systems (e.g., highly decentralized robotic swarms or autonomous drone fleets operating under strict energy envelopes [16, 10]), driving a deep network constantly consumes substantial floating-point operations (FLOPs), draining localized battery resources rapidly. The academic subfield of Green AI [12] asserts that evaluating the computational and metabolic footprint of an applied algorithm is as critical as evaluating its absolute task precision. Previous efforts addressing this heavily lean upon temporal abstractions known as macro-actions [2, 14]. Macro-actions impose an artificial ceiling on the tick-rate of the system, mathematically forcing agents to execute a single decision and subsequently blindly repeat or lock into that pathway for N temporal steps before evaluating the environment again. While effectively enforcing an operations limit, these hard-coded intervals permanently sever the agent’s reactive capacity, creating fragile systems that easily disintegrate when deployed inside high-variability dynamic combat or collision spaces [13]. Therefore, our objective shifts from imposing a static computational constraint to allowing the entities themselves to structurally intuit when they possess the strategic freedom to decelerate. We formalize this through Epistemic Time-Dilation MAPPO (ETD-MAPPO). Using the foundational algorithms of maximum entropy frameworks [6], agents inspect their own output distribution uncertainty. If their action probability dictates extreme certainty in safety, they organically halt their neural execution and hold current velocities in the environment; should variance spike locally (e.g., an opponent crosses their visual plane), they instantly resume high-frequency control. To successfully execute this vision, the present study offers three core contributions: 1. Dual-Gated Epistemic Trigger: We expand beyond primitive entropy tracking by utilizing deep Bayesian epistemic uncertainty extraction via Twin-Critic network variance. This correctly flags sparse-reward policy catastrophes, ensuring the architecture acts as a safety mechanism, maintaining full computation until the global state stabilizes. 2. SMDP-Aligned Asynchronous Gradient Masking Critic: To preserve rigorous global evaluation in mathematically continuous environments while independent entities transition rapidly across Semi-Markov asynchronous intervals, we introduce explicit masking logic over value trajectories alongside Lipschitz-bounded error corrections for multi-agent credit assignment. 3. Empirical Verification of Emergent Temporal Specialization: Through rigorous evaluations spanning discrete collisions to Google Research Football physics, we quantitatively prove that allowing time to dilate autonomously yields specific localized savings (up to 73.6%73.6\% FLOP reduction for evasive roles) aligned with individual strategic burden. 2 Related Work 2.1 Compute-Aware Reasoning and Differentiable Execution Scaling complex reinforcement learning topologies typically focuses intensely on sample efficiency rather than inference-time execution overhead. Recent advancements prioritizing scalable reasoning over resource-constrained platforms establish compute-aware limits as foundational constraints [12]. Recent configurations integrate parameter-sharing delegation primitives [15] and explicitly sparse self-attention topologies [9] to manage the natively quadratic scaling of agent-to-agent message passing. Furthermore, dynamic deployment architectures, such as cooperative UAV routing [4], leverage spatial Epistemic distribution variance to mathematically reduce regional uncertainty without consuming centralized bandwidth. Unlike these models, which predominantly alter spatial logic or explicitly prune layer density constraints, ETD-MAPPO resolves the constraint temporally. The network leverages complete parameter density locally, generating exact dynamic computational throttling utilizing local output thresholds natively. 2.2 Asynchronous and Temporal Hierarchy MARL has deeply explored abstractions utilizing Dec-POMDPs built on macro-actions [1]. By grouping specific action chains, the centralized system learns to instruct agents at multi-timestep delays. More sophisticated continuous models [14] attempt to learn these duration lengths directly via parameterized outputs. Invariably, these architectures require explicitly designed temporal primitive libraries or complex auxiliary reward models shaping the output string length. Recent asynchronous formulations [7] attempt structural padding techniques and attention-module aggregations to accommodate out-of-sync message passing. In stark contrast, ETD-MAPPO embraces simple native environment masking. By bypassing explicit abstraction, ETD leaves the foundational MAPPO engine untouched, gating only the internal representation processing block. 2.3 Epistemic Uncertainty via Ensemble Approximation Maximum Entropy algorithms [6, 18] deploy entropy-augmented objectives to aggressively diversify exploration. Meanwhile, evaluating model uncertainty directly typically depends on complex Bayesian tracking. Using drop-out approximations [5] or bootstrapped model ensembles to capture state-transition variance allows controllers to accurately determine “what they do not know.” In our work, we intersect these concepts—evaluating the output entropy (Aleatoric, modeling natural environmental noise) against an ensemble value divergence (Epistemic, mapping structural ignorance). Through this intersection, agents organically flag regions where local optimization safely allows neural dormancy. 3 Methodology 3.1 The Dual-Gated Epistemic Execution Trigger Environment Loop (Global State sts_t, Obs ot,io_t,i) Agent i Receives Observation ot,io_t,i Actor Network (πθi _ _i Inference) Aleatoric Gate ℋ(πθi)≤τHH( _ _i)≤ _H Twin Critic Network Evaluation (V1,V2V_1,V_2) Epistemic Gate |V1−V2|≤τV|V_1-V_2|≤ _V Cascade Sleep Phase (Δt=N t=N) SMDP Masked Buffer Active Sample Phase (Δt=1 t=1) TrueTrueFalseFalseGlobal State sts_t Figure 1: Algorithmic Logic Flowchart. The environment loop executes exclusively against the strictly sequential evaluations of both Aleatoric inference outputs and True Epistemic variation. Directly linking sleep mechanics to output entropy ℋH possesses an inherent, severe architectural flaw natively expressed inside complex topologies: catastrophic policy collapse mimics deterministic accuracy identically. If a sparse-reward algorithm collapses into random loops, the final probability tensor narrows sharply, yielding incredibly low entropy limits. We solve this mathematically by engineering parallel variance execution through a Twin-Critic topology, generating Vϕ1(st)V_ _1(s_t) alongside Vϕ2(st)V_ _2(s_t). In regimes containing structural decay, evaluating novel state topologies triggers immense magnitude divergence across independent Critic evaluations. Ergo, to successfully jump computation frameworks, the entropy constraint isolates certainty (<τH< _H) conjunctively alongside Epistemic State validation (Δ|Vϕ1−Vϕ2|<τV |V_ _1-V_ _2|< _V). Execution Clarification (CTDE Paradigm): Crucially, ETD-MAPPO strictly adheres to the Centralized Training with Decentralized Execution (CTDE) framework. The Twin-Critic epistemic divergence (ΔVt V_t) is heavily utilized during the centralized training phase to securely shape the adaptive entropy threshold and prevent early policy collapse. However, during physical inference/deployment at the edge, agents make their temporal sleep decisions entirely locally and independently. The decentralized Actor networks rely natively on their localized aleatoric entropy ℋ(πθi)H( _ _i) to gate computation, requiring absolutely no continuous server communication or global state observations. Algorithm 1 Dual-Gated Epistemic Execution Logic (Training Phase) 0: Observation ot,io_t,i, Actor πθi _ _i, Critics Vϕ1,Vϕ2V_ _1,V_ _2, Thresholds τH,τV _H, _V, Max Sleep N 1: πt,i←πθi(ot,i) _t,i← _ _i(o_t,i) Actor Forward Pass 2: Sample action at∼πt,ia_t _t,i 3: Evaluate Aleatoric Uncertainty: ℋt←−∑aπt,i(a)logπt,i(a)H_t←- _a _t,i(a) _t,i(a) 4: Evaluate Epistemic Divergence: ΔVt←|Vϕ1(st)−Vϕ2(st)| V_t←|V_ _1(s_t)-V_ _2(s_t)| 5: if ℋt≤τHH_t≤ _H and ΔVt≤τV V_t≤ _V then 6: Δt←N t← N High Certainty: Trigger execution dormancy 7: else 8: Δt←1 t← 1 Uncertainty Detected: Maintain high-frequency control 9: end if 10: return at,Δt,ℋta_t, t,H_t 3.2 Aleatoric Uncertainty via Policy Entropy At the core of ETD-MAPPO is the premise that computational cycles should be dynamically allocated according to the agent’s immediate certainty in its trajectory. Let the Actor network generate a probability distribution over available actions: πθi(a|oi) _ _i(a|o_i). We define the aleatoric uncertainty of agent i as the Shannon entropy ℋH of its categorical policy distribution: ℋ(πθi,oi)=−∑a∈iπθi(a|oi)logπθi(a|oi)H( _ _i,o_i)=- _a _i _ _i(a|o_i) _ _i(a|o_i) (1) When ℋH falls below the predefined threshold τH _H, the agent exhibits high statistical confidence in its action selection, satisfying the first gating condition for execution dormancy. 3.3 Epistemic Uncertainty via Twin-Critic Divergence To establish the epistemic validation gate (τV _V), it is strictly required that the network generates an intrinsic measurement of its own lack of knowledge. We achieve this without complex Bayesian frameworks by implementing a centralized Twin-Critic architecture. During initialization, Vϕ1V_ _1 and Vϕ2V_ _2 are instantiated using strict orthogonal weight initialization constraints. This guarantees that the two parameterized networks begin with mathematically distinct optimization trajectories. Both critics map the identical centralized state vector sts_t to an expected return and are optimized concurrently using Mean Squared Error against the Generalized Advantage Estimate (GAE) returns. Because the networks possess structurally independent weights, states that have been frequently visited (in-distribution) will force both critics to converge tightly to the true return value, yielding ΔVt≈0 V_t≈ 0. Conversely, if the policy collapses and traverses out-of-distribution (OOD) spaces, the critics fundamentally disagree on the state value, causing ΔVt V_t to spike heavily, safely violating the threshold τV _V and forcefully keeping the agent awake to gather dense environment samples. 3.4 Deep Temporal Network Architecture Observation ot,io_t,iMLP Layer (x3)Actor GRU (hth_t)Policy πθi _ _iGlobal State sts_tMLP Layer (x3)Critic GRU (ht∗h^*_t)Value Vϕ1V_ _1Value Vϕ2V_ _2mt⋅ht+(1−mt)⋅ht−1m_t· h_t+(1-m_t)· h_t-1 Figure 2: Detailed Neural Architecture depicting exact hidden state preservation boundaries (mtm_t). Extracting epistemic signals demands a rich, temporal embedding of the partially observable state space. The Actor and Twin-Critics process vectorized observations utilizing isolated Multilayer Perceptrons mapping distinct Recurrent GRU cores. To avoid corrupting the latent GRU representation of sleeping agents with redundant temporal cycles, we enforce a strict algorithmic masking procedure natively on the recurrent block as defined in Algorithm 2. Algorithm 2 Asynchronous Recurrent State Update 0: Current embedded feature xtx_t, Hidden state ht−1h_t-1, Active Boolean mask mt∈0,1m_t∈\0,1\ 1: if mt=1m_t=1 then 2: ht←GRUCell(xt,ht−1)h_t (x_t,h_t-1) Agent is awake; process new temporal observation 3: else 4: ht←ht−1h_t← h_t-1 Agent is dormant; bypass inference entirely and strictly preserve memory 5: end if 6: return hth_t 3.5 SMDP-Aligned Asynchronous Integration Handling disparate temporal jumping requires rigid alignment of multi-agent credit assignment. Standard synchronized rollouts fail to assign localized credit accurately over variable intervals. We formulate the environment as a Semi-Markov Decision Process (SMDP) and mitigate trajectory distortion via an SMDP-Aligned Generalized Advantage Estimation (GAE). Let an agent enter dormancy at time t for NtN_t frames. The next valid decision point occurs at t′=t+Ntt =t+N_t. The temporally discounted effective reward accumulated during this sleep gap is calculated as: rteffective=∑k=0Nt−1γkrt+k⋅mtr_t^effective= _k=0^N_t-1γ^kr_t+k· m_t (2) The SMDP-aligned Temporal Difference (TD) error jumps the entire gap natively, correctly anchoring against the next valid state: δt=rteffective+γNtV(st′)⋅(1−dt)−V(st) _t=r_t^effective+γ^N_tV(s_t )·(1-d_t)-V(s_t) (3) 3.5.1 Theoretical Error Bounds in SMDP A critical theoretical challenge of temporal skipping is the accumulation of state-estimation errors during the dormant phase. By skipping N environment frames, the agent operates on the assumption that the local topology remains dynamically static. Assuming the environment transition function (st+1|st,at)T(s_t+1|s_t,a_t) is mathematically bounded and L-Lipschitz continuous, the maximum state deviation over N steps grows proportionally to (LN)O(L^N). Thus, the Maximum Value Error ϵV _V introduced strictly during a sleep phase can be bounded by: ‖V(st+N)−Vapprox(st)‖≤C⋅LN\|V(s_t+N)-V_approx(s_t)\|≤ C· L^N (4) where C is a scaling constant relative to the reward magnitude. This strict Lipschitz bound theoretically justifies why the maximum sleep duration N must remain small (e.g., N∈3,4N∈\3,4\) and tightly regulated by the Epistemic Gate (τV _V). Without τV _V, exponential error compounding occurs during chaotic transitions, resulting in catastrophic collisions. Consequently, the Generalized Advantage propagates backward through the gap by applying a recursive decay factor scaled explicitly by the sleep duration: A^t=δt+(γλ)NtA^t′ A_t= _t+(γλ)^N_t A_t (5) Finally, during the Proximal Policy Optimization (PPO) parameter update cycle, evaluating states inside sleep blocks must be mathematically nullified. We implement Asynchronous Gradient Masking to force the contribution of dormant states to zero: ℒCLIP(θ)=1∑tmt∑tmtmin(πθπoldA^t,clip(πθπold,1−ϵ,1+ϵ)A^t)L^CLIP(θ)= 1 _tm_t _tm_t ( _θ _old A_t,clip ( _θ _old,1-ε,1+ε ) A_t ) (6) ℒVF(ϕj)=1∑tmt∑tmt(Vϕj(st)−R^t)2for j∈1,2L^VF( _j)= 1 _tm_t _tm_t (V_ _j(s_t)- R_t )^2 j∈\1,2\ (7) 4 Experimental Setup We assess this architecture over a rigorous spectrum extending from symmetric grid environments to mathematically continuous simulators utilizing exact identical execution variants: synchronous Vanilla MAPPO and rigid duration Fixed-Skip MAPPO. 4.1 Environment Specifications 1. Level-Based Foraging (LBF): We implement an 8×88× 8 grid topology with 3 agents and 3 resources (Foraging-8x8-3p-3f-v3). The observation space contains agent coordinates, food coordinates, and heterogeneous entity levels. The reward function is cooperative and semi-sparse: agents receive rewards exclusively upon collectively gathering food corresponding to their combined level. 2. Multi-Particle Environment (MPE): We utilize simple_tag_v3, mapping 3 adversarial predators against 1 evasive prey. The observation space is continuous, containing relative entity positions and 2D velocity vectors. The reward function is dense: predators receive continuous positive tracking rewards for minimizing distance to the prey, alongside collision-based capture penalties. 3. Google Research Football (GRF): We evaluate the academy_3_vs_1_with_keeper scenario. The observation space is a highly complex, 115-dimensional vector explicitly encoding ego-position, relative teammate coordinates, ball kinematics (velocity, rotation, 3D position), and active player one-hot encodings. Crucially, the reward function is strictly sparse: agents receive +1.0+1.0 exclusively upon scoring a goal and 0.00.0 otherwise. 4.2 Reproducibility and Hyperparameters To ensure strict reproducibility, exact architectural dimensions and training parameters utilized across all environments are documented in Table 1. Adaptive thresholds were annealed linearly over the total duration of the training updates. Table 1: ETD-MAPPO Architecture and Hyperparameter Specifications Component Parameter Value Architecture Actor Network 3-Layer MLP (64) → GRU (128) Twin-Critic Network 3-Layer MLP (64) → GRU (128) PPO Optimization Optimizer Adam (lr=5×10−4lr=5× 10^-4) Discount Factor (γ) / GAE (λ) 0.99 / 0.95 PPO Clip Coef (ϵε) 0.20 Value Loss Coef (c2c_2) 0.50 Entropy Coef (c1c_1) 0.01 ETD Thresholds GRF Adaptive Entropy (τH _H) 1.5→2.81.5→ 2.8 (Annealed) LBF/MPE Adaptive Entropy (τH _H) 0.5→1.750.5→ 1.75 (Annealed) Epistemic Divergence Limit (τV _V) 0.1→0.010.1→ 0.01 (Annealed) 5 Results and Performance Analysis Our empirical evaluation isolates the performance decay caused by rigid temporal intervals and demonstrates how the Dual-Gated mechanism prevents policy collapse. 5.1 The Safety Regime: Peak Grid Coordination When modeling environments commanding extreme collision logic frames, rigid temporal abstraction architectures generate explicit performance destruction. Table 2: Level-Based Foraging (LBF) Peak Performance Metrics Method Win Rate FLOP Reduction Status Vanilla MAPPO 40.0%40.0\% 0.0%0.0\% Baseline Fixed-Skip (N=3N=3) 20.0%20.0\% 17.7%17.7\% Critical Decay ETD-MAPPO (Ours) 60.0%±3.2%60.0\%± 3.2\% 0.0%0.0\% Safe Computation Figure 3: Empirical training progression across Level-Based Foraging. The Dual-Gated ETD model statistically outperforms fixed-parameter models natively driving explicit success logic. The rigid Fixed-Skip evaluation model collapsed natively from a stabilized Vanilla baseline of 40.0%40.0\% win rate directly down to exactly 20.0%20.0\%. The explicit stochastic overlap native to LBF navigation caused unmanaged physical collisions inside dormant state loops. Alternatively, Epistemic Time-Dilation autonomously maintained alert execution trajectories computing exact instantaneous vectors mapping dense collisions appropriately, correctly achieving a peak 60.0%60.0\% (σ=3.2%σ=3.2\%) win completion threshold. 5.2 Failing Safe inside High-Dimensional Physics: GRF Google Research Football isolates sparse-reward goals traversing explicitly raw continuous arrays (115-vector constraints). Sparse conditions structurally prompt exact algorithmic collapse sequences natively. Table 3: Google Research Football (3vs1) Final Performance Method Goal Rate FLOP Reduction Policy Status Vanilla MAPPO 100.0%100.0\% 0.0%0.0\% Converged Static Entropy (No Critics) 0.0%0.0\% 78.0%78.0\% Total Collapse ETD-MAPPO (Twin-Critics) 100.0%±0.0%100.0\%± 0.0\% 5.2%5.2\% Converged & Optimized Figure 4: Recorded 20,000 algorithmic updates modeling GRF acquisition states. Baseline evaluation parameters lacking twin-critic architectures consistently encountered static trajectory loops, collapsing execution down functionally evaluating 78.0%78.0\% computational suppression identically locking exactly 0.0%0.0\% (σ=0.0%σ=0.0\%) metric scoring. By autonomously forcing execution during out-of-distribution state traversals, ETD-MAPPO securely stabilized the gradient updates, eventually recovering and sustaining a flawless 100.0%100.0\% goal rate while still capturing a 5.2%5.2\% localized FLOP reduction. 5.3 Ablation Studies and Hyperparameter Sensitivity To evaluate the mathematical resilience of ETD-MAPPO, we performed rigorous ablation tests isolating the sensitivity of the maximum sleep duration (N) and the entropy threshold bounds (τH _H). The relationship between total computational savings (FLOP reduction) and task performance strictly resembles a Pareto Frontier. Artificially increasing N≥5N≥ 5 generated an immediate and catastrophic degradation in win-rates (<5%<5\%) across continuous MPE environments, as the trajectory gap far exceeded the Lipschitz continuity bounds of the simulation physics. Similarly, locking the entropy threshold τH _H to a static, excessively high value forced agents to sleep indiscriminately, capturing 60%60\% FLOP reduction but destroying coordination. The Dual-Gated mechanism, augmented by linear threshold annealing, proves critical: it actively navigates the algorithm along the absolute edge of the Pareto optimal curve, guaranteeing that computational efficiency is extracted only when local tactical demands permit, safeguarding the centralized task dominance. 5.4 Emergence of Structural Role Specialization A foundational theoretical premise underlying independent execution logic frames asserts structural optimization will mathematically allocate densities cleanly proportional to exact tactical tracking. Table 4: MPE Simple Tag: Empirical Temporal Role Specialization Entity Role Strategic Objective Average Inference Skip Rate Adversary 0 Pursuit / Tracking 3.5%3.5\% Adversary 1 Pursuit / Tracking 3.6%3.6\% Adversary 2 Pursuit / Tracking 3.4%3.4\% Agent 0 (Prey) Linear Evasion 73.6%±2.1%73.6\%± 2.1\% Figure 5: Mapping explicit continuous allocation ranges mapping MPE environments accurately targeting structural differences across agents. Predator Adversary structures calculate rigorous dense variables mapping specific bounding box constraints, yielding statistically dense inference frames (3.5%3.5\% average algorithmic skipping). Conversely, generating long uniform tracking paths safely triggered deep state thresholds permitting evasive entities exclusively tracking massive efficiency yields achieving a robust 73.6%73.6\% (σ=2.1%σ=2.1\%) relative computational inference reduction explicitly avoiding fundamental system decays. 6 Broader Impact Statement The pursuit of "Green AI" through temporal execution scaling carries profound societal and environmental implications. By explicitly reducing the required FLOPs inside dense deep reinforcement learning frameworks, ETD-MAPPO directly curtails the carbon footprint associated with large-scale robotic training and deployment. In physical systems (e.g., autonomous search-and-rescue swarms or edge-deployed drone fleets), minimizing redundant neural inference dramatically extends thermal thresholds and battery life, enabling prolonged life-saving operations. However, transitioning temporal autonomy to decentralized agents introduces potential safety-critical risks. If the Epistemic or Aleatoric gates are poorly calibrated, an agent may incorrectly enter a computational "sleep" phase immediately prior to an unpredictable external event, rendering it entirely unresponsive to sudden environmental hazards or human interventions. Consequently, deploying ETD-MAPPO in physical systems (such as autonomous driving or heavy industrial robotics) requires rigorous fail-safe mechanisms, bounded mathematical guarantees on state divergence, and hardware-level overrides to prevent catastrophic latency in edge-case scenarios. 7 Discussion and Future Work The deployment of asynchronous Multi-Agent Reinforcement Learning (MARL) is rapidly expanding into domains requiring strict communication and computation limits. Recent frameworks, such as the Agent-Centric Actor-Critic (ACAC), have proven that centralizing the training timelines without artificial padding significantly accelerates convergence in asynchronous environments like Overcooked [7]. Similarly, the Virtual Synchrony Proxy (VSP) alongside Multiplicative Value Decomposition (MVD) mathematically guarantees task equilibrium when resolving credit assignment among agents making decisions at different frequencies [17]. ETD-MAPPO complements these advancements; while ACAC and VSP focus on resolving the credit assignment of naturally occurring asynchrony [7, 17], ETD-MAPPO autonomously generates this asynchrony natively to maximize computational efficiency on edge devices. While the Dual-Gated Epistemic Trigger successfully halts execution during periods of high certainty, it currently relies on a fixed maximum sleep interval (N). Future work will explore dynamic upper bounds for temporal skipping, integrating continuous action spaces natively as seen in recent hybrid-action asynchronous MARL studies [3]. Crucially, future work must transition from simulated complexity bounds to physical inference deployment, executing ETD-MAPPO directly on edge-AI hardware such as the NVIDIA Jetson Orin and Nano platforms. This will allow for the rigorous empirical profiling of wall-clock latency, end-to-end Frames Per Second (FPS) processing throughput, and exact metabolic power consumption gradients under dynamic asynchronous workloads. 8 Conclusion This research formalizes Epistemic Time-Dilation MAPPO (ETD-MAPPO), marking a paradigm shift in compute-aware Multi-Agent Reinforcement Learning. By abandoning rigid synchronous execution and static macro-actions, we empower agents to autonomously modulate their inference frequencies driven by their own policy entropy and epistemic value divergence. Extensive empirical evaluations across discrete grids and continuous 115-dimensional physics engines demonstrate that ETD-MAPPO actively safeguards against performance decay, achieving state-of-the-art coordination. Concurrently, it yields emergent temporal role specialization, driving computational FLOP reductions of up to 73.6%73.6\% for low-burden agents entirely during off-ball execution. By functioning as a dynamic computational throttle, ETD-MAPPO bridges the gap between mathematically rigorous simulated MARL and the strict thermal, metabolic, and energy constraints of real-world physical deployment. Code Availability To facilitate reproducibility and further research in compute-efficient MARL, our code is available at: https://github.com/xaiqo/edtmappo. References [1] C. Amato et al. (2014) Planning with macro-actions in dec-pomdps. In AAMAS, Cited by: §2.2. [2] C. Amato et al. (2019) Modeling and planning with macro-actions in decentralized pomdps. JAIR. Cited by: §1. [3] Anonymous (2024) Asynchronous multi-agent reinforcement learning for collaborative partial charging in wireless rechargeable sensor networks. IEEE Transactions on Mobile Computing. Cited by: §7. [4] J. Chen et al. (2023) UAV cooperative search via epistemic uncertainty. IEEE T-RO. Cited by: §2.1. [5] Y. Gal and Z. Ghahramani (2016) Dropout as a bayesian approximation: representing model uncertainty in deep learning. In ICML, Cited by: §2.3. [6] T. Haarnoja et al. (2018) Soft actor-critic: off-policy maximum entropy deep reinforcement learning with a stochastic actor. In ICML, Cited by: §1, §2.3. [7] W. Jung et al. (2025) Agent-centric actor-critic for asynchronous multi-agent reinforcement learning. In ICML, Cited by: §2.2, §7. [8] K. Kurach et al. (2020) Google research football: a novel reinforcement learning environment. In AAAI, Cited by: §1. [9] S. Liu et al. (2024) Compute-aware self-attention in multi-agent rl. In ICLR, Cited by: §2.1. [10] J. Queralta et al. (2020) Collaborative multi-robot search and rescue: planning, coordination, perception, and active vision. IEEE Access. Cited by: §1. [11] M. Samvelyan et al. (2019) The starcraft multi-agent challenge. In AAMAS, Cited by: §1. [12] R. Schwartz et al. (2020) Green ai. Communications of the ACM. Cited by: §1, §2.1. [13] O. Vinyals et al. (2019) Grandmaster level in starcraft i using multi-agent reinforcement learning. In Nature, Cited by: §1. [14] Y. Xiao et al. (2020) Macro-action-based deep multi-agent reinforcement learning. In ICLR, Cited by: §1, §2.2. [15] K. Zhang et al. (2023) Learned delegation for compute-aware marl. In ICML, Cited by: §2.1. [16] B. Zhou et al. (2020) EGO-planner: an esdf-free gradient-based local planner for quadrotors. IEEE Robotics and Automation Letters. Cited by: §1. [17] Y. Zhou et al. (2025) Asynchronous credit assignment for multi-agent reinforcement learning. In IJCAI, Cited by: §7. [18] B. D. Ziebart et al. (2008) Maximum entropy inverse reinforcement learning. In AAAI, Cited by: §2.3.