Paper deep dive
Separating Decision Time from Decision Quality in the Real-Time Gap of Distilled Deciders: Evidence from a Game and a Conveyor Simulator
Chihoon Shin, Junyeong Lee, Kihyeok Jeong, Wonok Kwon
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Real-time agents are often evaluated with the world paused, or with decision time rounded to whole ticks (tick conversion). Tick conversion predicts almost no loss for a learned decider whose hit accuracy falls by 14.8 percentage points (pp) on the same games in asynchronous play. We instead split the decider's gap to a zero-latency teacher into time and quality components. In ViZDoom, we train a small decider by imitating a scripted teacher, run the game on wall-clock time, and add a control in which the teacher waits for a call to the decider's server before deciding. On a Windows host, three preregistered studies put the time component at 6.5-8.7 pp and the quality component at 6.3-8.6 pp. On a Linux host that skips 0.08-0.09% of ticks, the time component disappeared (-0.1 pp; paired 95% bootstrap interval [-0.4, 0.0]) while the quality component remained (5.6 pp [3.7, 7.5]). Retraining on teacher-labelled states from the decider's own play (DAgger) improved it on new games on both hosts (2.6 pp [0.4, 4.9] and 3.1 pp [1.1, 5.2]), a preregistered partial success. Imposing one delay schedule on every arm in live play left a quality component of 5.8 pp [4.0, 7.5] on 100 new games, and retraining cut the decider's disagreement with the teacher on its own states from 22.7% to 14.9%. In a conveyor simulator with a 400 ms deadline, delay past it erased the quality component. A latency-matched control whose delays match the decider's shows which remedy to try.
Tags
Links
- Source: https://arxiv.org/abs/2610.04810v1
- Canonical: https://arxiv.org/abs/2610.04810v1
Trouble viewing inline? Open PDF directly →
Full Text
149,022 characters extracted from source content.
Expand or collapse full text
Separating Decision Time from Decision Quality in the Real-Time Gap of Distilled Deciders: Evidence from a Game and a Conveyor Simulator Chihoon Shin a,∗ , Junyeong Lee a , Kihyeok Jeong b , Wonok Kwon c a Myeongseongsimjae AX Institute, Yeongyang-gun, Gyeongsangbuk-do, Republic of Korea b Department of Computer Science and Software Engineering, Korea University, Seoul, Republic of Korea c Electronics and Telecommunications Research Institute (ETRI), Daejeon, Republic of Korea Abstract Real-time agents are often evaluated with the world paused, or with decision time rounded to whole ticks (tick conversion). Tick conversion predicts al- most no loss for a learned decider whose hit accuracy falls by 14.8 percentage points (p) on the same games in asynchronous play. We instead split the decider’s gap to a zero-latency teacher into time and quality components. In ViZDoom, we train a small decider by imitating a scripted teacher, run the game on wall-clock time, and add a control in which the teacher waits for a call to the decider’s server before deciding. On a Windows host, three preregistered studies put the time component at 6.5–8.7 p and the quality component at 6.3–8.6 p. On a Linux host that skips 0.08–0.09% of ticks, the time component disappeared (−0.1 p; paired 95% bootstrap interval [−0.4, 0.0]) while the quality component remained (5.6 p [3.7, 7.5]). Retraining on teacher-labelled states from the decider’s own play (DAgger) improved it on new games on both hosts (2.6 p [0.4, 4.9] and 3.1 p [1.1, 5.2]), a preregis- tered partial success. Imposing one delay schedule on every arm in live play left a quality component of 5.8 p [4.0, 7.5] on 100 new games, and retrain- ing cut the decider’s disagreement with the teacher on its own states from 22.7% to 14.9%. In a conveyor simulator with a 400 ms deadline, delay past it erased the quality component. A latency-matched control whose delays ∗ Corresponding author. Email addresses: chihoon@mssj.ai (Chihoon Shin), junyeong@mssj.ai (Junyeong Lee), knp6464@korea.ac.kr (Kihyeok Jeong), happy@etri.re.kr (Wonok Kwon) arXiv:2610.04810v1 [cs.LG] 3 Oct 2026 match the decider’s shows which remedy to try. Keywords: real-time evaluation, inference latency, imitation learning, DAgger, covariate shift, ViZDoom, real-time systems 1. Introduction In real-time systems the world does not wait: a game engine keeps run- ning, and a part on an inspection line reaches its diverter whether or not a decision has arrived. Language-model agents are being tried as controllers in such live loops, games among them (Park et al., 2025; Golchinfar et al., 2026). When such a decider loses skill in real time, two different things may have gone wrong: its decisions may arrive too late, or they may be worse decisions. The remedies differ. The first calls for faster inference, predic- tion or latency-aware policies (Firoiu et al., 2018; Chen et al., 2021); the second calls for better training. Common evaluation approaches do not sep- arate them: paused evaluation removes time; tick conversion, which turns measured latency into held engine ticks, models time but not what the host does; and game studies report latency beside the score. We ask: when a fast distilled decider loses skill in real-time play, how much of the loss is decision time and how much is decision quality, and can training reduce the quality component? Because the answer turns out to depend on the host and the scene, we also ask which part of each component the host, the scene and the decider set. We use a first-person game as a controlled testbed, and a conveyor-inspection simulator with a hard deadline as a second environment. Our answer rests on a latency-matched control: a teacher arm that de- cides with the teacher’s rules but waits for a real call to the decider’s server before every decision. It is intended to reproduce the decider’s waiting time while retaining the teacher’s decisions. Comparing this latency-matched teacher with the zero-latency teacher gives the time component. Comparing the latency-matched teacher with the decider gives the quality component; Figure 1 shows both components in the three studies on host 1. Both com- ponents split the decider’s gap to the zero-latency teacher; the 14.8 p of the abstract is a different contrast, between two evaluations of the same decider (Section 6.9). This quality component also includes the effect of replacing exact coordinates with a textual description of the state; for hit accuracy we find no measurable input loss (Section 6.7). On the first host both components turn out to be large and of similar size. On a second host that almost never skips a tick, the time component disap- 2 0510 Decomposition (v4+v4b, 18 games) Intervention (v5, 18 games) Replication (v6b, 56 games) 8.7 8.2 6.5 time part: T0 S (waiting for the server) 0510 O original O original N DAgger O original N DAgger N32 DAgger 6.8 8.6 4.5 4.1 6.3 3.7 2.6 3.1 3.2 quality part: S decider (decider and its input) loss in hit accuracy, percentage points (dot: estimate; bar: 95% game-bootstrap interval) Figure 1: Time and quality components in three studies on host 1 (v4+v4b, v5, v6b; 18, 18 and 56 distinct games). Left: time component T0 − S. Right: quality component S − decider for O, N and N32, including the input change and, in v5, imperfect latency matching; arrows: reduction from O. Bars: paired 95% game-bootstrap intervals. Sym- bols: Table 1. pears and the quality component stays (Section 6.7): in defend_the_center the time component tracked the host while the quality component stayed with the decider; in deadly_corridor, where the scripted teacher is weaker, the quality component reversed. We interpret this disagreement as consis- tent with covariate shift, which imitation learning is known to suffer (Ross et al., 2011). Because the quality component persists where almost no ticks are skipped, the shift is not produced by skipped ticks alone; we attribute the remainder to the decider’s errors leading it into states absent from the teacher-played data, which we did not measure directly on the second host. Collecting those states in the live loop and labelling them with the teacher reduces the quality component. The reduction replicates on 56 new distinct games for two training seeds on the first host and for one on the second, and on the first host it is not explained by data volume: adding the same number of teacher-played states instead did not give the same gain (Sections 6.5 and 6.6). The time component is untouched, as it must be, since no learning enters the teacher arms; reducing it would require changes to inference, the execution loop or the policy’s handling of delay, which we do not attempt. Our contribution is therefore an evaluation method and what it reveals, not 3 a method that shortens decisions. The method needs a teacher that can be run under the decider’s timing, such as the scripted teachers used here; given one, it indicates whether to make the decider faster or to retrain it before either is attempted. We test it in imitation-learning evaluations with scripted teachers; whether the remedy it indicates is the better one for other decider families is untested. Our contributions are as follows. • A latency-matched decomposition of a real-time gap into a time and a quality component, with a check, for every teacher and decider run pair, of whether both lost a similar share of engine ticks to waiting (ratio within 0.8–1.25; pairs outside the band are flagged, not dropped), and an analysis in a model of asynchronous play: under a Lipschitz assumption with unknown constant the manipulation check becomes a bias bound, and, for additive return, the quality component is bounded by the decider’s disagreement with the teacher on its own asynchronous states (Sections 4 and 5, proofs in Section S2). • A preregistered measurement of both components for a distilled de- cider in asynchronous ViZDoom play, reported with game-level and run-level uncertainty and repeated on two hosts and two scenes, with a decision-period study, a live study that imposes one delay schedule on every arm, and a controlled-timing emulator, its fidelity confirmed on fresh seeds, that measures the share-matching bias directly (Sec- tions 6.1, 6.2, 6.7 and S1.13). • A diagnosis of the quality component as consistent with covari- ate shift, supported by the decider’s measured disagreement with the teacher on its own states, and an intervention, on-policy asynchronous distillation, evaluated against the original decider in interleaved triads with a data-volume control and a replication on 56 new distinct games. The intervention met its preregistered criteria only in part: the remaining quality component stayed above 2 p (Sections 6.3–6.6). • Evaluation lessons: tick conversion predicts almost no loss where the observed loss is 14.8 p, concurrent evaluation on one host fails a preregistered calibration, and in a conveyor-inspection simulator a deadline turns the time component into a step whose edge the host’s latency tail moves (Sections 6.8 and 6.9). 4 2. Related work Acting while the world moves. Real-time reinforcement learning starts from the observation that the state keeps changing while an action is selected (Ramstedt and Pal, 2019); neither the Markov nor the semi-Markov decision process captures a state that changes during the agent’s own computation (Travnik et al., 2018). Agents held to human reaction times lose performance quickly (Firoiu et al., 2018). Delay-aware formulations treat delay as a known constant, as actions that execute a fixed number of steps later (Derman et al., 2021) or through state augmentation (Chen et al., 2021). On physical systems the timing itself is uncertain and learning is sensitive to setup details that are rarely reported (Mahmood et al., 2018a; Dulac-Arnold et al., 2019). Benchmarks on real robots therefore specify action and read-write cycle times. They also separate environment and agent computation into different processes to control delays (Mahmood et al., 2018b). Sequential learning has been found to degrade relative to asynchronous learning when updates are expensive (Yuan and Mahmood, 2022). Work on concurrent control, observation delay and language-model game agents is reviewed in Section S1.1. Fast-slow and distilled agents. Several systems couple a slow reasoner with a fast controller. A reasoning vision-language model that needs about 1.5 s per response, too slow for a 15 Hz control loop, is paired with a reactive model that answers in milliseconds (Li and Shi, 2026). These works report scores and latencies side by side without attributing the loss between them. Our intervention uses dataset aggregation (DAgger) (Ross et al., 2011): we collect states reached by the decider during asynchronous play and label them with the teacher’s decisions for training. We use DAgger unchanged and apply it inside the asynchronous loop, where it can reduce the quality component but not the time component. Paused, converted and real-time evaluation. Pausing the game during infer- ence turns a real-time game into a turn-based one; vision-language models score higher on paused than on real-time versions of the same games, al- though both versions stay nearly unsolved (Zhang et al., 2025a), reasoning latency limits decision updates to second-level intervals (Chen et al., 2024), and a real-time variant of a paused benchmark makes latency part of task performance (Ouyang et al., 2026). Tick conversion, as we use the term, belongs to this family; we test it against asynchronous play rather than propose it. Closely related are frame skipping and action repetition: static 5 frame skip repeats an action for a fixed number of frames (Srinivas et al., 2016), action repetition requires open-loop action sequences but can shorten the effective horizon (Kalyanakrishnan et al., 2021), and learning the action duration adaptively did not beat a well-chosen fixed value in a fighting game (Nguyen et al., 2026). A skipped tick in asynchronous play is an action repetition that the agent did not choose. Evaluation methodology. Shared protocols (Bellemare et al., 2013; Machado et al., 2018), interval estimates and seed counts (Henderson et al., 2018; Agarwal et al., 2021; Colas et al., 2018), sources of benchmark variance (Bouthillier et al., 2021), environment diversity (Cobbe et al., 2020), pre- registration (Hofman et al., 2023) and negative results (Karl et al., 2024) all bear on how agents should be compared. Prior work models delays, de- signs systems to tolerate latency and reports real-time performance. We measure how much of a real-time performance loss is associated with latency and how much with decision quality. We add three checks specific to real time: a latency-matched arm, distinct games as the resampling unit, and a calibration of whether a host can run evaluations concurrently. 3. Setting Task and agents. We compare a scripted teacher that reads exact enemy coordinates with a learned decider that selects actions from short state sen- tences. The task uses aiming, firing and strafing actions in ViZDoom de- fend_the_center (Kempka et al., 2016) with two strafing buttons added, giving seven composite actions: turn left, turn right, fire, strafe left, strafe right, fire while strafing left, and fire while strafing right. Figure S33 shows the game and the conveyor simulator used as a second environment (Sec- tion 6.8). Teacher. The teacher is a script that reads exact coordinates: it targets the nearest enemy, fires only when the enemy lies inside an aim cone of arctan(16/d) degrees clipped to 1–6 degrees and at least 14 ticks after its previous shot, and alternates strafing direction every 10 ticks. Decider. The decider is the learned action-selection model. It never sees coordinates. A perception script writes a short state sentence (up to three enemies with side, 3-degree angle bin and 100-unit distance bin; enemy count; strafing phase; gun ready; ammunition in 5-bullet bins), which the decider encodes with hashed n-grams into 1,785 features. The decider has 265,665 parameters. It is trained by imitation for 2,500 steps on 300 teacher games 6 from training seeds. On a CPU, it decides in about 2 ms (p50 2.14 ms, p90 15.94 ms). Environment. The engine advances 35 ticks per second, one every ∆ = 28.6 ms; the pistol holds 26 bullets with no refill; an episode ends after 2100 ticks or at death. Figure 2 shows the evaluation loop, the arms and the training pipeline, and Figure 3 the arms of the latency-matched decomposition. Figure S16 shows the decider’s network, and Section S5 the loop of the earlier tick- converted evaluations. Table 1: Arms and symbols. Colours of these arms in the data figures: T0 dark gray, S gray, O orange, N dark blue, N32 light blue; other series use colours defined in their own figures. SymbolMeaning T0teacher with no added latency Sthe same teacher waiting for the decider’s server before acting (latency-matched teacher) SSthe teacher deciding from the state sentence the decider reads (v10) Da learned decider on the server, in Sections 4 and 5 (O, N, N32 or M) Ooriginal imitation decider Ndecider after two on-policy DAgger rounds, in which the de- cider plays, the teacher labels the states it reaches, and the decider is retrained on them; N1 after one round N32the same data as N, trained with seed 32 Mdata-volume control: the original recipe plus the same number of teacher-played rows HkO holding each action for k ticks after deciding (v15) L8O acting every tick on the decision it made 8 ticks earlier (v15) pppercentage points distinct games seeds whose teacher plays identical games form one unit; all intervals resample these units Two clocks. Under tick conversion, a decision that takes L ms holds the previous action for m(L) = ⌈L/∆⌉− 1 ticks, a constant execution delay in the sense of Derman et al. (2021). Under asynchronous play, the engine runs in ViZDoom’s asynchronous player mode at 35 Hz of wall-clock time and does not wait for the agent. A tick is skipped when it passes without a new 7 Game engine ViZDoom, two scenes Measurement counters, metrics, distinct games counters per game Decision arm one per run T0 teacher, no wait S teacher, waits O original decider N DAgger decider Hk O, holds action L8 O, stale decision Coordinates T0 and S Decider server reads the state sentence Clock mode tick conversion or asynchronous action decision delay: last action repeats offline, before runs Imitation O: demonstrations N: + own states Teacher labels Teacher games Own states weights run timeoffline Figure 2: The evaluation loop. The engine sends exact coordinates to the teacher arms and a state sentence to the decider’s server. Tick conversion holds the previous action for m(L) ticks; asynchronous play runs at 35 ticks per second and repeats the last action while a decision is pending (hatched bar). Dashed: offline training data. Symbols: Table 1. 8 state being read. Skipped ticks are observed from the engine’s episode clock; held ticks are modelled. Metric and unit. The primary metric is hit accuracy, P hits/ P shots, with shots counted from ammunition decreases and hits from the engine counter; before any learning we checked that it separates scripts known to differ (Sec- tion S10). Seeds are not independent games here: 200 seeds produced 103 distinct teacher games, one of which accounted for 98 seeds (Section S9). We group seeds by the signature of the tick-converted teacher’s game and treat each distinct game as one unit. We report two intervals: a percentile bootstrap over distinct games (10,000 resamples, seed 0, paired across arms), which captures game-to-game variation, and a t interval over runs (n = 5, df = 4), which captures run-to-run variation on the host. 4. Method 4.1. Latency-matched decomposition Three arms. Three arms play the same seeds asynchronously. T0 is the teacher with no added latency. S is the same teacher, but before applying each decision it sends the request that the decider would receive to the de- cider’s server and waits for, then discards, the answer. Each decision of S therefore carries the decider’s transport and inference time, including its tail (the occasional much longer delays). D is the decider. With accuracy a(·) over distinct games, a(T0)− a(D) = a(T0)− a(S) | z time component + a(S)− a(D) |z quality component .(1) The step from T0 to S adds only the wait for the decider’s server; the step from S to D replaces the teacher and its coordinate input with the decider and its sentence input (Figure 3). When the decomposition is valid. The time component measures what wait- ing for the decider’s server costs on this host, which includes transport and host jitter (variation in timing on the computer that runs the experiment) as well as inference. The decomposition is valid only if S and D experience similar delays. A tick is skipped when it passes without a new state being read, and an arm’s skipped-tick share is the fraction of all ticks of the episode that were skipped. We check each pair with the ratio of skipped-tick shares r = skip(S)/skip(D) against the band 0.8≤ r ≤ 1.25 (the skip match), and flag rather than drop pairs outside it. S and D runs are interleaved on one 9 host. In the two studies below, T0 comes from an earlier session on the same seeds (three runs, 0.1% skipped ticks); the replication (Section 6.6) measures T0 in the same session as the other arms. engine async, 35 Hz, wall clock T0 S D exact coordinates teacher rules exact coordinates + state sentence teacher rules state sentence send sentence to server, wait, discard answer state sentence (input changes) learned decider server replies; loop waits for the action decider server (dashed) swapped apply action apply action apply action next state; each arm is run separately (the engine keeps running) time quality + input Figure 3: The three arms of the latency-matched decomposition. S acts like T0 but calls the decider’s server and waits for, then discards, the answer; D is the learned decider on that server. T0 to S adds the wait (gray: time component); S to D swaps in the decider and its input (highlighted: quality component). Box widths do not encode time. 4.2. On-policy asynchronous distillation Idea. If the quality component comes from states the decider reaches only because ticks pass without its decisions, it should shrink when those states enter training. We therefore use DAgger (Ross et al., 2011): the decider plays, the teacher labels the states the decider reaches, and the decider is retrained on the teacher’s demonstrations plus these labelled states. We run it with the learner in control (β = 0) inside the asynchronous loop: 1. Let the current decider D i play n asynchronous games on collection seeds. At every decision, record the state sentence it saw; after its answer has been applied, record the teacher’s action for the same ob- servation. Labelling after the answer adds no time to the decision. 2. Train D i+1 from scratch with the original recipe on the teacher’s demonstrations plus all collected rounds, keeping each row in the same training, validation or calibration split. 3. Repeat for a number of rounds fixed before any evaluation, then eval- uate the last decider once. 10 We use n = 60 games per round and two rounds, and fix the training seed in advance (seed 31, as for the original decider). A decider trained with seed 32 on the same data serves as a replicate. 4.3. Protocol Preregistration. Before its seeds were opened, every experiment was frozen in a preregistration with numeric verdict lines, and its text was hashed with SHA-256 (Section S22). Figure S11 lists every study with its host, scene, size and status; the full ledger is Table S11. Every analysis script was run on positive and negative controls before it saw the results. Seed ranges are disjoint across design, metric qualification, training, collection and evalua- tion. Doom evaluation runs are executed one at a time (Section 6.9). The conveyor study (Section 6.8) ran four episodes concurrently, as preregistered; we report the consequences of that choice there. Intervention verdicts. For the intervention we preregistered three verdicts. Failure: the new decider N does not beat the original decider O, that is, the game-bootstrap lower bound of a(N)−a(O) is at most zero. Strong success: N beats O, the remaining quality component a(S)−a(N) is at most 2 p, and its upper bound lies below the lower bound of the time component. Partial success: every other outcome. How AI agents were used in the research, and the checks that kept their output from entering the results unchecked, are described in Section S1.2. 5. Theoretical analysis Questions and scope. We examine when the decomposition separates wait- ing effects from policy effects, and how disagreement on states visited by the decider relates to the quality component. The disagreement result con- cerns additive return such as expected hits; its connection to hit accuracy is empirical, not part of the proposition. Proofs are in Section S2. Model. Let s t be the engine state at tick t∈0,...,H−1 and let z t ∈0, 1 indicate whether the agent submits a new decision at tick t (z t = 0 is a skipped tick, on which the previous action a t−1 is repeated). The indicators are generated by a latent host state h t that evolves as a Markov chain; we call its law the skip process Z. The augmented state x t = (s t ,a t−1 ,z t ,h t ) is Markov. A policy π maps x t to an action (in practice it ignores h t ), and is constrained to π(x t ) = a t−1 whenever z t = 0; the teacher reads s t exactly and the decider reads a sentence computed from s t , so both are policies in 11 this sense. Let r(s t ,a t )∈ [0, 1] be a per-tick reward (for example, a hit) and J (π,Z) =E P t r(s t ,a t ) the expected return when π plays under Z. Assumption 1 (Exogenous skipping). The skip process does not depend on which action the policy chooses: the host state h t+1 , and hence z t+1 , is independent of (a 0 ,...,a t ) given h t and the decision latency. Under Assumption 1, S and D share one skip process Z D when their latencies match, T0 plays under Z 0 , and the three arms estimate J (π T ,Z 0 ), J (π T ,Z D ) and J (π D ,Z D ). Proposition 1 (Identification and bias). The time component J (π T ,Z 0 )− J (π T ,Z D ) changes only the skip process and the quality component J (π T ,Z D ) − J (π D ,Z D ) changes only the policy. If instead S runs under Z S ̸= Z D , the estimated quality component is Q + [J (π T ,Z S )− J (π T ,Z D )], where Q = J (π T ,Z D )− J (π D ,Z D ) is the quality component under Z D . If the teacher’s value of a metric (return, or accuracy) is κ-Lipschitz in the expected share of skipped ticks σ(Z) over the skip processes the host produces, the bias is at most κ|σ(Z S )− σ(Z D )|. Section S1.3 discusses how the manipulation check becomes a sensitivity estimate, why disagreements on skipped ticks cost nothing, and how the bound relates to DAgger and to hit accuracy. Proposition 2 (Quality component and on-policy disagreement). Let d D t be the distribution of x t when the decider plays under Z. Let ε D = 1 H P t Pr x∼d D t z t = 1, π D (x) ̸= π T (x) be its per-tick rate of disagree- ment with the teacher on its own states. Let ∆ Q = max t,x,a Q T t (x,π T (x))− Q T t (x,a) , where Q T t (x,a) is the teacher’s action value (expected return for a specified action followed by the teacher) under Z. Then J (π T ,Z)−J (π D ,Z) = X t E x∼d D t Q T t (x,π T (x))−Q T t (x,π D (x)) ≤ H ε D ∆ Q , (2) and the difference is non-negative if π T is greedy (choosing an action with the highest action value) with respect to Q T . 6. Results Figure 4 collects the primary contrasts of all studies on one axis. In every asynchronous study of defend_the_center, pooled hit accuracy is highest for the zero-latency teacher (T0), lower for the latency-matched teacher (S, the 12 −505101520 difference in hit accuracy (p) with 95% game-bootstrap interval v4b host 1 v5 host 1 v6b host 1 v12 host 2 v18 host 2, imposed v4b host 1 v5 host 1 v6b host 1 v12 host 2 emulator, exact schedule v18 host 2, imposed v5 N v6b N v6b N32 v12 N v18 N v18 N32 v7 N vs M v10 input share tick conversion loss time component quality component (O) DAgger gain other preregisteredexploratory or reanalysis Figure 4: Primary contrasts of all studies on one axis: time components, quality com- ponents of O, DAgger gains and two other contrasts, in hit-accuracy points with 95% game-bootstrap intervals. Filled: preregistered; open: reanalysis (tick conversion over distinct games). Host 2: v12 and v18; all others host 1. same teacher waiting for the decider’s server) and lowest for the deciders, although individual runs overlap (Figure S2). In both studies that include it, the decider retrained by DAgger (dataset aggregation: the decider plays, the teacher labels the states it reaches, and the decider is retrained on them) lies between S and the original decider. Four further metrics of the replication study (kills, reaction to a lone enemy, survival and hits taken) are shown in Figure S1 and discussed in Section S1.4. 6.1. Decision time and decision quality contribute comparably Five interleaved S/O pairs on seeds 850000–850029 (30 seeds, 18 dis- tinct games) give the decomposition in Table 2. The zero-latency teacher hits 95.5% of its shots, the latency-matched teacher 86.8% and the decider 13 Table 2: Latency-matched decomposition of hit accuracy Arm or contrast Hit accuracy (%)95% interval or difference (p) game bootstrap run-level t T095.5%– S86.8%– O80.0%– Time, T0 − S8.7 p[6.6, 10.9]– Quality, S − O6.8 p[4.5, 8.8][1.2, 12.4] Five S/O pairs, 18 distinct games; T0 from an earlier session on the same seeds. Differences are computed from unrounded values. Game bootstrap: paired resampling of distinct games; run-level t: over the five pairs. Symbols: Table 1. 80.0%. The time component (T0− S) is therefore 8.7 percentage points (p) and the quality component (S − O) 6.8 p. Counted in hits per game, an additive quantity, the same arms give a quality component larger than the time component (Figure S6; exploratory). The preregistered extension to five pairs allowed the reading “quality component below 5 p” only if the game-bootstrap upper bound was at most 5 p; it is 8.8 p, so we do not use that reading. The skip ratio fell outside the prespecified band in two of five pairs, with values of 1.59 and 0.62. The decider also varied more across runs than the latency-matched teacher (standard deviation 3.6 against 1.4 p). 6.2. Identical delays in live play Proposition 1 requires S and the decider to face the same skip process; the studies above matched only the share of skipped ticks. In a further preregistered study (v18) we imposed one delay schedule on every arm in live asynchronous play. On host 2, which almost never skips ticks on its own, each arm waited after every decision until a preset number of ticks had passed: the observation gaps that the original decider O produced on host 1 in v9 (seeds 860000–860049). The teacher (Tw), O, N and a second DAgger decider trained with another seed (N32) received the same schedule on 100 new seeds (887000–887099, 100 distinct games); T0 played without waiting. The preregistered checks passed: the realised gap equalled the target gap in 99.9% of decisions in every arm, and the waiting arms skipped 8.73–8.81% of ticks (Section S14.2). With delays identical by construction, the quality component of the orig- inal decider was 5.8 p [4.0, 7.5] (Figure 5). The time component, T0 − Tw, 14 0246810 difference in hit accuracy (p), 95% game bootstrap time T0 teacher quality O teacher O quality N teacher N quality N32 teacher N32 DAgger N O DAgger N32 O 9.8 [8.5, 11.2] 5.8 [4.0, 7.5] 2.8 [1.5, 4.1] 3.5 [2.0, 5.0] 3.1 [1.1, 5.0] 2.4 [0.6, 4.1] 2 p (strong-success line) Figure 5: Live play with one delay schedule imposed on every arm (v18; host 2; 100 distinct games, two sessions). Time: T0 − teacher on the schedule; quality: teacher − decider on the same schedule; DAgger: N or N32 − O. Paired 95% game-bootstrap intervals; dotted: 2 p strong-success line. was 9.8 p [8.5, 11.2]; on this host it is the cost of the imposed schedule, not of the host’s own delays. DAgger improved on O for both training seeds, by 3.1 p [1.1, 5.0] for N and 2.4 p [0.6, 4.1] for N32. The quality com- ponent left for N was 2.8 p [1.5, 4.1], above the 2 p of the preregistered strong-success criterion, so the intervention stays a partial success. The esti- mate for O is slightly below the range of the share-matched studies on host 1 (6.3–8.6 p) and within the interval of the synchronous emulator (7.1 p [3.7, 10.6], Section S1.13); for N the emulator’s estimate on the recorded host-1 schedules was lower (−0.2 p [−2.5, 2.1], 29 games). These were measured on other games. We ran the study in two sessions, restarting the servers and the runner in between. With the schedule imposed on this host, 498 of 500 seed and arm pairs produced the same episode in both sessions. The second session therefore shows that live play under an imposed schedule is reproducible; it does not add independent variation, and all intervals are over games. 6.3. The quality component is consistent with covariate shift When ticks pass without a decision, the agent keeps its previous ac- tion and drifts into positions and timings the teacher never produced. We measured how often the decider chooses the teacher’s action on two distribu- tions of states (Table S1). The decider agrees less often with the teacher on states encountered during its own play than on the teacher’s states. On the teacher’s own states the decider agrees 98.4% of the time. On the decisions 15 it took itself in 60 asynchronous collection games it agrees 88.8% of the time, an error rate about seven times higher. Between 44% and 51% of decisions with an enemy in view are taken after the ammunition has run out; there firing has no effect, and the teacher, which ignores ammunition, keeps labelling “fire”, so we report agreement on decisions with ammunition left. We read the gap as consistent with covariate shift: the decider encounters a different distribution of states during its own play than during teacher- generated training (Ross et al., 2011). A direct count points the same way (exploratory): of the state sentences with an enemy in view and ammunition left that the decider met in the two DAgger rounds, 49.2% and 47.8% do not occur verbatim among the 40,440 distinct sentences of the teacher data it was trained on, against 8.0% of the teacher’s own validation sentences. Figure 6 shows how familiar these states are to the decider. Two compet- ing explanations remain. One is data volume: the decider trained on the collected states also saw more data. The other is underfitting: the first re- trained decider matched only 74.0% of its own round-1 training rows when ammunition-empty rows are included. A data-volume control is reported in Section 6.5. teacher validation round 1round 2 0 10 20 30 40 50 60 state sentences never seen in training (%) 8.0% 49.2% 47.8% a new sentences 01 2–45–9 10–49 50–199 ≥200 occurrences in the training data 10 20 30 40 50 share of decisions (%) b how familiar each state is teacher validation round 1 (O plays) round 2 (N1 plays) Figure 6: A direct view of covariate shift (exploratory). (a) Share of state sentences, with an enemy in view and ammunition left, that never occur among the 40,440 distinct training sentences: 8.0% of the teacher’s validation sentences, 49.2% and 47.8% of those met in the two DAgger rounds. (b) How often each state occurs in the training data. Most on-policy errors are turns in the wrong direction (Section S4); Fig- ure S34 shows six of them, one per game. Figure S9 shows where the on- policy errors are: when the teacher turns right, the original decider turns left 16 in about a fifth of the cases; strafing and firing decisions are almost always right. Results from a second host qualify this mechanism: there almost no ticks pass without a decision and the quality component is of the same size (Sec- tion 6.7), so the drift is not produced by skipped ticks alone. The decider’s own wrong turns may also carry it into states absent from the teacher’s data, the setting in which DAgger was introduced (Ross et al., 2011). The live study also measured the mechanism directly. At every decision of a decider we recorded the action the teacher would have chosen in the same state. On its own states O chose differently from the teacher in 22.7% of its decisions and N in 14.9%, a difference of 7.8 p [6.0, 9.6]; N32 differed in 13.8%. Retraining on the decider’s own states thus reduced disagreement on the states it visits, the quantity that bounds the quality component in Proposition 2, although disagreement remained. Across games the relation between a decider’s disagreement and its quality loss was weak (Figure 7; correlation 0.24 for O, 0.19 for N and 0.06 for N32). 510152025303540 decisions differing from the teacher on own states (%) 20 10 0 10 20 teacher decider (p) O N N32 Figure 7: Disagreement with the teacher on the decider’s own states against its quality loss, per game (v18; 100 distinct games). Lines: least-squares fit per decider. 6.4. On-policy asynchronous distillation reduces the quality component Retraining on decisions collected during the decider’s own play reduced the quality component, with partial success under the preregistered criteria. Two rounds added 44,820 and 47,905 decisions collected in the loop. The final decider N was evaluated against the original decider O and the latency- matched teacher S in five interleaved triads on the same 18 distinct games 17 (Table 3, Figure S2). On the preregistered game bootstrap, N improves on O by 4.1 p (2.3–5.9), the quality component falls from 8.6 p for O to 4.5 p for N, and the time component is 8.2 p. The remaining quality component exceeds 2 p and its upper bound (6.3) lies above the lower bound of the time component (5.3), so the preregistered verdict is partial success. Table 3: Hit accuracy in the intervention study Arm or contrastHit accuracy (%)95% interval or difference (p) game bootstrap run-level t T095.5%– S87.3%– N82.8%– O78.7%– Improvement, N − O4.1 p[2.3, 5.9][−0.4, 8.8] Quality, S − O8.6 p[6.8, 10.4][1.4, 16.0] Quality, S − N4.5 p[2.6, 6.3][1.4, 7.6] Time, T0 − S8.2 p[5.3, 10.9]– Five interleaved triads (S, N, O), 18 distinct games; T0 from an earlier session. The last column is the preregistered run-level t interval over the five triads. Symbols: Table 1. The run-level interval of the improvement includes zero; run-level varia- tion between triads is reported in Section S1.4. 6.5. Data-volume control The improvement of N could come from the states it was trained on or simply from having seen more data. We therefore trained a control decider M using the same training procedure as N. M received the teacher’s demon- strations plus 92,725 additional rows, exactly as many as the two DAgger rounds added. These additional rows came from new teacher-played games (off-policy). We evaluated M and N alternately on the replication seeds (preregistration v7). Hit accuracy is 84.8% for N (84.2% in the replication session) and 77.2% for M (Figure S7), a difference of 7.6 p. By the preregistered rule the im- provement is not explained by data volume. M also scored below the original decider measured earlier on the same seeds (81.6%, Table 4), although the two were not run in the same session: more teacher demonstrations did not help, and may have hurt, while the same number of on-policy rows helped. More training data alone therefore does not explain the gain; whether the first decider was also undertrained is not excluded (Section 7). 18 6.6. Replication on 56 new distinct games A second preregistered evaluation on 100 new seeds (860000–860099, 56 distinct games) addressed three objections to the first two studies: the small number of games, the earlier session of T0, and the single training seed. All five arms ran in the same session, one run at a time, in a rotated order over four blocks of 25 seeds. N32 was trained with training seed 32 on exactly the same data as N and was fixed before evaluation. The replication again met the preregistered criteria for partial success (Table 4). The contemporaneous T0 (94.3%) is 1.2 p below the earlier- session T0 of the first two studies, so their time components may be slightly overstated. Both DAgger deciders improve on O: by 2.6 p for N (0.4–4.9; in the four blocks 5.0, 1.7, 1.7 and 2.4 p) and 3.2 p for N32 (0.8–5.4), leaving quality components of 3.7 p and 3.1 p. By the preregistered rules both are again a partial success: the improvement’s game-bootstrap interval excludes zero, and the remaining quality component is below the time component in point estimate, but its interval overlaps that of the time component and it is not below 2 p. Table 4: Replication with a second training seed Arm or contrastHit accuracy (%) Game-bootstrap Skipped or difference (p)95% intervalticks (%) T094.3%–0.1% S87.9%–8.5% N (seed 31)84.2%–7.8% N32 (seed 32)84.7%–8.2% O81.6%–7.3% Time, T0 − S6.5 p[4.6, 8.3] Quality, S − O6.3 p[3.9, 8.7] Quality, S − N3.7 p[0.8, 6.6] Improvement, N − O2.6 p[0.4, 4.9] Quality, S − N323.1 p[0.8, 5.5] Improvement, N32 − O3.2 p[0.8, 5.4] 56 distinct games with a contemporaneous zero-latency teacher and a second training seed (preregistration v6b); paired game-bootstrap 95% intervals. Differences are computed from unrounded values. Skipped ticks: share of engine ticks that passed without a new decision. Symbols: Table 1. 19 6.7. A second host, a second scene and the decision period Four further preregistered studies ask which parts of the decomposition belong to the host, which to the scene and which to the decider, and how often the decider should decide (Table 5). Host 2 is a Linux workstation (Section S21); the procedures, deciders, seeds and bootstrap are those of the earlier studies, and arms ran one at a time. Table 5: Host, scene and decision-period contrasts Study and contrastEstimate (p or points) 95% interval v12, host 2, defend_the_center (hit accuracy, p) time T0 − S−0.1[−0.4, 0.0] quality S − O5.6[3.7, 7.5] DAgger N − O3.1[1.1, 5.2] v11, host 2, deadly_corridor (mean reward, points) time T0 − S0.0[0.0, 0.0] quality S − O−4.6[−10.2, −0.1] DAgger N − O−1.8[−7.5, 3.5] v11, host 1, deadly_corridor (mean reward, points) time T0 − S−12.1[−25.7, 0.6] quality S − O2.5[−17.0, 21.0] DAgger N − O4.2[−15.7, 23.9] v10, host 1, defend_the_center (hit accuracy, p) input S − S−1.3[−3.7, 1.1] policy S − N3.2[0.7, 5.7] v15, host 2, defend_the_center (hit accuracy, p) hold 2 ticks, H2 − H0−58.9[−67.6, −51.4] fresh − stale, H8 − L8−7.8[−17.4, 1.2] v15, host 2, deadly_corridor (mean reward, points) hold 1 tick, H1 − H0−21.9[−35.6, −8.6] hold 4 ticks, H4 − H046.4[28.8, 64.2] fresh − stale, H8 − L835.1[19.3, 50.7] Preregistrations v10, v11, v12 and v15; paired game-bootstrap 95% intervals over distinct games. Hit accuracy in p for defend_the_center; mean reward for deadly_corridor. Hold contrasts are written as Hk − H0. Differences are computed from unrounded values. Symbols: Table 1. Host (v12). On host 2 we repeated the replication of Section 6.6 on the same 100 seeds (56 distinct games) with T0, S, N and O. The arms skipped 0.08– 0.09% of ticks instead of 7.3–8.5%. The time component is −0.1 p [−0.4, 20 0.0], below the preregistered 2 p threshold, while the quality component of O is 5.6 p [3.7, 7.5] (6.3 p on host 1) and N improves on O by 3.1 p [1.1, 5.2], again a partial success by the rule of Section 6.4. The decider’s per-episode median decision time was also steadier on host 2, 1.8–2.3 ms over 860 episodes, than on host 1, 2.4–21.2 ms over 1,300 episodes (exploratory, from the run logs), so the time component measured on host 1 includes that host’s scheduling. Figure S10 shows the two distributions. 02468 hit accuracy, percentage points +6.5 [+4.6, +8.3]time T0 S (host 1) +6.3 [+3.9, +8.7]quality S O (host 1) +2.6 [+0.4, +4.9]DAgger N O (host 1) 0.1 [0.4, +0.0]time T0 S (host 2) +5.6 [+3.7, +7.5]quality S O (host 2) +3.1 [+1.1, +5.2]DAgger N O (host 2) defend_the_center, 56 distinct games (v6b, v12) 20020 mean reward (progress) 12.1 [25.7, +0.6]time T0 S (host 1) +2.5 [17.0, +21.0]quality S O (host 1) +4.2 [15.7, +23.9]DAgger N O (host 1) +0.0 [+0.0, +0.0]time T0 S (host 2) 4.6 [10.2, 0.1]quality S O (host 2) 1.8 [7.5, +3.5]DAgger N O (host 2) deadly_corridor, 94 / 200 distinct games (v11) estimate [95% game bootstrap] Figure 8: Time (T0 − S), quality (S − O) and DAgger (N − O) contrasts with 95% game-bootstrap intervals; filled: interval excludes zero; gray: host 1, black: host 2. Top: defend_the_center, hit accuracy, 56 distinct games per host (v6b, v12). Bottom: deadly_corridor, mean reward, 94 games on host 1 and 200 on host 2 (v11). Scene (v11). deadly_corridor rewards progress towards an armour at the end of a corridor; its teacher runs forward, sidesteps near enemies and never 21 shoots, a script we know to be suboptimal. With 200 seeds (200 distinct games) on host 2, skipping stayed near 1.5% and T0 and S played identical games, so the time component is exactly zero; the decider scored 4.6 points more than the teacher ([0.1, 10.2]) and DAgger did not help (N − O = −1.8 [−7.5, 3.5]), which is expected when the teacher is the weaker player. Whether waiting costs or pays is a property of the scene; how much waiting occurs is a property of the host (Figure 8). On host 1 the same 200 seeds gave only 94 distinct games: 107 of them produced one and the same game, in tick conversion and in asynchronous play alike, whereas all 200 differed on host 2. The mapping from seed to game thus depends on the host’s installation; distinct games are identified within each host, so the two hosts’ corridor results are not measured on the same games. For hit accuracy, replacing exact coordinates with a sentence cost no measurable accuracy (v10, Section S1.5). How often to decide differs by scene (v15, Section S1.6): deciding ev- ery tick is best in defend_the_center, holding each action for four ticks in deadly_corridor. An exploratory selective-automation analysis is in Section S17.1. 6.8. A second environment with a deadline In both Doom scenes a late decision leaves the previous action running, so waiting changes the score by degrees. On a production line a late decision can be worth nothing: a part reaches the diverter at a fixed time and, without a decision, a default applies. We therefore ran the decomposition, under a new preregistration, in a conveyor-inspection simulator in which each part reaches the diverter 400 ms after the camera and passes by default (Section S18). A decider of the same family as the Doom decider reads a sentence describing each part and answers over HTTP; an exact teacher decides from the true state. We added a fixed delay after each answer and crossed it with the policy. Below the deadline the exact teacher paid nothing for delay: up to 380 ms its cost stayed at 0.03 per part. At zero delay the decider cost 0.84 more per part than the teacher [0.64, 1.08] and let 11.2% of defective parts through; this is the quality component. The decider itself began to pay for time before the nominal deadline. As preregistered, its live runs used four concurrent episodes on host 2, and under this load its round trips had a heavy tail (p99 66.7 ms against a median of 3.7 ms). On line A it was late on 6.0–13.1% of parts at 340–370 ms and on every part at 380 ms, where its round trip and rounding to the next 20 ms tick push answers past 400 ms. In an exploratory rerun with one episode at a time, its mean late shares at 340–370 ms fell to 22 0200400600 injected delay (ms) 0 2 4 6 8 10 weighted cost per part 400 ms deadline a cost, full range 340360380400 injected delay (ms) 0 2 4 6 8 10 b cost, 340400 ms 200300400 injected delay (ms) 0 25 50 75 100 late decisions per episode (%) c late decisions exact teacher exact teacher, ±50 ms jitter decider D (live HTTP) four episodes at once (preregistered) one episode at a time (exploratory) Figure 9: A deadline makes the time component a cliff (conveyor simulator, line A, asyn- chronous). (a) Cost per part by injected delay: exact teacher, teacher with ±50 ms jitter, decider D over HTTP; dots: episodes (10 per cell, 500 parts); dotted: 400 ms deadline. (b) 340–400 ms. (c) Late decisions: four episodes at once (preregistered), one at a time (exploratory). 23 0.2–0.4% (0.2–0.8% in single episodes) and its p99 to 4.1–4.3 ms. The early time cost thus came mainly from running episodes concurrently on the host, the failure of concurrent evaluation reported in Section 6.9, seen here in a second environment (Section S18.1). Past the deadline only time mattered: from 400 ms every decision of both policies was late, both passed every part, and their costs were equal (7.52 per part). The interaction at 600 ms,−0.84 [−1.07,−0.65], cancelled the quality component within the preregistered equivalence margin, so a better decider would not have helped. Timing noise also moves the edge for the teacher: with ±50 ms jitter it was already late on 30.4% of parts at 360 ms and cost more than the concurrent decider (2.06 against 1.83 per part; Figure 9). 6.9. Two shortcuts that hide the gap Tick conversion. Most of the decider’s calls finish within one tick (p50 2.14 ms, p90 15.94 ms), so tick conversion, which counts only whole ticks of delay, models almost no delay for it and predicts almost no loss. This is a prediction of the rule, not of asynchronous play. Under tick conversion it hit 99.2% of its shots; on the same 30 raw seeds in asynchronous play it hit 84.1%. Over the 18 distinct games of these seeds, the unit used everywhere else, the figures are 98.6% and 83.8%, a loss of 14.8 p [10.6, 19.4]. Figure S8 shows the loss game by game. We had preregistered that a difference of 10 p or more would retire tick conversion as a basis for real-time claims; on the preregistered unit, raw seeds, the difference was 15.1 p. Under tick conversion the teacher’s skill falls in steps at tick boundaries (Section S8.1). Concurrent evaluation on one host failed its preregistered calibration, so asynchronous evaluations ran one at a time (Section S1.7). 7. Limitations Scope. The results come from two scenes of one game and one conveyor sim- ulator, scripted teachers and one decider family, on two hosts. The simulator renders its own state into sentences; its results say nothing about camera- based inspection accuracy. A distant decider is represented only by the same decider acting on decisions 8 ticks old (v15), not by a separate model. Other games, human teachers and physical systems remain untested. The conveyor study ran four episodes concurrently, a setting that Section 6.9 shows can change timing, so the decider’s time cost below the deadline reflects that load. 24 Host variation. Skipped-tick shares varied between runs of the same arm. In earlier diagnostic asynchronous runs (Section S12), the zero-latency teacher skipped 6.3% to 15.3% of ticks, whereas it skipped about 0.1% in the main studies. In two of the five v5 triads, the skipped-tick shares of the two de- ciders ranged from 11.1% to 25.3%. We have not identified the cause of this variation. Variation in skipped-tick shares between runs is the main reason the run-level intervals are wide. Sessions also differ: the same de- cider on the same 18 games hit 80.0% of its shots in the decomposition study (Table 2) and 83.8% in the tick-conversion comparison (Section 6.9), a session-to-session difference comparable to the DAgger gains. Exogenous skipping. Proposition 1 requires the latency-matched teacher (S) and the decider (D) to face the same skip process. The studies on host 1 check this through the ratio of skipped-tick shares, not through identical schedules. The identical-schedule emulator (Section S1.13) put the bias term at−2.7 p [−6.2, 0.5] for the original decider, outside our ±2 p tolerance. Its fidelity was confirmed on fresh seeds only where the wait after the last observation was short; on a heavily loaded host, both earlier fidelity checks failed because of long waits. The live study with one imposed schedule (Section 6.2) ran on host 2 only, which almost never skips ticks on its own; whether a schedule can be imposed as exactly on a host with its own timing noise is untested, and so is how sensitive the share-matched design is to differences in the length of skipped runs. There the DAgger decider kept a quality component of 2.8 p [1.5, 4.1], whereas the emulator had put it at−0.2 p [−2.5, 2.1] on 29 other games. Unit count. The main studies use 18 distinct games, so their intervals are wide; the replication, the data-volume control, the input study, the host-2 study and the emulator’s synthetic schedules use 56. Other caveats. The underfitting explanation of Section 6.3 is not excluded: the first retrained decider matched only 74.0% of its own round-1 training rows. The first two studies used a zero-latency teacher from an earlier ses- sion, 1.2 p above the contemporaneous one, so their time components may be slightly overstated. The data-volume control compares its decider with O across sessions. The link from additive return, for which the bound of Section 5 holds, to hit accuracy is empirical; counted in hits per game, an additive quantity, the time and quality components are 2.5 and 3.3 hits in the decomposition study and 1.9 and 3.0 in the replication (exploratory). Each preregistration fixed its own primary hypotheses, and we did not correct for multiple comparisons across studies. 25 What the time component means. The time component measures the cost of waiting for the decider’s server on the host where it is measured, including transport and host jitter. Host jitter is variation in timing on the computer running the experiment. It is not a lower bound for all policies, and a policy that anticipates its own delay could beat S. Partial success. On-policy distillation reduced the quality component but did not remove it, and its effect over O is not significant at run level. We fixed two rounds in advance and did not tune them. 8. Implications For evaluation. Most benchmarks for learned agents pause the environment while the agent decides. In our studies such a pause removes one of the two costs that a real-time agent pays, and the other cost, lower decision quality on the agent’s own states, was of comparable size on one host (Section 6.1). A benchmark that reports only paused or tick-converted scores could therefore rank a slow, careful agent above a fast one that would do better on the real clock. We suggest that real-time claims report the time and quality components separately, on the host and schedule on which the agent is meant to run. For system design. Large language models are now a common choice for agent decisions. In our comparators, language models that chose every ac- tion were much slower per decision than the distilled decider and played far worse (Section S12). The small distilled decider answered within a tick; its remaining loss is consistent with covariate shift, and on-policy distillation reduced it only in part. Where a late decision is replaced by a default, as on the conveyor, a better decider would not have helped past the deadline (Section 6.8). This points to a division of labour that our results do not test directly: a large model can teach, check or handle rare cases away from the loop, while a fast learned decider acts inside it. Scope. These are implications of two simulated environments and one decider family, not tested claims about other systems (Section 7). 9. Conclusion Section 8 set out what these results suggest for evaluation and system design; here we summarise them. In our setting a real-time gap can be split into a time component, the cost of waiting for decisions, and a quality 26 component, the residual difference associated with the decider and its input representation; the two interact once skipping is large. For a fast distilled decider in asynchronous ViZDoom play, the two were comparable on one host; on another the time component vanished and the quality component did not. When one delay schedule was imposed on every arm in live play, the quality component of the original decider remained (5.8 p [4.0, 7.5] on 100 new games), and retraining reduced both the quality component and the decider’s disagreement with the teacher on its own states. In the identical- schedule emulator the quality component persisted with no skipping, and matching only the share of skipped ticks may have biased the split. The quality component is consistent with covariate shift: the decider meets a different distribution of states in its own play than in training, although we have not excluded underfitting. Training on states produced by that play reduced the quality component on new games (two training seeds on each host), but only to a partial success under our preregistered criteria, and it did not help in a scene where the teacher is the weaker player. On the first host the reduction was greater than that obtained with the same amount of teacher-played data. The measured cost of waiting differed between hosts and scenes. Where a late decision is replaced by a default, as on the conveyor, the time com- ponent behaved like a step: below the deadline, delay did not raise the exact teacher’s cost, although host timing noise moved where the edge fell; past it both policies defaulted and the quality difference vanished. In our stale-decision emulation, acting on decisions 8 ticks old cost reward in one scene and was inconclusive in the other, and the best decision period differed between scenes. A latency-matched arm, the teacher that waits for the decider before act- ing, indicates which of the two remedies to try, a faster decider or retraining on its own states, provided latency matching is adequate; equal skipped-tick shares alone do not establish this. The variation we saw between runs sug- gests that real-time evaluations should report run-level as well as game-level uncertainty. What carries over to other systems is the procedure, a latency- matched arm, a skip check and distinct games as the unit; the numbers are those of this game, these deciders and these hosts. CRediT authorship contribution statement Chihoon Shin: Conceptualization, Methodology, Investigation, Formal analysis, Data curation, Visualization, Writing – original draft, Writing – 27 review & editing, Project administration. Junyeong Lee: Software, Re- sources, Validation. Kihyeok Jeong: Validation, Supervision, Writing – review & editing. Wonok Kwon: Methodology, Validation, Writing – re- view & editing. Funding This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors. Declaration of competing interest Chihoon Shin and Junyeong Lee are affiliated with the Myeongseongsim- jae AX Institute, where the distilled decider evaluated in this study was developed. The other authors declare that they have no known compet- ing financial interests or personal relationships that could have appeared to influence the work reported in this paper. Data availability Preregistrations with their SHA-256 hashes, analysis scripts, decider weights and per-episode logs will be made publicly available upon publica- tion; they are available to reviewers on request. Declaration of generative AI and AI-assisted technologies in the manuscript preparation process During the preparation of this work the authors used Claude Opus 5 and, from 23 September 2026, Claude Opus 5.5 (Anthropic; Claude Code) and gpt-6-sol and gpt-6-astra (OpenAI; Codex command-line tool 0.156) in order to organise related literature, structure the manuscript, draft and edit text, and prepare figures from logged data. Their use in the research itself is summarised in Section 4.3 and described in Section S1.2. No AI tool created or altered images of experimental observations: all figures are plotted by scripts from logged results or rendered by the game engine or the simulator’s viewer. After using these tools, the authors reviewed and edited the content as needed and take full responsibility for the content of the publication. 28 References Agarwal, R., Schwarzer, M., Castro, P.S., Courville, A., Bellemare, M.G., 2021. Deep Reinforcement Learning at the Edge of the Statistical Precipice, in: Advances in Neural Information Processing Systems, p. 29304–29320. URL: https://papers.nips.c/paper_files/paper/ 2021/hash/f514cec81cb148559cf475e7426eed5e-Abstract.html. Anokhin, I., Rishav, R., Riemer, M., Chung, S., Rish, I., Kahou, S.E., 2025. Handling Delay in Real-Time Reinforcement Learning, in: The Thirteenth International Conference on Learning Representations. URL: https:// openreview.net/forum?id=YOc5t8PHf2. Bellemare, M.G., Naddaf, Y., Veness, J., Bowling, M., 2013. The Arcade Learning Environment: An Evaluation Platform for General Agents. Jour- nal of Artificial Intelligence Research 47, 253–279. doi:10.1613/jair. 3912. Bouthillier, X., Delaunay, P., Bronzi, M., Trofimov, A., Nichyporuk, B., Szeto, J., Sepah, N., Raff, E., Madan, K., Voleti, V., Kahou, S.E., Michalski, V., Serdyuk, D., Arbel, T., Pal, C., Varoquaux, G., Vincent, P., 2021. Accounting for Variance in Machine Learning Benchmarks, in: Proceedings of Machine Learning and Systems, p. 747–769. URL: https://proceedings.mlsys.org/paper_files/paper/ 2021/hash/0184b0cd3cfb185989f858a1d9f5c1eb-Abstract.html. Chan, S.C.Y., Fishman, S., Canny, J., Korattikara, A., Guadarrama, S., 2020. Measuring the Reliability of Reinforcement Learning Algorithms, in: International Conference on Learning Representations. URL: https: //openreview.net/forum?id=SJlpYJBKvH. Chen, B., Xu, M., Li, L., Zhao, D., 2021. Delay-Aware Model-Based Rein- forcement Learning for Continuous Control. Neurocomputing 450, 119– 128. doi:10.1016/j.neucom.2021.04.015. Chen, J., Wang, S., Zhu, G., Xu, C., 2026. Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning. arXiv preprint arXiv:2604.01681. Chen, P., Bu, P., Song, J., Gao, Y., Zheng, B., 2024. Can VLMs Play Action Role-Playing Games? Take Black Myth Wukong as a Study Case. arXiv preprint arXiv:2409.12889. 29 Clary, K., Tosch, E., Foley, J., Jensen, D., 2019. Let’s Play Again: Variability of Deep Reinforcement Learning Agents in Atari Environments. arXiv preprint arXiv:1904.06312. Cobbe, K., Hesse, C., Hilton, J., Schulman, J., 2020. Leveraging Pro- cedural Generation to Benchmark Reinforcement Learning, in: Interna- tional Conference on Machine Learning, p. 2048–2056. URL: https: //proceedings.mlr.press/v119/cobbe20a.html. Colas, C., Sigaud, O., Oudeyer, P.Y., 2018. How Many Random Seeds? Statistical Power Analysis in Deep Reinforcement Learning Experiments. arXiv preprint arXiv:1806.08295. Colas, C., Sigaud, O., Oudeyer, P.Y., 2019. A Hitchhiker’s Guide to Sta- tistical Comparisons of Reinforcement Learning Algorithms, in: ICLR 2019 Workshop on Reproducibility in Machine Learning. URL: https: //openreview.net/forum?id=ryx0N3IaIV. Derman, E., Dalal, G., Mannor, S., 2021. Acting in Delayed Environments with Non-Stationary Markov Policies, in: International Conference on Learning Representations. URL: https://openreview.net/forum?id= j1RMMKeP2gR. Dong, P., Hung, K.H., Sadigh, D., Finn, C., 2026. Reinforcement Learning for Real-Time Vision-Language-Action Policies. arXiv preprint arXiv:2609.18207. Dulac-Arnold, G., Mankowitz, D., Hester, T., 2019. Challenges of Real- World Reinforcement Learning. arXiv preprint arXiv:1904.12901. Firoiu, V., Ju, T., Tenenbaum, J., 2018. At Human Speed: Deep Reinforce- ment Learning with Action Delay. arXiv preprint arXiv:1810.07286. Golchinfar, D., Vaziri, D., Marquardt, A., 2026. Playing DOOM with 1.3M Parameters: Specialized Small Models vs Large Language Models for Real- Time Game Control. arXiv preprint arXiv:2604.07385. Henderson, P., Islam, R., Bachman, P., Pineau, J., Precup, D., Meger, D., 2018. Deep Reinforcement Learning that Matters. Proceedings of the AAAI Conference on Artificial Intelligence 32. doi:10.1609/aaai.v32i1. 11694. Hinton, G., Vinyals, O., Dean, J., 2015. Distilling the Knowledge in a Neural Network. arXiv preprint arXiv:1503.02531. 30 Hofman, J.M., Chatzimparmpas, A., Sharma, A., Watts, D.J., Hullman, J., 2023. Pre-registration for Predictive Modeling. arXiv preprint arXiv:2311.18807. Hu, S., Huang, T., Liu, G., Kompella, R.R., Ilhan, F., Tekin, S.F., Xu, Y., Yahn, Z., Liu, L., 2024. A Survey on Large Language Model-Based Game Agents. arXiv preprint arXiv:2404.02039. Huang, Z., Zhang, Y., Liu, J., Song, R., Tang, C., Ma, J., 2026. TIC-VLA: A Think-in-Control Vision-Language-Action Model for Robot Navigation in Dynamic Environments. arXiv preprint arXiv:2602.02459. Kalyanakrishnan, S., Aravindan, S., Bagdawat, V., Bhatt, V., Goka, H., Gupta, A., Krishna, K., Piratla, V., 2021. An Analysis of Frame-skipping in Reinforcement Learning. arXiv preprint arXiv:2102.03718. Kang, H., Zhang, Q., Cai, H., Xu, W., Krishna, T., Du, Y., Weissman, T., 2025a. Win Fast or Lose Slow: Balancing Speed and Accuracy in Latency-Sensitive Decisions of LLMs, in: Advances in Neural Information Processing Systems 38, p. 167482–167504. doi:10.52202/085713-5044. Kang, M., Jeong, J., Lee, S., Cho, J., Hwang, S.J., 2025b. Distilling LLM Agent into Small Models with Retrieval and Code Tools, in: Advances in Neural Information Processing Systems 38, p. 117920–117957. doi:10. 52202/085713-3553. Karamzade, A., Kim, K., Lanier, J., Corsi, D., Fox, R., 2025. Model- Based Reinforcement Learning under Random Observation Delays. arXiv preprint arXiv:2509.20869. Karl, F., Kemeter, L.M., Dax, G., Sierak, P., 2024. Position: Embracing Negative Results in Machine Learning, in: International Conference on Machine Learning, p. 23256–23265. URL: https://proceedings.mlr. press/v235/karl24a.html. Kempka, M., Wydmuch, M., Runc, G., Toczek, J., Jaśkowski, W., 2016. ViZDoom: A Doom-based AI Research Platform for Visual Reinforcement Learning, in: 2016 IEEE Conference on Computational Intelligence and Games (CIG), p. 1–8. doi:10.1109/cig.2016.7860433. Li, B., Shi, N., 2026. The Latent Bridge: A Continuous Slow-Fast Channel for Real-Time Game Agents. arXiv preprint arXiv:2606.24470. 31 Li, Y., Gong, J., Thompson, S., Javanmardi, E., Zhang, Q., Zeng, Z., Liu, S., Wang, P., Guo, Z., Tsukada, M., 2026. Think at 5 Hz, Act at 20 Hz: Asynchronous Fast-Slow Vision-Language-Action Inference for Closed-Loop Driving. arXiv preprint arXiv:2607.15621. Li, Y., Lin, C., Nasir, M.U., Bontrager, P., Liu, J., Togelius, J., 2025. GVGAI-LLM: Evaluating Large Language Model Agents with Infinite Games. arXiv preprint arXiv:2508.08501. Lin, Z., Sun, Z., Chen, S., Chen, X., Zhao, R., 2025. Accelerating Multi- modal LLM Gaming Performance via Input Prediction and Mishit Cor- rection. arXiv preprint arXiv:2512.17250. Liu, J., Yu, C., Gao, J., Xie, Y., Liao, Q., Wu, Y., Wang, Y., 2024. LLM- Powered Hierarchical Language Agent for Real-time Human-AI Coordi- nation, in: International Joint Conference on Autonomous Agents and Multiagent Systems, p. 1219–1228. doi:10.65109/zdxp6361. Machado, M.C., Bellemare, M.G., Talvitie, E., Veness, J., Hausknecht, M., Bowling, M., 2018. Revisiting the Arcade Learning Environment: Evalua- tion Protocols and Open Problems for General Agents. Journal of Artificial Intelligence Research 61, 523–562. doi:10.1613/jair.5699. Mahmood, A.R., Korenkevych, D., Komer, B.J., Bergstra, J., 2018a. Setting up a Reinforcement Learning Task with a Real-World Robot, in: 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), p. 4635–4640. doi:10.1109/iros.2018.8593894. Mahmood, A.R., Korenkevych, D., Vasan, G., Ma, W., Bergstra, J., 2018b. Benchmarking Reinforcement Learning Algorithms on Real-World Robots, in: Conference on Robot Learning, p. 561–591. URL: https: //proceedings.mlr.press/v87/mahmood18a.html. Motubrain Team, 2026. World Action Models in Real Time: An Empirical Study of Smooth Execution via Asynchronous Deployment. arXiv preprint arXiv:2608.01880. Nguyen, H.H., Driessens, K., Soemers, D.J.N.J., 2026. For How Long Should We Be Punching? Learning Action Duration in Fighting Games. arXiv preprint arXiv:2605.20911. Ouyang, M., Hu, S., Lin, K.Q., Ng, H.T., Shou, M.Z., 2026. GameWorld: Towards Standardized and Verifiable Evaluation of Multimodal Game Agents. arXiv preprint arXiv:2604.07429. 32 Park, D., Kim, M., Choi, B., Kim, J., Lee, K., Lee, J., Park, I., Lee, B.U., Hwang, J., Ahn, J., Mahabaleshwarkar, A.S., Kartal, B., Biswas, P., Suhara, Y., Lee, K., Cho, J., 2025. Orak: A Foundational Benchmark for Training and Evaluating LLM Agents on Diverse Video Games. arXiv preprint arXiv:2506.03610. Patterson, A., Neumann, S., White, M., White, A., 2024. Empirical Design in Reinforcement Learning. Journal of Machine Learning Research 25, 1–63. URL: https://jmlr.org/papers/v25/23-0183.html. Peng, D., Ma, F., Wang, B., Wang, S., Ma, J., 2026. Latency-Tolerant Cloud-Edge Collaborative Vision-Language-Action Models via Emergent Representational Specialization. arXiv preprint arXiv:2608.00569. Ramstedt, S., Pal, C., 2019.Real-Time Reinforcement Learn- ing, in: Advances in Neural Information Processing Systems. URL: https://papers.nips.c/paper_files/paper/2019/hash/ 54e36c5f5f6a1802925ca009f3ebb68-Abstract.html. Ross, S., Gordon, G.J., Bagnell, J.A., 2011. A Reduction of Imitation Learning and Structured Prediction to No-Regret Online Learning, in: Proceedings of the Fourteenth International Conference on Artificial In- telligence and Statistics, p. 627–635. URL: https://proceedings.mlr. press/v15/ross11a.html. Saha, S., Prasad, A., Chen, J.C.Y., Hase, P., Stengel-Eskin, E., Bansal, M., 2025. System-1.x: Learning to Balance Fast and Slow Planning with Language Models, in: The Thirteenth International Conference on Learning Representations. URL: https://openreview.net/forum?id= zd0iX5xBhA. Schwartz, E., Tennenholtz, G., Tessler, C., Mannor, S., 2019. Language is Power: Representing States Using Natural Language in Reinforcement Learning. arXiv preprint arXiv:1910.02789. Shao, X., Jiang, W., Zuo, F., Liu, M., 2024. SwarmBrain: Embodied agent for real-time strategy game StarCraft I via large language models. arXiv preprint arXiv:2401.17749. Srinivas, A., Sharma, S., Ravindran, B., 2016. Dynamic Frame skip Deep Q Network. arXiv preprint arXiv:1605.05365. 33 Travnik, J.B., Mathewson, K.W., Sutton, R.S., Pilarski, P.M., 2018. Reac- tive Reinforcement Learning in Asynchronous Environments. Frontiers in Robotics and AI 5, 79. doi:10.3389/frobt.2018.00079. Wen, Y., Ye, Y., Zhang, Y., Yang, D., Zhu, H., 2025. Real-Time Reasoning Agents in Evolving Environments. arXiv preprint arXiv:2511.04898. de Wynter, A., 2025. Will GPT-4 Run DOOM? IEEE Transactions on Games 17, 451–459. doi:10.1109/tg.2024.3497601. Xiao, T., Jang, E., Kalashnikov, D., Levine, S., Ibarz, J., Hausman, K., Herzog, A., 2020. Thinking While Moving: Deep Reinforcement Learning with Concurrent Control, in: International Conference on Learning Rep- resentations. URL: https://openreview.net/forum?id=SJexHkSFPS. Yu, P., Xu, J., Weston, J., Kulikov, I., 2024. Distilling System 2 into System 1. arXiv preprint arXiv:2407.06023. Yuan, Y., Mahmood, A.R., 2022. Asynchronous Reinforcement Learning for Real-Time Control of Physical Robots, in: 2022 International Confer- ence on Robotics and Automation (ICRA), p. 5546–5552. doi:10.1109/ icra46639.2022.9811771. Zhang, A.L., Griffiths, T.L., Narasimhan, K.R., Press, O., 2025a. VideoGameBench: Can Vision-Language Models complete popular video games? arXiv preprint arXiv:2505.18134. Zhang, S., Wang, X., Zhang, W., Li, C., Song, J., Li, T., Qiu, L., Cao, X., Cai, X., Yao, W., Zhang, W., Wang, X., Wen, Y., 2025b. Leveraging Dual Process Theory in Language Agent Framework for Real-time Simultaneous Human-AI Collaboration, in: Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 4081–4108. doi:10.18653/v1/2025.acl-long.206. Zheng, Y., Mao, S., Zhang, D., Cai, W., 2025. Reflex First, Reflect Later: Latency-Aware Embodied LLM Agents for Dynamic Response. arXiv preprint arXiv:2506.07223. Zhu, B., Khalil, M., Harrison, E., Poggi, E., Schmitt, P., Kast, B., Meis- ter, P., Atreya, P., Li, Q., Ferchau, F., Colmenero, C., Shahapurkar, Y., Narayanan, G., Erdogan, M., Wurm, K., von Wichert, G., Mees, O., Solowjow, E., Wagenmaker, A., Levine, S., 2026. Learning to Act While Waiting: RL Finetuning of Generalist Robot Policies Under Inference La- tency. arXiv preprint arXiv:2608.23831. 34 Supplementary material Supplementary Material Separating Decision Time from Decision Quality in the Real-Time Gap of Distilled Deciders: Evidence from a Game and a Conveyor Simulator S1. Material moved from the main text This section holds analyses and discussion that the main text summarises in a sentence. S1.1. Further related work Agents that must choose the next action while the previous one is still executing face concurrent control (Xiao et al., 2020). Recent work formalises observational delay caused by the network’s own inference (Anokhin et al., 2025), random observation delays that arrive out of sequence (Karamzade et al., 2025), the violation of the Markov assumption when an executed action depends on an earlier state (Zhu et al., 2026), and the distribution shift created when the observation behind an action is stale by the time it executes (Dong et al., 2026); the remedies are state augmentation, history- augmented observations, architectural skip connections and fast action edits on top of a slow policy. Our study instead measures what a real host adds and separates its effect from that of decision quality. Language-model game agents. Surveys single out low-latency control as a requirement of some genres (Hu et al., 2024); benchmarks render games as text (Li et al., 2025; Park et al., 2025); and real-time reasoning environ- ments change while the model is still thinking (Wen et al., 2025). ViZDoom states have been represented in natural language before (Schwartz et al., 2019). Inference latency is a recognised obstacle for language-model agents in real-time games (Liu et al., 2024; Shao et al., 2024; de Wynter, 2025), and dual-process designs combine fast rules with asynchronous reasoning (Zhang et al., 2025b). We report language-model controllers only as peripheral com- parators (Section S12). Latency in deployed vision-language-action systems. Policies trained under idealised synchronous supervision can degrade severely under realistic infer- ence latency, which has motivated injecting inference delays during imitation and reinforcement learning (Huang et al., 2026). Such designs reduce the cost of latency without estimating how much of a measured loss it explains. 35 S1.2. Use of AI agents in the research Use of AI agents in the research. Large language model agents (Claude Opus 5 until 23 September 2026 and Claude Opus 5.5 from that day, through the Claude Code environment; gpt-6-sol and gpt-6-astra through the Codex command-line tool 0.156) were used as research assistants: they proposed and critiqued experiment designs, implemented the experiment and analysis scripts, launched and monitored runs, and drafted analyses. To keep their output from entering the results unchecked, every experiment was frozen as a preregistration with a SHA-256 hash before its seeds were opened; every analysis script was run on positive and negative controls before it saw the results; every number reported in this paper is produced by a script from logged episode files and checked by an automated gate against those files; and the authors reviewed each design and result. Two errors introduced during the project (an agreement statistic that initially included decisions taken without ammunition, and an agent-produced draft that silently shortened the manuscript) were caught by these checks and corrected; they are reported in Supplementary Section S11. Use of AI agents. Large language model agents served as research assis- tants: they proposed and critiqued experiment designs, implemented the experiment and analysis scripts, launched and monitored runs, and drafted analyses. Their output entered the results only through fixed checks: every experiment was preregistered and hashed before its seeds were opened, ev- ery analysis script was run on positive and negative controls before it saw the results, and every reported number is produced by a script from logged episode files and checked by an automated gate against those files. The agents and versions used, and the errors these checks caught, are described in Section S1.2. S1.3. Notes on the analysis Interpretation of Proposition 1. The Lipschitz condition requires changes in the teacher’s metric to be no greater than a constant times changes in expected skipped-tick share. Proposition 1 turns the manipulation check into a bias term. The Lipschitz constant is not known; as a sensitivity estimate we plug in the slope fitted on all 450 episodes of the intervention study (ˆκ ≈ 0.61 accuracy per unit skipped-tick share, common to all arms; Section S1.4). In triad 1, the skipped-tick shares differ by 8.3 p. Using the fitted slope, this difference corresponds to an estimated accuracy bias of about 5 p. This is an estimate, not a certified bound, and matched mean 36 skipped-tick shares do not guarantee matched skip processes; we therefore report skip-adjusted estimates beside, not instead of, the preregistered ones. Two consequences. First, disagreements on skipped ticks cost nothing, be- cause both policies must repeat a t−1 ; only disagreements at decision ticks enter ε D . Second, the relevant error is measured on the decider’s own asyn- chronous states, d D , not on the teacher’s; a decider can agree with the teacher on 98.4% of teacher states and still carry an ε D several times larger, which is what Table S1 shows. Table S1: Decider agreement with the teacher’s actions State sourceDeciderAgreementError version(%) rate (%) Teacher demonstrations (validation split, 2,500 sampled) original98.41.6 On-policy, round 1 (24,547 decisions, 60 games) original88.811.2 On-policy, round 2 (25,529 decisions, 60 games) after one round90.010.0 Decisions with an enemy in view and ammunition left: 24,547 and 25,529 of the 44,820 and 47,905 decisions collected in the two rounds. On-policy states were collected in asynchronous play on seeds disjoint from evaluation. Ammunition filtering. Disagreements after the ammunition has run out have Q T t (x,π T (x))− Q T t (x,a) ≈ 0 for the firing part of the action, which is why we exclude them from the agreement statistic. DAgger and its bound. This is the performance-difference argument behind DAgger (Ross et al., 2011), applied to the augmented process with the skip process folded into the dynamics. DAgger trains on states drawn from d D , so it targets the disagreement term of the bound in Proposition 2. It min- imises a surrogate loss on aggregated data, which does not guarantee lower disagreement for the final policy on its own states, nor a lower actual gap; the bound also concerns an additive return, not accuracy. Whether the gap falls is the empirical question of Section 6.4. Additive return versus accuracy. The proposition concerns an additive return such as expected hits. Our primary metric is accuracy, the ratio of hits to shots. The two move together here because the arms fire almost the same number of shots (Section S1.14). This link is empirical and is not part of the proposition. 37 849096 O N32 N S T0 hit accuracy (%, distinct games) 2124 kills per episode 243036 lone-enemy reaction (ticks) 6.07.5 hits taken per episode 105012001350 ticks survived per episode Figure S1: Five metrics in the replication study (v6b; 100 seeds, 56 distinct games). Dots: means; bars: 95% intervals from resampling distinct games. Lone-enemy reaction: ticks from the first tick with one enemy in view to the first shot aimed within 6 degrees of it. Hits taken depends on survival time and is not ranked. S1.4. Further metrics, run-level variation and a skip-adjusted estimate Four further metrics in the replication study provide context for the accuracy result (Figure S1). Kills per episode and reaction to a lone enemy follow the same broad teacher-versus-decider pattern: the original decider kills fewest enemies and reacts slowest. The zero-latency teacher survives about as long as S. Hits taken does not follow the ordering, because arms that survive longer are exposed to enemies for longer. The accuracy result is therefore not an artefact of firing fewer shots. Host variation. The run-level interval of the improvement includes zero. In triads 1 and 2, both deciders skipped far more ticks than S (Figure S2, middle panel): N 15.7% and 11.1%, O 19.0% and 25.3%, S 7.4% and 6.8%. O skipped more than N, whichever order the arms ran in. These triads failed the S/N check (ratios 0.47 and 0.61) and also the S/O check (0.39 and 0.27); in triads 3–5 all three arms skipped 6.7–9.9% of ticks. Excess skipping plausibly lowers a decider’s accuracy, so these triads likely inflate both the quality component and the improvement over O. In triads 3–5 the improvement was 4.9, 1.5 and−0.1 p and the remaining quality component 2.3, 3.0 and 2.7 p; this split was not preregistered. Figure S3 shows how accuracy relates to skipping across runs in the three studies. The fitted slopes are negative for most arms, while the original de- cider in the replication study (v6b) has a small positive slope whose interval 38 T0*SO 75 80 85 90 95 hit accuracy, distinct games (%) v4+v4b · 18 games 5 S/O pairs 95.5 86.8 80.0 T0*SNO v5 · 18 games 5 S/N/O triads 95.5 87.3 82.8 78.7 triad failing the S/N skip match T0SNN32O v6b · 56 games 4 blocks of 25 seeds 94.3 87.9 84.2 84.7 81.6 Figure S2: Hit accuracy by run in the three studies on host 1: runs (v4+v4b: 5 S/O pairs; v5: 5 S/N/O triads; 18 games each) or 25-seed blocks (v6b: 4 blocks, 56 games). Squares: the three earlier-session T0 runs; filled diamonds: pooled, with 95% game- bootstrap intervals; dashed: the two v5 triads outside the skip band 0.8–1.25. includes zero. Only three of the nine intervals exclude zero (N and O in the intervention study, v5, and N32 in v6b), and the negative slope of O in v5 is driven by the two triads that failed the skip match. With four or five runs per slope, and with skipping that may itself depend on the policy, these slopes are descriptive only. 39 43210123 change in hit accuracy (points) per 1-point increase in the skipped-tick share S (n=5) O (n=5) S (n=5) N (n=5) O (n=5) S (n=4) N (n=4) N32 (n=4) O (n=4) v4+v4b (runs) 0.13 [1.36, +1.11] 1.31 [4.23, +1.60] v5 (runs) 0.95 [2.61, +0.70] 0.71 [1.40, 0.01] 0.57 [0.78, 0.36] v6b (25-seed blocks) 0.47 [1.12, +0.18] 0.27 [3.10, +2.57] 0.53 [0.67, 0.39] +0.34 [0.03, +0.70] slope [95% CI] lower accuracy with more skippinghigher Figure S3: Slope of hit accuracy on skipped-tick share, run by run, with 95% t-intervals; n: runs (v4+v4b, v5: 18 games) or 25-seed blocks (v6b: 56 games). Filled circles: intervals that exclude zero. T0 is omitted because it skips almost no ticks. Scatter: Figure S4. 510152025 skipped ticks in run (%) 72.5 75.0 77.5 80.0 82.5 85.0 87.5 90.0 hit accuracy, distinct games (%) v4+v4b · runs S -0.13 [-1.36, +1.11] O -1.31 [-4.23, +1.60] 510152025 skipped ticks in run (%) v5 · runs (hollow: failed skip match) S -0.95 [-2.61, +0.70] N -0.71 [-1.40, -0.01] O -0.57 [-0.78, -0.36] 510152025 skipped ticks in run (%) v6b · 25-seed blocks S -0.47 [-1.12, +0.18] N -0.27 [-3.10, +2.57] N32 -0.53 [-0.67, -0.39] O +0.34 [-0.03, +0.70] Figure S4: Run-level data behind Figure S3: hit accuracy over distinct games against the skipped-tick share of each run (v4+v4b, v5) or 25-seed block (v6b), with least-squares lines per arm and the slope with its 95% t-interval in each legend entry; hollow v5 markers are the triads that failed the S/N skip match. T0 is omitted because it skips almost no ticks. 40 Skip-adjusted estimate (exploratory). Because skipping varies within as well as between runs, we regressed each episode’s hit accuracy on arm indicators and the episode’s skipped-tick share (450 episodes, weighted by shots, stan- dard errors clustered by the 18 distinct games). The regression estimates an accuracy decrease of 6.1 p for each additional 10% of skipped ticks (stan- dard error 0.4 p). At equal skipping, N exceeds O by 3.9 p (standard error 0.8) and falls 2.1 p below S (standard error 1.0). This analysis was not preregistered, and a policy can influence its own skipping, so we treat it as supporting evidence only. S1.5. Input and policy shares (v10) Input and policy (v10). The quality component compares a teacher reading exact coordinates with a decider reading a sentence. On host 1 we added a teacher S that decides from the same sentence the decider reads (it agrees with the coordinate teacher on every decision outside the lowest ammunition bin). The input share, S− S, is−1.3 p [−3.7, 1.1], below the preregistered 2 p threshold; the policy share, S− N, is 3.2 p [0.7, 5.7]. For accuracy we found no measurable input loss, and the policy share accounts for most of the quality component. The sentence does cost kills: S does not fire with 1–4 bullets left and kills 2.96 fewer enemies per game [2.26, 3.62]. S1.6. Decision period and stale decisions (v15) Decision period and stale decisions (v15). A decider inside the game can choose how often to decide; a decider far from the game decides only as of- ten as its round trip allows, and on a state that is already old. On host 2 we held each of O’s actions for k = 0, 1, 2, 4 or 8 ticks after deciding (arms H0 to H8), and added an arm L8 that acts every tick on the decision O made 8 ticks (228.6 ms) earlier. The best period differs by scene (Figure S5). In defend_the_center (29 distinct games) deciding every tick is best: holding each action for two ticks lowers hit accuracy from 87.1% to 28.3% (H2− H0 = −58.9 p [−67.6, −51.4]). In deadly_corridor (200 distinct games) hold- ing for four ticks raises the mean reward from 58.8 to 105.2 (H4− H0 = 46.4 [28.8, 64.2]), while holding for one tick lowers it (H1 − H0 = −21.9 [−35.6, −8.6]), against our preregistered prediction of a gain. A fresh decision held for 8 ticks (H8, a new decision every 8 ticks) against a decision 8 ticks old applied every tick (L8): H8 scores 35.1 points more than the stale decision in deadly_corridor [19.3, 50.7]; in defend_the_center the difference is incon- clusive (−7.8 p [−17.4, 1.2]). A distant decider pays the stale-decision cost by construction; a fast local one can choose the period that suits the scene. 41 01248 0 20 40 60 80 100 hit accuracy (%) defend_the_center (29 games) 87.1 66.8 28.3 31.5 29.0 stale decision, 8 ticks late: 36.8 01248 0 25 50 75 100 mean episode reward deadly_corridor (200 games) 58.8 36.9 44.9 105.2 105.0 stale decision, 8 ticks late: 69.9 12488 fresh minus stale ticks held after each decision (k) 60 40 20 0 difference from k = 0 12488 fresh minus stale ticks held after each decision (k) 25 0 25 50 difference from k = 0 a b Figure S5: How often to decide depends on the scene: O on host 2 holds each action for k ticks (v15). (a) Hit accuracy (defend_the_center) and mean reward (deadly_corridor) by k; dashed: O acting every tick on a decision 8 ticks (228.6 ms) old. (b) Differences from k = 0 and fresh versus stale (square), with 95% game-bootstrap intervals. 42 T0SOT0SO 60 70 80 90 100 hit accuracy (%) 95.5 86.8 80.0 time quality v4+v4b 94.3 87.9 81.6 time quality v6b a ratio: hit accuracy T0SOT0SO 14 16 18 20 22 24 26 hits per game 24.7 22.1 18.9 time quality v4+v4b 24.3 22.4 19.4 time quality v6b b additive: hits per game Figure S6: The decomposition on two scales (exploratory for b). (a) Hit accuracy of T0, S and O in the decomposition study (v4+v4b; 95.5, 86.8, 80.0%) and the replication (v6b; 94.3, 87.9, 81.6%). (b) Hits per game in the same studies (24.7, 22.1, 18.9 and 24.3, 22.4, 19.4). Arrows mark the time and quality components. S1.7. Concurrent evaluation on one host Concurrent evaluation on one host. To shorten wall-clock time, we prereg- istered a calibration of concurrent evaluation with each lane pinned to its own processor cores (core pinning). Three lanes, each with its own decider server, ran the latency-matched teacher at the same time. We would use the setup only if every lane stayed within 86.2–89.2% raw accuracy and 5.5– 12.1% skipped ticks. These bands came from five solo runs. All three lanes failed: they skipped 82.5–83.2% of ticks and hit 53.2–58.3% of their shots. We did not separate whether pinning, concurrency or both caused this. On this host, asynchronous evaluations had to run one at a time. 43 76788082848688 hit accuracy on distinct games (%) N M in p [pooled 95% interval] pooled (4 blocks) block 4 block 3 block 2 block 1 O (v6b) N (v6b) S (v6b) N M +7.6 [4.8, 10.4] +9.4 +6.8 +7.1 +7.6 77.284.8 M: +92,725 rows from teacher-played games N: +92,725 rows from its own asynchronous play (DAgger) Figure S7: The data-volume control (v7; 56 distinct games, 4 blocks): N adds 92,725 rows from its own asynchronous play, labelled by the teacher; M adds 92,725 rows from new teacher-played games. Gray segments: blocks; brackets: paired 95% game-bootstrap interval of the pooled difference. Dotted: O, N and S from v6b; T0 lies off the axis at 94.3. the 18 distinct games, sorted by the loss 60 70 80 90 100 hit accuracy of the decider (%) pooled 98.6% pooled 83.8% tick conversion asynchronous play Figure S8: Tick conversion against asynchronous play for the same decider on the 18 distinct games of seeds 850000–850029, sorted by the loss. Pooled hit accuracy: 98.6% under tick conversion and 83.8% in asynchronous play. 44 TLTRSLSRFLFR decider action turn left (TL) turn right (TR) strafe left (SL) strafe right (SR) fire+strafe L (FL) fire+strafe R (FR) teacher action 9341011 21741112 0099010 00010000 0000991 0001099 11,635 7,190 992 819 1,658 2,253 n round 1 (O plays) TLTRSLSRFLFR decider action TL TR SL SR FL FR 9520011 18761122 00100000 00010000 0030970 0000099 11,777 7,974 926 785 1,697 2,370 n round 2 (N1 plays) ON1 N1 O = 2.4 [ 5.5, +0.6] 15 20 TL share of TR (%) 20.7 18.2 TRTL, 95% CI Figure S9: On-policy errors in the DAgger rounds of v5 (enemy in view, ammunition left), driven by O and N1; row percentages (sums 99–100, errors on one 0–25% scale). Right: TR→TL rate, O 20.7% [18.7, 22.7], N1 18.2% [16.0, 20.5], difference −2.4 [−5.5, 0.6] (descriptive). No F row: the teacher never chose it. S1.8. Further figures of the main studies S1.9. Further diagnostics of the quality component S1.10. Decision time on the two hosts 235102030 median decision time within an episode (ms, log scale) 0.0 0.2 0.4 0.6 0.8 1.0 share of episodes one tick (28.6 ms) host 1 (Windows), 1,300 episodes host 2 (Linux), 860 episodes Figure S10: Decision time on the two hosts (exploratory): cumulative share of episodes by the median decision time within the episode, 1,300 decider episodes on host 1 and 860 on host 2 (log scale). The dotted line marks one engine tick (28.6 ms). S1.11. Study ledger figure S1.12. The two environments S1.13. Identical schedules: a controlled-timing emulator The live study of Section 6.2 imposed one schedule in live play on host 2. To impose the recorded host-1 schedules exactly, and synthetic ones as 45 1101001,00010,000 episodes played (log scale; 1 = not executed) S1/E1 E2 v3 v4 v4b v5 v6a v6b v6b+ v7 v8 v9 v10 v11 v11-h1 v12 v13p v14 v14b v15 v16 v16b v17 v18 f1 f1-solo corr-att corr-att2 host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 1 · defend host 2 · corridor host 1 · corridor host 2 · defend host 2 · corridor host 2 · both host 2 · both host 2 · both host 1 · defend host 1 · defend host 1 · defend host 2 · defend host 2 · conveyor host 2 · conveyor host 1 · corridor host 1 · corridor preregisteredexploratoryfailedwithdrawn Figure S11: All 28 studies of this paper (25,550 episodes in total): episodes played, host and scene. Blue: preregistered and run; lilac: exploratory or without a frozen hash; orange: failed its own calibration or fidelity check; gray: withdrawn. Bars at 1: not executed (v8, v14b) or counted in another row (v4 in v4b, v6b+ in v6b). 46 well, we also rebuilt asynchronous play in the synchronous engine, where one schedule can be imposed on every policy. How the timing was rebuilt is described in Section S14.1. With each policy deciding again on its own recorded schedule, hit accuracy matched in all 150 fresh episodes of two preregistered checks and all five game counts in 148, but the death tick differed in some episodes and both preregistered fidelity gates failed narrowly. The recorder had not logged the wait after the last observation, which the emulator then filled with the recorded gaps; in the second check, whose episodes skipped 19.4–39.4% of ticks, every failing episode had such a wait of at least 5 ticks. A third preregistered check on fresh seeds (v17) logged that wait and imposed it: all five game counts matched and every emulated episode ended within its recorded window in 75 of 75 episodes. Its episodes skipped 8.1–9.8% of ticks, as the v9 schedules used below did (7.9–11.3%), and no wait exceeded 2 ticks, so fidelity is confirmed at the main studies’ level of skipping but not for long waits. The results below come from the emulation rerun with that schedule (Section S14). On synthetic schedules imposed identically on T, O and N (Figure S12), a quality component remained without any skipping: O hit 10.0 p less than the teacher [8.0, 12.2] and N 7.1 p less [5.4, 9.0]. The time component rose with the skipped share, from 2.7 p [1.0, 4.4] at 4.0% of ticks skipped to 9.2 p [7.2, 11.1] at 19.9%, while N’s quality component shrank from 7.1 to 2.1 p: time and quality interact. DAgger improved on O on every schedule, by 2.5 to 4.5 p. 47 0.02.55.07.510.012.515.017.520.0 skipped ticks on the identical schedule (%) 0 2 4 6 8 10 12 hit accuracy difference (p) time componentquality component, Oquality component, N Figure S12: Time and quality components on identical synthetic skip schedules imposed on T, O and N (56 distinct games, 100 seeds). Time: T without skipping minus T on the schedule; quality: T minus O and T minus N on the same schedule. Paired 95% game-bootstrap intervals. On the schedules recorded on host 1 in v9 (Table S2), the teacher scored 2.7 p less on the schedule S had met than on the original decider’s own schedule [−6.2, 0.5], with 9.6% and 8.6% of ticks skipped. This bias term is not within our preregistered ±2 p tolerance, so matching shares did not rule it out. With schedules matched exactly, O’s quality component was 7.1 p [3.7, 10.6], against 4.4 p [0.2, 8.9] under the share-matched design: the original decomposition may have understated the quality component and overstated the time component. 48 Table S2: Recorded host-1 schedules replayed identically Contrast (host-1 recorded schedules, 29 games) Original decider O DAgger decider N Teacher on S schedule − teacher on own schedule (bias term) −2.7 [−6.2, 0.5] −2.2 [−6.2, 1.6] Quality, schedules matched exactly7.1 [3.7, 10.6] −0.2 [−2.5, 2.1] Quality, shares matched (original design) 4.4 [0.2, 8.9] −2.5 [−6.7, 1.8] DAgger gain N − O on N’s schedule 4.8 [1.8, 7.6] Schedules recorded on host 1 in v9 (seeds 860000–860049, 29 distinct games). Hit-accuracy differences in p with paired game-bootstrap 95% intervals. S: the teacher waiting for the decider’s server; its schedule is the one the original design compared against. For the DAgger decider the picture differs. With schedules matched ex- actly, N’s quality component on the recorded host-1 schedules was −0.2 p [−2.5, 2.1], indistinguishable from zero, against 3.7 p in the asynchronous replication on 56 games (Section 6.6). Taken alone, this would move the intervention towards the preregistered strong-success criterion. We do not read it that way. First, it rests on the 29 distinct games of these seeds, and its interval still allows a residual of about 2 p. Second, on the synthetic schedules over 56 games N kept a quality component at every skip rate, from 7.1 p [5.4, 9.0] without skipping to 2.1 p at 19.9% (Figure S12). The recorded asynchronous episodes of the same 29 games point to the games rather than to the emulator: in them N’s quality component was also indis- tinguishable from zero (−2.4 p [−6.7, 1.8]; exploratory), while O’s was the 4.4 p above. The preregistered verdict therefore stays partial success. The two emulations agree on direction: DAgger improved on O on the recorded schedules (4.8 p [1.8, 7.6]) and on every synthetic schedule. The live study of Section 6.2, run on other games and on host 2, put N’s quality compo- nent under one imposed schedule at 2.8 p [1.5, 4.1]; whether the emulator’s lower value on these 29 games reflects the games or the emulation is open (Section 7). S1.14. Details shortened in the main text Details that support single sentences of the main text, grouped by the section they belong to. Introduction. A game agent that decides well but slowly and one that decides quickly but less well can look identical on a benchmark that pauses while 49 they think. Fast decision services now make it cheap to put a learned decider inside a live loop (Golchinfar et al., 2026; Park et al., 2025). Related work. Cloud-edge designs remove blocking synchronisation so that the controller always consumes the latest available feature (Peng et al., 2026), and asynchronous deployment of world action models overlaps computation with execution and makes the alignment of commands with observation timestamps an engineering requirement (Motubrain Team, 2026). Other approaches combine language-model planning with model-predictive control (Chen et al., 2026), routing between fast and slow planners (Saha et al., 2025), and speculative execution with a distilled corrector (Lin et al., 2025). Distillation moves knowledge into cheaper models (Hinton et al., 2015; Yu et al., 2024; Kang et al., 2025b), and a 1.3M-parameter model trained on human demonstrations has been reported to outplay language models in de- fend_the_center (Golchinfar et al., 2026). A time-conversion protocol that maps inference time to elapsed simulation time has been proposed for em- bodied agents (Zheng et al., 2025); as Li et al. (2026) note, driving agents are commonly invoked on alternate simulation ticks with the previous command replayed in between; and the speed-accuracy trade-off of agents has been studied in real-time fighting games with local inference (Kang et al., 2025a). Performance is better reported as a distribution than as a point estimate (Clary et al., 2019), comparisons need statistical tests with sample sizes set from the expected effect (Colas et al., 2019), reliability has its own metrics of variability and worst-case risk (Chan et al., 2020), and offline and online evaluation create different pressures (Patterson et al., 2024). Results. Our first preregistration (three pairs) asked whether the decider fell more than 5 p below S in all three pairs; it did not, and the verdict was “mixed”. Its three pairs are the first three of the five-pair study (v4b). The three-pair estimate had suggested that about three quarters of the gap was time; the two added pairs reversed that reading, and we report both. In the intervention study, N does not trade accuracy for fewer shots: it fires 25.6 shots per game against 25.7 for S and 23.5 for O, and kills 21.5 enemies per game against 22.5 for S and 18.4 for O. In the data-volume control, the difference between N and M has a game-bootstrap interval of 4.8–10.4 p and an interval over the four blocks of 5.9–9.6 p; the difference lies between 6.8 and 9.4 p in every block. In the replication, the time component, now measured against a contemporaneous zero-latency teacher, is 6.5 p (4.6– 8.3). On host 2 the decider’s accuracy rose (O 88.6% against 81.6%), but its distance from a teacher that waits as long as it does hardly changed. 50 In deadly_corridor on host 1 (94 distinct games) S skipped 10.0% of ticks against 1.4% for T0 and scored 12.1 points more than T0 ([−0.6, 25.7]): in this scene a held forward step keeps earning progress, so waiting helps. In the controlled-timing emulator, on N’s own recorded schedule, DAgger improved on O by 4.8 p [1.8, 7.6]. In the conveyor simulator, cost per part weights an escaped defect 100, a false rejection 20, a line stop 10, an escalation 5 and a rework 1 (Figure 9). When the belt waited for each decision, cost did not change with delay, but the decider’s throughput fell from 148.7 to 59.8 parts per minute at 600 ms, against 249.7 without waiting. In the tick-conversion comparison, the decider also killed fewer enemies per game in asynchronous play than under tick conversion (19.90 against 23.87) and reacted to a lone enemy more slowly (32.8 ticks against 24.9). Limitations. A preregistered attempt to impose identical schedules by recording and replaying them (v9) failed its own fidelity check and is reported in Section S13; it is not used as evidence. S2. Proofs Throughout, the horizon H is finite, rewards lie in [0, 1], and an episode that ends early is extended with an absorbing zero-reward state, so every sum runs over t = 0,...,H − 1. Proof of Proposition 1. Write J T (Z) = J (π T ,Z) and J D (Z) = J (π D ,Z). The decomposition J T (Z 0 ) − J D (Z D ) = [J T (Z 0 ) − J T (Z D )] + [J T (Z D ) − J D (Z D )] is an identity. In the first bracket the policy is fixed at π T and only the skip process changes; in the second the skip process is fixed at Z D and only the policy changes. Assumption 1 is what makes Z D a well-defined object shared by the two policies: without it, the skip process experienced by a policy would depend on the policy’s own actions and the second bracket would mix a policy change with a skip change. If S runs under Z S , the estimator of the quality component is ˆ Q = J T (Z S )− J D (Z D ) = J T (Z D )− J D (Z D ) | z Q + J T (Z S )− J T (Z D ) . If J T depends on the skip process only through σ(Z) up to a κ-Lipschitz function,|J T (Z S )−J T (Z D )|≤ κ|σ(Z S )−σ(Z D )|, which is the stated bound. The same argument applied to the time component with Z 0 and Z S in place of Z 0 and Z D bounds the bias of the time component by κ|σ(Z S )− σ(Z D )| with the opposite sign, so the two biases cancel in the total gap. 51 Performance difference in the augmented process. Fix Z and let V T t (x) = E P u≥t r(s u ,a u )| x t = x under π T , and Q T t (x,a) = r(s,a)+E[V T t+1 (x t+1 )| x t = x,a t = a], with V T H ≡ 0. Because π T is deterministic, V T t (x) = Q T t (x,π T (x)). Lemma 1. For any policy π D in the augmented process, J (π T ,Z) − J (π D ,Z) = P H−1 t=0 E x∼d D t Q T t (x,π T (x))− Q T t (x,π D (x)) . Proof. Let x 0 ,a 0 ,x 1 ,... be a trajectory of π D . Then J (π D ,Z) = E P t r(s t ,a t ) and, since x 0 has the same law under both policies, J (π T ,Z) =E[V T 0 (x 0 )]. Telescoping, J (π T ,Z)− J (π D ,Z) =E h X t V T t (x t )− r(s t ,a t )− V T t+1 (x t+1 ) i , because P t (V T t (x t )− V T t+1 (x t+1 )) = V T 0 (x 0 )− V T H (x H ) = V T 0 (x 0 ). Condi- tioning each term on (x t ,a t ) with a t = π D (x t ) and using the Markov property of the augmented state,E[r(s t ,a t ) + V T t+1 (x t+1 ) | x t ,a t ] = Q T t (x t ,π D (x t )). Substituting V T t (x t ) = Q T t (x t ,π T (x t )) gives the claim. Proof of Proposition 2. Consider one term of Lemma 1. If z t = 0, both policies are constrained to a t−1 , which is part of x t , so π T (x t ) = π D (x t ) and the term is zero. If z t = 1 and π D (x t ) = π T (x t ), the term is zero. Otherwise it is at most ∆ Q by definition. Hence each term is at most ∆ Q Pr x∼d D t [z t = 1,π D (x)̸= π T (x)], and summing over t gives H ε D ∆ Q . If π T is greedy with respect to Q T , every term is non-negative. Remarks. (i) The bound uses the decider’s own state distribution d D , which under asynchronous play includes the states produced by skipped ticks; agreement measured on the teacher’s distribution does not enter it. (i) ∆ Q can be much smaller than H: a wrong turn at one decision tick is usually corrected at the next, so its effect on the teacher’s value is local. Disagree- ments whose action difference has no effect on the reward (firing without ammunition) contribute nothing. (i) Proposition 2 is an upper bound: it names the disagreement term that DAgger targets, but a smaller bound does not imply a smaller gap; whether the gap falls is shown by the experiments (Sections 6.6 and 6.2). S3. Per-run results Tables S3 and S4 list every asynchronous evaluation run of the decom- position and intervention studies with all recorded metrics. “Acc. dist.” is 52 hit accuracy over distinct games, “Acc. raw” over raw seeds; kills, shots, hits taken and ticks survived are per-episode means over raw seeds; “Lone react.” is the mean reaction to a lone enemy in ticks; “Skip” is the share of engine ticks skipped. Runs are listed in execution order. Table S3: Decomposition study (v4, v4b): five S/D pairs in execution order, seeds 850000– 850029. Run Acc. dist. Acc. raw Kills Shots Hits taken Ticks Lone react. Skip % S185.387.3 22.50 25.75.53 125029.2 11.3 D480.781.6 19.37 23.75.20 111235.07.1 S286.687.9 22.87 26.06.10 123525.55.9 D583.781.9 18.83 23.06.00 107238.79.6 S386.386.7 22.13 25.55.53 124325.96.9 D683.282.4 19.17 23.36.03 108736.06.7 S489.289.7 23.17 25.86.17 127326.48.8 D776.878.0 18.13 23.25.47 106340.8 10.2 S586.788.4 22.30 25.25.23 120726.48.9 D875.877.8 18.40 23.75.53 106636.4 10.5 Table S4: Intervention study (v5): five triads in execution order (triads 2 and 4 in reverse order), seeds 850000–850029. Run Acc. dist. Acc. raw Kills Shots Hits taken Ticks Lone react. Skip % S186.587.2 22.47 25.85.53 123427.07.4 N179.580.3 20.50 25.55.57 115932.9 15.7 O174.574.2 17.60 23.75.70 105035.1 19.0 O272.471.1 16.10 22.65.17 99738.8 25.3 N282.083.3 21.37 25.65.47 119227.2 11.1 S289.588.2 22.40 25.46.00 124530.16.8 S388.687.8 22.63 25.85.93 124726.08.0 N386.387.1 22.37 25.75.43 116428.08.1 O381.382.0 19.43 23.75.13 108131.77.2 O482.582.1 19.47 23.75.77 108434.06.7 N484.085.1 21.67 25.56.23 119128.08.3 S487.087.9 22.83 26.05.80 131428.49.6 S584.986.1 22.10 25.65.40 119527.49.9 N582.284.0 21.70 25.86.07 116329.89.4 O582.382.2 19.57 23.85.10 109237.59.3 53 Table S5: Skip-adjusted contrasts per triad of the intervention study (exploratory). Ad- justed contrasts move each decider to the skipped-tick share of the arm it is compared with using the pooled slope κ of Section S1.4 (Proposition 1). Values in percentage points, skipped-tick shares in percent. Triad Skip S Skip N Skip O S−N S−N adj. N−O N−O adj. 17.415.719.07.01.95.13.0 26.811.125.37.54.89.61.0 38.08.17.22.32.24.95.5 49.68.36.73.03.81.52.5 59.99.49.32.73.0-0.1-0.0 1020304050 skipped ticks in episode (%) 50 60 70 80 90 100 episode hit accuracy (%) S N O Figure S13: Episode hit accuracy against the share of skipped ticks in that episode for the 450 episodes of the intervention study. S4. DAgger collection and training Table S6 summarises the two collection rounds and Figure S14 the train- ing curves. Table S7 gives the confusion between the teacher’s action and the decider’s action, as a percentage of each teacher action. It covers the decider’s own asynchronous states: decisions with an enemy in view and ammunition left. Most on-policy errors are turns in the wrong direction when the teacher turns right; strafing and firing decisions are almost always correct. Table S6: DAgger collection rounds. “Engaged” counts decisions with an enemy in view. Round DriverSeedsRows Engaged Skip % Minutes 1a814587e03 720000–720059 44,82043,70711.531 2240805651d 720100–720159 47,90546,72511.133 54 Table S7: On-policy confusion (rows: teacher action, columns: decider action, % of row). TL/TR turn left/right, F fire, SL/SR strafe, FSL/FSR fire while strafing. Teachern TL TRFSL SR FSL FSR Round 1 (original decider in control), 24,547 decisions TL11,635 93.2 4.0 0.0 0.6 0.4 1.00.9 TR7,190 20.7 73.5 0.0 1.3 1.2 1.32.0 SL992 0.2 0.2 0.0 98.8 0.1 0.70.0 SR819 0.2 0.2 0.0 0.0 99.5 0.00.0 FSL1,658 0.1 0.1 0.0 0.2 0.0 99.10.5 FSR2,253 0.2 0.0 0.0 0.0 0.8 0.0 98.9 Round 2 (decider after one round in control), 25,529 decisions TL11,777 95.5 2.4 0.0 0.3 0.4 0.70.7 TR7,974 18.2 75.7 0.0 1.1 0.9 1.92.1 SL926 0.0 0.0 0.0 99.9 0.0 0.10.0 SR785 0.0 0.0 0.0 0.0 99.9 0.00.1 FSL1,697 0.3 0.1 0.0 2.7 0.0 96.80.1 FSR2,370 0.0 0.1 0.0 0.1 0.4 0.1 99.2 05001000150020002500 training step 0.0 0.2 0.4 0.6 0.8 validation Brier original (teacher data only) after round 1 after round 2 Figure S14: Validation Brier score during training for the original decider and after each DAgger round (seed 31). The validation split of the later rounds includes on-policy rows. 55 S5. Evaluation loop of the tick-converted evaluations Game engine ViZDoom, defend_the_center one arm per run A Scripted teacher B Distilled decider C Small LLM D LLM agent E Slowfast hybrid F Action queue Coordinates teacher only State sentence arms BF tick conversion asynchronous clock mode, one per run action Measurement counters, metrics, distinct games counters per game offline, before runs Imitation Teacher labels Teacher games played by arm A weights run time offline Figure S15: The evaluation loop of the earlier tick-converted evaluations, including the language-model comparator arms (labelled A–F there; in the main text D denotes the original decider). The engine sends exact coordinates to the scripted teacher and a state sentence, written by a perception script, to the learned arms. One arm plays each run, under tick conversion (which waits for each decision and holds the previous action for m(L) ticks) or asynchronous play on wall-clock time. 56 S6. Decider network Request state sentence and candidates Encoder hashed n-grams and overlaps Embedding shared layer, each candidate Scoring context layer: candidate + set Reply softmax: choice and probabilities Set pooling mean and max over candidates learned layer Figure S16: The decider scores each candidate action against the state sentence. A request carries the state sentence and the seven candidate actions with their descriptions. The encoder turns each candidate, with the sentence, into 1,785 hashed word and character n- gram and overlap features; a shared layer (1,785 to 128, ReLU) embeds each candidate; set pooling (mean and max, 256) summarises all candidates; a context layer (the candidate’s 128 values with the 256 pooled values, 384 to 96, ReLU) and a linear score (96 to 1) rate each candidate; a softmax with a fitted temperature returns the choice and a probability for every candidate. 265,665 parameters; about 2 ms per decision on one CPU core; the same code serves both scenes and only the weights change. Training uses the teacher’s labels with a sampled policy-gradient estimate of the Brier score, keeps the step with the best validation Brier score and fits the temperature on a calibration split. No pretrained language model is used. S7. Examples of on-policy errors S8. Latency dose-response under tick conversion S8.1. Tick-converted latency dose response Under tick conversion, latency lowers skill in steps at tick boundaries, a property of the rule that real play need not follow. When the teacher is given a fixed latency under tick conversion, its skill holds and then drops at discrete latencies (Figure S17). With 0, 10 or 20 ms it hits 99.5% of its shots, kills 25.03 enemies per game and reacts to a lone enemy in 25.4 ticks. At 30 ms accuracy falls to 87.8%, kills to 16.97 and reaction rises to 42.5 ticks; 50 ms produces exactly the same numbers. Accuracy is 44.4% at 100 ms, 16.1% at 200 ms and zero at 500 ms. Any latency under one tick gives m = 0, and 30 and 50 ms both give m = 1. 57 0210301003001k5k decision latency (ms, log; 0 left of gap) 0 20 40 60 80 100 hit accuracy (%) one tick (28.6 ms) axis break 0210301003001k5k decision latency (ms, log; 0 left of gap) 0 5 10 15 20 25 kills per game one tick (28.6 ms) axis break teacher + injected latency (measured points) distilled decider (median latency) small LLM (median latency) action queue, 16 per call (median latency) LLM agent (median latency) Figure S17: Under tick conversion, injected latency leaves the teacher’s skill intact below one tick and removes it quickly above, and slower controllers fall on the degraded side. Comparator arms are placed at the median of their per-episode median latency. They are context rather than latency-only comparisons, because their policies differ. The arms are the distilled decider; a 1.7B-parameter open-weight large language model (LLM) choosing every action (small LLM); the same model returning 16 actions per call (action queue); and a command-line language-model agent (LLM agent), on the 20 raw seeds of the comparator evaluation. The gray band is an axis break: 0 ms is plotted to its left. Teacher with fixed injected latency (one run per latency on the 100 seeds of the second tick-converted evaluation; points are measured values joined by lines, not a fitted step); These tick-converted runs were preregistered as single runs without intervals. 58 S9. Duplicated seeds Running the deterministic teacher on 200 seeds produced 103 distinct games, and a single game accounted for 98 of the seeds; two further ranges of 100 seeds gave 47 and 58 distinct games (Figure S18). The 30 evaluation seeds of the main studies give 18, and the 100 seeds of the replication give 56. Intervals computed over seeds treat copies of one game as independent evidence (Colas et al., 2018). 0255075100125150175200 seeds run 0 50 100 150 200 distinct games ruler range (200 seeds) first evaluation second evaluation every seed distinct Figure S18: Distinct teacher games as seeds are added. The dotted line is the case in which every seed gives a new game. S10. Metric qualification Before training, we checked that the event metrics separate scripts known to differ (Figure S19). On 200 seeds the teacher hit 99.7% of its shots and reacted in 25.3 ticks; a variant without strafing hit 99.8% and reacted in 22.3 ticks; a variant that fires whenever the target is within 15 degrees hit 31.9% and reacted in 45.2 ticks; the teacher with 100 ms latency hit 43.1% and reacted in 104.6 ticks, and with 500 ms it hit nothing. 59 teacherno strafe 15° coneteacher +100 ms teacher +500 ms 0 20 40 60 80 100 hit accuracy (%) 99.7 99.8 31.9 43.1 0.0 teacherno strafe 15° coneteacher +100 ms teacher +500 ms 0 20 40 60 80 100 reaction to a lone enemy (ticks) 25.3 22.3 45.2 104.6 none Figure S19: Qualifying the metrics on 200 seeds before any learning. S11. Pilot measurement and corrections The DAgger preregistration cited a pilot on two design games in which the original decider agreed with the teacher on 69.7% of its own engaged de- cisions against 95.8% on teacher states. That pilot included decisions taken with no ammunition left, where the teacher’s label has no effect on accuracy; Table S1 replaces it with agreement on decisions with ammunition left, com- puted before any evaluation of the new decider. A run-level summary field for held ticks recorded zero in every asynchronous run because it counted only modelled delays; all skipped-tick figures come from the per-episode en- gine clock. Training of the first DAgger round was interrupted by the host and restarted from the saved collection data before any evaluation. During manuscript preparation, an AI agent asked to typeset the paper returned a version shortened from 7,545 to about 1,100 words while reporting comple- tion, and another dropped most citations from a draft; both outputs were detected by word, page and citation counts and discarded. S12. Peripheral comparators and host incidents Under tick conversion, a 1.7B-parameter open-weight language model (qwen3:1.7b served by Ollama; Table S8) choosing every action fired no shots without a notation legend or with eight worked examples and hit nothing with the legend; a commercial language-model agent (gpt-6-sol through the Codex command-line tool 0.156.1 with its default settings, run on 2026-09- 23; Table S8) at a median of more than five seconds per decision killed 0.43 enemies per game; asking the small model for 16 actions at a time raised accuracy to 8.6% and kills to 1.93. A slow-fast hybrid in which the small 60 model picks a target about once per second while the teacher’s rules aim and fire hit 100.0% of its shots; its skill comes from the teacher’s rules. Per-decision cost differs by orders of magnitude (Figure S20). In earlier diagnostic asynchronous runs the zero-latency teacher hit 92.4%, 84.8% and 82.8% of its shots with 6.3%, 15.3% and 15.3% skipped ticks at low mean processor load; raising the timer resolution did not change the skip rate. 10 0 10 1 10 2 10 3 10 4 median latency of one model call (ms, log) 0 5 10 15 20 25 kills per game (distinct games) teacher script one tick distilled decider (0 tokens) slow-fast hybrid (308 tok/call) queue k=16 (323 tok/call) small LLM (490 tok/call) LLM agent (20,791 tok/call) Figure S20: Kills per game against median latency of one model call, under tick conversion (peripheral comparison). 61 Table S8: Experimental environment. Weight digests are the first 10 hexadecimal digits of SHA-256. Host 1AMD Ryzen 7 7800X3D (8 cores, 16 threads), 31 GB RAM, NVIDIA GeForce RTX 3060, Windows 11 Edu- cation 10.0.26200 Host 2AMD Ryzen 5 5600G (6 cores, 12 threads), 31 GB RAM, Ubuntu 24.04.4 LTS, Linux 6.11.0-29-generic Game and runtimeViZDoom 1.3.1 (asynchronous player mode, 35 ticks per second), NumPy 2.5.3; Python 3.14.0 (host 1), 3.12.3 (host 2) TrainingPyTorch 2.13.0 CPU (host 1), 2.14.0 CPU (host 2); 2,500 steps, training seed 31 (and 32 for N32) Deciderhashed n-gram encoder to 1,785 features, 265,665 parameters, served over local HTTP; original O a814587e03, after one DAgger round 240805651d, after two rounds N 8ce6b41475, seed-32 replicate 2f4785728c, data-volume control M 2061ae8b; second scene O a164d90514, N 11409b5c60 Open-weight com- parator qwen3:1.7b through Ollama on host 1 (model layer 3d0b790534) Commercialcom- parator gpt-6-sol through the Codex command-line tool 0.156.1, default settings, run on 2026-09-23 S13. Trace replay (preregistration v9): a failed fidelity check To enforce identical skip schedules by design, preregistration v9 recorded asynchronous runs of the arms N, O and S (seeds 860000–860049, two chunks of 25) together with the tick at which each decision was observed and applied, and replayed these schedules tick by tick in synchronous mode for six policies (teacher, original decider, the deciders after one and two DAgger rounds, the seed-32 replicate and the data-volume control), plus synthetic schedules that skip each tick independently with probability 0 to 0.30 or in bursts. The preregistered first check (B1) replays each recorded arm’s own policy on its own schedule and counts episodes whose shots, hits and kills match exactly; below 90% the replay was not to be read as the asynchronous run. 62 Table S9: Fidelity of trace replay (v9, B1). Accuracy and skipped-tick share pooled over the 50 episodes of each arm. Recorded armExact matches Hit accuracy (%) Skipped ticks (%) recorded replayed recorded replayed N, DAgger decider 0 of 5087.899.28.28.2 O, original decider 0 of 5080.998.88.68.7 S, teacher0 of 5085.299.39.69.6 Replay reproduced the skipped-tick share of every arm but none of the 150 episodes, and it raised hit accuracy to about 99% for all three policies. In 124 of the 150 replayed episodes the number of kills exceeds the number of shots counted, which happens in none of the recorded asynchronous episodes. The replay loop presses the previous button again on every tick without a new decision, while the shot counter records shots only on ticks where a new decision is applied; shots fired while a button is held therefore kill enemies without being counted. The replay thus implements the held-action rule of tick conversion rather than asynchronous play, and its own self-test, which compared it with the tick-conversion runner, could not detect this. Under the preregistered rules none of the three hypotheses is supported: the gain of the DAgger decider over the original on the schedules recorded for N is −0.25 points [−1.01, +0.52] (29 distinct games), the time components do not follow the order of the skipped-tick shares, and the time component at a 30% synthetic skip rate is [−0.08, +1.06] points. Because the fidelity check failed, we do not use these numbers as evidence about asynchronous play. Because the replay’s shot counter misses shots fired while a button is held, its near-99% accuracy is not evidence about held actions either. A separate tick-level audit of the asynchronous engine settled what happens on pending ticks. With a fixed wait of 0, 60 or 120 ms before each action, each action step spanned 1, 3 or 5 engine ticks. Among decisions that chose no turn right after a turn (163, 40 and 14 decisions), the view angle changed in every one; among those that followed another action (704, 18 and 2 decisions), it changed in none. The engine therefore keeps the previous action on pending ticks, as stated in Section 3. The cause was found later: in asynchronous mode the engine refreshes its clock only when an action is sent, so the recorded application ticks always equalled the observation ticks and the replay acted on fresh states. An emulator that applies each decision one tick after the next observation reproduces the recorded games (Section S1.13). A curve study over added delays (v8) was withdrawn before any episode ran, in favour of this design. 63 S14. Controlled-timing emulator The emulator (emu_s1.py) runs the engine synchronously and rebuilds asynchronous timing from a schedule of observation ticks. Decision i reads the state at its observation tick t i ; its action takes effect from tick t i+1 + L and holds until the next action takes effect; before the first action no button is pressed. Metrics are taken where the asynchronous runner takes them. Replaying the recorded actions of the six v9 traces (150 episodes) reproduced kills and the death tick in 0, 136 and 0 episodes for L = 0, 1 and 2, and we fixed L = 1 before the preregistered checks. Two preregistered fidelity checks recorded fresh asynchronous episodes on host 1 and let each policy decide again on its own recorded schedule. v16 (seeds 884000–884024) required all six counts (shots, hits, kills, hits taken, damage, death tick) to match exactly in at least 85% of episodes per policy; the teacher, O and N reached 20, 20 and 21 of 25, and all five game counts matched in 75 of 75. The mismatches were in the death tick: the recorder does not log the pending interval after its last observation, so the emulator fills it from the recorded gaps. v16b (seeds 885000–885024) required the five game counts to match and the death tick to be within two ticks; the teacher, O and N reached 24, 21 and 24 of 25 against the same 85% threshold, and all five game counts matched in 25, 23 and 25. Hit accuracy matched in all 150 episodes of the two checks. Both gates failed. In v16b every failing episode had a wait of at least 5 ticks after its last observation (up to 19), and its episodes skipped 19.4–39.4% of ticks, against 7.9–11.3% in the v9 schedules. v17 (seeds 886000–886024) recorded that wait and appended it to the schedule as a final gap, so that the next observation cannot come before the recorded end. Its criterion, fixed before recording, required the five game counts to match and the emulated end to fall after the last observation and no later than the recorded end, in at least 85% of episodes per policy. The teacher, O and N reached 25, 25 and 25 of 25, and also 25 of 25 under the v16b criterion. The v17 episodes skipped 8.1–9.8% of ticks and no wait exceeded 2 ticks, so the correction itself was not exercised on fresh data; applied after the fact to the v16b episodes, the recorded wait and the window criterion together match all 75 (exploratory). The main emulation imposed the v9 schedules of S, O and N on every policy (seeds 860000–860049, 29 distinct games) and synthetic schedules with independent skipping at 0, 4, 8, 12, 16 and 20% and bursty skipping at 10% (seeds 860000–860099, 56 distinct games). We reran it with the final gap of each recorded schedule imposed (v17); the synthetic schedules have no recorded end and are unchanged, and on the recorded schedules the estimates 64 moved by at most 0.15 p. S14.1. Controlled-timing emulator: how asynchronous timing was rebuilt Proposition 1 requires S and the decider to face the same skip process, and the studies above matched only the share of skipped ticks. We therefore rebuilt asynchronous play in the synchronous engine, where one schedule can be imposed on every policy. In ViZDoom’s asynchronous mode the state and the episode clock are refreshed only when an action is sent, so the ticks at which v9 recorded decisions being applied always equalled the observation ticks, and its replay acted on fresh states (Section S13). We rebuilt the timing from the recorded observation ticks alone: the action chosen on the state at one observation takes effect one tick after the next observation, and the previous action holds until then. Replaying recorded actions this way reproduced kills and the death tick in 136 of 150 recorded episodes, and in none with a lag of zero or two ticks. With each policy deciding again on its own recorded schedule, hit accuracy matched in all 150 fresh episodes of two preregistered checks and all five game counts in 148, but the death tick differed in some episodes and both preregistered fidelity gates failed narrowly. A third check on fresh seeds (v17), with the wait after the last observation recorded and imposed, passed (Section S14); the results in Section S1.13 come from the emulation rerun with that schedule. S14.2. Live identical-delay study (v18) v18 ran on host 2 with seeds 887000–887099. Distinct games were identi- fied by the tick-converted teacher on the same seeds (100 reference episodes); all 100 seeds gave distinct games. Seed 887000+j used the observation gaps of episode j mod 50 of the v9 host-1 recordings of O, repeated from the start if an episode lasted longer. After choosing, each arm waited until (gap − 1) × 28.571 ms plus a fixed offset had passed since the observation; the offset, 14 ms, was chosen in a 40-episode pilot on other seeds as the value with the highest share of realised gaps equal to the target (99.91–99.912% for all four values tried). Arms ran one at a time in blocks of 25 seeds, in an order fixed before recording, and each session checked the frozen hashes of code, schedules and weights before it started. The preregistered checks were that the realised gap equals the target gap in at least 90% of decisions in every waiting arm (99.9% in each) and that at least 85% of decider episodes skip within 0.8–1.25 times the share of the teacher on the same seed and session (100% for O, N and N32). Figure S21 shows target against realised gaps and Figure S22 the lengths of skipped runs and the per-episode skipped share. 65 12345 realized gap 1 2 3 4 5 target gap (ticks) 100 991 5446 100 100 teacher, schedule 12345 realized gap 100 991 5248 100 O, schedule 12345 realized gap 100 991 4555 100 N, schedule 12345 realized gap 100 991 5248 100 100 N32, schedule Figure S21: Schedule adherence in v18: share of decisions by target and realised gap (ticks), per waiting arm. 12345678 length of a skipped run (ticks) 10 4 10 3 10 2 10 1 10 0 share of runs length a run lengths 4681012 skipped-tick share per episode (%) 0.0 0.2 0.4 0.6 0.8 1.0 cumulative share of episodes b per-episode skipping teacher, scheduleO, scheduleN, scheduleN32, schedule Figure S22: Skipping in v18: (a) share of skipped runs at least as long as a given length; (b) cumulative share of episodes by skipped-tick share. Figure S23 shows the quality component of O game by game. 66 teacher, schedule O, schedule 55 60 65 70 75 80 85 90 95 hit accuracy per game (%) a paired games 020406080100 games, sorted (100) 15 10 5 0 5 10 15 20 25 teacher O (p) b 76% of games lose accuracy Figure S23: Quality component of O per game in v18: (a) teacher and O on the same schedule, one line per game; (b) teacher − O sorted, dashed: mean. Figure S24 compares the two sessions game by game. 1001020 session 1 (p) 10 0 10 20 session 2 (p) a teacher O per game (r = 1.00) 20100102030 session 1 (p) 20 10 0 10 20 30 session 2 (p) b N O per game (r = 1.00) Figure S24: Session 1 against session 2 in v18, per game: (a) teacher − O; (b) N − O. Dashed: equality. 67 75 100 acc. (%) game 80game 53game 65game 30game 98 75 100 acc. (%) game 96game 49game 14game 27game 69 75 100 acc. (%) game 31game 59game 85game 51game 88 75 100 acc. (%) game 57game 94game 99game 23game 83 20 of the games, from the smallest to the largest quality component; points left to right: T0, teacher, O, N T0 (no wait)teacher, scheduleO, scheduleN, schedule Figure S25: T0, teacher, O and N in 20 of the v18 games, from the smallest to the largest quality component of O. Figure S25 shows the four arms in 20 of the games. Figure S26 sets the live estimates beside those of the synchronous emu- lator. 20246810 teacher decider (p) quality O quality N live, host 2 (v18) emulator, host-1 schedules (v17) Figure S26: Quality components in live play (v18, host 2, 100 games) and in the syn- chronous emulator on host-1 schedules (v17 rerun, 29 games); different games. Figure S27 shows accuracy against disagreement within quartiles of the skipped share. 68 2040 disagree (%) 60 70 80 90 accuracy (%) skipped 4.27.4% 2040 disagree (%) skipped 7.48.4% 2040 disagree (%) skipped 8.410.3% 2040 disagree (%) skipped 10.311.8% Figure S27: Accuracy against disagreement with the teacher per v18 episode, in quartiles of the skipped-tick share; colours as in Figure 7. S15. Additional views of the data The figures in this section are generated from the per-episode result files and the offline decision logs by make_figs_v5.py; they show the same stud- ies from angles the main text summarises in a single number. A separate data atlas (Supplementary File 2, 34 pages, 46 figures) applies one standard battery to every study of Table S11, including the failed, withdrawn and ex- ploratory ones: episode distributions, episode-level relation to skipped ticks, secondary metrics, decision time, run order by block, live agreement with the teacher where it was logged, and every interval stored in each analysis file. It is generated by make_study_battery.py and build_atlas_pdf.py from the result files; a coverage check (results_sprint.py) confirms that every study has an artifact or a stated reason for each of ten angles. 69 01020304050 skipped ticks in the episode (%) 60 70 80 90 100 hit accuracy in the episode (%) host 1 (v6b) 0.050.100.150.200.250.300.35 skipped ticks in the episode (%) host 2 (v12) T0 S N O Figure S28: Per-episode skipped-tick share and hit accuracy in defend_the_center, same 100 seeds on both hosts (v6b, v12). On host 1 the arms that wait for a server skip 5–12% of ticks in 90% of episodes (5th–95th percentile), with single episodes up to 56%; the zero- latency teacher and every arm on host 2 skip below 0.4%. Note the different horizontal ranges: the hosts differ by two orders of magnitude. T0SNOT0SNO 70 75 80 85 90 95 100 host 1host 2 hit accuracy (%) T0SNOT0SNO 14 16 18 20 22 24 26 host 1host 2 kills per game T0SNOT0SNO 0 2 4 6 8 10 12 host 1host 2 hits taken per game Figure S29: Episode-level distributions of three metrics by arm and host (de- fend_the_center, v6b and v12; boxes: quartiles; whiskers: 1.5 interquartile ranges; out- liers hidden). On host 2 the latency-matched teacher S is indistinguishable from T0 on all three metrics, while the ordering S > N > O in accuracy and kills persists. 70 T0SONT0SONH0H4L8 100 0 100 200 300 400 episode reward (progress) v11 host 1v11 host 2v15 host 2 (decider O) Figure S30: Episode rewards in deadly_corridor (v11 on both hosts, v15 on host 2); diamonds: episode means. Episode rewards vary far more between games than between arms, which is why the game bootstrap intervals of this scene are wide (Table 5). 10 1 median decision time in an episode (ms, log) 0.0 0.2 0.4 0.6 0.8 1.0 share of episodes O, local decider (host 2, v15), 50 episodes S, teacher waiting for the server (host 1, v6b), 100 episodes S, teacher waiting for the server (host 2, v12), 100 episodes Figure S31: Decision time: distribution over episodes of the median decision time (log scale) for the local decider on host 2 and the latency-matched teachers on both hosts, which wait for a real call to the decider’s server. 71 0.40.60.81.0 decider confidence (bin mean) 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1.0 agreement with the teacher defend_the_center (30,385 states) deadly_corridor (518 states) 020406080100 states handled by the decider, most confident first (%) 90 92 94 96 98 100 agreement on handled states (%) defend_the_center deadly_corridor Figure S32: Confidence of the original deciders on all held-out teacher states. Left: agree- ment with the teacher by confidence bin (bins with at least 20 states); the dotted line is perfect calibration. Right: agreement on the states the decider would handle alone when it takes the most confident ones first; the remaining states would be handed to a slower judge (Table S10). Table S10: Confidence thresholds on the validation split (all states). Automated: share of states at or above the threshold; agreement: teacher agreement on those states; handed off: the rest. Scene and threshold Automated (%) Agreement (%) Handed off (%) defend_the_center (30,385 states) none100.096.10.0 ≥ 0.895.197.84.9 ≥ 0.990.898.49.2 ≥ 0.9584.898.815.2 ≥ 0.9963.499.436.6 deadly_corridor (518 states) none100.099.20.0 ≥ 0.899.0100.01.0 ≥ 0.998.7100.01.4 ≥ 0.9597.9100.02.1 ≥ 0.9988.4100.011.6 S16. Study ledger Table S11 lists every study of the project, including withdrawn, failed and exploratory ones, with the episodes on disk. It is generated by a script (study_ledger.py) from the preregistration files, the frozen hash lists and the raw result files; no cell is typed by hand. “Amended” means that text was appended to the preregistration after it was frozen (results, addenda or a withdrawal note, each marked as such in the file). Episode counts 72 include runs that were repeated after an infrastructure failure. v14 and v14b are withdrawn under the terms of the third-party service they used; their rows keep the episodes on disk and report no results. Their preregistrations committed us to report the results whatever they were, so this withdrawal is a deviation from them. Table S11: All studies. Host 1: Windows desktop; host 2: Linux workstation. Scene: defend = defend_the_center, corridor = deadly_corridor. Verdict: the preregistered verdict keys stored in each analysis file. IDStudyHost Scene Episodes Frozen text Verdict S1/E1 first real-time demonstration: paused, tick-converted and asynchronous play 1 defend320 yessee text E2competing methods (hybrid, action queue, language models) on one real-time ruler 1 defend1,365 amended see text v3asynchronous replication of teacher and decider 1 defend210 yes– v4latency-matched decomposition, three pairs (episodes in v4b: its first three pairs) 1 defend0 yes– v4bextension to five pairs; run- and game-level intervals 1 defend300 yesclaim of quality component <5 p allowed: no v5on-policy asynchronous DAgger, five triads 1 defend450 yespartial success v6acalibration of core-pinned concurrent evaluation (failed) 1 defend90 amended – v6breplication on 56 distinct games, seed-32 replicate 1 defend600 yespartial success v6b+ v6b addendum: arms run one at a time after the failed calibration (episodes in v6b) 1 defend0 yes– v7data-volume control1 defend200 yesnot explained by volume v8asynchronous latency curve (withdrawn before execution) 1 defend0 amended – v9trace replay of skip schedules (fidelity check failed) 1 defend5,250 yesfidelity check failed; H9-1 supported: no; H9-3 supported: no v10sentence-reading teacher: input versus policy share 1 defend300 yesinput share small v11second scene: decomposition and DAgger 2 corridor1,460 yesinconclusive v11-h1 second scene on host 1 (addendum) 1 corridor1,000 yesinconclusive v12second host, same seeds as v6b2 defend500 yespartial success v13p injected-delay pilot, exploratory, no preregistration 2 corridor175 – v14withdrawn: third-party service terms; results not reported 2 both1,500 yesnot reported v14b withdrawn: third-party service terms; results not reported 2 both0 yesnot reported 73 IDStudyHost Scene Episodes Frozen text Verdict v15action holding for k ticks and stale decisions 2 both1,800 yesnegative; positive v16controlled-timing emulator: fidelity check and identical-schedule decomposition 1 defend2,625 yesfidelity gate failed; rerun confirmed in v17 v16b confirmatory fidelity recheck of the emulator 1 defend75 yesfidelity gate failed v17emulator fidelity on fresh seeds, censored final interval; identical-schedule rerun 1 defend75 yesfidelity gate passed v18live play, one delay schedule imposed on every arm; second DAgger seed; disagreement 2 defend1,140 yeschecks passed; all four hypotheses held f1conveyor simulator: policy× injected delay, four concurrent episodes 2 conveyor600 yessee text f1-solo conveyor simulator: decider one episode at a time, exploratory 2 conveyor15 yes– corr-att corridor: learning when to shoot, rounds 1–2 (paused evaluation) 1 corridor500 –negative corr- att2 corridor: learning when to shoot, branch-comparison labels 1 corridor5,000 –negative S17. Selective automation with a confidence threshold The decider returns a probability for each action. If its largest probability is below a threshold, the decision can be passed to a slower judge. Table S10 gives, for each threshold, the share of held-out teacher states the decider would handle alone and its agreement with the teacher on those states. The measurement is offline (script cascade_coverage.py, weights a814587e03 and a164d90514); it says how often the decider is confident and right, not how a cascade would play live. S17.1. Selective automation (exploratory) Selective automation (exploratory). A local decider that returns a confidence can hand only its uncertain cases to a slower judge. On the 30,385 held-out teacher states of defend_the_center, O acts alone on 90.8% of them at a confidence threshold of 0.9, with 98.4% agreement on those, against 96.1% on all states; the remaining 9.2% would go to a larger model or a person (Supplementary Table S10). These are offline agreements with the teacher, not live scores. 74 S18. Second environment: a conveyor with a deadline LineTempo is a Python conveyor-inspection simulator written before this study for a separate project; we used a frozen version without changing its code. Parts enter at a camera every 0.24 m on a belt moving at 1 m/s and reach a diverter 0.4 m (400 ms) later; the simulator runs at 50 ticks per second. Each part has a true defect class, severity, size and location. A deterministic perception script, with detection misses, false alarms and blur, turns the state into a sentence such as “part: family housing | defect scratch | size small | location edge | severity minor | image clear | line: belt normal | . . . ”. The decider receives the sentence over HTTP and chooses PASS, REJECT (with one of four bins), REWORK or ESCALATE; the teacher applies a fixed disposal table to the true state. The factory decider uses the same model family and training code as the Doom decider and was trained on 3,000 teacher-labelled sentences from separate seeds. On 300 held-out sentences (seed 920000) it chose the teacher’s operation in 97.7% of cases, but only 21 of the 26 rejections; this is agreement with a synthetic teacher, not inspection accuracy. A decision that has not arrived when the part reaches the diverter is late and the part passes by default. Cost per part weights an escaped defect 100, a false rejection 20, a line stop 10, an escalation 5 and a rework 1. 75 Table S12: Preregistrations, seeds and frozen hashes. IDContentEvaluation seeds Hash v3zero-latency teacher (T0) and decider, asynchronous replication 850000–850029 299dafd18395 v4latency-matched decomposition, three pairs 850000–850029 d4c13a3bcfce v4b extension to five pairs; run-level and game-level intervals 850000–850029 a317edf316e7 v5on-policy asynchronous distillation, five triads 850000–850029 404982264335 v6acalibration of core-pinned concurrent evaluation 850000–850029 b96753c944f2 v6b replication with contemporaneous T0 and seed-32 replicate 860000–860099 1ab89caac991 v6b+ sequential execution after failed calibration 860000–860099 d9876210356f v7data-volume control860000–860099 75852f3614f v9trace replay; fidelity check failed (Section S13) 860000–860099 3b2f9fcad9d4 v10sentence-reading teacher: input versus policy share 860000–860099 b8d822e52778 v11second scene (deadly_corridor), host 2; host-1 addendum 870000–870199 59819d9d7802 / 36896cdebded v12second host (Linux workstation), defend_the_center 860000–860099 01b7e9d4902 v14withdrawn (third-party service terms; results not reported, a deviation) 860000–860099; 870000–870199 fcf1420fa783 v14b withdrawn (third-party service terms; results not reported, a deviation) 860000–860099; 870000–870199 80040d299869 v15action holding for k ticks and stale decisions, host 2; resume addendum 860000–860049; 870000–870199 1d15c665c9ce / e85b906894b1 f1conveyor simulator, policy × injected delay, host 2 950000–950009 f194ae8008ed v16controlled-timing emulator: fidelity check and identical-schedule decomposition, host 1 884000–884024; 860000–860099 0139149b996c v16b confirmatory fidelity recheck of the emulator, host 1 885000–885024 d8abf70af5d6 v17emulator fidelity on fresh seeds with the final wait recorded; identical-schedule rerun, host 1 886000–886024; 860000–860099 d70ce2bab648 v18live play with one delay schedule imposed on every arm; second DAgger seed; disagreement on own states, host 2 887000–887099 3a94ac70cf68 76 The preregistration (hash in Table S12) crossed the policy (exact teacher; the decider D over live HTTP; an untrained decider; a random policy) with a delay added after each answer arrived, so that delay did not queue later requests: 0, 200, 340, 360, 370, 380, 400 and 600 ms. Each cell ran 10 episodes of 500 parts on seeds 950000–950009, paired by seed. The teacher was also run with a uniform ±50 ms jitter on the delay, and both policies were run with the belt waiting for each decision (synchronous mode) at 0, 380 and 600 ms. Line B adds two part families and a packing buffer; in this configuration its settings otherwise equal line A’s and the buffer never filled, so it is close to a repeat of line A. Live runs used four concurrent episodes on host 2. Table S13: LineTempo, asynchronous mode: mean cost per part and, in parentheses, late decisions (%), by injected delay. Teacher: exact teacher; teacher ±50: with ±50 ms uni- form jitter; D: the decider over live HTTP, four concurrent episodes; D, solo: exploratory rerun of D one episode at a time (5 seeds). Other cells: 10 episodes of 500 parts. Line ALine BLine A δ (ms) teacherteacher ±50 DteacherDD, solo 0 0.03 (0.0) 0.03 (0.0) 0.88 (0.0) 0.03 (0.0) 0.88 (0.0) – 200 0.03 (0.0) 0.03 (0.0) 0.88 (0.0) 0.03 (0.0) 0.88 (0.0) – 340 0.03 (0.0) 0.85 (9.6) 1.37 (7.3) 0.03 (0.0) 0.90 (0.2) 0.92 (0.2) 360 0.03 (0.0) 2.06 (30.4) 1.83 (13.1) 0.03 (0.0) 2.99 (31.2) 0.88 (0.4) 370 0.03 (0.0) 3.16 (40.0) 1.25 (6.0) 0.03 (0.0) 1.38 (6.7) 0.88 (0.3) 380 0.03 (0.0) 4.00 (51.0) 7.52 (100.0) 0.03 (0.0) 7.50 (100.0) – 400 7.52 (100.0) 5.43 (69.9) 7.52 (100.0) 7.50 (100.0) 7.50 (100.0) – 600 7.52 (100.0) 7.52 (100.0) 7.52 (100.0) 7.50 (100.0) 7.50 (100.0) – untrained decider, δ = 05.00 (0.0) untrained decider, δ = 6007.52 (100.0) random policy8.87 (0.0) Table S13 lists every asynchronous cell. The preregistered prediction that the decider would change sharply between 370 and 380 ms failed: with four concurrent episodes it was late on 7.3% of parts at 340 ms (line A), and on up to 82% in single episodes near the deadline. A probe of the server alone ruled out a transport stall. The exploratory rerun of the decider at 340, 360 and 370 ms one episode at a time (seeds 950000–950004) gave mean late shares of 0.2–0.4% (0.2–0.8% in single episodes) and round-trip p99 of 4.1–4.3 ms, against 66.7 ms over the concurrent runs. In synchronous mode, where the belt waits for each decision, delay barely changed the cost of either policy (decider 0.88, 0.88 and 0.90 per part at 0, 380 and 600 ms; exact teacher 0.03 throughout), but it cut throughput: the 77 decider line passed 148.7, 76.5 and 59.8 parts per minute and the teacher line 150.0, 78.9 and 61.2, against 249.7 in asynchronous mode. Waiting for each decision thus moves the cost of time from the score into throughput. S18.1. Deadline edge and host latency in the conveyor simulator The deadline also included the host. We had predicted the decider’s edge between 370 and 380 ms, where its round trip (median 3.7 ms) and rounding to the next 20 ms tick first push answers past 400 ms. With four episodes running at once on host 2, round trips had a heavy tail (p99 66.7 ms), and the decider was already late on 7.3% of parts at 340 ms. In an exploratory rerun with one episode at a time, mean late shares fell to 0.2–0.4% (0.2–0.8% in single episodes) and p99 to 4.1–4.3 ms. As in Doom, the host sets the time component; with a deadline, a change in the latency tail moves the decider over a cliff rather than along a slope. S18.2. Simulator checks (exploratory) Before the preregistered study, the Python reference of the simulator passed its 58 reference tests. On line B, in a check configuration that differs from the preregistered study (defect rate 0.15 rather than 0.08, 50 parts per episode), we ran one episode on the same seed for each of three policies. The exact teacher, run in process, let none of the 5 defective parts escape. The decider D over live HTTP, with a median decision time of 20 ms, let 1 escape. A mock slow model whose first answer takes 600 ms, also over HTTP, was late on every decision and let all 5 escape. These single episodes check the pipeline; they are not estimates. Figure S33b is a frame of the decider’s episode, replayed from its log in the simulator’s viewer. a ViZDoomb conveyor simulator camera (inspection station) diverter, 0.4 m downstream (400 ms at 1 m/s) sorting bins parts Figure S33: (a) ViZDoom from the player’s view, with an opposite-turn error of Sup- plementary Figure S34 (gray arrow teacher, orange arrow decider). (b) The conveyor simulator of Section 6.8, a frame of a logged 50-part run of the decider on line B. A cam- era inspects each part; a diverter 0.4 m downstream (400 ms at 1 m/s) sorts it. 78 Figure S34: First observed opposite-turn errors of the original decider, one per game: the teacher turns toward the nearest enemy and the decider turns away. Key: gray arrow teacher, orange arrow decider, white circle the nearest enemy; in the state sentence, a is the angle bin in degrees and d the distance bin in 100-unit steps. Illustrative, not a measurement: the first opposite-turn decision with ammunition left in each of the first six games from seed 860000, under synchronous replay with 10% i.i.d. skipped ticks. The frame is shown for the reader; the decider itself received only the state sentence printed below it. S19. Injected-delay pilot An exploratory pilot on host 2 (25 seeds of deadly_corridor, 890000– 890024, disjoint from all evaluations; no preregistration) added a fixed delay after every call of S and O to the decider’s server. Table S14 shows mean reward rising with the delay, for the teacher and the decider alike, as more ticks pass under the previous action. The delay here makes the decision stale; the preregistered study of holding a freshly chosen action for k ticks (v15) is reported in Section 6.7. 79 Table S14: Injected-delay pilot, deadly_corridor, host 2 (exploratory). T0 has no server call and is the same at every delay. Delay (ms) T0SO S skipped (%) O skipped (%) 0 61.6 61.6 63.01.61.6 30 61.6 89.4 91.548.648.6 60 61.6 82.4 84.563.163.1 S20. Reproducing the studies Every study is run by one shell script per host that freezes the hashes of its preregistration, runner, analysis script and decider weights before the first seed, runs the arms one at a time in the preregistered order, and calls the analysis script at the end. The evaluators are arms_s1.py (de- fend_the_center) and arms_corr.py (deadly_corridor) in the asynchronous mode; the decider is served by doom_server.py; injected delay and action holding are wrappers (inject_delay.py, hold_ticks.py) that leave the evaluators unchanged. Analyses (analyze_v4b.py to analyze_v15.py) run a self-test on synthetic positive and negative controls before reading results. The study ledger, the tables of this supplement and the figures are regener- ated from the result files by study_ledger.py, make_tables_v021.py and make_figs_v4.py to make_figs_v6.py. When a host failed during a run, the remaining jobs were rerun from their first seed under an addendum frozen before resumption (v10, v11 host 1, v15). S21. Experimental environment Table S8 lists the hosts, software and models. The scene-1 studies v3–v10 (defend_the_center) ran on host 1; the second scene (deadly_corridor, v11) and the second-host replication of scene 1 (v12) ran on host 2. The decider and the game engine run on the CPU; the GPU of host 1 served only the open-weight language-model comparator. S22. Preregistrations Table S12 lists the preregistrations used in this paper with the first 12 hexadecimal digits of the SHA-256 hash of each text as frozen before execu- tion (results were appended to some texts afterwards, in a separate section). Decider weights: original a814587e03, after one DAgger round 240805651d, after two rounds 8ce6b41475, seed-32 replicate 2f4785728c. 80