Paper deep dive
Selecting Long-Horizon Trajectories for Reliable and Efficient Terminal-Agent Training
Cuong Dang, Hoang Anh Just, Ruoxi Jia
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Terminal agents are commonly trained by imitating long teacher trajectories, yet how much of each trajectory to supervise remains unexplored. We study the \emph{supervision horizon}, the number of trajectory tokens retained for training, and show that it is a key design axis for reliability and cost. Reliability improves with longer horizons but saturates: on Terminal-Bench, a 12K-token horizon solves more tasks than 16K ($29\pm0.7$ vs.\ $26\pm0.8$) while requiring 30\% less training time. The horizon also shapes agent behavior: short horizons cause premature termination, intermediate horizons yield productive error recovery, and long horizons induce over-persistence. We analyze this saturation through a bias--complexity bound, in which longer supervision reduces temporal supervision bias but increases finite-sample estimation error from more heterogeneous late-stage histories. Guided by this analysis, we propose \emph{selective long-horizon refinement}, which first trains on short prefixes and then refines only on continuations that are most likely under the warm-start model. It consistently outperforms full long-horizon training. At 16K, it raises successful attempts from $110\pm2.7$ to $126\pm2.1$ and tasks solved in at least six of eight attempts from $9\pm0.7$ to $14\pm0.6$; with half of the long-horizon data, it still reaches $122\pm2.4$ while cutting training time by 23\%. The gains transfer across benchmarks, from $64\pm2.6$ to $73\pm2.1$ on Terminal-Bench v2.0 and from $137\pm2.7$ to $155\pm2.2$ on OpenThoughts-TBLite. For long-horizon supervision, selecting the right trajectories matters more than training on all of them.
Tags
Links
- Source: https://arxiv.org/abs/2610.05831v1
- Canonical: https://arxiv.org/abs/2610.05831v1
Trouble viewing inline? Open PDF directly →
Full Text
87,679 characters extracted from source content.
Expand or collapse full text
SELECTING LONG-HORIZON TRAJECTORIES FOR RELI- ABLE AND EFFICIENT TERMINAL-AGENT TRAINING Cuong Dang & Hoang Anh Just & Ruoxi Jia Department of Electrical and Computer Engineering Virginia Tech Blacksburg, VA, USA cuongdc,just,ruoxijia@vt.edu ABSTRACT Terminal agents are commonly trained by imitating long teacher trajectories, yet how much of each trajectory to supervise remains unexplored. We study the supervision horizon, the number of trajectory tokens retained for training, and show that it is a key design axis for reliability and cost. Reliability improves with longer horizons but saturates: on Terminal-Bench, a 12K-token horizon solves more tasks than 16K (29± 0.7vs.26± 0.8) while requiring 30% less training time. The horizon also shapes agent behavior: short horizons cause premature termination, intermediate horizons yield productive error recovery, and long horizons induce over-persistence. We analyze this saturation through a bias–complexity bound, in which longer supervision reduces temporal supervision bias but increases finite- sample estimation error from more heterogeneous late-stage histories. Guided by this analysis, we propose selective long-horizon refinement, which first trains on short prefixes and then refines only on continuations that are most likely under the warm-start model. It consistently outperforms full long-horizon training. At 16K, it raises successful attempts from110± 2.7to126± 2.1and tasks solved in at least six of eight attempts from9± 0.7to14± 0.6; with half of the long-horizon data, it still reaches122± 2.4while cutting training time by 23%. The gains transfer across benchmarks, from64± 2.6to73± 2.1on Terminal-Bench v2.0 and from 137± 2.7to155± 2.2on OpenThoughts-TBLite. For long-horizon supervision, selecting the right trajectories matters more than training on all of them. 1INTRODUCTION Terminal-based coding agents such as OpenAI Codex (OpenAI, 2025) and Anthropic Claude Code (Anthropic, 2026) are becoming a common interface for software development, operating directly over terminals and repositories to execute commands, modify code, and iteratively solve tasks. Recent works have therefore begun to study how to train smaller, capable terminal agents efficiently (Pi et al., 2026; Cheng et al., 2026; Gandhi et al., 2026; Wu et al., 2026). Table 1: Training time and accuracy across supervision horizons. Cutoff4K8K12K16K Time (h)↓10.1721.6535.4450.81 Accuracy (%)↑24252926 A common way to train terminal agents is imitation learning from teacher-generated trajectories, where the student is supervised to reproduce the teacher’s actions conditioned on the preceding interaction his- tory. These long-horizon terminal trajectories involve extended sequences of interdependent decisions in which later actions depend on earlier observations, modifications, errors, and intermediate outcomes. As trajectories unfold, the agent may need to recover from failed commands, revise hypotheses, verify in- termediate results, or switch strategies based on accumulated interaction history. Training with a short horizon can therefore omit these later-stage behaviors and provide insufficient supervision for recovery, correction, verification, and strategy switching. In contrast, longer horizons expose the student to richer temporal dependencies and may improve persistence and long-range reasoning, but they also incur substantially higher training cost and introduce increasingly heterogeneous histories 1 arXiv:2610.05831v1 [cs.LG] 5 Oct 2026 4K8K12K16K ~ 12K context $ agent.run(task) PlanErrorAct Recover Diagnose SolveSTOP ... useful persistence ... ~ 4K context $ agent.run(task) PlanErrorFAIL ... gives up early, fail to recover ... Act ~ 16K context $ agent.run(task) PlanErrorAct Recheck Recover SolveBREAK ... overshoots, unnessary continuation ... 4K(Under-persistent)12K(Balanced)16K(Over-persistent) Persistence Too little (premature termination) Productive persistence Excessive (post-solution overreach) B. Horizon controls Behaviors 70 90 110 Shorter inference trajectories Less training compute than 16K Sweet spot Successful Attempts A. Reliability Emerges with Diminishing Returns C. Less is More: Quality over Quantity in Long-Horizon Training Supervision Horizon Lower Likelihood Higher Likelihood Keep High- likelihood Data Training Time Reliability Successful Runs Stable Tasks 110 126 9 14 Selective Training Full Training Modify Supervision Horizon Error / Learning Difficulty Temporal Supervision Bias Estimation Error Total Error Effective Training Horizon -23% Figure 1: Overview of our three main findings on effective training horizons for terminal agents. (A) Reliability Improves with Diminishing Returns: increasing the training horizon substantially improves reliability, but the marginal gains diminish beyond an intermediate sweet spot, where 12K provides a favorable balance between successful attempts, training compute, and inference length. (B) Horizon Controls Behaviors: increasing the training horizon shifts agents from under-persistence at 4K, to productive persistence at 12K, and over-persistence at 16K; theoretically, this reflects a trade-off between decreasing temporal supervision bias and increasing estimation difficulty. (C) Less is More: Quality over Quantity in Long-Horizon Training: simply training on more long-horizon trajectories is not necessarily beneficial; training on a smaller set of appropriately selected long-horizon trajectories can achieve higher reliability than using the full long-horizon data, showing that data quality matters more than data quantity for exploiting long-horizon supervision. and more complex decision situations. As shown in Table 1, increasing the cutoff from 8K to 16K more than doubles training time, from 21.65 to 50.81 hours, while Terminal-Bench accuracy improves only marginally from 25 to 26. Moreover, the 12K model achieves the highest accuracy of 29 with 35.44 hours of training, suggesting that simply extending the supervision horizon does not yield proportional performance gains and may even exhibit diminishing returns. This motivates two important design questions that remains largely unresolved: Q1: Is supervising the longer teacher trajectory always better? Q2: Can selectively chosen long-horizon trajectories provide better and more efficient supervision than using all long-horizon data? To answer these questions, we study the effective training horizon for terminal agents: a horizon that is sufficiently long to capture useful temporal dependencies, yet not so long that additional supervision becomes inefficient or counterproductive. As summarized in Figure 1, our study reveals three main findings.❶First, increasing the training horizon substantially improves reliability, but the gains diminish beyond an intermediate sweet spot, indicating that full-horizon supervision is not necessarily optimal.❷Second, the training horizon systematically shapes the agent’s behavioral regime: short horizons induce under-persistence, intermediate horizons encourage productive persistence, while excessively long horizons lead to over-persistence, causing agents to continue acting unnecessarily and resulting in longer test-time inference trajectories. We further explain this intermediate optimum theoretically by decomposing the total error into two competing quantities: temporal supervision bias, which decreases as longer horizons expose the model to richer later-stage behaviors, and finite-sample estimation error, which increases as the learner must model a more complex set of long-horizon 2 histories. The effective training horizon therefore emerges at an intermediate point that balances the decreasing temporal supervision bias against the increasing estimation error. These findings lead to a declarative answer to Q1: No, supervising a longer teacher trajectory is not always better; the effective training horizon lies in the middle.❸Third, we show that more long-horizon data is not always better. A two-stage strategy that first trains on the full short-horizon data and then refines the model using only half of the long-horizon trajectories achieves higher reliability than full long-horizon training while reducing both training and inference costs. This leads to a declarative answer to Q2: Yes, selectively chosen long-horizon trajectories can provide more effective and efficient supervision than using the full long-horizon dataset. Our contributions are fourfold and organized as follows.❶First, we characterize the relation- ship between training horizon and agent reliability, showing that reliability improves with longer supervision but exhibits diminishing returns (Section 4).❷Second, we explain this scaling behavior empirically through the notion of persistence, revealing a transition from under-persistence to pro- ductive persistence and eventually over-persistence (Section 5.1).❸Third, we theoretically explain why longer supervision eventually exhibits diminishing returns, characterizing the effective training horizon through a trade-off between temporal supervision bias and finite-sample estimation error (Section 5.2).❹Fourth, we propose a simple, efficient two-stage training strategy that combines full short-horizon training with selective long-horizon refinement to further improve reliability (Section 6). 2RELATED WORKS Terminal-Agent Training and Data Engineering.Recent work improves terminal agents through data scaling, environment construction, and richer supervision. Gandhi et al. (2026), Cheng et al. (2026), and Shrivastava et al. (2026) improve environments and supervision signals, while Pi et al. (2026) study long-context training and observe that extending context does not always improve performance. However, their analysis measures aggregate accuracy and leaves open how longer supervision changes agent reliability. We show that aggregate accuracy hides a qualitative shift: beyond an intermediate horizon, longer supervision unlocks few new tasks but makes already-solvable tasks more repeatable, redistributing success from fragile to consistent solving. Building on this obser- vation, we characterize how the supervision horizon shapes agent behavior, from under-persistence to productive recovery and eventually over-persistence, explain the saturation theoretically through a bias–complexity trade-off, and use this analysis to motivate selective refinement of newly exposed long-horizon trajectory segments. Likelihood-based data selection for SFT. Data quality often matters more than quantity: LIMA (Zhou et al., 2023) shows that a small, well-chosen dataset can rival much larger ones. One line of work selects data by how well it fits the model being trained. Ren et al. (2024) show that LLM-generated responses outperform human-written ones largely because their lower perplexity makes them easier to learn, and Zhang et al. (2025) select, among candidate responses, those with the highest likelihood under the target model. Other criteria instead favor harder examples, e.g., instruction-following difficulty (Li et al., 2024), and perplexity-based pruning in pretraining finds that the best subset depends on the reference model and the retained fraction (Marion et al., 2023; Ankner et al., 2024). Reconciling these views, Cuong et al. (2026) show that the preference for high-likelihood data is not universal: in mathematical reasoning, it yields faster early gains, especially for small students, whereas low-likelihood data becomes more beneficial for larger students under longer training. These methods score whole examples, typically in single-turn settings. In contrast, we score only the newly exposed continuation(H 1 ,H 2 ]of each long-horizon trajectory, using a warm-start model trained on shorter prefixes, and treat likelihood as compatibility with the learned short-horizon policy rather than as intrinsic quality. 3BACKGROUND 3.1TERMINAL AGENTS AS SEQUENTIAL DECISION MAKING In this work, we formulate the interaction following the Terminus-2 scaffold (Merrill et al., 2026), which provides the model with a single headless terminal interface and requires it to complete tasks through Bash commands. This lightweight setup minimizes additional scaffold-specific mechanisms, 3 allowing the learned sequential behavior to be more directly attributed to the underlying language model. At each interaction stept, the agent observes the current terminal outputo t and selects an actiona t , such as executing a command, inspecting or modifying a file, installing a dependency, or running a test. The action is sampled from a policyπ θ conditioned on the complete interaction history h t = (o 1 ,a 1 ,...,o t−1 ,a t−1 ,o t ), i.e.,a t ∼ π θ (· | h t ). A complete interaction forms a trajectory τ = (o 1 ,a 1 ,o 2 ,a 2 ,...,o T ,a T ), where T denotes the number of environment-level actions. 3.2TERMINAL AGENT TRAINING AS IMITATION LEARNING A common paradigm for training terminal agents is to learn from trajectories generated by a stronger teacher model, which can be viewed as offline imitation learning. Letπ T denote the teacher policy andπ θ the student policy. At each stept, the teacher selects an action according toa t ∼ π T (·| h t ), whereh t = (o 1 ,a 1 ,...,o t )denotes the interaction history. Rolling out the teacher policy produces a trajectoryτ = (o 1 ,a 1 ,o 2 ,a 2 ,...,o T ,a T ). A dataset ofNteacher trajectories is denoted by D =τ (i) N i=1 , where τ (i) = (o (i) 1 ,a (i) 1 ,...,o (i) T i ,a (i) T i ). Given these offline demonstrations, behavior cloning trains the student to imitate the teacher action at each visited history. The student assigns probabilityπ θ (a (i) t | h (i) t ) to the teacher action observed at step t, with the standard log-loss objective L(θ) =− 1 N N X i=1 T i X t=1 logπ θ a (i) t | h (i) t .(1) 3.3FROM DECISION HORIZON TO TOKEN-LEVEL SUPERVISION In practice, LLM-based agents are trained with supervised fine-tuning (SFT) over tokenized interac- tion sequences. In SFT, each actiona t = (y t,1 ,...,y t,ℓ t ) is generated autoregressively at the token level. Let p θ (y t,j | h t ,y t,<j ) denote the next-token distribution. Then, π θ (a t | h t ) = ℓ t Y j=1 p θ (y t,j | h t ,y t,<j ),logπ θ (a t | h t ) = ℓ t X j=1 logp θ (y t,j | h t ,y t,<j ).(2) A common cutoff at a particular decision step does not correspond to a common sequence length across trajectories. Instead, the supervision horizon is implemented as a token budgetK, retaining only the prefix of the serialized trajectory within the firstKtokens. Lettingn t (τ )denote the cumulative number of tokens through actiona t , a token budgetKinduces the decision horizon H K (τ ) = maxt : n t (τ )≤ K, which is non-decreasing inK. Throughout the empirical analysis, we therefore use the token supervision horizonKas the controllable training variable, while the theoretical analysis is expressed in terms of the induced decision horizon H . 4EMERGENCE AND SATURATION OF RELIABILITY We first investigate how the amount of trajectory supervision affects the reliability of terminal agents. We vary only the token-level supervision horizon while keeping the base model, underlying training trajectories, and evaluation protocol fixed. The resulting trend is clear: extending the supervision horizon initially improves both the breadth and repeatability of task solving, but the gains diminish beyond an intermediate regime. In particular, substantially longer supervision incurs considerably higher training cost without a corresponding improvement in aggregate reliability. 4.1EXPERIMENTAL SETTING Model and Training Data.Following prior work on terminal-agent training (Pi et al., 2026; Gandhi et al., 2026; Cheng et al., 2026; Shrivastava et al., 2026), we use Qwen/Qwen3-8Bas the base model throughout our experiments. We train on trajectories from nvidia/Nemotron-Terminal-Corpus, which contains terminal-interaction trajectories gen- erated by DeepSeek-V3.2 (Liu et al., 2025). Following Pi et al. (2026), we restrict training to completed trajectories, allowing us to isolate the effect of supervision horizon from confounding 4 effects introduced by incomplete or imperfect trajectories. All training variants are constructed from the same underlying trajectory set, with supervision truncated at4096,8192,12288, or16384 response tokens. We refer to these settings as 4K, 8K, 12K, and 16K, respectively. Evaluation Protocol. We evaluate the resulting models on Terminal-Bench v1.0 (Merrill et al., 2026), using the Terminus-2 agent scaffold throughout. Each task is evaluated with eight independent attempts. Repeated evaluation allows us to distinguish between coverage — whether a model can solve a task at all — and repeatability — whether it can solve the same task consistently across independent runs. Reliability Metrics. Lets i = P 8 r=1 1attempt r succeeds on task ibe the number of successes for task i. We define Rel ≥k = M X i=1 1s i ≥ k, ̄s + = P i:s i ≥1 s i Rel ≥1 ,(3) whereRel ≥k counts tasks solved in at leastkof eight attempts, and ̄s + is the mean success count among solved tasks. We also report fragile tasks, defined by 1≤ s i ≤ 2. 4.2EXPERIMENTAL RESULTS Table 2: Reliability across supervision horizons. Values are mean±variation over four evaluations. Metric4K8K12K16K Aggregate Succ.↑78±2.897±2.6 111±2.3110±2.7 Rel ≥1 ↑24±0.825±0.8 29±0.726±0.8 ̄s + ↑3.25±0.13.88±0.13.83±0.1 4.23±0.1 Repeatability Rel ≥4 ↑9±0.710±0.713±0.8 14±0.8 Rel ≥6 ↑4±0.58±0.6 9±0.6 9±0.7 Rel ≥8 ↑2±0.45±0.5 6±0.54±0.5 Fragile↓15±0.811±0.712±0.8 9±0.7 Reliability emerges with longer supervision. As shown in Table 2, increasing the supervision horizon from4K to12K substantially improves reliability. Successful attempts increase from 78±2.8to111±2.3, while task coverage rises fromRel ≥1 = 24±0.8to29±0.7. The improve- ment also extends to repeated success:Rel ≥6 increases from4±0.5to9±0.6, andRel ≥8 from 2±0.4to6±0.5. In every case, the gain far ex- ceeds the variation across evaluations. Thus, longer supervision improves both the breadth and consistency of task solving. Reliability saturates beyond an intermediate horizon. As shown in Table 2, extending supervision from12K to16K yields little additional aggregate benefit. Successful attempts remain nearly unchanged (111±2.3vs.110±2.7, within variation), while task coverage decreases from29±0.7 to26±0.8andRel ≥8 from6±0.5to4±0.5. However, the reliability profile shifts: the number of fragile tasks decreases from12±0.8to9±0.7, whileRel ≥4 increases slightly from13±0.8to 14±0.8. Together with the higher mean success count ( ̄s + = 3.83±0.1vs.4.23±0.1), this suggests that after the reliability peak, additional supervision primarily redistributes success toward more consistent performance on already-solvable tasks rather than expanding overall coverage. We defer task-level reliability and further analyses in Appendix B. Takeaway 1 (Emergence-and-Saturation of Reliability). Reliability improves markedly up to an intermediate horizon, after which additional supervision mainly redistributes success from fragile to more repeatable task solving rather than increasing aggregate reliability. 5HORIZON CONTROLS BEHAVIORS 5.1EMPIRICAL ANALYSIS Table 3: Proactive vs. reactive successful tra- jectories across four context lengths. Proac- tive share peaks at 12K and declines at 16K. Context length4K8K12K16K Proactive17252927 Reactive61728283 Proactive share21.79%25.77%26.13%24.55% Longer horizons strengthen reactive problem solv- ing. We classify a successful trajectory as proactive if the target failure never manifests, and as reactive if the failure manifests but the agent subsequently di- agnoses and recovers from it to solve the task; details in Section A.4. As shown in Table 3, increasing the supervision horizon consistently increases the num- ber of successful reactive trajectories, from 61 at 4K to 83 at 16K. This suggests that longer-horizon 5 Figure 2: Episodes per run for successful trials, all and reactive only across training horizons. 0 2 4 6 8 10 12 14 16 Successful Number of runs 4K model n = 78 Mean = 11.5 Median = 9.0 P25–P75 = 5–15 8K model n = 97 Mean = 11.5 Median = 10.0 P25–P75 = 6–14 12K model n = 111 Mean = 10.8 Median = 9.0 P25–P75 = 6–12 16K model n = 110 Mean = 12.7 Median = 10.0 P25–P75 = 6–16 01020304050 Episodes per run 0 2 4 6 8 10 12 14 Reactive Number of runs n = 61 Mean = 12.8 Median = 11.0 P25–P75 = 7–17 01020304050 Episodes per run n = 72 Mean = 13.6 Median = 12.0 P25–P75 = 8–15 01020304050 Episodes per run n = 82 Mean = 12.6 Median = 11.0 P25–P75 = 8–15 01020304050 Episodes per run n = 83 Mean = 15.1 Median = 12.0 P25–P75 = 9–20 supervision supports successful diagnosis and recovery after errors occur. Proactive successes also increase from 17 at 4K to 29 at 12K before slightly decreasing to 27 at 16K, with the proactive share peaking at 26.13% at 12K. Overall, longer horizons are associated with more successful recovery trajectories while preserving strong proactive solving. Longer horizons can induce over-persistence. Together, Table 3 and Figure 2 suggest that 12K provides a favorable balance between proactive and reactive behavior. At 12K, proactive successes peak at 29 (26.13% of successful trajectories), while reactive successes reach 82. It also produces the shortest successful trajectories, averaging 10.8 episodes overall and 12.6 episodes among reactive runs. Scaling to 16K yields only a marginal increase in reactive successes (82→83), while proactive successes decrease (29→27) and successful trajectories become longer on average, both overall (10.8→12.7 episodes) and among reactive runs (12.6→15.1 episodes). These results suggest that the additional persistence induced by longer supervision becomes less efficient beyond 12K and is consistent with over-persistence at 16K. We leave detailed task-specific analyses in Appendix C. Takeaway 2 (Persistence Emerges and Eventually Overshoots). Increasing the supervision horizon is associated with more successful recovery trajectories, with the most favorable balance around 12K. Further scaling provides little additional reactive success while producing longer trajectories, suggesting over-persistence at 16K. 5.2WHY DOES RELIABILITY SATURATE? Longer supervision exposes later-stage behavior absent under short-horizon training, but these stages involve increasingly diverse histories. We formalize this tension as a trade-off between temporal supervision bias and finite-sample estimation difficulty. LetL H (π)be the expected imitation loss over the first H teacher decisions, and bπ H the empirical risk minimizer at horizon H . Definition 5.1 (Temporal supervision bias). LetΠ ⋆ H be the set of prefix-optimal policies andπ ⋆ T a full-horizon optimal policy. The temporal supervision bias at horizon H is Π ⋆ H = arg min π∈Π L H (π), π ⋆ T ∈ arg min π∈Π L T (π), B(H) := sup π∈Π ⋆ H L T (π)− L T (π ⋆ T ), B(H)≥ 0, B(T ) = 0. (4) B(H) is the worst-case full-horizon risk left undetermined by supervising only the firstHdecisions. We assume it is non-increasing inH, while the horizon-induced policy complexityC(H)is non- decreasing (Lemma E.3). Theorem 5.2 (Bias–Complexity Bound). AssumeNindependent teacher trajectories and a bounded loss0 ≤ ℓ ≤ M. LetL ⋆ H := min π∈Π L H (π) , and assume the prefix-to-full-horizon transfer 6 coefficient is bounded: Γ H :=sup π∈Π:L H (π)>L ⋆ H L T (π)− sup π ′ ∈Π ⋆ H L T (π ′ ) + L H (π)− L ⋆ H ≤ Γ <∞, ∀H ≤ T.(5) Then, with probability at least 1− δ, L T (bπ H )− L T (π ⋆ T )≤ B(H) + √ 2 ΓM r C(H) + log(2/δ) N .(6) The proof, assumptions, and scope are in Appendix E. The result bounds imitation risk rather than task success, so we use it as a lens on the trade-off and test it empirically. Figure 3: Supervision Bias b B(·) and Tem- poral Difficultyℓ loc H (·; 4)for the 8K, 12K and 16K models. Response log-likelihood Cutoff / Horizon (response tokens) Avg. LLH of the last 4 tokens Empirical diagnostics. The two terms in Theo- rem 5.2 are not directly observable, so we measure one diagnostic for each on training trajectories longer than16K tokens. For each horizon-Hmodelθ H , we compute ̄ ℓ H (K) = 1 K X t≤K logp θ H (x t | x <t ), b B(H) = ̄ ℓ 16K (K max )− ̄ ℓ H (K max ), ℓ loc H (K;w) = 1 w K+w X t=K+1 logp θ H (x t | x <t ), (7) whereK max = 16K and we use a local window of w = 4tokens throughout. The bias proxy b B(H) targets the first term, measuring how much worse a model supervised up toHfits the full16K trajectory than one supervised on all of it, thereby mirroring the truncation-induced excess risk inB(H). The local difficultyℓ loc H (K; 4)targets the second term: tracking a short window’s likelihood asKincreases reveals how difficult each position becomes to model, reflect- ing the growing heterogeneity of late-stage histories that drives estimation burden. These quantities are diagnostics, not direct estimates of the theoretical terms (Section A.3). As shown in Figure 3,ℓ loc H (K; 4)drops substantially as the trajectory progresses, even within supervised regions (red oval regions), so later-stage behavior is harder to model. Meanwhile, b B(12K) < b B(8K) : longer supervision narrows the gap to the16K reference, consistent with the assumption that temporal supervision bias is non-increasing inH. Together with Theorem 5.2, this suggests that longer supervision reduces truncation bias but adds behavior that is harder to estimate from finite data. Interpretation. Theorem 5.2 and Figure 3 point to the same tension: longer supervision reduces the deficit caused by truncating later-stage behavior, but the newly introduced behavior is harder to estimate from finite data. This is consistent with the saturation observed in Section 4, and it motivates the selective refinement strategy in Section 6. The bound does not explain the behavioral shift toward over-persistence, which we regard as an empirical finding. Takeaway 3 (Estimation Burden). Longer supervision reduces temporal bias but increases esti- mation burden by exposing the model to more heterogeneous late-stage behaviors. Empirically, supervision beyond the effective horizon coincides with over-persistence (Section 5.1). 6SELECTIVE LONG-HORIZON REFINEMENT Motivation & Method Overview. Longer supervision exposes useful later-stage behaviors, but training on all long-horizon continuations does not consistently improve reliability. Motivated by the 7 bias–complexity trade-off in Theorem 5.2 and increasing temporal difficulty in Figure 3, we propose a two-stage selective refinement strategy. We first train on all trajectories up to a shorter horizon H 1 , then extend supervision toH 2 using continuations with high likelihood under the warm-start model. We interpret likelihood as compatibility with behavior learned atH 1 , rather than as a universal measure of trajectory quality. This provides a conservative way to introduce long-horizon behavior and improve the reliability–efficiency trade-off without uniformly training on difficult continuations. and Select Top-LLH trajectories 0 Train on all samples up to 1 Dataset 3 Score Ranking Train on all selected samples 2 Warm-start at 0 Score only new tokens between and high low high medium 4 Refine at Figure 4: Overview of the proposed two-stage selective long-horizon training method. Stage I: Data Preparation. Following Section 3, letD = τ (i) N i=1 denote the offline teacher dataset, where each trajectoryτ (i) = (x (i) 1 ,...,x (i) L i )containsL i supervised response tokens. We consider two supervision horizons,H 1 < H 2 , whereH 1 is used for short-horizon warm-start training andH 2 for subsequent long-horizon refinement. Since scoring requires observing the complete continuation fromH 1 toH 2 , we restrict the selection pool to trajectories of length at leastH 2 , D H 2 = τ (i) ∈ D : L i ≥ H 2 . For each candidate trajectory, the interval(H 1 ,H 2 ]is used for likelihood-based scoring and selection. Stage I: Short-Horizon Warm Start. We train on the full dataset with supervision truncated atH 1 : L H 1 (θ;D) =− P N i=1 P min(H 1 ,L i ) t=1 logp θ (x (i) t | x (i) <t ) P N i=1 min(H 1 ,L i ) , θ 1 = arg min θ L H 1 (θ;D).(8) The resulting modelθ 1 provides both the initialization for long-horizon training and the reference model for selecting learnable continuations. Stage I: Continuation Scoring and Selection. For eachτ (i) ∈D H 2 , we score the continuation between H 1 and H 2 using the warm-start model: s i (H 1 ,H 2 ) = 1 H 2 − H 1 H 2 X t=H 1 +1 logp θ 1 (x (i) t | x (i) <t ).(9) Given a selection ratio ρ, we retain the highest-scoring trajectories to formD sel ⊆D H 2 . Stage IV: Selective Long-Horizon Refinement. Starting from the warm-start checkpointθ 1 , we continue training onD sel with the supervision horizon extended toH 2 . Since all selected trajectories satisfy L i ≥ H 2 , the refinement objective is L H 2 (θ;D sel ) =− 1 |D sel |H 2 X τ (i) ∈D sel H 2 X t=1 logp θ x (i) t | x (i) <t , θ 2 = arg min θ ;θ←θ 1 L H 2 (θ;D sel ). (10) 7EXPERIMENTS We compare selective long-horizon refinement with one-stage training at 4K, 8K, 12K, and 16K. For 12K and 16K refinement, we warm-start from the 8K checkpoint, rank candidate continuations by likelihood under the warm-start model, and retain the top or bottom25%,50%, or75%for training. All models are trained for two epochs, with each stage of two-stage training run for two epochs. 8 Table 5: Reliability of one-stage and selective two-stage training on Terminal-Bench v1.0. Two- stage models warm-start from 8K and refine at 12K or 16K on the highest- (Top) or lowest-likelihood (Bot.)ρ%of continuations. Values are reported with estimated run-to-run variation.Shaded: improves over one-stage training at the same horizon; green: best in row. One-stage12K refinement (from 8K)16K refinement (from 8K) training horizonρ=25%ρ=50%ρ=75%ρ=25%ρ=50%ρ=75% Metric4K8K12K16KTopBot.TopBot.TopBot.TopBot.TopBot.TopBot. Aggregate reliability Succ. attempts↑78±2.897±2.6111±2.3 110±2.7125±2.2 108±2.8121±2.3112±2.6119±2.5117±2.7114±2.6 108±2.9122±2.4120±2.5126±2.1114±2.7 Rel ≥1 ↑24±0.825±0.829±0.726±0.826±0.728±0.827±0.728±0.829±0.731±0.827±0.724±0.828±0.725±0.826±0.730±0.8 ̄s + ↑3.25±0.1 3.88±0.1 3.83±0.1 4.23±0.14.81±0.13.86±0.14.48±0.14.00±0.14.10±0.1 3.77±0.1 4.22±0.14.50±0.14.36±0.14.80±0.14.85±0.1 3.80±0.1 Repeatability Rel ≥4 ↑9±0.710±0.713±0.814±0.819±0.713±0.815±0.715±0.816±0.814±0.814±0.713±0.815±0.716±0.815±0.715±0.8 Rel ≥6 ↑4±0.58±0.69±0.69±0.710±0.67±0.611±0.68±0.69±0.69±0.79±0.69±0.711±0.612±0.714±0.611±0.7 Rel ≥8 ↑2±0.45±0.56±0.54±0.55±0.54±0.56±0.53±0.46±0.55±0.55±0.54±0.56±0.56±0.57±0.54±0.5 Fragile (1≤ s i ≤ 2)↓15±0.811±0.712±0.89±0.76±0.613±0.89±0.79±0.710±0.714±0.810±0.79±0.79±0.78±0.69±0.713±0.8 Following Section 4, we report successful attempts,Rel≥ 1,Rel≥ 4,Rel≥ 6,Rel≥ 8, ̄s + , and fragile tasks. For each checkpoint, we evaluate eight attempts per task, reporting the mean and run-to- run variation across four evaluations. Our main results use Terminal-Bench v1.0, while results on Terminal-Bench v2.0 and OpenThoughts-TBLite (Merrill et al., 2026) are deferred to Appendix D.4. 7.1MAIN RESULTS Table 4: Training cost of one-stage and selec- tive two-stage training. Top-50% models are warm-started from 8K; Total includes the warm- start stage. Parentheses give the change vs. one- stage training at the same horizon. Hor.Strategy#SamplesTime (h)Total (h)↓ 8KFull59,40821.6521.65 12KFull59,40835.4435.44 12KTop-50%19,56811.8333.48 (−5.5%) 16KFull59,40850.8150.81 16KTop-50%19,60017.4839.13 (−23.0%) Selective refinement improves long-horizon training. Withρ ≥ 50%, Top-ρrefinement improves over one-stage training at the same horizon (Table 5). At 12K, Top-50 increases successful attempts from111±2.3to121±2.3, raisesRel ≥6 from9±0.6to11±0.6, and re- duces fragile tasks from12±0.8to9±0.7. At 16K, Top-75 performs best, increasing success- ful attempts from110±2.7to126±2.1,Rel ≥6 from9±0.7to14±0.6, andRel ≥8 from4±0.5 to7±0.5. These gains exceed run-to-run varia- tion, showing that selective long-horizon super- vision is more effective than uniform training. Selective refinement also reduces training cost. The refinement stage uses only a third of the samples of one-stage training (19,568 vs. 59,408; Table 4). Including the 8K warm start, 8K→12K Top-50% takes 33.48 h in total, less than one-stage 12K (35.44 h) and 16K (50.81 h) training, while solving more attempts (121 vs. 111 and 110). At 16K, selective refinement cuts total training time by 23.0% (39.13 vs. 50.81 h). 7.2ABLATION STUDY Does high-likelihood selection improve long-horizon learnability? Figure 5 provides direct evidence that high-likelihood selection improves long-horizon learnability. At the same selection ratio, Top-likelihood models consistently achieve higher local likelihood than their Bottom-likelihood counterparts (Top-25%>Bottom-25%, Top-50%>Bottom-50%, and Top-75%>Bottom-75%). Notably, Top-25% even surpasses full 12K training around the target horizon. These results suggest that warm-start likelihood identifies continuations that are more compatible with the learned short- horizon policy and therefore easier to model during long-horizon refinement. How much long-horizon data should be retained?Table 5 and Figure 5 show a horizon-dependent quality–quantity trade-off. At 12K, Top-25 gives the most successes (125±2.2), while Top-50 yields slightly strongerRel ≥6 (11vs.10); at 16K, Top-75 is best (126±2.1successes,Rel ≥6 = 14). This matches Theorem 5.2: smallerρimproves continuation compatibility but reducesN sel , increasing the estimation burdenO p C(H)/N sel . SinceC(H)also grows with horizon, longer horizons require retaining more data to control estimation error. 9 Figure 5: Temporal Supervision Difficultyℓ loc H (·; 4)of the base, selective 12K-refined, and full 12K models. Cutoff / Horizon (response tokens) Avg. LLH of the last 4 tokens 12K-Top25 surpasses 12K Does likelihood-based selection matter? Likelihood-based selection matters, but its effect depends on the selection ratio and target horizon (Table 5). At 12K withρ = 50%, Top-50 clearly outperforms Bottom- 50, with121±2.3vs.112±2.6successful attempts,Rel ≥6 = 11±0.6vs.8±0.6, and Rel ≥8 = 6±0.5vs.3±0.4. These gaps exceed the estimated run-to-run variation, whereas at other settings the differences narrow or reverse. Thus, likelihood is a useful compatibility signal, but should be considered jointly with the retained data and target horizon. Is a longer refinement horizon always beneficial? No. Atρ = 50%, 12K-Top and 16K-Top are statistically comparable (121±2.3vs.122±2.4successes; bothRel ≥6 = 11±0.6). Atρ = 75%, however, extending to 16K improves119±2.5→ 126±2.1successes andRel ≥6 from9±0.6to 14±0.6. Thus, longer refinement helps only when enough continuation data are retained. Additional ablations. We provide other ablations, covering task-level and selection analy- ses, schedule/data-quantity controls, and cross-benchmark results on Terminal-Bench v2.0 and OpenThoughts-TBLite in Appendix D. 8CONCLUSION We study the supervision horizon in terminal-agent training and identify an emergence-and-saturation pattern: reliability improves with longer supervision but plateaus beyond an intermediate horizon. Based on this, we propose selective long-horizon refinement, using continuation likelihood to retain more learnable long-horizon trajectories while reducing unnecessary training burden. 10 AI USE STATEMENT Generative AI tools were used to assist with language polishing, LaTeX formatting, and code debugging. All research ideas, experimental design, analyses, results, and conclusions were developed and verified by the authors. ETHICS STATEMENT This work studies the training and evaluation of terminal agents using existing datasets and bench- marks. It does not involve human subjects or personally identifiable information. While more capable terminal agents could potentially be misused for harmful automation, our work focuses on improving training reliability and efficiency rather than expanding access to harmful capabilities. 11 REFERENCES Zachary Ankner, Cody Blakeney, Kartik Sreenivasan, Max Marion, Matthew L Leavitt, and Mansheej Paul. Perplexed by perplexity: Perplexity-based pruning with small reference models. In ICLR 2024 Workshop on Navigating and Addressing Data Problems for Foundation Models, 2024. URL https://openreview.net/forum?id=wNseMYxwpq. Anthropic.Introducing claude opus 4.8.https://w.anthropic.com/news/ claude-opus-4-8, May 2026. Accessed: 2026-09-20. Zihao Cheng, Hongru Wang, Zeming Liu, Xinyi Wang, Xiangrong Zhu, Yuhang Guo, Wei Lin, Jeff Z Pan, and Yunhong Wang. Terminal-world: Scaling terminal-agent environments via agent skills. arXiv preprint arXiv:2605.20876, 2026. Dang Cao Cuong, Hoang Anh Just, and Ruoxi Jia. Capacity-dependent effects of data selection for mathematical reasoning. In Third Conference on Language Modeling, 2026. URLhttps: //openreview.net/forum?id=2QZkhzstnL. Kanishk Gandhi, Shivam Garg, Noah D Goodman, and Dimitris Papailiopoulos. Endless terminals: Scaling rl environments for terminal agents. arXiv preprint arXiv:2601.16443, 2026. Ming Li, Yong Zhang, Zhitao Li, Jiuhai Chen, Lichang Chen, Ning Cheng, Jianzong Wang, Tianyi Zhou, and Jing Xiao. From quantity to quality: Boosting LLM performance with self-guided data selection for instruction tuning. In Kevin Duh, Helena Gomez, and Steven Bethard (eds.), Proceedings of the 2024 Conference of the North American Chapter of the Association for Compu- tational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 7602–7635, Mexico City, Mexico, June 2024. Association for Computational Linguistics. doi: 10.18653/v1/ 2024.naacl-long.421. URL https://aclanthology.org/2024.naacl-long.421/. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, et al. Deepseek-v3. 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556, 2025. Max Marion, Ahmet Üstün, Luiza Pozzobon, Alex Wang, Marzieh Fadaee, and Sara Hooker. When less is more: Investigating data pruning for pretraining llms at scale. arXiv preprint arXiv:2309.04564, 2023. Mike A Merrill, Alexander Glenn Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E. Kelly Buchanan, Junhong Shen, Guanghao Ye, Haowei Lin, Jason Poulos, Maoyu Wang, Marianna Nezhurina, Di Lu, Orfeas Menis Mastromichalakis, Zhiwei Xu, Zizhao Chen, Yue Liu, Robert Zhang, Leon Liangyu Chen, Anurag Kashyap, Jan-Lucas Uslu, Jeffrey Li, Jianbo Wu, Minghao Yan, Song Bian, Vedang Sharma, Ke Sun, Steven Dillmann, Akshay Anand, Andrew Lanpouthakoun, Bardia Koopah, Changran Hu, Etash Kumar Guha, Gabriel H. S. Dreiman, Jiacheng Zhu, Karl Krauth, Li Zhong, Niklas Muennighoff, Robert Kwesi Amanfu, Shangyin Tan, Shreyas Pimpalgaonkar, Tushar Aggarwal, Xiangning Lin, Xin Lan, Xuandong Zhao, Yiqing Liang, Yuanli Wang, Zilong Wang, Changzhi Zhou, David Heineman, Hange Liu, Harsh Trivedi, John Yang, Junhong Lin, Manish Shetty, Michael Yang, Nabil Omi, Negin Raoof, Shanda Li, Terry Yue Zhuo, Wuwei Lin, Yiwei Dai, Yuxin Wang, Wenhao Chai, Shang Zhou, Dariush Wahdany, Ziyu She, Jiaming Hu, Zhikang Dong, Yuxuan Zhu, Sasha Cui, Ahson Saiyed, Arinbjörn Kolbeinsson, Christopher Michael Rytting, Ryan Marten, Yixin Wang, Jenia Jitsev, Alex Dimakis, Andy Konwinski, and Ludwig Schmidt. Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces. In The Fourteenth International Conference on Learning Representations, 2026. URLhttps://openreview. net/forum?id=a7Qa4CcHak. OpenAI. Introducing codex.https://openai.com/index/introducing-codex/, May 2025. Accessed: 2026-09-20. Renjie Pi, Grace Lam, Mohammad Shoeybi, Pooya Jannaty, Bryan Catanzaro, and Wei Ping. On data engineering for scaling LLM terminal capabilities. In Deep Learning for Code: Towards Human-Centered Coding Agents, 2026. URLhttps://openreview.net/forum?id= BykU9KPyQ1. 12 Xuan Ren, Biao Wu, and Lingqiao Liu. I learn better if you speak my language: Understanding the superior performance of fine-tuning large language models with LLM-generated responses. In Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (eds.), Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 10225–10245, Miami, Florida, USA, November 2024. Association for Computational Linguistics. doi: 10.18653/v1/2024.emnlp-main. 571. URL https://aclanthology.org/2024.emnlp-main.571/. Vaishnavi Shrivastava, Piero Kauffmann, Ahmed Awadallah, and Dimitris Papailiopoulos. Echo: Terminal agents learn world models for free. arXiv preprint arXiv:2605.24517, 2026. Siwei Wu, Yizhi LI, Yuyang Song, Wei Zhang, Yang Wang, Riza Batista-Navarro, Xian Yang, Mingjie Tang, Bryan Dai, Jian Yang, and Chenghua Lin. Large-scale terminal agentic trajectory generation from dockerized environments. In Forty-third International Conference on Machine Learning, 2026. URL https://openreview.net/forum?id=PeFSCRulgy. Dylan Zhang, Qirun Dai, and Hao Peng. The best instruction-tuning data are those that fit. In The Thirty-ninth Annual Conference on Neural Information Processing Systems, 2025. URL https://openreview.net/forum?id=4jFSekBaDT. Chunting Zhou, Pengfei Liu, Puxin Xu, Srini Iyer, Jiao Sun, Yuning Mao, Xuezhe Ma, Avia Efrat, Ping Yu, LILI YU, Susan Zhang, Gargi Ghosh, Mike Lewis, Luke Zettlemoyer, and Omer Levy. LIMA: Less is more for alignment. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL https://openreview.net/forum?id=KBMOKmX2he. 13 APPENDIX CONTENTS ASetting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 A.1Training Setting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .14 A.2Evaluation Protocol and Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . .15 A.3Empirical Measures of the Bias–Complexity Trade-off . . . . . . . . . . . . . . . .15 A.4Proactive vs. Reactive Classification . . . . . . . . . . . . . . . . . . . . . . . . . .15 BTask-Level Emergence and Saturation of Reliability . . . . . . . . . . . . . . . . . . .15 B.1From Fragile to Repeatable Success . . . . . . . . . . . . . . . . . . . . . . . . . . .15 B.2Saturation as Cancellation, Not a Plateau . . . . . . . . . . . . . . . . . . . . . . . .16 CHorizon and Agent Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .17 C.1Longer Supervision Enables Recovery . . . . . . . . . . . . . . . . . . . . . . . . .18 C.2Beyond the Effective Horizon: Over-Persistence . . . . . . . . . . . . . . . . . . . .18 DAdditional Results for Selective Refinement . . . . . . . . . . . . . . . . . . . . . . . .19 D.1Task-Level Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .19 D.2Top- vs. Bottom-Likelihood Selection . . . . . . . . . . . . . . . . . . . . . . . . . .19 D.3How do training schedule, data quantity, and likelihood selection interact? . . . . .19 D.4Results on Other Benchmarks . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .20 EMissing Proofs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .22 ASETTING This section provides the detailed setup used throughout our paper, including the training configura- tion, evaluation protocol, reliability metrics, and auxiliary analyses underlying the main results. A.1TRAINING SETTING For the experiments in Section 4, Section 5 and Section 6, we train all models with full-parameter supervised fine-tuning using LLaMA-Factory on the Nemotron terminal-agent corpus, which contains 59,407 ShareGPT-format trajectories. The base model for one-stage training isQwen/Qwen3-8B. Unless otherwise specified, all runs use the same hyperparameters: two training epochs, BF16 precision, DeepSpeed ZeRO-3 on four NVIDIA H100 GPUs, per-device batch size 1 with no gradient accumulation (global batch size 4), AdamW with learning rate1× 10 −5 , cosine decay without warmup, gradient clipping at 1.0, and no sequence packing. We train complete-data models with supervision cutoffs of 4,096, 8,192, 12,288, and 16,384 tokens, respectively, while keeping the underlying 59,407 trajectories fixed so that these runs isolate the effect of supervision horizon. For selective refinement, we warm-start from the complete 8K checkpoint and continue training for two additional epochs at either 12K or 16K on likelihood-filtered subsets of long trajectories. Candidate trajectories are those extending beyond 8,192 tokens and are ranked by the average teacher-forced log-probability assigned by the 8K model to assistant tokens in the newly exposed continuation, i.e., 8K–12K or 8K–16K. We retain either the highest- or lowest-likelihood 25%, 50%, or 75% of the scored trajectories, yielding approximately 9.8K, 19.6K, and 29.4K training examples, respectively. Trajectories not extending beyond 8K are excluded from the selective-refinement stage. As shown in Table 4, the 12K Top-50% setting contains slightly fewer samples than the 16K Top-50% 14 setting because some candidate trajectories do not contain enough tokens in the required 8K–12K continuation and are therefore skipped during scoring and selection. A.2EVALUATION PROTOCOL AND METRICS We evaluate all models on Terminal-Bench v1.0 using the Terminus-2 scaffold, with eight independent attempts per task. For each taski,s i ∈ 0,..., 8denotes the number of successful attempts; the reliability metricsRel ≥k , ̄s + , and fragile tasks (1≤ s i ≤ 2) are defined in Section 4. Because each task has only eight attempts, per-task differences of one or two successes are within sampling noise; in task-level analyses we therefore interpret only large or consistent shifts. A.3EMPIRICAL MEASURES OF THE BIAS–COMPLEXITY TRADE-OFF The metrics in Equation 7 are empirical diagnostics rather than estimates of the quantities in The- orem 5.2. The biasB(H)is defined through population risks over the unknown prefix-optimal set Π ⋆ H and the full-horizon optimumπ ⋆ T , while the complexity term depends on the policy class. In particular, b B(H)measures the gap to the trained16K model, itself a finite-sample estimate, rather than toπ ⋆ T , and corresponds to a single element ofΠ ⋆ H rather than the supremum inB(H). We therefore construct observable metrics that capture the same intuition. Specifically, b B(H)measures the full-horizon modeling gap between the empirically trained horizon-Hpolicy and the 16K ref- erence policy, mirroring the population risk gap inB(H). Meanwhile,ℓ loc H (K;w)measures the average log-likelihood of a local token window on the training trajectories. Tracking this quantity as Kincreases reveals how the empirical learnability of behavior changes deeper into the trajectory: decreasing local likelihood indicates that later-stage behavior is increasingly difficult for the learned policy to model. Thus, these metrics do not numerically instantiate the theoretical terms, but provide empirical diagnostics for the same bias–difficulty trade-off. A.4PROACTIVE VS. REACTIVE CLASSIFICATION For the experiment in Section 5.1, we classify each trial according to whether the agent encounters an execution error during interaction. A trial is proactive if no genuine command error appears in any terminal observation returned to the agent, and reactive if at least one such error occurs and the agent must respond to it. We inspect the terminal observation after every interaction step, excluding the initial task prompt, and match lines against common error signatures, including Python tracebacks and * Error/ * Exception,command not found,No such file or directory, per- mission and access failures,fatal/error, segmentation faults, missing modules, syntax errors, linker failures,FAILED,Killed, connection failures, andapt E:messages. To avoid false posi- tives, we ignore lines that merely echo the agent’s own commands or code, harness/parser messages about response formatting, and warnings. This classification is independent of task outcome: a resolved trial may therefore be either proactive, meaning it succeeds without encountering an error, or reactive, meaning it succeeds after encountering and recovering from an error. The procedure is intentionally mechanical and identifies whether an error occurred, rather than inferring how the agent reasoned about it. BTASK-LEVEL EMERGENCE AND SATURATION OF RELIABILITY Aggregate metrics in Section 4 summarize how reliability changes with the supervision horizon but not which tasks drive the change. Table 6 reports per-task successes for all twelve configurations; this section uses the four one-stage columns, and Appendix D discusses the selective ones. With eight attempts per task, differences of one or two successes are within sampling noise (Appendix A.2), so we focus on large or consistent shifts. B.1FROM FRAGILE TO REPEATABLE SUCCESS At 4K, success is sparse and fragile:15 of the 24 solved tasks succeed in only one or two of eight attempts.Extending the horizon to 12K mostly moves tasks up the reliability distribution rather than adding isolated successes.For example, 15 Table 6: Task-level reliability across one-stage and selective training. Each cell reports the number of successful attempts out of eight; shading scales with success count (1358), and·denotes zero. T/B denote selection of the highest-/lowest-likelihood continuations during refinement. #Runs counts the configurations (out of 12) that solve each task at least once; green marks the best value in each summary row. One-stage12K refinement16K refinement Task4K8K12K16KT50B50T75B75T50B50T75B75#Runs Solved by all 12 configurations crack-7z-hash.easy41643665486712 create-bucket13232342331612 csv-to-parquet88878788888812 extract-safely57887786888712 fibonacci-server 22348555767612 fix-pandas-version77888887888812 fix-permissions78888888788812 git-workflow-hack23735433357612 grid-pattern-transform12125113221112 hello-world88888888888812 heterogeneous-dates 58468534666612 incompatible-python-fasttext23221326353312 incompatible-python- fasttext.base_with_hint 24866368878512 modernize-fortran-build23375552467112 new-encrypt-command 22545666667512 simple-web-scraper28356558747712 sqlite-with-gcov12426454553412 swe-bench-langcodes56767785878412 Solved by a subset of configurations configure-git-webserver·11221122 ·2110 crack-7z-hash· · 453443314310 vim-terminal-task45131141 · ·1110 conda-env-conflict-resolution· ·1111211 ·219 blind-maze-explorer-algorithm ·1 · ·12 ·211 ·17 processing-pipeline·11 ·23321 · · ·7 blind-maze-explorer-algorithm.hard12 ·1 · · ·12 · ·16 get-bitcoin-nodes 2 ·11 ·111 · · · ·6 tmux-advanced-workflow2 ·1 · · ·11 ·12 ·6 blind-maze-explorer-5x5· · ·231 ·1 · · ·15 oom2 ·1 · ·2 ·2 · ·1 ·5 simple-sheets-put· · · · · · 11122 ·5 git-multibranch 11 · · · · · ·1 ·1 ·4 openssl-selfsigned-cert· ·32 · · ·14 · · ·4 fix-git· · 1 · ·1 · · · · ·13 password-recovery·1 · · · · · · ·2 ·13 blind-maze-explorer-algorithm.easy · ·1 ·1 · · · · · · ·2 prove-plus-comm· · · · · ·1 ·1 · · ·2 crack-7z-hash.hard· · · · · · · · ·1 · ·1 decommissioning-service-with- sensitive-data · · · · · · · · · · ·11 eval-mteb· · · · · · · · · · ·11 eval-mteb.hard· · · · · · · · · · ·11 organization-json-generator· · · · · · 1 · · · · ·1 path-tracing· · · · · · · · ·2 · ·1 path-tracing-reverse· · · ·1 · · · · · · ·1 swe-bench-fsspec· · · · · ·1 · · · · ·1 Correct tasks (Rel ≥1 )242529262728293128252630 Successful attempts7897111110121112119117122120126114 incompatible-python-fasttext.base_with_hintimproves from2/8to8/8, git-workflow-hackfrom2/8to7/8,new-encrypt-commandfrom2/8to5/8, and sqlite-with-gcov from 1/8 to 4/8. Some tasks become solvable only once the horizon is long enough.crack-7z-hashis never solved at 4K or 8K but reaches4/8at 12K and5/8at 16K;conda-env-conflict-resolutionis first solved at 12K; andblind-maze-explorer-5x5is solved only at 16K. Successful runs on these tasks take 20–33 episodes of sustained trial and adjustment (Table 8), so short-horizon training rarely observes the teacher behavior they require. Consistently, the 16K model produces 21 successful trials lasting at least 20 episodes, spread across 12 tasks, whereas the 8K model produces 11 across 6 tasks. B.2SATURATION AS CANCELLATION, NOT A PLATEAU From 12K to 16K, the aggregate success count barely moves (111→ 110), but this hides substantial task-level movement. Of the 34 tasks solved by at least one one-stage model, 11 improve, 13 worsen, 16 Table 7: Recovery persistence across supervision horizons. Taxonomy: successful trajectories over the 36 manually analyzed tasks, classified as proactive (pitfall anticipated) or reactive (error observed, then repaired). Reactive tasks: successes out of eight on tasks that require recovery; every modernize-fortran-buildsuccess is reactive. Pass rate: over the five tasks whose eight trials are all classified. Green: best in row. Metric4K8K12K16K Taxonomy (36 tasks) Proactive successes42535654 Reactive successes27384142 Reactive share39%42%42%44% Reactive tasks modernize-fortran-build2/83/83/87/8 fibonacci-server2/82/83/84/8 simple-web-scraper (reactive only)0/82/80/84/8 Pass rate (5 fully classified tasks)50%70%63%80% Table 8: Task-level behavior across supervision horizons. Successes out of eight, with mean episodes per run in parentheses (shown for solved settings). Counts are a subset of Table 6; green: best in row. Task4K8K12K16K Long-horizon emergence crack-7z-hash004 (27.1)5 (32.6) conda-env-conflict-resolution001 (32.5)1 (25.1) blind-maze-explorer-5x50002 (20.6) Efficient recovery modernize-fortran-build2 (10.5)3 (15.5)3 (11.6)7 (12.9) Over-persistence simple-web-scraper2 (6.9)8 (7.4)3 (7.3)5 (9.3) vim-terminal-task4 (28.3)5 (13.0)1 (18.8)3 (23.1) and 10 are unchanged. The improving tasks gain 18 successes and the degrading tasks lose 19: +18 |z gained successes + (−19) |z lost successes =−1.(11) The gains concentrate on tasks that require long recovery, such asmodernize-fortran-build (3/8 →7/8) andblind-maze-explorer-5x5(0 →2/8).The losses fall mainly on tasks that peak at 12K:git-workflow-hack(7/8 →3/8), incompatible-python-fasttext.base_with_hint(8/8 →6/8), sqlite-with-gcov(4/8 → 2/8), andopenssl-selfsigned-cert(3/8 → 2/8). In addition, five tasks solved once at 12K become unsolved at 16K. This cancellation explains the mixed aggregate profile in Table 2. Coverage (Rel ≥1 ) and perfect repeatability (Rel ≥8 ) fall because marginal and 12K-peak tasks lose ground. Meanwhile,Rel ≥4 , ̄s + , and the number of fragile tasks improve because other tasks become more consistent. Saturation is therefore not a uniform plateau: beyond 12K, additional supervision redistributes reliability across tasks rather than increasing it. Caveat. A few tasks are solved most often at the shortest horizon; for example,oomand tmux-advanced-workflowsucceed in2/8attempts at 4K and in at most1/8at longer horizons. These differences are within sampling noise, and we do not interpret them as horizon effects. CHORIZON AND AGENT BEHAVIOR This section complements Section 5.1 with task-level evidence on how the horizon shapes persistence: the agent’s tendency to keep acting after an error, diagnose it, and recover rather than terminate. Table 3 in the main text uses the automatic error-signature classifier of Appendix A.4 over all runs. Here we additionally use a finer manual taxonomy over 36 tasks, which labels a successful trajectory 17 Table 9: Recovery persistence across supervision horizons. Successful trajectories over the 36 manually analyzed tasks, classified as proactive (pitfall anticipated) or reactive (error observed, then repaired). Pass rate is computed over the five tasks whose eight trials are all classified. green: best in row. 4K8K12K16K Proactive successes42535654 Reactive successes27384142 Reactive share39%42%42%44% Pass rate (5 tasks)50%70%63%80% proactive if the relevant pitfall is anticipated before it manifests and reactive if the error first occurs and is then diagnosed and repaired (Table 9). C.1LONGER SUPERVISION ENABLES RECOVERY Reactive successes grow steadily with the horizon, from 27 at 4K to 42 at 16K, and their share of successful trajectories rises from 39% to 44% (Table 9). Because some tasks are structurally proactive or reactive, this ratio alone is not evidence of persistence; the more informative comparison is across checkpoints on the same task. modernize-fortran-buildis structurally reactive: the build must fail before it can be repaired, so every success is a recovery. Successes rise from2/8at 4K to3/8at 8K and 12K, and then to7/8at 16K. Mean episodes at 16K (12.9) remain below those at 8K (15.5), so the gain reflects better-targeted recovery rather than more attempts (Table 8).fibonacci-servershows a similar trend, rising from 2/8 to 4/8. The contrast is sharpest onsimple-web-scraper. The 4K model never recovers after observing the target bug: it dismisses the observed behavior as acceptable, edits the downstream report instead of the corrupted CSV, or spends its remaining budget on an unrelated error. At 16K, four of the five successful trajectories recover reactively, one after 22 episodes of revisiting its diagnosis. Across the five tasks whose eight trials are all classified, the pass rate rises from 50% at 4K to 80% at 16K. Longer supervision therefore exposes the learner to later trajectory stages in which an initial action has already failed, shifting behavior fromAct → Observe Failure → Terminatetoward Act→ Observe Failure→ Diagnose→ Retry→ Recover. C.2BEYOND THE EFFECTIVE HORIZON: OVER-PERSISTENCE The same tendency to continue becomes costly on tasks that shorter horizons already solve. On simple-web-scraperandvim-terminal-task, the 8K model is best, and longer horizons both lower success and lengthen runs. For example, 8K solvesvim-terminal-taskin5/8 attempts with 13.0 episodes per run, whereas 16K solves it in 3/8 with 23.1 (Table 8). Failed 16K trajectories onsimple-web-scraperreveal the mechanism. The agent often pro- duces a working script, notices a minor formatting issue, and then either enters repetitive correction or introduces a new error that breaks the solution; one failure repeats the same incorrect diagnosis over several episodes. In other runs, the agent verifies only coarse properties and terminates while semantic defects remain. The same tendency appears in aggregate: 89% of successful 8K runs finish within 20 episodes, compared with 81% at 16K, and mean episodes of successful runs rise from 10.8 at 12K to 12.7 at 16K (Figure 2). Together, these results reconcile the two faces of 16K. It shows the strongest recovery persistence on structurally reactive tasks, but this persistence is not selective: the same drive to continue helps tasks that require recovery and hurts tasks that were already solved. Behavior therefore moves from under-persistence at 4K, through productive persistence around 12K, to over-persistence at 16K, consistent with Section 5.1. 18 DADDITIONAL RESULTS FOR SELECTIVE REFINEMENT This section provides additional analyses supporting the selective long-horizon refinement results in Section 6 and Section 7 of the main paper. We further examine the effect of selection ratio, likelihood-based filtering, refinement horizon, and evaluation benchmark, complementing the main results and ablations on when and why selective refinement improves long-horizon agent training. D.1TASK-LEVEL RESULTS Selective refinement consistently improves aggregate success. The advantage of selective re- finement is not confined to a single configuration. All eight filtered models in Table 6 achieve 112–126 successful attempts, exceeding the best one-stage result of 111. Uniformly supervising all long-horizon continuations is therefore unnecessary for obtaining the benefits of long-horizon training. Selection trades coverage against repeatability.Successful attempts and task coverage need not move together. 16K-T75 obtains the most successful attempts (126) but solves 26 tasks, whereas 16K- B75 obtains fewer (114) while solving 30. Likelihood-based selection thus redistributes reliability: high-likelihood subsets favor repeated success on solvable tasks, whereas low-likelihood subsets expand the set of tasks solved. D.2TOP- VS. BOTTOM-LIKELIHOOD SELECTION Top-likelihood refinement repairs regressions and shortens runs. Top-likelihood selection re- covers several tasks that regress under full 16K training, usually with shorter runs (16K columns of Table 6).fibonacci-serverimproves from4/8under full 16K training to7/8un- der both T50 and T75, while mean episodes per run nearly halve (14.0 → 7.9under T50). incompatible-python-fasttext.base_with_hintreturns to8/8, matching its 12K peak, andswe-bench-langcodesrises from6/8to8/8.fix-pandas-versionisolates the efficiency effect: every 16K variant solves it8/8, but T50 needs 9.9 episodes per run versus 17.5 for full 16K training. In aggregate, T75 yields more completed runs than B75 (209 vs. 197), with shorter completed trajectories (10.7 vs. 12.2 mean episodes). Selection is not uniformly beneficial, however.vim-terminal-task, already an over-persistence case, degrades further under Top selection (3/8under full 16K training vs.1/8under T75 and 0/8under T50).modernize-fortran-buildis highly sensitive to which continuations are retained: it keeps7/8under T75 but collapses to1/8under B75, suggesting that the recovery segments it depends on can be filtered out entirely. Bottom-likelihood refinement trades repeatability for breadth. B75 solves the most tasks of any 16K model (Rel ≥1 = 30), includingeval-mteb,eval-mteb.hard, anddecommissioning-service-with-sensitive-data, which no one-stage or Top- selected model solves. B50 similarly addscrack-7z-hash.hardandpath-tracing. These new successes are mostly isolated (1–2/8): relative to T75, B75 has fewer consistently solved tasks (Rel ≥6 : 11 vs. 14) and more fragile ones (13 vs. 9; Table 5). Low-likelihood continuations thus appear to carry rare, diverse behaviors that expand coverage, whereas high-likelihood continua- tions consolidate repeatable solving, consistent with the capacity- and budget-dependent effects of likelihood-based selection reported by Cuong et al. (2026). D.3HOW DO TRAINING SCHEDULE, DATA QUANTITY, AND LIKELIHOOD SELECTION INTERACT? To disentangle the effects of the two-stage schedule, data quantity, and likelihood-based selection, we add two controls. First, we refine the 8K warm-start model at 12K and 16K using all eligible trajectories, isolating the effect of the two-stage schedule from filtering. Second, for each retention ratioρ ∈ 25%, 50%, 75%, we uniformly sample the same number of eligible trajectories as the corresponding Top/Bottom-ρsetting, so comparisons among Top, Random, and Bottom are made at a fixed data budget. Table 10 supports the bias–complexity trade-off in Theorem 5.2. First, two-stage 19 Table 10: Disentangling two-stage training, data quantity, and likelihood selection. All two- stage models warm-start from 8K and refine at the target horizon on a random, highest- (Top), or lowest-likelihood (Bottom)ρ%of continuations. One-stage and Top/Bottom results are measured. Shaded: improves over one-stage training at the same target; green: best in column within each target. SelectionρSucc.↑Reliability↑Fragile↓ Rel ≥4 Rel ≥6 Rel ≥8 12K target One-stage (full) — 111±2.3 13±0.8 9±0.6 6±0.5 12±0.8 Two-stage All100%120±2.416±0.810±0.6 6±0.59±0.7 Random116±2.515±0.88±0.64±0.510±0.7 Top125±2.219±0.710±0.65±0.56±0.6 Bottom 25% 108±2.813±0.87±0.64±0.513±0.8 Random117±2.515±0.8 9±0.6 5±0.510±0.7 Top121±2.315±0.711±0.6 6±0.59±0.7 Bottom 50% 112±2.615±0.8 8±0.6 3±0.49±0.7 Random118±2.615±0.89±0.75±0.510±0.8 Top119±2.516±0.89±0.66±0.510±0.7 Bottom 75% 117±2.714±0.89±0.75±0.514±0.8 16K target One-stage (full) — 110±2.7 14±0.8 9±0.7 4±0.5 9±0.7 Two-stage All100%123±2.416±0.812±0.76±0.58±0.7 Random111±2.814±0.89±0.75±0.510±0.7 Top114±2.614±0.79±0.65±0.510±0.7 Bottom 25% 108±2.913±0.89±0.74±0.59±0.7 Random121±2.515±0.811±0.76±0.5 9±0.7 Top 122±2.415±0.711±0.66±0.5 9±0.7 Bottom 50% 120±2.516±0.812±0.76±0.58±0.6 Random122±2.416±0.812±0.76±0.59±0.8 Top126±2.115±0.714±0.67±0.59±0.7 Bottom 75% 114±2.715±0.811±0.74±0.513±0.8 training with all eligible data improves over one-stage training from111± 2.3to120± 2.4successes at 12K and from110± 2.7to123± 2.4at 16K, indicating that the warm-start schedule itself provides a substantial benefit. Second, the Random controls reveal a horizon-dependent data-quantity effect. At 12K, increasing retention from 25% to 75% changes success only from116± 2.5to118± 2.6, which is within the estimated run-to-run variation. At 16K, however, the corresponding increase is much larger, from111±2.8to122±2.4, consistent with the finite-sample termO p C(H)/N sel : asC(H) grows with horizon, using too few trajectories becomes increasingly costly. Finally, fixed-budget comparisons isolate selection quality. At 12K-25%, Top reaches125± 2.2successes, compared with116± 2.5for Random and108± 2.8for Bottom, a gap larger than the reported variation. At 16K-75%, Top similarly reaches126± 2.1, versus122± 2.4for Random and114± 2.7for Bottom. In contrast, some settings show little separation, for example, at 16K-50%, Top, Random, and Bottom obtain122± 2.4,121± 2.5, and120± 2.5successes, respectively. Overall, the controls support a joint quality–quantity view: the two-stage schedule improves initialization, likelihood selection can provide additional gains when the separation exceeds evaluation variation, and retaining sufficient data becomes increasingly important at longer horizons. D.4RESULTS ON OTHER BENCHMARKS The emergence–saturation pattern generalizes across benchmarks. As shown in Table 11 and Table 12, the one-stage results reproduce the same qualitative trend observed in the main experiments. On Terminal-Bench v2.0, successful attempts increase from44±2.6at 4K to65±2.3at 12K, but remain essentially unchanged at 16K (64±2.6); similarly, on OpenThoughts-TBLite they increase 20 Table 11: Reliability of one-stage and selective two-stage training on Terminal-Bench v2.0. Two-stage models warm-start from 8K and refine at 12K or 16K on the highest- (Top) or lowest- likelihood (Bot.)ρ%of continuations. Values are reported with estimated run-to-run variation. Shaded : improves over one-stage training at the same horizon; green: best in row. One-stage12K refinement (from 8K)16K refinement (from 8K) training horizonρ=25%ρ=50%ρ=75%ρ=25%ρ=50%ρ=75% Metric4K8K12K16KTopBot.TopBot.TopBot.TopBot.TopBot.TopBot. Aggregate reliability Succ. attempts↑44±2.655±2.565±2.364±2.672±2.263±2.771±2.366±2.570±2.469±2.666±2.563±2.871±2.370±2.573±2.166±2.6 Rel ≥1 ↑13±0.714±0.717±0.615±0.715±0.716±0.816±0.716±0.817±0.718±0.816±0.714±0.816±0.715±0.815±0.717±0.8 ̄s + ↑3.38±0.1 3.93±0.1 3.82±0.1 4.27±0.14.80±0.13.94±0.14.44±0.14.13±0.14.12±0.13.83±0.1 4.13±0.14.50±0.14.44±0.14.67±0.14.87±0.1 3.88±0.1 Repeatability Rel ≥4 ↑5±0.56±0.58±0.69±0.611±0.68±0.69±0.69±0.69±0.68±0.68±0.68±0.69±0.69±0.69±0.69±0.6 Rel ≥6 ↑2±0.44±0.55±0.55±0.56±0.54±0.56±0.55±0.55±0.55±0.55±0.55±0.56±0.57±0.68±0.66±0.5 Rel ≥8 ↑1±0.32±0.43±0.42±0.43±0.42±0.44±0.52±0.44±0.53±0.43±0.42±0.43±0.43±0.44±0.52±0.4 Fragile (1≤ s i ≤ 2)↓7±0.66±0.67±0.75±0.64±0.58±0.75±0.65±0.66±0.68±0.76±0.65±0.65±0.65±0.65±0.68±0.7 Table 12: Reliability of one-stage and selective two-stage training on OpenThoughts-TBLite. Two-stage models warm-start from 8K and refine at 12K or 16K on the highest- (Top) or lowest- likelihood (Bot.)ρ%of continuations. Values are reported with estimated run-to-run variation. Shaded : improves over one-stage training at the same horizon; green: best in row. One-stage12K refinement (from 8K)16K refinement (from 8K) training horizonρ=25%ρ=50%ρ=75%ρ=25%ρ=50%ρ=75% Metric4K8K12K16KTopBot.TopBot.TopBot.TopBot.TopBot.TopBot. Aggregate reliability Succ. attempts↑96±2.9118±2.7 139±2.4 137±2.7153±2.2 135±2.8151±2.3141±2.6149±2.5147±2.7141±2.6 135±2.9151±2.4149±2.5155±2.2141±2.7 Rel ≥1 ↑28±0.830±0.836±0.732±0.832±0.734±0.834±0.734±0.836±0.738±0.834±0.730±0.834±0.732±0.832±0.736±0.8 ̄s + ↑3.43±0.1 3.93±0.1 3.86±0.1 4.28±0.1 4.78±0.13.97±0.14.44±0.14.15±0.14.14±0.13.87±0.1 4.15±0.14.50±0.14.44±0.14.66±0.14.84±0.1 3.92±0.1 Repeatability Rel ≥4 ↑11±0.713±0.717±0.819±0.8 23±0.717±0.819±0.719±0.819±0.817±0.817±0.717±0.819±0.719±0.819±0.719±0.8 Rel ≥6 ↑4±0.59±0.611±0.611±0.713±0.68±0.613±0.611±0.611±0.611±0.711±0.611±0.713±0.615±0.717±0.613±0.7 Rel ≥8 ↑2±0.44±0.56±0.54±0.56±0.54±0.58±0.54±0.58±0.56±0.56±0.54±0.56±0.56±0.59±0.54±0.5 Fragile (1≤ s i ≤ 2)↓15±0.813±0.715±0.811±0.79±0.617±0.811±0.711±0.713±0.717±0.813±0.711±0.711±0.711±0.711±0.717±0.8 from96±2.9to139±2.4before saturating at137±2.7. The reliability profile also exhibits the same redistribution beyond the effective horizon: from 12K to 16K, task coverage decreases (17→ 15 on TB v2.0;36→ 32on TBLite), while mean success among solved tasks increases (3.82→ 4.27; 3.86→ 4.28),Rel ≥4 increases, and the number of fragile tasks decreases. Thus, across benchmarks, longer supervision initially broadens task solving, whereas supervision beyond the intermediate horizon primarily makes already-solvable tasks more repeatable rather than increasing aggregate reliability. Selective refinement consistently improves long-horizon reliability.Table 11 and Table 12 further show that the benefit of selective refinement transfers across benchmarks. On Terminal-Bench v2.0, 12K Top-50 improves successful attempts from65±2.3to71±2.3, while 16K Top-75 improves from 64±2.6to73±2.1. The same pattern is stronger on OpenThoughts-TBLite, where the corresponding improvements are139±2.4→ 151±2.3and137±2.7→ 155±2.2. These gains are larger than the estimated run-to-run variation, supporting the main-paper observation that uniformly supervising all long-horizon continuations is unnecessary: selecting a subset of compatible continuations can yield higher reliability than full long-horizon training. The effect of selection depends on both likelihood and retained data.As shown in Table 11 and Table 12, the cross-benchmark results also preserve the interaction between continuation likelihood and selection ratio observed in the main experiments. At 12K andρ = 50%, Top-likelihood selection consistently exceeds Bottom-likelihood selection in aggregate success (71vs.66on TB v2.0 and 151vs.141on TBLite), while also improving higher-order repeatability. However, the advantage is not uniform across every metric or selection ratio: Bottom-likelihood subsets can retain broader task coverage in some settings, and the strongest 16K result emerges only when sufficient data are retained, with Top-75 achieving73±2.1successes on TB v2.0 and155±2.2on TBLite. This is consistent with the main conclusion that likelihood is a useful compatibility signal, but effective refinement must jointly balance continuation learnability and data sufficiency. 21 EMISSING PROOFS In this section, we provide the detailed derivation of Theorem 5.2 (or Theorem E.11) and establish several consequences for choosing the supervision horizon. In particular, Corollary E.14 characterizes when shorter-horizon supervision yields a tighter bound than full-horizon training, Corollary E.16 and Corollary E.18 characterize when an intermediate horizon minimizes a surrogate bound, and Corollary E.19 describes how this horizon changes with the amount of expert data. Our analysis proceeds in three steps. First, we formalize the benefit of longer supervision through temporal supervision bias, which captures the full-horizon performance gap induced by truncating teacher trajectories. Second, we show that extending the supervision horizon exposes the learner to a richer set of decision histories, increasing the effective policy complexity and hence the finite-sample estimation burden. Third, we connect prefix-level estimation error to full-horizon performance and combine these two effects to obtain the bias–complexity bound in Theorem E.11. We then use this result to answer three questions about the bound: when shorter-horizon supervision yields a tighter guarantee than full-horizon training, under what conditions an intermediate horizon minimizes a surrogate of the bound, and how this horizon shifts as the amount of expert data grows. These results characterize the guarantee rather than the true error, consistent with the scope stated below. Finally, we connect the decision-level analysis to the token-level supervision horizon used in our experiments. Setup. We build on the decision-level imitation-learning formulation in Section 3. LetΠdenote the student policy class andπ T the teacher policy. LetHdenote the number of teacher decisions retained for supervision. We assume thatNteacher trajectories are sampled independently fromπ T and have lengthsT i ≤ T. Fort > T i , we setℓ ≡ 0, so thatL H and b L H are well defined for every H ≤ T. For anyH ≤ T, supervision is restricted to the firstHdecisions, while the policy continues to condition on the complete history h t at each step. Scope of the analysis. The analysis concerns imitation risk under teacher-visited histories, for an empirical risk minimizer over a finite policy class with bounded loss. For log-loss, boundedness requires clipping probabilities below at someε > 0, givingM = ℓ max log(1/ε)for actions of at mostℓ max tokens. Neural networks trained for a finite number of optimizer steps on a token-averaged objective (Equation 8) only approximate this setting. The resulting bound does not by itself imply that the true error, or task success under the agent’s own rollouts, is non-monotone inH, and it makes no prediction about trajectory length or persistence. We therefore use it to identify two competing quantities and treat the behavioral findings of Section 5.1 as empirical. Definition E.1 (Horizon-Himitation risk). Letℓ(π(·| h t ),a t )denote an action-level imitation loss satisfying 0≤ ℓ(π(·| h t ),a t )≤ M.(12) The population and empirical imitation risks under supervision horizon H are L H (π) =E τ∼π T " 1 H H X t=1 ℓ(π(·| h t ),a t ) # , b L H (π) = 1 N N X i=1 1 H H X t=1 ℓ π(·| h (i) t ),a (i) t , (13) whereD =τ (i) N i=1 denotes the N teacher trajectories. Behavior cloning with horizon H learns bπ H ∈ arg min π∈Π b L H (π).(14) Definition E.2 (Horizon-induced policy complexity). Let H ≤H = h t : 1≤ t≤ H, Pr τ∼π T (h t ) > 0 (15) denote the set of interaction histories reachable under the teacher policy within the firstHdecisions. Two policies π,π ′ ∈ Π are equivalent at horizon H , written π ≡ H π ′ , if π(·| h) = π ′ (·| h), ∀h∈H ≤H .(16) The corresponding quotient class and horizon-induced policy complexity are Π H = Π/≡ H , C(H) = log|Π H |.(17) 22 For infinite policy classes, the union bound in Lemma E.7 must be replaced by a covering or symmetrization argument, withC(H)replaced by the corresponding complexity of the loss class restricted toH ≤H . We do not pursue this extension here. Lemma E.3 (Policy complexity increases with supervision horizon). For any H 1 < H 2 , C(H 1 )≤C(H 2 ).(18) Proof. Since H ≤H 1 ⊆H ≤H 2 ,(19) any two policies distinguishable within horizonH 1 remain distinguishable within horizonH 2 . Thus, increasing the supervision horizon may split equivalence classes but cannot merge them, implying |Π H 1 |≤|Π H 2 |, C(H 1 )≤C(H 2 ).(20) Lemma E.3 formalizes that longer supervision exposes the learner to a richer set of distinguishable behaviors. This increase in complexity reflects the diversity of observed decision histories, not an increase in the number of model parameters. Definition E.4 (Prefix-optimal set and pessimistic reference risk). Let L ⋆ H := min π∈Π L H (π),Π ⋆ H := arg min π∈Π L H (π), π ⋆ T ∈ arg min π∈Π L T (π),(21) and define the pessimistic full-horizon risk of the prefix-optimal set as R H := sup π∈Π ⋆ H L T (π).(22) Since 0≤ ℓ≤ M , R H ≤ M is finite. Supervision over the firstHdecisions constrains a policy only on histories inH ≤H (Definition E.2). Policies that agree on these histories have identical prefix risk but may behave differently afterward, soΠ ⋆ H generally contains policies with different full-horizon risks.R H accounts for all of them rather than for a single tie-broken minimizer. Definition E.5 (Temporal supervision bias). We define the temporal supervision bias at horizonH (restated from Definition 5.1) as B(H) := R H − L T (π ⋆ T ) = sup π∈Π ⋆ H L T (π)− L T (π ⋆ T ).(23) SinceΠ ⋆ H is non-empty andπ ⋆ T minimizesL T overΠ⊇ Π ⋆ H , we haveB(H)≥ 0. Moreover, every π ∈ Π ⋆ T (the caseH = TofΠ ⋆ H ) is a minimizer ofL T and therefore attainsL T (π) = L T (π ⋆ T ), so R T = L T (π ⋆ T )andB(T ) = 0. Thus,B(H)measures the worst-case excess full-horizon risk left undetermined by supervising only the first H decisions. Assumption E.6 (Monotone temporal supervision bias). For any H 1 < H 2 ≤ T , R H 2 ≤ R H 1 ,equivalently B(H 2 )≤ B(H 1 ).(24) Lemma E.7 (Finite-sample estimation error). Assume that theNteacher trajectories are indepen- dently sampled and that 0≤ ℓ≤ M . Then, with probability at least 1− δ, L H (bπ H )− L ⋆ H ≤ M s 2 C(H) + log 2 δ N .(25) Proof. Define the trajectory-level prefix loss as Z H (π;τ ) = 1 H H X t=1 ℓ(π(·| h t ),a t ),0≤ Z H (π;τ )≤ M.(26) Although decisions within a trajectory are dependent, the variablesZ H (π;τ (1) ),...,Z H (π;τ (N ) ) are independent across teacher trajectories. 23 For a fixed policy π, Hoeffding’s inequality gives Pr L H (π)− b L H (π) > ε ≤ 2 exp − 2Nε 2 M 2 .(27) BothL H (π)and b L H (π)depend onπonly through its behavior onH ≤H , and are therefore constant on each equivalence class of Π H . A union bound over the|Π H | classes thus yields Pr sup π∈Π L H (π)− b L H (π) > ε ≤ 2|Π H | exp − 2Nε 2 M 2 .(28) Setting the right-hand side to δ and usingC(H) = log|Π H | gives sup π∈Π L H (π)− b L H (π) ≤ M r C(H) + log(2/δ) 2N .(29) Fix any π ′ ∈ Π ⋆ H . Since bπ H minimizes the empirical risk, b L H (bπ H )≤ b L H (π ′ ), and hence L H (bπ H )− L ⋆ H = L H (bπ H )− b L H (bπ H ) + b L H (bπ H )− b L H (π ′ ) + b L H (π ′ )− L H (π ′ ) ≤ 2 sup π∈Π L H (π)− b L H (π) . (30) Substituting the uniform bound above proves the result. Training controls performance only over the supervised prefix, whereas our goal is performance over the full task horizon. Note that the full-horizon riskL T , unlikeL H , is not constant on the equivalence classes ofΠ H : prefix-equivalent policies may differ after stepH. The transfer coefficient below is therefore defined relative to the pessimistic referenceR H , which covers every prefix-optimal policy. Definition E.8 (Prefix-to-full-horizon transfer coefficient). For supervision horizonH, define (re- stated from (5)) Γ H :=sup π∈Π: L H (π)>L ⋆ H [L T (π)− R H ] + L H (π)− L ⋆ H ,[x] + := maxx, 0,(31) with Γ H := 0 if no policy has L H (π) > L ⋆ H . The coefficientΓ H measures how much prefix-level excess risk can amplify when evaluated over the full horizon. We assume there exists a constant Γ such that Γ H ≤ Γ, ∀H ≤ T.(32) For a finite policy class, eachΓ H is a maximum over finitely many ratios and is therefore finite, but it can be arbitrarily large, andΓenters the bound multiplicatively throughC 1 . The assumption is therefore substantive as a quantitative condition. Because supervision constrains a policy only on H ≤H , a policy with small prefix excess risk may still behave very differently after stepH, which inflatesΓ H . A moderateΓthus requires the policy class to couple late-stage behavior to early-stage behavior, for example through parameter sharing across histories. DefiningΓ H relative toR H , rather than to a single minimizer, ensures that the transfer inequality also holds for policies that tie on the prefix (Lemma E.9). Lemma E.9 (Prefix-to-full-horizon transfer). For every π ∈ Π, L T (π)− R H ≤ Γ H [L H (π)− L ⋆ H ].(33) Consequently, under Γ H ≤ Γ, with probability at least 1− δ, L T (bπ H )− R H ≤ ΓM s 2 C(H) + log 2 δ N .(34) Proof. Since L H (π)≥ L ⋆ H for every π ∈ Π, two cases are exhaustive. 24 Case 1:L H (π) = L ⋆ H . Thenπ ∈ Π ⋆ H , soL T (π)≤ R H by definition ofR H . The left-hand side of (33) is non-positive and the right-hand side is zero, so the inequality holds. Case 2: L H (π) > L ⋆ H . By definition of Γ H , L T (π)− R H ≤ [L T (π)− R H ] + ≤ Γ H [L H (π)− L ⋆ H ].(35) Applying (33) to π = bπ H , then Γ H ≤ Γ and Lemma E.7, proves the second claim. Lemma E.10 (Error decomposition). For any supervision horizon H , L T (bπ H )− L T (π ⋆ T ) = B(H) +E N (H), E N (H) := L T (bπ H )− R H .(36) Proof. Adding and subtracting R H gives L T (bπ H )− L T (π ⋆ T ) = [L T (bπ H )− R H ] + [R H − L T (π ⋆ T )].(37) The two terms areE N (H)andB(H), respectively. Note thatE N (H)may be negative, e.g., when bπ H is a prefix-optimal policy with better late-stage behavior than the worst one in Π ⋆ H . Theorem E.11 (Detailed Bias–Complexity Bound). Consider the history-conditioned terminal-agent imitation-learning problem above. Assume that: 1. the N teacher trajectories are sampled independently; 2. the imitation loss is bounded, 0≤ ℓ≤ M ; 3. the policy class Π is finite, so that minimizers exist andC(H) = log|Π H | <∞; 4. the transfer coefficient satisfies Γ H ≤ Γ <∞. Then, for every 1≤ H ≤ T , with probability at least 1− δ, L T (bπ H )− L T (π ⋆ T )≤ B(H) + C 1 r C(H) + log(2/δ) N , C 1 = √ 2 ΓM.(38) Proof. By Lemma E.10 and Lemma E.9, L T (bπ H )− L T (π ⋆ T ) = B(H) +E N (H) ≤ B(H) + ΓM r 2 [C(H) + log(2/δ)] N . (39) Substituting C 1 = √ 2 ΓM proves the result. For the remainder of this section, we denote the estimation term of the bound by ε N (H) := C 1 r C(H) + log(2/δ) N ,(40) which upper-bounds the estimation errorE N (H)of Lemma E.10 with probability at least1 − δ. By Lemma E.3,ε N (H)is non-decreasing inH. Note that Theorem E.11 does not require Assumption E.6; monotonicity is used only to interpret the bound as a trade-off. Interpretation. Theorem E.11 identifies two competing effects of increasing the supervision horizon. The first term,B(H), is the temporal supervision bias, the worst-case full-horizon risk left undeter- mined by truncating supervision atH. Under Assumption E.6, it is non-increasing inH, reflecting the additional information provided by later-stage teacher behavior. The second term,ε N (H), is the finite-sample estimation term. By Lemma E.3, it is non-decreasing inH, since longer horizons expose the learner to a richer set of behaviorally distinguishable histories. The bound therefore reflects a bias–complexity trade-off: additional supervision tightens the guarantee when the reduction in temporal supervision bias outweighs the increase in the estimation term. This is consistent with the empirical observation that performance improves with horizon initially but shows diminishing returns beyond an intermediate regime, although the bound alone does not establish that the true error follows this pattern. 25 Definition E.12 (Bound-optimal horizon). The bound-optimal horizon is H ⋆ N ∈ arg min 1≤H≤T B(H) + ε N (H) .(41) It is the horizon at which the guarantee of Theorem E.11 is tightest, and it need not coincide with the empirically best horizon. Corollary E.13 (Full-horizon supervision). At H = T , the temporal supervision bias vanishes and the bound becomes B(T ) = 0, L T (bπ T )− L T (π ⋆ T )≤ C 1 r C(T ) + log(2/δ) N .(42) Moreover, by Lemma E.3, C(T ) = max H≤T C(H).(43) Thus, full-horizon supervision removes temporal supervision bias but incurs the largest horizon- dependent estimation term. Corollary E.14 (When shorter supervision is preferable). For anyH < T, the horizon-Hbound is tighter than the full-horizon bound if and only if B(H) < ε N (T )− ε N (H),(44) whereε N is the estimation term in (40). In particular, sinceB(H)≥ 0, this requiresC(H) <C(T ). Proof. By Theorem E.11 and Corollary E.13, the horizon-H and full-horizon bounds are B(H) + ε N (H), B(T ) + ε N (T ) = ε N (T ),(45) respectively. Hence, horizon H gives a tighter bound if and only if B(H) + ε N (H) < ε N (T ),(46) which is equivalent to the stated condition. The second claim follows becauseε N is strictly increasing inC(H), so ε N (T )− ε N (H) > 0 requiresC(H) <C(T ). To characterize diminishing returns, we impose an additional assumption on the temporal benefit contributed by later stages of a trajectory. Assumption E.15 (Diminishing temporal novelty). There exists a non-increasing functionβ(H) such that B(H)≤ β(H),∆ β (H) := β(H)− β(H + 1),∆ β (H + 1)≤ ∆ β (H).(47) A useful special case is geometric decay,β(H) = Aρ H withA > 0and0 < ρ < 1, for which ∆ β (H) = A(1− ρ)ρ H . Under Assumption E.15, the bound of Theorem E.11 is itself bounded by the surrogate U N (H) := β(H) + ε N (H),∆ ε (H) := ε N (H + 1)− ε N (H)≥ 0.(48) Corollary E.16 (Marginal characterization of the sweet spot). For every H < T , U N (H + 1) < U N (H) ⇐⇒ ∆ β (H) > ∆ ε (H).(49) Thus, extending the supervision horizon tightens the surrogate bound only while the marginal reduction in temporal supervision bias exceeds the marginal increase in the estimation term. Proof. By definition, U N (H + 1)− U N (H) =−∆ β (H) + ∆ ε (H). Assumption E.17 (Single crossing). There existsH 0 ∈1,...,Tsuch that∆ β (H) > ∆ ε (H)for all H < H 0 , and ∆ β (H)≤ ∆ ε (H) for all H 0 ≤ H < T . Corollary E.18 (Intermediate surrogate-optimal horizon). Under Assumption E.17,H 0 ∈ arg min 1≤H≤T U N (H). Moreover, H 0 < T if and only if ∆ β (T − 1)≤ ∆ ε (T − 1). Proof.By Corollary E.16,U N is strictly decreasing on1,...,H 0 and non-decreasing on H 0 ,...,T, so H 0 is a minimizer. The second claim follows from the definition of H 0 . 26 Corollary E.19 (Effect of additional expert data). For a fixed horizonH,ε N (H) = O p C(H)/N → 0asN → ∞. Moreover, letH 0 (N )denote the crossing point of Assump- tion E.17 at sample sizeN. If the assumption holds at bothNandN ′ > N, thenH 0 (N ′ )≥ H 0 (N ). Proof.The first claim is immediate from (40). For the second, writec δ = log(2/δ)and∆ (N ) ε for ∆ ε at sample size N . Then ∆ (N ) ε (H) = C 1 √ N h p C(H + 1) + c δ − p C(H) + c δ i ,(50) where the bracket is non-negative by Lemma E.3 and independent ofN. Hence∆ (N ) ε (H) is non- increasing in N , while ∆ β (H) does not depend on N . It follows that for every H < H 0 (N ), ∆ β (H) > ∆ (N ) ε (H)≥ ∆ (N ′ ) ε (H).(51) Suppose, for contradiction, thatH 0 (N ′ ) < H 0 (N ). Applying Assumption E.17 atN ′ with H = H 0 (N ′ )gives∆ β (H 0 (N ′ )) ≤ ∆ (N ′ ) ε (H 0 (N ′ )) , which contradicts the display above since H 0 (N ′ ) < H 0 (N ). Therefore H 0 (N ′ )≥ H 0 (N ). Thus, more expert data weakens the statistical penalty of longer horizons and moves the surrogate- optimal horizon weakly later. Connection to token-level supervision. The analysis above is expressed in terms of a decision horizonH, while our implementation controls a token budgetK. As defined in Section 3.3,K induces the trajectory-dependent decision horizon H K (τ ) = maxt : n t (τ )≤ K, K 1 < K 2 =⇒ H K 1 (τ )≤ H K 2 (τ ),(52) wheren t (τ )is the cumulative number of tokens through actiona t . Thus, increasingKmonotonically extends the supervised portion of every trajectory. Two differences from the analysis remain. First, H K (τ )varies across trajectories, whereas the analysis uses a commonH. Second, an action that straddles the budget is partially supervised, whichH K (τ )ignores. The analysis should therefore be read as describing the qualitative effect of extending supervision, not as a quantitative model of token-level truncation. 27