Paper deep dive
Cross-Benchmark Transfer from RL on Agentic Coding Tasks
Sushant Mehta, Logan Ritchie, Edwin Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 96%
Last extracted: 10/3/2026, 1:41:38 AM
Summary
This paper investigates whether reinforcement learning (RL) on a small set of expert-built agentic coding tasks enables cross-benchmark transfer. The authors post-train the Kimi K2.7 Code model (1T parameters) using Group Sequence Policy Optimization (GSPO) on 1,700 tasks (1,000 repository-based, 700 terminal-based). The training employs a partial-credit reward gated by pass-to-pass tests to prevent regression. Results show significant improvements in pass@1 across six external benchmarks (SWE-Bench Pro, DeepSWE, Terminal-Bench 2.1/3/4, SWE-Marathon) and under different agent harnesses, indicating the model learned generalizable problem-solving behaviors rather than memorizing specific benchmarks. Trajectory analysis reveals the trained model avoids common failure modes like dropping requirements or breaking existing behavior.
Entities (13)
Relation Signals (15)
Kimi-K2.7-Code → evaluatedon → DeepSWE
confidence 99% · improves pass@1 on each of the six external benchmarks... DeepSWE
Kimi-K2.7-Code → evaluatedon → Terminal-Bench 4
confidence 99% · improves pass@1 on each of the six external benchmarks... Terminal-Bench 4
Kimi-K2.7-Code → evaluatedon → SWE-bench Pro
confidence 99% · improves pass@1 on each of the six external benchmarks... SWE-Bench Pro
Kimi-K2.7-Code → evaluatedon → SWE-Marathon
confidence 99% · improves pass@1 on each of the six external benchmarks... SWE-Marathon
Kimi-K2.7-Code → evaluatedon → Terminal-Bench 2.1
confidence 99% · improves pass@1 on each of the six external benchmarks... Terminal-Bench 2.1
Kimi-K2.7-Code → evaluatedon → Terminal-Bench 3
confidence 99% · improves pass@1 on each of the six external benchmarks... Terminal-Bench 3
Kimi-K2.7-Code → finetunedwith → LoRA
confidence 95% · One epoch of GSPO on a rank-32 LoRA adapter
Kimi-K2.7-Code → trainedwith → GSPO
confidence 95% · We post-train Kimi K2.7 Code... with RL alone... One epoch of GSPO
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was supposed to stay intact, or validate against an unchecked assumption. We ask whether reinforcement learning (RL) on expert-built agentic coding tasks closes this gap, and whether what the agent learns transfers beyond the training distribution. We post-train Kimi K2.7 Code, a 1T-parameter (32B active) open-weight mixture-of-experts model, with RL alone on 1,700 tasks: 1,000 repository tasks graded by hidden fail-to-pass tests and by pass-to-pass tests of existing behavior, and 700 terminal tasks graded by expert-written hidden verifiers. The reward is the fraction of target checks passed and drops to zero if any pass-to-pass test fails. One epoch of GSPO on a rank-32 LoRA adapter improves pass@1 on each of the six external benchmarks we evaluated, across three agent harnesses: SWE-Bench Pro (60.1 to 64.8), DeepSWE (31.0 to 43.4), Terminal-Bench 2.1 (67.4 to 82.0), Terminal-Bench 3 (1.4 to 12.1), Terminal-Bench 4 (0.0 to 7.6), and SWE-Marathon (5.0 to 25.0). Pooled over the five independent task sets (Terminal-Bench 4 revises Terminal-Bench 3), the improvement is significant (p < 0.001), and it remains significant on the three sets released after the training data was collected (p = 0.004); the model also improves under both harnesses never used in training. Median trajectories on DeepSWE and Terminal-Bench 3 are 24-35% shorter in agent steps. The base model's failed DeepSWE runs are mostly near-misses, and on the tasks the trained model newly solves, paired trajectories show it avoiding each of the four failure modes above.
Tags
Links
- Source: https://arxiv.org/abs/2610.00890v1
- Canonical: https://arxiv.org/abs/2610.00890v1
Trouble viewing inline? Open PDF directly →
Full Text
54,528 characters extracted from source content.
Expand or collapse full text
Cross-Benchmark Transfer from RL on Agentic Coding Tasks Sushant Mehta Logan Ritchie Edwin Chen Affiliation: Surge AI Email: research@surgehq.ai Abstract Coding agents often fail in the last mile: they build most of a feature but drop a requirement, test only the cases their implementation already handles, break behavior that was supposed to stay intact, or validate against an unchecked assumption. We ask whether reinforcement learning (RL) on expert-built agentic coding tasks closes this gap, and whether what the agent learns transfers beyond the training distribution. We post-train Kimi K2.7 Code, a 1T-parameter (32B active) open-weight mixture-of-experts model, with RL alone on 1,700 tasks: 1,000 repository tasks graded by hidden fail-to-pass tests and by pass-to-pass tests of existing behavior, and 700 terminal tasks graded by expert-written hidden verifiers. The reward is the fraction of target checks passed and drops to zero if any pass-to-pass test fails. One epoch of GSPO on a rank-32 LoRA adapter improves pass@1 on each of the six external benchmarks we evaluated, across three agent harnesses: SWE-Bench Pro (60.1→ 64.8), DeepSWE (31.0→ 43.4), Terminal-Bench 2.1 (67.4→ 82.0), Terminal-Bench 3 (1.4→ 12.1), Terminal-Bench 4 (0.0→ 7.6), and SWE-Marathon (5.0→ 25.0). Pooled over the five independent task sets (Terminal-Bench 4 revises Terminal-Bench 3), the improvement is significant (p<0.001p<0.001), and it remains significant on the three sets released after the training data was collected (p=0.004p=0.004); the model also improves under both harnesses never used in training. Median trajectories on DeepSWE and Terminal-Bench 3 are 24–35% shorter in agent steps. The base model’s failed DeepSWE runs are mostly near-misses, and on the tasks the trained model newly solves, paired trajectories show it avoiding each of the four failure modes above. 1 Introduction Reinforcement learning with verifiable rewards (RLVR) is now a standard stage in training coding agents [DeepSeek-AI et al., 2025; Wei et al., 2025; Luo et al., 2025; Kimi Team et al., 2025]. The usual evidence that a run worked is a higher score on a benchmark whose format the training data shares. That leaves two questions open. What did the model learn: domain knowledge, or something more general about carrying a task to completion? And does the improvement survive a change of benchmark, agent harness, and horizon? An agent that has absorbed one benchmark’s conventions is worth less than one that has learned to finish realistic work. Figure 1: Pass@1 on six external benchmarks before and after RL on 1,700 expert-built coding tasks, grouped by task format, with SWE-Marathon’s multi-hour builds shown separately. Bold numbers are gains in percentage points. Harnesses are in italics, in bold for the two not used in training. Hatched baselines are publicly reported scores for the same checkpoint and harness (DeepSWE: benchmark leaderboard; Terminal-Bench 2.1: Artificial Analysis); the others are in-house runs. Asterisks mark benchmarks released after the training data was collected; Terminal-Bench 4 revises Terminal-Bench 3. We study both questions with one training run and six external benchmarks. We post-train Kimi K2.7 Code [Moonshot AI, 2026] with RL alone on 1,700 expert-built agentic coding tasks from two families (§3). Each of the 1,000 repository tasks pairs a real repository at a pinned commit with a change request; hidden fail-to-pass tests grade the change, and pass-to-pass tests guard existing behavior. Each of the 700 terminal tasks pairs a working environment with a deliverable graded by expert-written hidden verifiers. The recipe is deliberately simple, so that what the run tests is the tasks and their reward: Group Sequence Policy Optimization (GSPO) [Zheng et al., 2025] on a rank-32 LoRA adapter [Hu et al., 2022], no supervised warm-up, no KL penalty, and a partial-credit reward, the fraction of satisfied checks, that drops to zero whenever a pass-to-pass test regresses (§4). We compare the base and trained models on SWE-Bench Pro, DeepSWE, Terminal-Bench 2.1, 3, and 4, and SWE-Marathon, fixing one harness per benchmark, three harnesses in all (§5). The six cover both training formats, new task sources and domains, and a horizon absent from training: SWE-Marathon’s multi-hour builds. Four were released after the training data was collected. The trained model improves on all six (§6; Figure 1), by 4.7 to 20.0 percentage points (p), including under both harnesses never used in training: Claude Code on SWE-Marathon (1 to 5 of 20 tasks) and Terminus 2 on Terminal-Bench 2.1 (+14.6 p over its public baseline). Because Terminal-Bench 4 revises Terminal-Bench 3, the six benchmarks form five independent task sets; pooled over them, the improvement is significant (p<0.001p<0.001), and it remains significant on the three sets released after the training data was collected (p=0.004p=0.004). The trained model also takes fewer agent steps: the median falls from 150 to 98 on DeepSWE and from 102 to 78 on Terminal-Bench 3. To see what changed, we paired base and trained trajectories on the same tasks (§7). The base model’s DeepSWE failures are mostly near-misses: the median failed run still passes 86% of its target tests, and 84% of failed runs keep every pass-to-pass test. In the pairs we read, all from tasks the trained model newly solves, the base model’s failures recur in four modes that follow a standard engineering loop: lost requirements (understand), narrow testing (test), silent regressions (maintain), and weak ground truth (verify). On the same tasks, the trained model implements the entire specification, derives its tests from the requirement instead of from its own implementation, tracks what must keep working, and builds its own ground truth when none is provided. Each behavior is one the training tasks and reward favor. The evidence points to what transfers: not domain knowledge, which the base model largely had, but a way of finishing work. Contributions. 1. Transfer from a small expert-built RL dataset: one epoch on 1,700 tasks improves pass@1 on all six external benchmarks we evaluated, under three harnesses, including four released after the data was collected and one whose multi-hour horizon has no counterpart in training (§6). 2. A paired trajectory analysis showing that the base model’s DeepSWE failures are mostly last-mile near-misses and that, on tasks the trained model newly solves, the base model’s failures recur in four modes that the trained model avoids (§7). 3. A partial-credit reward gated on regressions, with an account that links each behavior change to a property of the training tasks and reward (§3–4, §7). 2 Related Work RL for software-engineering agents. RLVR was named and established on mathematics and instruction following [Lambert et al., 2025]; DeepSeek-R1 scaled rule-based rewards, optimized with GRPO [Shao et al., 2024], to mathematics and short-form code [DeepSeek-AI et al., 2025], and RL has since moved to long-horizon agentic software engineering. SWE-RL [Wei et al., 2025] learns from software-evolution data with a rule-based patch-similarity reward; SWE-Gym [Pan et al., 2025], SWE-smith [Yang et al., 2025], and R2E-Gym [Jain et al., 2025] build executable training environments from real repositories at scale; DeepSWE-Preview [Luo et al., 2025], an agent unrelated to the DeepSWE benchmark, trains with RL alone in such environments; SkyRL [Cao et al., 2025] provides an RL pipeline for long-horizon agents; and frontier open models such as Kimi K2 [Kimi Team et al., 2025] combine large-scale agentic data synthesis with RL. These efforts mostly mine or synthesize tasks at scale and evaluate on SWE-bench-style benchmarks [Jimenez et al., 2024; Chowdhury et al., 2024] whose format matches the training environments. We train on a small set of expert-built tasks whose specifications enumerate individually gradable requirements, with partial credit gated on regressions, and measure transfer across benchmarks, harnesses, and horizons. Benchmarks for agentic coding. SWE-bench [Jimenez et al., 2024] and its human-validated subset [Chowdhury et al., 2024] made issue resolution on real repositories the standard task; SWE-Bench Pro [Deng et al., 2026] raises difficulty with long-horizon, multi-file, human-verified problems. Terminal-Bench [Merrill et al., 2026] evaluates agents on hard tasks in containerized command-line environments and, through the Harbor framework [Harbor Framework Team, 2026], has become a continuously versioned benchmark (versions 2.0, 2.1, 3, and 4) [Terminal-Bench Team, 2026; Marten et al., 2026; Marten, 2026]. DeepSWE [Huang et al., 2026] asks for original features in existing repositories, and SWE-Marathon [Desai et al., 2026] for complete systems over multi-hour horizons. Scores on all of these depend on the agent harness [Merrill et al., 2026; Desai et al., 2026], which fixes the tools, prompts, and control loop the model operates in. Common harnesses include SWE-agent and mini-swe-agent [Yang et al., 2024; Lieret and Jimenez, 2025], OpenHands [Wang et al., 2025], Terminus 2 [Merrill et al., 2026], and commercial CLIs such as Claude Code [Anthropic, 2025]. Partial-credit and rubric-based rewards. Rewards proportional to the fraction of unit tests passed are established in code RL [Liu et al., 2023]. Rubric-based rewards have been used to train instruction following [He et al., 2026; Mehta et al., 2026a] and open-ended reasoning [Gunjal et al., 2026]; Mehta et al. [2026a] reward the fraction of expert-written criteria satisfied. Proxy rewards can be gamed [Skalse et al., 2022], and partial credit opens a specific trade: satisfying new checks at the expense of existing behavior. Our reward keeps partial credit on the requested behavior and removes that trade with a gate that assigns zero reward whenever a pass-to-pass test fails. Transfer from post-training. Chu et al. [2025] find that RL generalizes to unseen rule and visual variants where supervised fine-tuning memorizes; Huan et al. [2026] find that RL on mathematics transfers to other domains better than supervised fine-tuning on the same problems; Yue et al. [2025] argue that much of RLVR’s pass@1 gain on reasoning tasks comes from concentrating probability on solutions the base model could already sample. Prior work from our group reports transfer across task distributions: post-training on long-horizon multi-tool office work, with no software-engineering tasks, produced gains on five external evaluations, including SWE-Bench Pro from 20.5% to 26.3% [Mehta et al., 2026c; Ritchie et al., 2026]; single-turn constraint-following training produced its largest gains on multi-turn dimensions, led by +22.1 p on MultiChallenge instruction retention [Mehta et al., 2026a; Deshpande et al., 2025]; and training on an enterprise simulation improved out-of-distribution pass rates [Mehta et al., 2026b]. Here training and evaluation are both coding, but the benchmarks, harnesses, and horizons differ. 3 Training Data The training set consists of 1,700 tasks from Surge AI’s expert-built agentic coding collections, in the two formats most common in agentic coding evaluation. Every task pairs a specification the agent sees with hidden checks of the requested behavior. 3.1 Two task families Repository tasks (1,000). Each task is a real repository at a pinned commit and a request for a specific, realistic change. Two groups of tests grade the result: hidden fail-to-pass tests encode the requested behavior, failing on the original commit and passing with the task author’s reference patch; pass-to-pass tests guard existing behavior and pass on both, drawing on the repository’s own suite and, in about half the tasks, on hidden tests written by the task author. The format is that of SWE-bench [Jimenez et al., 2024], but each task is newly written by an expert, and its specification states concrete, individually gradable requirements: edge cases, exact interfaces, output formats, restrictions, and what must remain unchanged. Terminal tasks (700). Each task places the agent in a working containerized environment and asks for a concrete deliverable, in the style of Terminal-Bench [Merrill et al., 2026]. Expert-written hidden verifiers inspect the final state of the environment and score a set of criteria. 3.2 Properties that shape the training signal Four properties of the tasks determine what the reward can teach; §7 returns to each. (P1) Gradable requirements. Each stated requirement maps to one or more checks, so the reward can credit partial progress (§4.1). (P2) Hidden graders. The agent sees the specification but never the fail-to-pass tests or verifiers that grade the requested behavior, so it cannot code against visible cases and has to decide for itself what to check. (P3) Protected behavior. Repository specifications state what must not change, and pass-to-pass tests enforce it; a single regression zeroes the reward. (P4) No oracle. The agent receives no reference solution, so any evidence that its result is correct has to be produced inside the rollout. Hidden graders and the absence of an oracle are common to executable coding environments. What is specific to these tasks is that their specifications enumerate individually gradable requirements, including what must not change, and that the reward gives partial credit per check, gated on regressions. 3.3 Separation from the evaluation benchmarks All training tasks were collected before DeepSWE, Terminal-Bench 3 and 4, and SWE-Marathon were released. The two families mirror the formats of SWE-bench and Terminal-Bench, but neither was selected to improve a benchmark score. We checked every training task against all six benchmarks [Deng et al., 2026; Huang et al., 2026; Terminal-Bench Team, 2026; Marten et al., 2026; Marten, 2026; Desai et al., 2026]: no training task overlaps with or is derived from a benchmark task. 4 Method The recipe is about one RL epoch over the 1,700 tasks (108 rollout steps, 216 optimizer updates), with no supervised stage. The tasks follow a coarse two-stage curriculum, the 1,000 repository tasks first and the 700 terminal tasks after them, ordered by difficulty: the base model’s pass rate was slightly higher on repository tasks. Prompts are shuffled within each stage. 4.1 Reward Each rollout ends with the task’s checks executed against the final environment state. Let Pi∈0,1P_i∈\0,1\ (i≤NPi≤ N_P) be the outcomes of the pass-to-pass tests (existing behavior) and Fj∈[0,1]F_j∈[0,1] (j≤NFj≤ N_F) the outcomes of the fail-to-pass tests or verifier criteria (requested behavior), all weighted equally. The reward is R=[∑i=1NPPi=NP]⋅1NF∑j=1NFFj.R\;=\;1\! [\, _i=1^N_PP_i=N_P ]· 1N_F _j=1^N_FF_j. (1) The second factor gives partial credit: a rollout that satisfies 3 of 12 requested checks scores 0.25. The first is a hard gate: a rollout that satisfies all 12 but fails 1 of 100 pass-to-pass tests scores 0. Terminal tasks have no pass-to-pass tests (NP=0N_P=0), so their reward is the fraction of verifier criteria satisfied. Rollouts truncated by a length limit are excluded from the loss, similar to DAPO’s overlong filtering [Yu et al., 2025]. Why partial credit matters under group-relative advantages. The reward enters the update only through advantages standardized within the G rollouts of a prompt. Under a binary pass/fail reward, a group carries no signal whenever all G rollouts fail or all pass. For a task on which each rollout passes with probability q, that happens with probability qG+(1−q)Gq^G+(1-q)^G, which for G=8G=8 is 43% at q=0.1q=0.1 and 66% at q=0.05q=0.05: the harder the task, the more groups are silent. Under Equation (1), a group in which no rollout passes every check still ranks the rollouts that keep existing behavior by the fraction of target checks each satisfies. The gate places every regressing rollout at zero regardless of how many new requirements it meets, so there is no exchange rate between new and existing behavior for the policy to exploit. Both properties reappear as behaviors in §7. 4.2 Policy optimization We optimize the policy with GSPO [Zheng et al., 2025]. For a prompt x with rollouts y1,…,yGy_1,…,y_G and rewards R1,…,RGR_1,…,R_G, GSPO uses the length-normalized sequence ratio and the group-standardized advantage, si(θ)=(πθ(yi∣x)πθold(yi∣x))1/|yi|,A^i=Ri−meanjRjstdjRj,s_i(θ)= ( _θ(y_i x) _ _old(y_i x) )^1/|y_i|, A_i= R_i-mean_jR_jstd_jR_j, (2) with A^i=0 A_i=0 when all rewards in the group are equal, and maximizes (θ)=[1G∑i=1Gmin(si(θ)A^i,clip(si(θ),1−εlow,1+εhigh)A^i)].J(θ)=E [ 1G _i=1^G \! (s_i(θ)\, A_i,\ clip (s_i(θ),1- _low,1+ _high )\, A_i ) ]. (3) In a multi-turn rollout, yiy_i comprises the tokens the model generates; tool outputs are masked out of the likelihoods and the loss. In mixture-of-experts models, the token-level ratios of GRPO [Shao et al., 2024] fluctuate with expert routing, which motivates GSPO’s sequence-level ratio [Zheng et al., 2025]. The numerical mismatch between the rollout engine and the trainer is corrected with truncated importance sampling [Yao et al., 2025]. There is no KL penalty and no entropy bonus. Each rollout step samples 16 prompts with G=8G=8 rollouts each, and the 128 trajectories are consumed in two optimizer updates of 64. Groups that contain a rollout aborted by the infrastructure (for example, by a container crash) are discarded and replaced by groups sampled from additional prompts; groups whose rollouts all receive the same reward are kept and contribute no gradient. Low-rank adaptation. Instead of updating all parameters of the mixture-of-experts model, we train a rank-32 LoRA adapter [Hu et al., 2022] with α=32α=32 on the attention and MLP projections, using Adam with decoupled weight decay [Kingma and Ba, 2015; Loshchilov and Hutter, 2019]. Schulman and Thinking Machines Lab [2025] find that LoRA applied to all weight matrices, the MLP layers included, matches full fine-tuning for RL even at small rank, because an RL episode carries little information. 4.3 Rollouts and infrastructure Rollouts run in a minimal bash-only agent loop in the style of mini-swe-agent [Lieret and Jimenez, 2025], inside an isolated sandbox that provides the task’s repository or environment and runs the task’s checks when the rollout ends. The model runs in thinking mode at maximum reasoning effort with nucleus sampling (top-p 0.95); each model turn is capped at 65,536 generated tokens, and a trajectory can use the model’s full 262,144-token context window. Training runs on 64 NVIDIA H200 GPUs, with rollout generation and policy updates co-located. The stack is built from open-source components: Miles [Miles Team, 2026], an RL framework derived from slime [Zhu et al., 2025], orchestrates SGLang [Zheng et al., 2024] for generation and Megatron-LM with Megatron-Bridge [Shoeybi et al., 2019; NVIDIA, 2025] and FlashAttention [Dao et al., 2022] for training, combining tensor, pipeline, context, and expert parallelism to fit the 1T-parameter model. All results use the final checkpoint, fixed in advance; no hyperparameter was tuned on any benchmark. 5 Evaluation Protocol Table 1: The six external benchmarks and how each departs from the training data. Training used repository and terminal tasks in a mini-swe-agent-style loop, with nothing at the horizon and scope of SWE-Marathon’s builds. ∗Released after the training data was collected. Benchmark Tasks What it asks for Harness Departure from training SWE-Bench Pro 731 Issue resolution in 11 repositories (public split); human-verified problems that require multi-file changes mini-swe-agent Same format DeepSWE v1.1∗ 113 Original features in 91 repositories and five languages; reference solutions average 668 added lines mini-swe-agent Larger features Terminal-Bench 2.1 89 Hard command-line tasks in software engineering, system administration, scientific computing, security, and data science Terminus 2 Same format; unseen harness Terminal-Bench 3∗ 74 Harder, longer tasks in seven domains (software, science, ML, operations, security, hardware, media); open internet mini-swe-agent Harder tasks; open internet Terminal-Bench 4∗ 66 Terminal-Bench 3 with 8 tasks removed and 19 fixed; 8-hour agent limit mini-swe-agent As Terminal-Bench 3 SWE-Marathon v1.1∗ 20 Complete systems (library reproductions, product clones, ML engineering, algorithmic optimization); agent limits up to 10 hours; expert estimates up to 400 hours Claude Code Multi-hour horizon; unseen harness Benchmarks. Table 1 lists the six benchmarks and their harnesses [Deng et al., 2026; Huang et al., 2026; Terminal-Bench Team, 2026; Marten et al., 2026; Marten, 2026; Desai et al., 2026]. Terminal-Bench 4 revises Terminal-Bench 3 (8 tasks removed, 19 fixed), so we treat the two as one family. These six are the only benchmarks we evaluated. Harnesses. Scores vary substantially across harnesses [Merrill et al., 2026; Desai et al., 2026], so we fix one per benchmark and use it unchanged for both models: mini-swe-agent [Lieret and Jimenez, 2025] for SWE-Bench Pro, DeepSWE (the leaderboard harness for all models), and Terminal-Bench 3 and 4 (the harness Artificial Analysis [2026b] uses for Terminal-Bench 4); Terminus 2 [Merrill et al., 2026] for Terminal-Bench 2.1, the harness of its public baseline; and, for SWE-Marathon, Claude Code [Anthropic, 2025], one of the harnesses the benchmark authors evaluated, connected to the model through an API-compatible endpoint. Terminus 2 and Claude Code differ from the training loop in tool interface, prompts, and control flow. Metric and baselines. We report pass@1 from a single run per benchmark with the final checkpoint, in thinking mode at maximum reasoning effort. In-house runs of both models use the same inference stack and sampling settings; only the LoRA adapter differs. Two baselines are publicly reported scores for the same checkpoint and harness: Terminal-Bench 2.1 (67.4%) from Artificial Analysis [2026a], which averages three runs under Terminus 2 (Vals AI [2026] independently reports 67.0%), and DeepSWE from the benchmark leaderboard [Huang et al., 2026]. Our in-house DeepSWE base run under the same harness scored 26.5% (30 of 113), below the public 31.0%, so the headline DeepSWE gain is conservative; the trajectory analysis in §7 uses this in-house run. All other baselines are in-house runs, which makes Terminal-Bench 2.1 the only benchmark without an in-house base run. Scoring rules and statistics are in Appendix A. 6 Results Table 2: Pass@1 (%) of Kimi K2.7 Code before and after RL on 1,700 expert-built coding tasks. Gains are differences of the rounded scores. Benchmark Harness Tasks Base +RL Gain (p) SWE-Bench Pro mini-swe-agent 731 60.1 64.8 +4.7 DeepSWE mini-swe-agent 113 31.0† 43.4 +12.4 Terminal-Bench 2.1 Terminus 2 89 67.4† 82.0 +14.6 Terminal-Bench 3 mini-swe-agent 74 1.4 12.1‡ +10.7 Terminal-Bench 4 mini-swe-agent 66 0.0 7.6 +7.6 SWE-Marathon Claude Code 20 5.0 25.0 +20.0 † Publicly reported for the same checkpoint and harness (§5); our in-house DeepSWE base run scored 26.5. ‡ Of the trained model’s 74 runs, 8 failed for harness reasons and are not scored; 8 of the remaining 66 pass. 6.1 The trained model improves on all six benchmarks The gains in Table 2 and Figure 1 correspond to 35 more solved tasks on SWE-Bench Pro, 14 on DeepSWE (19 against our in-house base run), 13 on Terminal-Bench 2.1, 7 on Terminal-Bench 3, 5 on Terminal-Bench 4, and 4 on SWE-Marathon. Without correction for multiple comparisons, the gains on Terminal-Bench 2.1, 3, and 4, and on DeepSWE against our in-house base run, are individually significant at the 0.05 level (Boschloo’s exact test [Boschloo, 1970]). A Cochran–Mantel–Haenszel test [Mantel and Haenszel, 1959] over the five independent task sets, with the Terminal-Bench 3/4 family counted once through Terminal-Bench 4, rejects the null of equal pass rates at p<0.001p<0.001 (Appendix A.2). 6.2 Where the gains appear The gains are not confined to the benchmarks closest to training (Table 1), SWE-Bench Pro (+4.7 p) and Terminal-Bench 2.1 (+14.6 p). The four released after the training data was collected improve by 7.6 to 20.0 p, and pooled over their three independent task sets the improvement is significant on its own (p=0.004p=0.004). Both benchmarks evaluated under harnesses absent from training improve by more than 14 p: Terminal-Bench 2.1 under Terminus 2, and SWE-Marathon, whose multi-hour builds have no counterpart in training, under Claude Code. Only these two test transfer across harnesses: the other four run under mini-swe-agent, whose bash-only loop the training loop follows. The gains also span the difficulty range. On Terminal-Bench 2.1, where the base model passes two-thirds of the tasks, RL closes 45% of the remaining gap; on Terminal-Bench 3 and 4, where the base model passes 1 of 74 and 0 of 66 tasks, the trained model passes 8 of 66 and 5 of 66. 6.3 Fewer agent steps On the two benchmarks where we counted agent steps, the trained model reaches these scores in fewer steps (Figure 2b). Over all 113 DeepSWE tasks, the median trajectory falls from 150 agent steps for the in-house base run to 98 (−-35%), and over Terminal-Bench 3 from 102 to 78 (−-24%). Figure 2: (a) Failed base runs on DeepSWE are mostly near-misses: of the 83 failed in-house base runs, 84% kept every pass-to-pass test and 59% passed at least 80% of their target tests; the median failed run passed 86%. (b) Median agent steps per task over all tasks, passed or failed; the DeepSWE base is the in-house run. 7 What Changed: A Paired Trajectory Analysis We paired base and trained trajectories on the same tasks and read the models’ reasoning, shell commands, tests, and outputs, concentrating on tasks from DeepSWE, Terminal-Bench 3, and SWE-Marathon that the base model failed and the trained model passed. The cases below therefore come from tasks the trained model newly solves; the near-miss statistics in §7.1 cover every failed in-house DeepSWE base run. 7.1 The base model’s DeepSWE failures are mostly near-misses Of the 83 failed in-house DeepSWE base runs, 59% were near-misses that passed at least 80% of their target tests (the tests of the requested behavior); the median failed run passed 86%, and 84% of failed runs preserved every pass-to-pass test (Figure 2a). In the pairs we read, the typical failure had most of the feature built and one or two requirements missing. A test fraction is not a requirement fraction: in the first case below, one dropped requirement fails 16 of 137 tests. 7.2 Four failure modes and the behaviors after RL The base model’s misses recur in four modes, and in each the trained model’s trajectory on the same task shows a different behavior (Table 3). The four follow a standard engineering loop (understand the requirement, test it, maintain what already works, verify against sound assumptions) and line up with the four task properties of §3.2. Table 3: Failure modes of the base model in paired trajectories, the trained model’s behavior on the same tasks, and the task property (§3.2) that favors the change. Stage Failure mode before RL Behavior after RL Property Understand Lost requirements. Built most of the feature but dropped a required constraint, often a subtle one. Implements the entire specification. Carries every stated requirement through implementation and final review. (P1) Gradable requirements Test Narrow testing. Tested the cases its own incomplete implementation was built around. Tests the requirement, not the implementation. Derives positive, negative, and alternate-form cases from the specification. (P2) Hidden graders Maintain Silent regressions. Satisfied the new requirement while breaking behavior that had to stay intact. Protects existing behavior. Tracks what must change and what must keep working before editing. (P3) Protected behavior Verify Weak ground truth. Validated against a false assumption or a weak proxy. Reconstructs ground truth. Builds an independent reference, an emulator, or its own test inputs. (P4) No oracle 7.2.1 Lost requirements → implementing the entire specification Before. A FastAPI task listed 20 requirements. Requirement 18 asked for a tracking middleware that exposes get_stats() and reset_stats() methods. The base model built nearly the entire feature, with the right architecture and working code, but defined the two functions at module level instead of as methods on the middleware object. It passed 121 of 137 target tests; the remaining 16 called the methods where the specification said they would be, and all 16 failed. After. The trained model made them instance methods and passed all 137 tests. The base model’s understanding of the feature was not wrong: one interface requirement was lost between reading the specification and finishing the implementation, and the trained model carried it through. Link to training (P1). Every requirement maps to checks, so a rollout that drops one loses that requirement’s share of the reward (on a task graded like this one, 16 of 137 checks). Partial credit keeps that pressure on when no rollout in a group finishes: of two rollouts that keep existing behavior, the one that passes more checks receives the higher advantage, where a pass/fail reward would give the whole group no signal (§4.1). 7.2.2 Narrow testing → testing the requirement, not the implementation Before. Both models test often; they differ in what they choose to test. One task required default parameter values to work for both anonymous and named function definitions. The base model chose an implementation that handled the anonymous form, func(a = 1), and every test it wrote used that form. The specification also required the named form, func f(a = 1); the implementation did not handle an identifier between func and the parenthesis, and both hidden test suites failed. The base model had designed its tests around the path its implementation already supported. After. The trained model began with the named example from the specification itself, then tried anonymous functions, multi-line parameter lists, spread calls, immediately invoked functions, and other variants. It looked for cases that could prove its reading of the requirement wrong, not for confirmation that the cases it already supported worked. Link to training (P2). With the grader hidden, a rollout’s own tests of the stated requirements are its best evidence about the checks that determine its reward. Rollouts that test what was asked, not only what was built, catch more of their own errors and score higher within their group, and the advantage credits the testing along with the fix. 7.2.3 Silent regressions → protecting existing behavior Before. A task required eliminating layout shift across a site’s pages while preserving the visible elements, styling, and analytics side effects. The base model reached zero layout shift on every page, partly by deleting two components. Its own final review judged the deletion harmless because some of the components’ styling had been inlined elsewhere. After. Before editing, the trained model noted what the specification required to remain, then preserved those components while changing how their visual effects were applied. It worked from two lists: what had to change and what was not allowed to. Link to training (P3). On repository tasks, a rollout that fails any pass-to-pass test earns zero however many new requirements it meets, so it ties for last in its group and receives a negative advantage whenever another rollout earns credit. 7.2.4 Weak ground truth → reconstructing ground truth Before. Some tasks offer no reference implementation or check to validate against. There the base model would sometimes adopt an assumption and then build a test that inherited it. On a two-dimensional spectral-solver task whose reference functions were unavailable inside the container, the base model decided the problem was effectively one-dimensional, built a one-dimensional solver, and validated it against reference data constructed under the same assumption. The test had no way to expose the assumption. After. The trained model used the method of manufactured solutions: it chose an analytic two-dimensional solution, derived the input for which that solution holds exactly, and used the pair as an independent check, which exposed an instability before submission. On a Verilog console task, it tested the hardware implementation differentially, against a Python emulator it wrote, instruction by instruction. On a SWE-Marathon task that required a Zstandard decompressor in an environment with no reference zstd binary, it first wrote a compressor, so that it could generate test inputs where none existed and check, round trip, that its decompressor recovered the original. Link to training (P4). The training set has no tasks on partial differential equation solvers, hardware emulators, or compressors, but every task puts the model in the same position: a precise specification, a hidden grader, and no oracle. Rollouts that produce their own evidence that a result is correct earn reward more consistently, which is the behavior these trajectories show. The SWE-Marathon authors report that frontier-agent failures often arise from poor self-verification [Desai et al., 2026]. 7.3 A common thread In all four modes the base model stopped short of establishing that the whole job was done: it declared completion without checking every requirement, tested what it had built instead of what was asked, changed what it should have preserved, or verified against its own assumption. The trained model treats the specification as a contract to be met in full and its own work as a hypothesis to be tested. It adds this verification and still takes fewer agent steps on DeepSWE and Terminal-Bench 3 (§6.3). These are engineering habits, not the conventions of any one benchmark, which is consistent with gains that hold across benchmarks and harnesses. 8 Conclusion One epoch of RL on 1,700 expert-built coding tasks, with GSPO and a rank-32 LoRA adapter, improved Kimi K2.7 Code on all six external benchmarks we evaluated, by 4.7 to 20.0 p, including on four benchmarks released after the data was collected, under two harnesses absent from training, and on multi-hour builds with no counterpart in training (1 to 5 of 20 tasks). On the newly solved tasks we read, the trained model implements the entire specification, tests the requirement, protects existing behavior, and builds its own ground truth, each a behavior the training tasks and reward favor. On this evidence, what transferred was not domain knowledge, which the base model largely had, but a way of finishing work. References Anthropic [2025] Anthropic. Claude Code, 2025. URL https://github.com/anthropics/claude-code. Agentic coding tool; first released February 24, 2025. Documentation: https://code.claude.com/docs/en/overview. Accessed September 2026. Artificial Analysis [2026a] Artificial Analysis. Terminal-Bench 2.1 benchmark leaderboard, 2026a. URL https://artificialanalysis.ai/evaluations/terminalbench-2-1. Independent evaluation using the Terminus 2 harness; pass@1 averaged over three runs per task. Accessed September 2026. Artificial Analysis [2026b] Artificial Analysis. Terminal-Bench 4.0 benchmark leaderboard, 2026b. URL https://artificialanalysis.ai/evaluations/terminalbench-4-0. Independent evaluation of all 66 tasks with the mini-swe-agent harness; pass@1 averaged over three runs per task. Accessed September 2026. Boschloo [1970] R. D. Boschloo. Raised conditional level of significance for the 2×22× 2-table when testing the equality of two probabilities. Statistica Neerlandica, 24(1):1–9, 1970. Cao et al. [2025] Shiyi Cao, Sumanth Hegde, Dacheng Li, Tyler Griggs, Shu Liu, Eric Tang, Jiayi Pan, Xingyao Wang, Akshay Malik, Graham Neubig, et al. SkyRL-v0: Train real-world long-horizon agents via reinforcement learning, 2025. URL https://novasky-ai.notion.site/skyrl-v0. NovaSky (UC Berkeley Sky Computing Lab) blog post, May 7, 2025. Code: https://github.com/NovaSky-AI/SkyRL. Accessed September 2026. Chowdhury et al. [2024] Neil Chowdhury, James Aung, Chan Jun Shern, Oliver Jaffe, Dane Sherburn, Giulio Starace, Evan Mays, Rachel Dias, Marwan Aljubeh, Mia Glaese, et al. Introducing SWE-bench Verified, 2024. URL https://openai.com/index/introducing-swe-bench-verified/. OpenAI blog post, August 13, 2024. Accessed September 2026. Chu et al. [2025] Tianzhe Chu, Yuexiang Zhai, Jihan Yang, Shengbang Tong, Saining Xie, Dale Schuurmans, Quoc V. Le, Sergey Levine, and Yi Ma. SFT memorizes, RL generalizes: A comparative study of foundation model post-training. In International Conference on Machine Learning (ICML), 2025. URL https://arxiv.org/abs/2501.17161. Dao et al. [2022] Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with IO-awareness. In Advances in Neural Information Processing Systems (NeurIPS), 2022. URL https://arxiv.org/abs/2205.14135. DeepSeek-AI et al. [2025] DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Peiyi Wang, Qihao Zhu, Runxin Xu, Ruoyu Zhang, Shirong Ma, et al. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. URL https://arxiv.org/abs/2501.12948. Also published in Nature, 645(8081):633–638, 2025. Deng et al. [2026] Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Chetan Rane, Karmini Sampath, et al. SWE-Bench Pro: Can AI agents solve long-horizon software engineering tasks? In International Conference on Machine Learning (ICML), 2026. URL https://arxiv.org/abs/2509.16941. Public leaderboard: https://labs.scale.com/leaderboard/swe_bench_pro. Desai et al. [2026] Rishi Desai, Jesse Hu, Joan Cabezas, Neel Harsola, Pratyush Shukla, Roey Ben Chaim, Adnan El Assadi, Omkaar Mukund Kamath, Fenil Faldu, Prannay Hebbar, et al. SWE-Marathon: Can agents autonomously complete ultra-long-horizon software work? arXiv preprint arXiv:2606.07682, 2026. URL https://arxiv.org/abs/2606.07682. Benchmark website (v1.1, July 2026): https://w.swe-marathon.org/. Deshpande et al. [2025] Kaustubh Deshpande, Ved Sirdeshmukh, Johannes Baptist Mols, Lifeng Jin, Ed-Yeremai Hernandez-Cardona, Dean Lee, Jeremy Kritz, Willow E. Primack, Summer Yue, and Chen Xing. MultiChallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier LLMs. In Findings of the Association for Computational Linguistics: ACL 2025, pages 18632–18702, 2025. URL https://aclanthology.org/2025.findings-acl.958/. Greenland and Robins [1985] Sander Greenland and James M. Robins. Estimation of a common effect parameter from sparse follow-up data. Biometrics, 41(1):55–68, 1985. Gunjal et al. [2026] Anisha Gunjal, Anthony Wang, Elaine Lau, Vaskar Nath, Yunzhong He, Bing Liu, and Sean Hendryx. Rubrics as rewards: Reinforcement learning beyond verifiable domains. In International Conference on Learning Representations (ICLR), 2026. URL https://arxiv.org/abs/2507.17746. Harbor Framework Team [2026] Harbor Framework Team. Harbor: A framework for evaluating and optimizing agents and models in container environments, 2026. URL https://github.com/harbor-framework/harbor. First released November 7, 2025. Concept DOI 10.5281/zenodo.20953922. Documentation: https://w.harborframework.com. Accessed September 2026. He et al. [2026] Yun He, Wenzhe Li, Hejia Zhang, Songlin Li, Karishma Mandyam, Sopan Khosla, Yuanhao Xiong, Nanshu Wang, Xiaoliang Peng, Beibin Li, et al. AdvancedIF: Rubric-based benchmarking and reinforcement learning for advancing LLM instruction following. In Annual Meeting of the Association for Computational Linguistics (ACL), pages 18003–18022, 2026. URL https://aclanthology.org/2026.acl-long.820/. Hu et al. [2022] Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022. URL https://arxiv.org/abs/2106.09685. Huan et al. [2026] Maggie Ziyu Huan, Yuetai Li, Tuney Zheng, Xiaoyu Xu, Seungone Kim, Minxin Du, Radha Poovendran, Graham Neubig, and Xiang Yue. Does math reasoning improve general LLM capabilities? Understanding transferability of LLM reasoning. In International Conference on Machine Learning (ICML), 2026. URL https://arxiv.org/abs/2507.00432. Huang et al. [2026] Wenqi Huang, Charley Lee, Leonard Tng, and Serena Ge. DeepSWE: Measuring frontier coding agents on original, long-horizon engineering tasks, 2026. URL https://deepswe.datacurve.ai/. Datacurve, May 26, 2026; v1.1 released June 2026. Also arXiv:2607.07946. Code: https://github.com/datacurve-ai/deep-swe. Accessed September 2026. Jain et al. [2025] Naman Jain, Jaskirat Singh, Manish Shetty, Tianjun Zhang, Liang Zheng, Koushik Sen, and Ion Stoica. R2E-Gym: Procedural environment generation and hybrid verifiers for scaling open-weights SWE agents. In Conference on Language Modeling (COLM), 2025. URL https://arxiv.org/abs/2504.07164. Jimenez et al. [2024] Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2310.06770. Kimi Team et al. [2025] Kimi Team, Yifan Bai, Yiping Bao, Y. Charles, Cheng Chen, Guanduo Chen, Haiting Chen, Huarong Chen, Jiahao Chen, Ningxin Chen, et al. Kimi K2: Open agentic intelligence. arXiv preprint arXiv:2507.20534, 2025. URL https://arxiv.org/abs/2507.20534. Kingma and Ba [2015] Diederik P. Kingma and Jimmy Ba. Adam: A method for stochastic optimization. In International Conference on Learning Representations (ICLR), 2015. URL https://arxiv.org/abs/1412.6980. Lambert et al. [2025] Nathan Lambert, Jacob Morrison, Valentina Pyatkin, Shengyi Huang, Hamish Ivison, Faeze Brahman, Lester James V. Miranda, Alisa Liu, Nouha Dziri, Xinxi Lyu, et al. Tülu 3: Pushing frontiers in open language model post-training. In Conference on Language Modeling (COLM), 2025. URL https://openreview.net/forum?id=i1uGbfHHpH. Lieret and Jimenez [2025] Kilian Lieret and Carlos E. Jimenez. mini-SWE-agent: The minimal AI software engineering agent, 2025. URL https://github.com/SWE-agent/mini-swe-agent. GitHub repository; first PyPI release July 2025. Accessed September 2026. Liu et al. [2023] Jiate Liu, Yiqin Zhu, Kaiwen Xiao, Qiang Fu, Xiao Han, Wei Yang, and Deheng Ye. RLTF: Reinforcement learning from unit test feedback. Transactions on Machine Learning Research (TMLR), 2023. URL https://arxiv.org/abs/2307.04349. Loshchilov and Hutter [2019] Ilya Loshchilov and Frank Hutter. Decoupled weight decay regularization. In International Conference on Learning Representations (ICLR), 2019. URL https://arxiv.org/abs/1711.05101. Luo et al. [2025] Michael Luo, Naman Jain, Jaskirat Singh, Sijun Tan, Ameen Patel, Qingyang Wu, Alpay Ariyak, Colin Cai, Tarun Venkat, Shang Zhu, et al. DeepSWE: Training a fully open-sourced, state-of-the-art coding agent by scaling RL, 2025. URL https://w.together.ai/blog/deepswe. Agentica and Together AI blog post, July 2, 2025. Accessed September 2026. Mantel and Haenszel [1959] Nathan Mantel and William Haenszel. Statistical aspects of the analysis of data from retrospective studies of disease. Journal of the National Cancer Institute, 22(4):719–748, 1959. Marten [2026] Ryan Marten. Terminal-Bench 4.0, 2026. URL https://w.tbench.ai/news/terminal-bench-4-0. Blog post, August 28, 2026. Removes 8 tasks from Terminal-Bench 3.0 and fixes 19. Accessed September 2026. Marten et al. [2026] Ryan Marten, Alex Shaw, and Andy Konwinski. Terminal-Bench 3.0, 2026. URL https://w.tbench.ai/news/terminal-bench-3-0. Blog post, July 30, 2026. Dataset: https://github.com/harbor-framework/terminal-bench. Accessed September 2026. Mehta et al. [2026a] Sushant Mehta, Liudas Panavas, Suhaas Garre, and Edwin Chen. ComplexConstraints and beyond: Expert rubrics for RLVR. arXiv preprint arXiv:2606.09118, 2026a. URL https://arxiv.org/abs/2606.09118. Also in the GEM Workshop at ACL 2026. Mehta et al. [2026b] Sushant Mehta, Logan Ritchie, Suhaas Garre, Ian Niebres, Nick Heiner, and Edwin Chen. EnterpriseBench Corecraft: Training generalizable agents on high-fidelity RL environments. arXiv preprint arXiv:2602.16179, 2026b. URL https://arxiv.org/abs/2602.16179. Mehta et al. [2026c] Sushant Mehta, Logan Ritchie, Liudas Panavas, and Edwin Chen. Cross-benchmark generalization in long-horizon agents. arXiv preprint arXiv:2608.00181, 2026c. URL https://arxiv.org/abs/2608.00181. To appear in the Workshop on Agent Behavior at COLM 2026. Merrill et al. [2026] Mike A. Merrill, Alexander G. Shaw, Nicholas Carlini, Boxuan Li, Harsh Raj, Ivan Bercovich, Lin Shi, Jeong Yeon Shin, Thomas Walshe, E. Kelly Buchanan, et al. Terminal-Bench: Benchmarking agents on hard, realistic tasks in command line interfaces. In International Conference on Learning Representations (ICLR), 2026. URL https://arxiv.org/abs/2601.11868. Miles Team [2026] Miles Team. Miles: Enterprise-grade reinforcement learning for large-scale model post-training, 2026. URL https://github.com/radixark/miles. Forked from slime; announced November 19, 2025 (https://w.lmsys.org/blog/2025-11-19-miles/). Accessed September 2026. Moonshot AI [2026] Moonshot AI. Kimi K2.7 Code, 2026. URL https://huggingface.co/moonshotai/Kimi-K2.7-Code. Hugging Face model card. Accessed September 2026. Newcombe [1998] Robert G. Newcombe. Interval estimation for the difference between independent proportions: Comparison of eleven methods. Statistics in Medicine, 17(8):873–890, 1998. NVIDIA [2025] NVIDIA. NeMo Megatron Bridge, 2025. URL https://github.com/NVIDIA-NeMo/Megatron-Bridge. GitHub repository; first PyPI release August 2025. Accessed September 2026. Pan et al. [2025] Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with SWE-Gym. In International Conference on Machine Learning (ICML), 2025. URL https://arxiv.org/abs/2412.21139. Ritchie et al. [2026] Logan Ritchie, Sushant Mehta, Liudas Panavas, and Edwin Chen. Post-training on office work improves software engineering: A behavioral account of cross-domain transfer. arXiv preprint arXiv:2608.01604, 2026. URL https://arxiv.org/abs/2608.01604. Schulman and Thinking Machines Lab [2025] John Schulman and Thinking Machines Lab. LoRA without regret, 2025. URL https://thinkingmachines.ai/blog/lora/. Thinking Machines Lab: Connectionism, September 29, 2025. DOI 10.64434/tml.20250929. Accessed September 2026. Shao et al. [2024] Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. URL https://arxiv.org/abs/2402.03300. Shoeybi et al. [2019] Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-LM: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019. URL https://arxiv.org/abs/1909.08053. Skalse et al. [2022] Joar Skalse, Nikolaus Howe, Dmitrii Krasheninnikov, and David Krueger. Defining and characterizing reward gaming. In Advances in Neural Information Processing Systems (NeurIPS), 2022. URL https://arxiv.org/abs/2209.13085. Terminal-Bench Team [2026] Terminal-Bench Team. Terminal-Bench 2.1: A revision of Terminal-Bench 2.0 that fixes 28 tasks, 2026. URL https://w.tbench.ai/news/terminal-bench-2-1. Blog post, May 6, 2026. Accessed September 2026. Vals AI [2026] Vals AI. Terminal-Bench 2.1 benchmark, 2026. URL https://w.vals.ai/benchmarks/terminal-bench-2-1. Terminus 2 harness; pass@1. Kimi K2.7 Code: https://w.vals.ai/models/kimi_kimi-k2.7-code. Accessed September 2026. Wang et al. [2025] Xingyao Wang, Boxuan Li, Yufan Song, Frank F. Xu, Xiangru Tang, Mingchen Zhuge, Jiayi Pan, Yueqi Song, Bowen Li, Jaskirat Singh, et al. OpenHands: An open platform for AI software developers as generalist agents. In International Conference on Learning Representations (ICLR), 2025. URL https://arxiv.org/abs/2407.16741. Wei et al. [2025] Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I. Wang. SWE-RL: Advancing LLM reasoning via reinforcement learning on open software evolution. In Advances in Neural Information Processing Systems (NeurIPS), 2025. URL https://arxiv.org/abs/2502.18449. Yang et al. [2024] John Yang, Carlos E. Jimenez, Alexander Wettig, Kilian Lieret, Shunyu Yao, Karthik Narasimhan, and Ofir Press. SWE-agent: Agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL https://arxiv.org/abs/2405.15793. Yang et al. [2025] John Yang, Kilian Lieret, Carlos E. Jimenez, Alexander Wettig, Kabir Khandpur, Yanzhe Zhang, Binyuan Hui, Ofir Press, Ludwig Schmidt, and Diyi Yang. SWE-smith: Scaling data for software engineering agents. In Advances in Neural Information Processing Systems (NeurIPS), Datasets and Benchmarks Track, 2025. URL https://arxiv.org/abs/2504.21798. Yao et al. [2025] Feng Yao, Liyuan Liu, Dinghuai Zhang, Chengyu Dong, Jingbo Shang, and Jianfeng Gao. Your efficient RL framework secretly brings you off-policy RL training, 2025. URL https://fengyao.notion.site/off-policy-rl. Blog post, August 2025. Accessed September 2026. Yu et al. [2025] Qiying Yu, Zheng Zhang, Ruofei Zhu, Yufeng Yuan, Xiaochen Zuo, Yu Yue, Weinan Dai, Tiantian Fan, Gaohong Liu, Juncai Liu, et al. DAPO: An open-source LLM reinforcement learning system at scale. In Advances in Neural Information Processing Systems (NeurIPS), 2025. URL https://arxiv.org/abs/2503.14476. Yue et al. [2025] Yang Yue, Zhiqi Chen, Rui Lu, Andrew Zhao, Zhaokai Wang, Yang Yue, Shiji Song, and Gao Huang. Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model? In Advances in Neural Information Processing Systems (NeurIPS), 2025. URL https://arxiv.org/abs/2504.13837. Zheng et al. [2025] Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy optimization. arXiv preprint arXiv:2507.18071, 2025. URL https://arxiv.org/abs/2507.18071. Zheng et al. [2024] Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. SGLang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URL https://arxiv.org/abs/2312.07104. Zhu et al. [2025] Zilin Zhu, Chengxing Xie, Xin Lv, and slime Contributors. slime: An LLM post-training framework for RL scaling, 2025. URL https://github.com/THUDM/slime. GitHub repository. Accessed September 2026. Appendix A Evaluation Details and Statistics A.1 Scoring All in-house results are single runs. On Terminal-Bench 3, runs that fail for harness reasons are not scored; for the trained model this excludes 8 of the 74 runs, and 8 of the remaining 66 pass. Counting those 8 runs as failures instead gives 10.8% (8 of 74), a gain of 9.4 p (p=0.016p=0.016). The base model passes 1 task; any base denominator from 69 to 74 gives 1.4% and p≤0.013p≤ 0.013 (p≤0.021p≤ 0.021 with the 8 runs counted as failures). On SWE-Marathon, one task requires an A100 GPU that was not available to us and is scored as failed for both models. A.2 Statistics Table 4 gives pass counts, 95% confidence intervals for each gain (hybrid score interval for a difference of independent proportions; Newcombe, 1998), and two-sided p-values from Boschloo’s exact test [Boschloo, 1970]. Public baselines enter as one run with the nearest count (35/113 for DeepSWE; 60/89 for Terminal-Bench 2.1, a mean of three runs), which is conservative for a multi-run mean. The pooled tests are Cochran–Mantel–Haenszel tests [Mantel and Haenszel, 1959] without continuity correction, stratified by benchmark; they use the public DeepSWE baseline and count the Terminal-Bench 3/4 family once, through Terminal-Bench 4, the more conservative choice (Terminal-Bench 3 instead gives p=5×10−4p=5× 10^-4). Over the five independent task sets, p=7×10−4p=7× 10^-4 (exact conditional version: 8×10−48× 10^-4), and the Mantel–Haenszel risk difference is 7.0 p (95% CI [3.0, 11.0]; Greenland and Robins, 1985). Over the three task sets released after the training data was collected (DeepSWE, Terminal-Bench 4, SWE-Marathon), p=0.004p=0.004 (exact conditional version: 0.005). Substituting the in-house DeepSWE run gives p=3×10−4p=3× 10^-4 over all five sets; dropping both benchmarks with public baselines gives p=0.02p=0.02. The intervals and tests treat the two runs as independent samples, which is conservative when outcomes of the two checkpoints on the same task are positively correlated. Table 4: Pass counts and uncertainty for Table 2: 95% confidence interval of each gain and two-sided p-value of Boschloo’s exact test. Benchmark Base +RL 95% CI of gain (p) p SWE-Bench Pro 439/731 474/731 [−-0.2, 9.7] 0.059 DeepSWE, public baseline 35/113 49/113 [−-0.2, 24.4] 0.055 DeepSWE, in-house baseline 30/113 49/113 [4.4, 28.5] 0.008 Terminal-Bench 2.1 60/89 73/89 [1.8, 26.8] 0.028 Terminal-Bench 3 1/74 8/66 [2.5, 20.8] 0.010 Terminal-Bench 4 0/66 5/66 [0.6, 16.5] 0.029 SWE-Marathon 1/20 5/20 [−-3.2, 42.3] 0.089