Paper deep dive
Cross-Benchmark Generalization in Long-Horizon Agents
Sushant Mehta, Logan Ritchie, Liudas Panavas, Edwin Chen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 8/4/2026, 4:04:49 AM
Summary
This paper investigates cross-benchmark generalization in long-horizon AI agents by post-training the Qwen3.5-122B-A10B model on 363 long-horizon Model Context Protocol (MCP) tasks using a two-stage SFT-then-RL pipeline. The study demonstrates that the trained model improves performance on five external benchmarks (Toolathlon, τ²-Bench, BFCL-V4, SWE-Bench Pro, Terminal-Bench 2) despite no external tasks or graders being used in training. Behavioral analysis reveals four recurring improvements: careful local-goal formation, building goal-relevant working state, maintaining parent goal stability, and verifying completion, suggesting that long-horizon multi-tool post-training fosters transferable working strategies.
Entities (10)
Relation Signals (10)
Qwen3.5-122B-A10B → trainedon → LHMTA
confidence 98% · We post-train an open-weight mixture-of-experts model (Qwen3.5-122B-A10B) on 363 long-horizon Model Context Protocol (MCP) tasks
Qwen3.5-122B-A10B → evaluatedon → Toolathlon
confidence 97% · The trained model improves over the base on five reported external evaluations: Toolathlon
Qwen3.5-122B-A10B → evaluatedon → SWE-bench Pro
confidence 97% · Both software-engineering benchmarks improve... SWE-Bench Pro
Qwen3.5-122B-A10B → evaluatedon → Terminal-Bench 2
confidence 97% · Terminal-Bench 2 (+2.8 pp)
Qwen3.5-122B-A10B → evaluatedon → τ-bench
confidence 97% · τ²-Bench (+5.3 pp)
Qwen3.5-122B-A10B → evaluatedon → BFCL v4
confidence 97% · BFCL-V4 (+3.5 pp)
Qwen3.5-122B-A10B → improvesperformanceon → SWE-bench Pro
confidence 95% · Both software-engineering benchmarks improve... SWE-Bench Pro (+5.8 pp)
Qwen3.5-122B-A10B → usesadaptation → LoRA
confidence 95% · We use LoRA (Hu et al., 2022) (rank 32,α =32) on all attention and MLP projections
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:For reinforcement learning (RL) in self-contained environments, a policy can get rewards by exploiting environment-specific regularities (tool schemas, grader parsing, task templates) rather than by acquiring transferable skill, and an in-distribution holdout shares those regularities. We argue that the discriminating question is behavioral, namely how a trained agent acts, and that cross-benchmark transfer is the right place to look for it. We post-train an open-weight mixture-of-experts model (Qwen3.5-122B-A10B) on 363 long-horizon Model Context Protocol (MCP) tasks across 27 categories, using a two-stage SFT-then-RL pipeline. Toolathlon performance informed the initial base-family and SFT-teacher choices, but no external-benchmark task or grader entered training and no external score informed the reward, training hyperparameters, trained-checkpoint selection, or stopping. At greedy pass@1, the trained model improves over the base on five reported external evaluations: Toolathlon (+9.6 pp), $\tau^2$-Bench (+5.3 pp), BFCL-V4 (+3.5 pp), SWE-Bench Pro (+5.8 pp), and Terminal-Bench 2 (+2.8 pp). Both software-engineering benchmarks improve despite the training collection containing no software-engineering tasks. An exploratory paired-trajectory analysis identifies four recurring behavioral differences (more careful local-goal formation, building goal-relevant working state, keeping parent goals stable through local repairs, and verifying completion) that appear in analogous forms across office workflows and code. These results provide descriptive evidence that long-horizon multi-tool post-training can change ways of working that transfer beyond its training domain.
Tags
Links
- Source: https://arxiv.org/abs/2608.00181v1
- Canonical: https://arxiv.org/abs/2608.00181v1
Trouble viewing inline? Open PDF directly →
Full Text
40,618 characters extracted from source content.
Expand or collapse full text
Accepted at the COLM 2026 Workshop on Agent Behavior Cross-Benchmark Generalization in Long-Horizon Agents Sushant Mehta ∗ Logan RitchieLiudas PanavasEdwin Chen Surge AI Abstract For reinforcement learning (RL) in self-contained environments, a pol- icy can get rewards by exploiting environment-specific regularities (tool schemas, grader parsing, task templates) rather than by acquiring trans- ferable skill, and an in-distribution holdout shares those regularities. We argue that the discriminating question is behavioral, namely how a trained agent acts, and that cross-benchmark transfer is the right place to look for it. We post-train an open-weight mixture-of-experts model (Qwen3.5-122B- A10B) on 363 long-horizon Model Context Protocol (MCP) tasks across 27 categories, using a two-stage SFT-then-RL pipeline. Toolathlon perfor- mance informed the initial base-family and SFT-teacher choices, but no external-benchmark task or grader entered training and no external score informed the reward, training hyperparameters, trained-checkpoint selec- tion, or stopping. At greedy pass@1, the trained model improves over the base on five reported external evaluations: Toolathlon (+9.6 p),τ 2 -Bench (+5.3 p), BFCL-V4 (+3.5 p), SWE-Bench Pro (+5.8 p), and Terminal- Bench 2 (+2.8 p). Both software-engineering benchmarks improve despite the training collection containing no software-engineering tasks. An ex- ploratory paired-trajectory analysis identifies four recurring behavioral differences (more careful local-goal formation, building goal-relevant work- ing state, keeping parent goals stable through local repairs, and verifying completion) that appear in analogous forms across office workflows and code. These results provide descriptive evidence that long-horizon multi- tool post-training can change ways of working that transfer beyond its training domain. 1 Introduction The dominant way to report progress on AI agents is capability-centric: an agent is trained, its task-success rate is measured, and a higher number is taken as evidence of a better agent. For reinforcement learning (RL) on a curated, self-built environment, the success rate alone is a weak signal of what was learned. A policy optimized against a fixed environment may exploit consistent features of that environment’s reward landscape (idiosyncrasies of the tool surface, grader implementation, or task templates) rather than improve the underlying skills the environment is meant to teach. Crucially, the natural internal control, a held-out split of the same environment, does not exclude this possibility: an in-distribution holdout shares many environment-specific degrees of freedom with the training set. This is fundamentally a question about agent behavior: not only what an agent achieves but how it achieves it, and whether the “how” is a reusable way of working or a brittle exploit. We take two complementary positions. First, transfer should be tested on external task instances and graders, with results withheld from reward design, training hyperparameters, trained-checkpoint selection, and stopping. Second, the interpretation should be behavioral: we should point to recurring changes in how the agent acts, ask whether those changes appear across superficially different domains, and identify where they do not. We study both. We post-train an open-weight mixture-of-experts (MoE) model, Qwen3.5- 122B-A10B, on Long-Horizon Multi-Tool Agent tasks (LHMTA): 363 training tasks spanning ∗ Corresponding author: sushantmehta@surgehq.ai. 1 arXiv:2608.00181v1 [cs.SE] 31 Jul 2026 Accepted at the COLM 2026 Workshop on Agent Behavior 27 categories of realistic professional workflows, exposed through Model Context Protocol (MCP) servers (Anthropic, 2024) and scored by deterministic per-criterion graders. We then evaluate on five external benchmarks whose task instances and graders were excluded from training and analyze how the observed behavior changed. Our contributions are: •An evaluation stance and a consistent transfer result. We treat the in-distribution holdout as a pipeline check and report five externally maintained benchmark evaluations. The trained model improves on Toolathlon,τ 2 -Bench, BFCL-V4, SWE- Bench Pro, and Terminal-Bench 2, including two software-engineering benchmarks despite no software-engineering training tasks (Section 6). •A behavioral account of the transfer. Through paired base-vs-trained trajectory analysis we identify four recurring behavioral differences (goal formation, goal- relevant state-building, goal stability, and verification), show that each appears across general tool use and code, and quantify related signals on software trajecto- ries (Section 7). • Calibrated reporting. We disclose the complete reward, external-evaluation firewall, diagnostic-result provenance, exploratory nature of the trajectory analysis, and single-run limitation (Sections 4–7). 2 Related Work RL for agents and verifiable rewards.RL with verifiable rewards has become the standard recipe for eliciting multi-step reasoning and tool use from language models (DeepSeek- AI, 2025; Shao et al., 2024). Group-relative estimators such as GRPO (Shao et al., 2024) and the sequence-level GSPO (Zheng et al., 2025) we use avoid a learned value model by normalizing rewards within a group of rollouts. Most open work in this line trains on a single domain (typically code or math) and reports in-domain gains; our focus is whether the behaviors learned in one environment transfer outside it. Coding-agent RL and environments. A productive recent line grounds code LLMs in execution feedback (Gehring et al., 2024; Wei et al., 2025) and builds executable training environments and verifiers for software agents (Pan et al., 2024; Jain et al., 2025; Cao et al., 2025). These environments are powerful but largely code-centric. LHMTA instead spans general professional software work (office suites, terminals, calendars, file systems, web), and we ask what transfers from that broader distribution to SWE-Bench Pro (Deng et al., 2025), a repository-level issue-resolution benchmark in the SWE-bench lineage (Jimenez et al., 2024), and Terminal-Bench 2 (Merrill et al., 2026), which evaluates difficult work in command-line environments. Agentic benchmarks. Realistic, long-horizon agent benchmarks include web environ- ments (Zhou et al., 2024), broad agent suites (Liu et al., 2024b), tool-agent-user interaction benchmarks (Yao et al., 2024; Barres et al., 2025), function-calling evaluation (Patil et al., 2025), and long-horizon tool execution (Li et al., 2026). We use external tasks and graders fromτ 2 -Bench, BFCL-V4, and Toolathlon to test whether changes extend beyond LHMTA; Section 5 states the known Toolathlon surface overlap. Cross-domain post-training. Recent work directly tests whether agentic post-training gains extend beyond the training distribution, including high-fidelity enterprise environ- ments, unseen-domain reinforcement post-training, agent reinforcement fine-tuning, and tool-integrated RL (Mehta et al., 2026; Hu et al., 2026; Xi et al., 2026; Chen et al., 2025). Our focus is complementary: we connect cross-benchmark score changes to an exploratory account of recurring trajectory-level behavior. Data selection and what makes data generalize.Instruction-tuning research moved from “more data” to “better-selected data”: selecting prompt-sensitive tasks (Kung et al., 2023), 2 Accepted at the COLM 2026 Workshop on Agent Behavior balancing complexity/quality/diversity (Liu et al., 2024a), and choosing examples by their influence on a target capability (Xia et al., 2024). A recurring finding is that surface difficulty and surface diversity are poor proxies; what matters is the underlying reasoning a task exercises. Our behavioral analysis is the agentic analogue of that lesson: we characterize the task pressures an environment exerts rather than its surface domains. Behavior over scores.Our framing aligns with the behavioral study of machines (Rahwan et al., 2019): we seek trajectory-level behavioral explanations of how an agent acts, not only aggregate success. It also builds on failure-derived accounts of agentic capabilities (Ritchie et al., 2026), process-supervision work (Lightman et al., 2024), and parameter-efficient studies of what fine-tuning does and does not change (Hu et al., 2022; Biderman et al., 2024). 3 The Training Environment LHMTA is a suite of long-horizon agentic tasks designed to exercise realistic professional work rather than single tool calls. It spans 27 categories covering office suites (spreadsheets, documents, slides), terminals andbash, calendars and scheduling, search and retrieval, file systems, and browser automation; it contains no software-engineering tasks. Each environment is exposed as one or more MCP servers (Anthropic, 2024), so a single trajectory routinely composes several servers (for example, a browser, a spreadsheet reader, a file system, and a calendar) inside one rollout. Verifiable, per-criterion graders. Every task ships a deterministic Python grader that scores completion from the final environment state (e.g., the contents of files the agent was asked to produce). Graders are structured per criterion (e.g., 8 of 10 sub-conditions satisfied). This structure is central to our reward design (Section 4): it supports both a sparse pass/fail reward and a dense partial-completion reward, and it raises the fraction of tasks that yield usable training signal. Long-horizon and multi-tool.Successful trajectories typically take 30–40 tool-calling turns and 80K–100K tokens, stressing planning and context management rather than one-shot tool selection. This length also drives real training cost: full trajectories above 50K tokens substantially increase context-parallel sharding overhead, making the RL runs compute- intensive. Task construction.Tasks were authored to produce meaningful agentic failures rather than trivial format errors: creators targeted work that strong models could not reliably complete, and the collection was monitored for diversity of tools, task types, and failure modes. Of an initial 430 environment/task pairs, we excluded 27 that depended on rate-limited external APIs (e.g., mapping, calendar, and finance services) or that failed to run deterministically. This left 403 usable tasks: 363 for training and a disjoint 40-task in-distribution holdout. We discuss the implications of this curation-by-failure-mode philosophy, and its limits, in Section 7. 4 Training Method Base model and adapter.We post-train Qwen3.5-122B-A10B (Qwen Team, 2026), an open- weight MoE checkpoint with 122B total and roughly 10B active parameters, chosen as a favorable size/quality trade-off among open models at project start. 1 We use LoRA (Hu et al., 2022) (rank 32,α =32) on all attention and MLP projections (q,k,v,o,gate,up,down). LoRA is motivated both by compute efficiency and by evidence that low-rank adaptation forgets less of the base model’s capabilities than full fine-tuning (Biderman et al., 2024). 1 We use the released checkpoint without modification and refer to it by its public name; the contribution here is the environment and the analysis, not the base model. 3 Accepted at the COLM 2026 Workshop on Agent Behavior A two-stage recipe (SFT then RL). The base policy rarely solved these tasks even across four attempts, so binary rewards were too sparse to support RL from that initialization. We therefore used a supervised warm-up before RL. The open-weight teacher, Kimi K2.6 (Moonshot AI, 2026), generated trajectories only on the 363 LHMTA training tasks; we retained 3,000 trajectories scoring above 0.9. We then ran RL on the same 363 tasks with the GSPO sequence-level estimator (Zheng et al., 2025). We evaluate the one-epoch output of the complete SFT+RL pipeline, called the trained model throughout; exploratory continuation through later epochs was not used for the reported evaluations. This experiment was not designed to attribute the observed transfer to either stage in isolation. Dense criterion reward and low-score effort floor. Letr g ∈ [0, 1]be the fraction of deterministic grader criteria satisfied. For trajectories withr g <0.5, the implemented reward also had an effort floor e = min(0.30, 0.05 min(n calls , 6)) + 0.05 1 normal + 0.05 1 ≥3 distinct tools ,r = max(r g , e); forr g ≥0.5,r = r g . Thus the floor was at most 0.40 and was a lower bound, not a bonus added to successful grader reward. In a four-attempt diagnostic, 16.8% of tasks had at least one full pass, whereas 82.7% yielded at least one nonzero dense or effort-shaped reward; mean reward increased from 0.30 to 0.51. Reward was assigned once per full trajectory. An earlier per-prefix implementation was abandoned after the policy optimized rewarded prefixes instead of full completion. This shaping was used only for training; evaluation used each benchmark’s native scoring. Stability adaptations. Two rollout-time issues shaped the recipe. (i) Context blow-ups: a few tools return very large payloads (full spreadsheets, paper bodies), so we offload large tool outputs to files the agent cangrep, trading turns for context. (i) Looping: the model sometimes repeated malformed tool calls (e.g., passing a string where a server demanded a number) until it exhausted context; we compact corrected retry sequences and retain only the most recent reasoning trace in the rolling context, matching the model’s chat template and reducing context bloat. Hyperparameters.SFT: max length 131,072, AdamW (betas=0.9/0.95), LR 1×10 −5 cosine, warmup ratio 0.05, weight decay 0.01, 1 epoch. RL: max sequence length 80K, GSPO advantage estimator, 8 samples per prompt, rollout temperature 1.0, LR 5×10 −5 constant, KL coefficient 0 (KL-loss coefficient 0.001, low-variancek3 estimator), entropy coefficient 0.001, gradient clip 1.0, 8 actor nodes×8 H200 GPUs. Infrastructure was built onslime, Megatron-LM (Shoeybi et al., 2019), and SGLang (Zheng et al., 2024); all evaluations served the model in BF16 with SGLang. Additional configuration details are in Appendix A. 5 Evaluation Protocol What counts as evidence. The holdout (40 in-family tasks) is an in-distribution pipeline check: it confirms that the pipeline changes target-environment performance, but it shares the environment’s degrees of freedom and cannot by itself establish transfer. We report five external benchmarks: Toolathlon (Li et al., 2026),τ 2 -Bench (Barres et al., 2025), BFCL-V4 (Patil et al., 2025), SWE-Bench Pro (Deng et al., 2025), and Terminal-Bench 2 (Merrill et al., 2026). No external-benchmark task or grader appears in SFT or RL. After the initial base- family and teacher choices, no external score informed the training reward, hyperparameters, trained-checkpoint selection, or early stopping. Overlap and contamination.The training collection was Toolathlon-inspired and reused some MCP server implementations and tool schemas from its repository. Toolathlon leader- board performance was considered when choosing the initial base-model family and SFT teacher, but no Toolathlon task instance or grader entered training, and later benchmark results did not determine the reward, training hyperparameters, trained checkpoint, or stop- ping. The other benchmarks also share capabilities and, in places, tool/API surfaces with 4 Accepted at the COLM 2026 Workshop on Agent Behavior Table 1: Tool-use results (%);∆is base→trained. Greedy values are from the updated final evaluation, exceptτ 2 -Bench, which is retained from the accepted-paper snapshot. Four-sample diagnostics are from that earlier snapshot and should not be treated as decom- positions of the updated greedy scores. BaseTrained∆ pass@1 (greedy) LHMTA holdout10.027.5+17.5 Toolathlon22.231.8+9.6 τ 2 -Bench54.860.1+5.3 BFCL-V455.759.2+3.5 Accepted-paper four-sample diagnostic Holdout15.937.2+21.3 Toolathlon25.936.1+10.2 τ 2 -Bench57.162.9+5.8 BFCL-V467.772.2+4.5 LHMTA. We cannot audit the base model’s pretraining corpus, so we do not claim the bench- marks were unseen during pretraining. Our empirical claim is narrower: under matched evaluation conditions, the trained checkpoint differs from the base after post-training that excluded the external benchmark tasks and graders. Sampling and statistics.The primary results use greedy decoding and each benchmark’s native scoring and aggregation. Base and trained checkpoints were evaluated with the same protocol within each benchmark. We report descriptive point estimates: the benchmarks are correlated, several use native weighted aggregation, and a single post-training run does not estimate training-run variance, so we do not attach a formal cross-benchmark hypothesis test. We also retain scores reported as pass@4 in the accepted-paper snapshot, obtained from four temperature-1.0 trajectories and the archived harness’s native aggregation. The raw units needed to identify these values with a standard unweighted any-of-four estimator are unavailable, so we refer to them only as native four-sample diagnostics. The retained τ 2 -Bench configuration aggregates Airline, Retail, Telecom, and Banking. These diagnostics are distinct from the updated greedy evaluation. 6 Results Transfer to external tool-use benchmarks.Table 1 reports the base and trained models on the holdout and three external tool-use evaluations. The updated greedy evaluation gives a +17.5 p holdout change,+9.6 p on Toolathlon, and+3.5 p on BFCL-V4. We additionally retain the accepted-paperτ 2 -Bench result (+5.3 p) and four-sample diagnostics, while separating their provenance from the updated greedy scores. Transfer to software-engineering benchmarks. The strongest cross-domain test is im- provement on software work absent from LHMTA. The same trained checkpoint improves on SWE-Bench Pro by+5.8 p and Terminal-Bench 2 by+2.8 p (Table 2). The claim con- cerns the direction of change from general-agent post-training, not absolute state of the art. Across the five reported external evaluations, all aggregate changes are positive; we treat that consistency descriptively rather than as a formal test because the benchmarks are correlated and τ 2 -Bench comes from the retained evaluation snapshot. Interpreting the point estimates.The results are descriptive, not uncertainty-free. Greedy decoding removes sampling temperature from the primary comparison, but finite task sets, native benchmark weighting, evaluation infrastructure, and training-run variation remain. We ran one full post-training experiment at this scale and do not have the raw paired units needed for benchmark-appropriate confidence intervals across all rows. We 5 Accepted at the COLM 2026 Workshop on Agent Behavior Table 2: Updated cross-domain software-engineering results (greedy pass@1, %). The training collection contains no software-engineering tasks;∆ is base→trained. BaseTrained∆ SWE-Bench Pro20.526.3+5.8 Terminal-Bench 244.947.7+2.8 Table 3: Toolathlon pass@1 by category (%) from the accepted-paper diagnostic snapshot; these values do not arithmetically decompose the updated aggregate in Table 1. AcademiaCampusDailyFinanceOfficeShoppingTech Base30.715.421.425.00.050.026.7 Trained38.119.529.235.116.561.536.6 ∆+7.4+4.1+7.8 +10.1 +16.5+11.5+9.9 therefore emphasize the direction and magnitude of the observed changes and avoid claims of statistical significance. Per-category structure and a capability-vs-attempt caveat. Toolathlon gains in the accepted-paper diagnostic snapshot are uneven across categories (Table 3). The largest is Office, 0.0→16.5 p. This zero baseline leaves greater room for changes in task initiation and completion, behaviors that were also encouraged by the low-score effort floor; 16.5% absolute remains low. Finance (+10.1), Shopping (+11.5), and Tech (+9.9) improve from non-trivial baselines, which is less readily explained by task initiation alone. BFCL slice structure. In the retained pass@1 breakdown, BFCL-V4 improves on Agentic (41.8→46.9,+5.1 p), Multi-Turn (57.1→62.1,+5.0 p), and Live (73.1→75.7,+2.6 p), while Non-Live and Hallucination regress slightly. We report these as diagnostics without uncertainty estimates. 7 Behavioral Analysis: Recurring Differences Aggregate scores tell us that measured performance changed; they do not tell us how behav- ior differed. We compared base and trained trajectories for the same task under the same prompt, environment, scaffold, and evaluation protocol, focusing on fail-to-pass pairs across LHMTA, Toolathlon, SWE-Bench Pro, and Terminal-Bench 2. An automated root-cause- analysis agent (Claude Opus 4.8) traced failures to evidence-supported divergences; the authors then checked candidate reports against the raw reasoning, tool calls, environment responses, edits, tests, and artifacts. The four categories below were developed iteratively as an interpretive framework, not as mutually exclusive labels or claims about internal model mechanisms. We organize the observed differences with a simple goal-directed loop: set a goal→take an action→update working state from feedback→compare to the goal→repeat, applied recursively as high-level goals decompose into sub-goals and concrete tool calls. Tool use, normal completion, and tool diversity were directly shaped for low-scoring trajectories by the effort floor. Precise local-goal formation, faithful state construction, parent-goal stability, and test-based verification were not explicit reward terms. 1. Forming the correct goal at every scale. The trained model grounds local goals more carefully in the actual environment state. On a Toolathlon clickstream A/B-test task, the base model defined conversion asclicks/storeviews(yielding rates above 100%), while the trained model related the columns to the event order and usedstoreviews/clicks. In a SWE-Bench Pro NodeBB task, both models found helpers and tests encoding the required selected-field semantics. The base reimplemented filtering in backend paths and changed missing-object behavior; the trained model formed the narrower goal of routing 6 Accepted at the COLM 2026 Workshop on Agent Behavior Table 4: Behavioral metrics over 731 matched SWE-Bench Pro task pairs. Reference patches are not unique ground truth, and smaller patches are not inherently better. MetricBaseTrainedInterpretation Mean retrieval calls25.623.9similar retrieval effort Repeated retrieved information22.5%14.3%less repeated investigation Reference-patch files touched2.693.08greater reference overlap Mean added lines415.5111.7smaller edit footprint Runs containing a formal test37.5%73.3%more verification the public methods through the established helpers. The surfaces differ, but both cases concern translating a parent objective and observed state into the correct local target. 2. Building goal-relevant working state.On a Toolathlon task computing year-over-year growth from a workbook, the base model treated formula strings as missing data and discarded values needed by the report; the trained model reopened the workbook using cached formula values. In a SWE-Bench Pro Ansible task, the base introduced a local helper that calledkeyword.iskeywordeven though onlyiskeywordwas imported, then changed the test setup after the resultingNameError. The trained model connected the requirement to the repository’s central collection-name validator and reused that path. In both cases, the difference is whether observed environment structure enters the working representation that guides the next action. 3. Keeping parent goals stable through local repairs. On a course-scheduling holdout task, the base model first recognized the “only 2x-level courses” constraint, then violated it (selecting a 1x course) while fixing a credit-cap problem, and then falsely self-certified. In a SWE-Bench Pro PowerShell CLIXML task requiring decoded control characters to be preserved, the base model encountered a stale test expecting the old behavior and “fixed” it by reintroducing exactly the behavior it was told to change; the trained model recognized the test as stale relative to the new requirement. The pressure, holding the parent requirement fixed while repairing a local problem, is domain-independent. 4. Verifying completion at each boundary.On a library-catalogue holdout task, the base model generated a partial 44-book intermediate file, read it back, and declared the catalogue complete (checking an artifact against itself); the trained model reconciled two workbook views and confirmed both contained the same 347 authors before proceeding. In a SWE- Bench Pro refactor that moved logic out of a class component, the base model checked only that the new helpers were exported, while the trained model searched for stale call sites and ran a targeted type-check over the changed files. The behavior is verifying the property the downstream task actually depends on, not a convenient proxy. Quantifying the behaviors.Table 4 reports deterministic metrics over 731 matched SWE- Bench Pro task pairs. With similar retrieval volume, the trained model repeats less retrieved information. Its changed files overlap the reference patch more often and its patches add far fewer lines, patterns consistent with more targeted investigation and editing but not proof of correctness. Verification changes most clearly: the share of runs containing a formal test rises from 37.5% to 73.3%. Retrieval calls are read-only inspection commands. Repeated information is the share of lowercased, overlapping 12-token spans already seen in an earlier retrieval output from the same run. Reference-patch overlap and added lines are computed from the final submitted diff, and formal tests are commands invoking standard test runners. Capability or propensity? One reading is that post-training mainly teaches the model to attempt tasks and emit gradeable artifacts. The effort floor explicitly encouraged tool use, normal completion, and tool diversity for low-scoring trajectories, so changes in attempting and finishing may partly reflect direct shaping; we did not run a no-floor 7 Accepted at the COLM 2026 Workshop on Agent Behavior ablation. Precise local-goal formation, faithful state construction, parent-goal stability, and test-based verification were not explicit reward terms. Their recurrence across domains and the aggregate correlates in Table 4 are consistent with broader transfer, but do not establish a causal mechanism. 8 Discussion: Designing for Transferable Behavior Reading transfer behaviorally has a direct payoff for data design. If the unit of value is a task pressure (a way the environment forces the agent to form a goal, build state, hold requirements, or verify), then surface diversity (different prompts, tools, file types) can be a trap: a dataset can look diverse while clustering on one or two failure modes, and two superficially unrelated tasks can train the same transferable pressure. This mirrors the instruction-tuning shift from “more data” to “data selected for the reasoning it exercises” (Kung et al., 2023; Liu et al., 2024a; Xia et al., 2024), now at the level of environments rather than prompt-response pairs. A practical next step is a flywheel: build tasks that produce meaningful failures, tag the pressure each is meant to exercise, train, evaluate on held-out, nearby, and far benchmarks, analyze trajectories rather than only score deltas, and use the observed pattern to generate testable hypotheses for the next data round. One such hypothesis is that adding tasks requiring preservation of mature-system contracts may improve code transfer; the present experiment does not test it. 9 Conclusion The value of an agentic post-training environment is not only how high it pushes its own score but whether the ways of working it encourages are reusable. We post-trained an open MoE model on long-horizon, MCP-based professional workflows and observed positive changes across five external evaluations, including two software-engineering benchmarks despite no software-engineering training tasks. We then provided an exploratory behavioral account: more careful local-goal formation, faithful state-building, goal stability under repair, and evidence-based verification recur across office workflows and code. Explicit effort shaping can account for some changes in attempting and finishing; the remaining behavioral observations and aggregate correlates are consistent with, but do not prove, a broader mechanism of transfer. Together, cross-benchmark evaluation and trajectory-level analysis offer a practical way to study not only what agents achieve, but how their behavior changes. References Rishabh Agarwal, Nino Vieillard, Yongchao Zhou, Piotr Stanczyk, Sabela Ramos, Matthieu Geist, and Olivier Bachem. On-policy distillation of language models: Learning from self-generated mistakes. In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2306.13649. Anthropic. Introducing the model context protocol.https://w.anthropic.com/news/ model-context-protocol, 2024. Victor Barres, Honghua Dong, Soham Ray, Xujie Si, and Karthik Narasimhan.τ 2 - bench: Evaluating conversational agents in a dual-control environment. arXiv preprint arXiv:2506.07982, 2025. URL https://arxiv.org/abs/2506.07982. Dan Biderman, Jacob Portes, Jose Javier Gonzalez Ortiz, Mansheej Paul, Philip Greengard, Connor Jennings, Daniel King, Sam Havens, Vitaliy Chiley, Jonathan Frankle, Cody Blakeney, and John P. Cunningham. LoRA learns less and forgets less. Transactions on Machine Learning Research (TMLR), 2024. URL https://arxiv.org/abs/2405.09673. Shiyi Cao, Dacheng Li, Fangzhou Zhao, Shuo Yuan, Sumanth R. Hegde, Connor Chen, Charlie Ruan, Tyler Griggs, Shu Liu, Eric Tang, Richard Liaw, Philipp Moritz, Matei Zaharia, Joseph E. Gonzalez, and Ion Stoica. SkyRL-agent: Efficient RL training for 8 Accepted at the COLM 2026 Workshop on Agent Behavior multi-turn LLM agent. arXiv preprint arXiv:2511.16108, 2025. URLhttps://arxiv.org/ abs/2511.16108. Zhengyu Chen, Jinluan Yang, Teng Xiao, Ruochen Zhou, Luan Zhang, Xiangyu Xi, Xiaowei Shi, Wei Wang, and Jinggang Wang. Can tool-integrated reinforcement learning generalize across diverse domains? arXiv preprint arXiv:2510.11184, 2025. URLhttps://arxiv.org/ abs/2510.11184. DeepSeek-AI. DeepSeek-R1: Incentivizing reasoning capability in LLMs via reinforcement learning. arXiv preprint arXiv:2501.12948, 2025. URLhttps://arxiv.org/abs/2501.12948. Xiang Deng, Jeff Da, Edwin Pan, Yannis Yiming He, Charles Ide, Kanak Garg, Niklas Lauffer, Andrew Park, Nitin Pasari, Chetan Rane, Karmini Sampath, Maya Krishnan, Srivatsa Kundurthy, Sean Hendryx, Zifan Wang, Vijay Bharadwaj, Jeff Holm, Raja Aluri, Chen Bo Calvin Zhang, Noah Jacobson, Bing Liu, and Brad Kenstler. SWE-bench pro: Can AI agents solve long-horizon software engineering tasks? arXiv preprint arXiv:2509.16941, 2025. URL https://arxiv.org/abs/2509.16941. Jonas Gehring, Kunhao Zheng, Jade Copet, Vegard Mella, Quentin Carbonneaux, Taco Cohen, and Gabriel Synnaeve. RLEF: Grounding code LLMs in execution feedback with reinforcement learning. arXiv preprint arXiv:2410.02089, 2024. URLhttps://arxiv.org/ abs/2410.02089. Chuxuan Hu, Yuxuan Zhu, Antony Kellermann, Caleb Biddulph, Suppakit Waiwitlikhit, Jason Benn, and Daniel Kang. Breaking barriers: Do reinforcement post training gains transfer to unseen domains? In International Conference on Learning Representations (ICLR), 2026. URL https://openreview.net/forum?id=mvLhN0veUd. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022. URLhttps://arxiv.org/ abs/2106.09685. Naman Jain, Jaskirat Singh, Manish Shetty, Liang Zheng, Koushik Sen, and Ion Stoica. R2E-Gym: Procedural environments and hybrid verifiers for scaling open-weights SWE agents. arXiv preprint arXiv:2504.07164, 2025. URL https://arxiv.org/abs/2504.07164. Carlos E. Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. SWE-bench: Can language models resolve real-world GitHub issues? In International Conference on Learning Representations (ICLR), 2024. URLhttps: //arxiv.org/abs/2310.06770. Po-Nien Kung, Fan Yin, Di Wu, Kai-Wei Chang, and Nanyun Peng. Active instruction tuning: Improving cross-task generalization by training on prompt sensitive tasks. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2023. URL https://aclanthology.org/2023.emnlp-main.112/. Junlong Li, Wenshuo Zhao, Jian Zhao, Weihao Zeng, Haoze Wu, Xiaochen Wang, Rui Ge, Yuxuan Cao, Yuzhen Huang, Wei Liu, Junteng Liu, Zhaochen Su, Yiyang Guo, Fan Zhou, Lueyang Zhang, Juan Michelini, Xingyao Wang, Xiang Yue, Shuyan Zhou, Graham Neubig, and Junxian He. The tool decathlon: Benchmarking language agents for diverse, realistic, and long-horizon task execution. In International Conference on Learning Representations (ICLR), 2026. URL https://openreview.net/forum?id=z53s5p0qhf. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. In International Conference on Learning Representations (ICLR), 2024. URLhttps: //arxiv.org/abs/2305.20050. Wei Liu, Weihao Zeng, Keqing He, Yong Jiang, and Junxian He. What makes good data for alignment? a comprehensive study of automatic data selection in instruction tuning. In International Conference on Learning Representations (ICLR), 2024a. URL https://arxiv. org/abs/2312.15685. 9 Accepted at the COLM 2026 Workshop on Agent Behavior Xiao Liu, Hao Yu, Hanchen Zhang, Yifan Xu, Xuanyu Lei, Hanyu Lai, Yu Gu, Hangliang Ding, Kaiwen Men, Kejuan Yang, Shudan Zhang, Xiang Deng, Aohan Zeng, Zhengxiao Du, Chenhui Zhang, Sheng Shen, Tianjun Zhang, Yu Su, Huan Sun, Minlie Huang, Yuxiao Dong, and Jie Tang. AgentBench: Evaluating LLMs as agents. In International Conference on Learning Representations (ICLR), 2024b. URL https://arxiv.org/abs/2308.03688. Sushant Mehta, Logan Ritchie, Suhaas Garre, Ian Niebres, Nick Heiner, and Edwin Chen. En- terpriseBench Corecraft: Training generalizable agents on high-fidelity RL environments. arXiv preprint arXiv:2602.16179, 2026. URL https://arxiv.org/abs/2602.16179. Mike A. Merrill et al. Terminal-bench: Benchmarking agents on hard, realistic tasks in command line interfaces. In International Conference on Learning Representations (ICLR), 2026. URL https://openreview.net/forum?id=a7Qa4CcHak. Moonshot AI. Kimi K2.6 model card. Hugging Face, 2026. URLhttps://huggingface.co/ moonshotai/Kimi-K2.6. Jiayi Pan, Xingyao Wang, Graham Neubig, Navdeep Jaitly, Heng Ji, Alane Suhr, and Yizhe Zhang. Training software engineering agents and verifiers with SWE-Gym. arXiv preprint arXiv:2412.21139, 2024. URL https://arxiv.org/abs/2412.21139. Shishir G. Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. The berkeley function calling leaderboard (BFCL): From tool use to agentic evaluation of large language models. In Proceedings of the 42nd International Conference on Machine Learning (ICML), 2025. URLhttps://proceedings.mlr.press/v267/ patil25a.html. Qwen Team.Qwen3.5-122B-A10B model card.Hugging Face, 2026.URLhttps: //huggingface.co/Qwen/Qwen3.5-122B-A10B. Iyad Rahwan, Manuel Cebrian, Nick Obradovich, Josh Bongard, Jean-Franc ̧ois Bonnefon, Cynthia Breazeal, Jacob W. Crandall, Nicholas A. Christakis, Iain D. Couzin, Matthew O. Jackson, Nicholas R. Jennings, Ece Kamar, Isabel M. Kloumann, Hugo Larochelle, David Lazer, Richard McElreath, Alan Mislove, David C. Parkes, Alex Sandy Pentland, Mar- garet E. Roberts, Azim Shariff, Joshua B. Tenenbaum, and Michael Wellman. Machine behaviour. Nature, 568(7753):477–486, 2019. doi: 10.1038/s41586-019-1138-y. Logan Ritchie, Sushant Mehta, Nick Heiner, Mason Yu, and Edwin Chen. The hierarchy of agentic capabilities: Evaluating frontier models on realistic RL environments. arXiv preprint arXiv:2601.09032, 2026. URL https://arxiv.org/abs/2601.09032. Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y.K. Li, Y. Wu, and Daya Guo. DeepSeekMath: Pushing the limits of mathematical reasoning in open language models. arXiv preprint arXiv:2402.03300, 2024. URL https://arxiv.org/abs/2402.03300. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-LM: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019. URLhttps://arxiv.org/abs/ 1909.08053. Yuxiang Wei, Olivier Duchenne, Jade Copet, Quentin Carbonneaux, Lingming Zhang, Daniel Fried, Gabriel Synnaeve, Rishabh Singh, and Sida I. Wang. SWE-RL: Advancing LLM reasoning via reinforcement learning on open software evolution. arXiv preprint arXiv:2502.18449, 2025. URL https://arxiv.org/abs/2502.18449. Zhiheng Xi, Xin Guo, Jiaqi Liu, Jiazheng Zhang, Yutao Fan, Zhihao Zhang, Shichun Liu, Mingxu Chai, Xiaowei Shi, Yitao Zhai, Xunliang Cai, Tao Gui, Qi Zhang, and Xuanjing Huang. Does reinforcement fine-tuning improve generalization of LLM agents? an empirical study. In International Conference on Machine Learning (ICML), 2026. URL https://openreview.net/forum?id=A4wPhPe6nW. 10 Accepted at the COLM 2026 Workshop on Agent Behavior Mengzhou Xia, Sadhika Malladi, Suchin Gururangan, Sanjeev Arora, and Danqi Chen. LESS: Selecting influential data for targeted instruction tuning. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. URLhttps://arxiv.org/abs/ 2402.04333. Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan.τ-bench: A benchmark for tool-agent-user interaction in real-world domains. arXiv preprint arXiv:2406.12045, 2024. URL https://arxiv.org/abs/2406.12045. Chujie Zheng, Shixuan Liu, Mingze Li, Xiong-Hui Chen, Bowen Yu, Chang Gao, Kai Dang, Yuqiong Liu, Rui Men, An Yang, Jingren Zhou, and Junyang Lin. Group sequence policy optimization. arXiv preprint arXiv:2507.18071, 2025. URLhttps://arxiv.org/abs/2507. 18071. Lianmin Zheng, Liangsheng Yin, Zhiqiang Xie, Chuyue Sun, Jeff Huang, Cody Hao Yu, Shiyi Cao, Christos Kozyrakis, Ion Stoica, Joseph E. Gonzalez, Clark Barrett, and Ying Sheng. SGLang: Efficient execution of structured language model programs. In Advances in Neural Information Processing Systems (NeurIPS), 2024. URLhttps://arxiv.org/abs/2312.07104. Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Tianyue Ou, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. WebArena: A realistic web environment for building autonomous agents. In International Conference on Learning Representations (ICLR), 2024. URL https://arxiv.org/abs/2307.13854. A Reported Training Configuration SFT (framework:ms-swift): max length 131,072; optimizer AdamW (fused), betas(0.9, 0.95); LR 1×10 −5 , cosine schedule, warmup ratio 0.05; weight decay 0.01; max grad norm 1.0; 1 epoch; batch size 4; gradient accumulation 4. SFT corpus: 3,000 Kimi-K2.6 trajectories, rejection-sampled at reward> 0.9. RL (GSPO): max sequence length 80K; RoPE scaling factor 8; context-parallel size 4; expert- parallel size 8; tensor-parallel size 4; pipeline-parallel size 2; 8 actor nodes×8 GPUs; SGLang TP/EP size 8; samples per prompt 8; rollout temperature 1.0, top-p1.0; LoRA targets q,k,v,o,gate,up,down, rank 32,α32; LR 5×10 −5 constant; Adam betas(0.9, 0.98), eps 1×10 −8 ; weight decay 0.1; clip 1.0; KL coefficient 0; KL-loss coefficient 0.001 (low-variance k3); entropy coefficient 0.001; rollout routing replay on; BF16. SFT trajectory generation and RL used the same 363 training tasks; the 40 holdout tasks were excluded from both stages. Evaluation: BF16 on H200 with SGLang; primary pass@1 greedy (temperature 0); retained four-sample diagnostics at temperature 1.0. B Continued Training and Task Saturation Extending RL to a second and third epoch on the same tasks did not improve and eventually regressed: entropy fell from≈0.9 to≈0.6 by epoch two with declining training reward, and performance regressed further by epoch three. This is consistent with overfitting and suggests that additional task diversity may be more useful than additional passes over the same 363 tasks. On-policy distillation on self-generated successful trajectories (Agarwal et al., 2024) is a natural next step. 11