Paper deep dive
BaT: Towards Self-Evolving Medical Research Agent with Stage Rubrics
Junqi Liu, Yufan He, Yexiao He, Pengfei Guo, Dong Yang, Andriy Myronenko, Can Zhao, Hanrong Ye, Tianhao Qi, Yuyin Zhou, Daguang Xu, Yucheng Tang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%
Last extracted: 8/23/2026, 1:41:41 AM
Summary
The paper introduces Benchmark-as-Teacher (BaT), a recursive self-improvement system for medical research agents. BaT utilizes an asynchronous Stage Bank data pipeline to synthesize content-isolated training states and Bilevel Curriculum Reinforcement Learning (BiCuRL) for post-training. BiCuRL selects stage curricula based on held-out evaluation scores, verifies rollouts with task rubrics, and updates policies using GRPO. The system significantly improves performance on AutoMedBench-Lite, with BaT-9B outperforming Claude Opus 4.6.
Entities (8)
Relation Signals (5)
BaT → contains → Stage Bank
confidence 95% · BaT contains two linked components: the asynchronous Stage Bank data pipeline and BiCuRL...
BaT → contains → BiCuRL
confidence 95% · BaT contains two linked components: ... and BiCuRL (Bilevel Curriculum Reinforcement Learning), its self-improving post-training method.
BaT-9B → evaluatedon → AutoMedBench-Lite
confidence 95% · On AutoMedBench-Lite, BaT-4B and BaT-9B more than double the Overall scores...
BaT-9B → outperforms → Claude Opus 4.6
confidence 95% · BaT-9B Agent reaches 79.6 Overall, exceeding Claude Opus 4.6 with Claude Code at 77.5.
BiCuRL → uses → GRPO
confidence 90% · updates the policy with GRPO
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Long-horizon agents are beginning to automate complete workflows that produce code, reports, and research artifacts. Medical imaging workflows are multi-stage and data-sensitive, while expert trajectories remain scarce and difficult to share. Structured benchmarks can localize failures through stage-level rubrics, but standard post-training discards these diagnostics before the next training round. We present Benchmark-as-Teacher (BaT), a recursive self-improvement system for agent post-training. BaT contains two linked components: the asynchronous Stage Bank data pipeline and BiCuRL (Bilevel Curriculum Reinforcement Learning), its self-improving post-training method. Stage Bank synthesizes content-isolated training states outside the policy-update loop. BiCuRL uses a fixed held-out evaluation to select the next stage curriculum, verifies rollouts with task rubrics, updates the policy with GRPO, and returns the candidate checkpoint to evaluation. On AutoMedBench-Lite, BaT-4B and BaT-9B more than double the Overall scores of their Qwen Instruct baselines. BaT-9B Agent reaches 79.6 Overall, exceeding Claude Opus 4.6 with Claude Code at 77.5.
Tags
Links
- Source: https://arxiv.org/abs/2608.16211v1
- Canonical: https://arxiv.org/abs/2608.16211v1
Trouble viewing inline? Open PDF directly →
Full Text
63,506 characters extracted from source content.
Expand or collapse full text
BaT: Towards Self-Evolving Medical Research Agent with Stage Rubrics Junqi Liu Affiliation: University of California, Santa Cruz Yufan He Affiliation: NVIDIA Yexiao He Affiliation: NVIDIA Pengfei Guo Affiliation: NVIDIA Dong Yang Affiliation: NVIDIA Andriy Myronenko Affiliation: NVIDIA Can Zhao Affiliation: NVIDIA Hanrong Ye Affiliation: NVIDIA Tianhao Qi Affiliation: University of California, Santa Cruz Yuyin Zhou Affiliation: University of California, Santa Cruz Daguang Xu Affiliation: NVIDIA Yucheng Tang Affiliation: NVIDIA Abstract Long-horizon agents are beginning to automate complete workflows that produce code, reports, and research artifacts. Medical imaging workflows are multi-stage and data-sensitive, while expert trajectories remain scarce and difficult to share. Structured benchmarks can localize failures through stage-level rubrics, but standard post-training discards these diagnostics before the next training round. We present Benchmark-as-Teacher (BaT), a recursive self-improvement system for agent post-training. BaT contains two linked components: the asynchronous Stage Bank data pipeline and BiCuRL (Bilevel Curriculum Reinforcement Learning), its self-improving post-training method. Stage Bank synthesizes content-isolated training states outside the policy-update loop. BiCuRL uses a fixed held-out evaluation to select the next stage curriculum, verifies rollouts with task rubrics, updates the policy with GRPO, and returns the candidate checkpoint to evaluation. On AutoMedBench-Lite, BaT-4B and BaT-9B more than double the Overall scores of their Qwen Instruct baselines. BaT-9B Agent reaches 79.6 Overall, exceeding Claude Opus 4.6 with Claude Code at 77.5. We release the code at https://github.com/AutoMedBench/Benchmark-as-Teacher. Figure 1: Benchmark-as-Teacher turns an output-only benchmark into post-training infrastructure. (a) Top: conceptual overview. BaT turns the usual output-only benchmark into training infrastructure with the Stage Bank data-synthesis pipeline, Curriculum Teacher, and BiCuRL post-training method. (b) Lower left: BiCuRL loop. The system forms a closed loop with four operational phases. First, after an SFT cold start, benchmarking splits into stage-wise scores. Second, the aggregate scores feed the Curriculum Teacher, shown as the Domain-Shift Detector and Report for next round. Third, the environment sandbox pool selects targeted-stage (S-target) sandboxes together with mix-stage (S-mix) and end-to-end (E2E) sandboxes. Fourth, the agent executes rollouts in these sandboxes, an LLM verifier scores them with rubric-as-rewards (10), and GRPO (42) updates the policy. The updated checkpoint loops back to benchmarking and restarts the cycle. (c) Lower right: training rounds. Dashed curves show each round’s evaluation, solid step lines track the best checkpoint so far, stars mark the auto-selected best checkpoints. 1 Introduction Long-horizon medical agents must plan, configure tools, validate data, run inference, and submit a checked artifact. A mistake in one stage can invalidate the work that follows, while expert trajectories remain scarce and difficult to share. Medical-agent benchmarks increasingly expose the structure of these workflows. MedAgentBench evaluates 300 physician-authored EHR tasks in a FHIR-compliant environment, HealthAgentBench covers 54 healthcare tasks across seven categories, and AutoMedBench scores planning, setup, validation, inference, and submission (12; 16; 15). These benchmarks identify the stage where an agent fails. Standard post-training discards most of that signal. Group-relative policy optimization (GRPO) commonly assigns one outcome reward to a complete trajectory (42). A single AutoMedBench-Lite run averages 33 interaction turns, so one score cannot identify which stage needs more practice. Iterative methods such as Self-Rewarding Language Models and SPIN improve a model over repeated updates, but their training schedules omit held-out stage diagnostics (50; 6). This paper asks one question: can a structured benchmark teach an agent while its task content stays outside training? Benchmark-as-Teacher (BaT) turns the benchmark signal into a recursive post-training cycle. BaT contains an asynchronous data pipeline, called Stage Bank, and a self-improving post-training method, called Bilevel Curriculum Reinforcement Learning (BiCuRL). Stage Bank synthesizes fictional tasks, reconstructs executable stage states, and applies leakage checks outside the policy-update loop. It exposes three content-isolated training pools: S-target for the selected weak stage, S-mix for the remaining stages, and an end-to-end (E2E) surrogate for the complete workflow. BiCuRL connects evaluation, training, and re-evaluation. Its outer loop reads stage scores from a fixed held-out evaluation, selects the next target stage, and retains or rejects candidate checkpoints. Its inner loop samples Stage Bank states, scores new rollouts with rubric items and artifact evidence, and updates the policy with GRPO. Only aggregate scores cross the evaluation boundary; task IDs, answers, paths, reports, and traces remain held out. Figure 1 summarizes this RSI cycle. A trained policy becomes a BaT Agent when paired with a fixed execution environment that includes public stage skills. We keep this engineering layer fixed across BaT Agent comparisons so BiCuRL changes only the policy. We evaluate medical performance on AutoMedBench-Lite, ABRA, and MedXpertQA-Text, and test transfer on eight external benchmarks (15; 20; 54). BaT-4B and BaT-9B more than double their Qwen Instruct Overall scores. The full Stage Bank mixture leads every evaluated partial mixture. BaT-9B Agent reaches 79.6 Overall, 2.1 points above Claude Opus 4.6 with Claude Code (Figure 2). On the external suite, the 9B policy remains within 3.4–5.8 points of its baseline on three reasoning tasks and improves τ2τ^2-Bench, SWE-bench Verified, and Terminal Bench 2.0. BaT makes the benchmark part of the training system while preserving a content boundary around held-out tasks. Our work makes four contributions: • Benchmark-as-Teacher. BaT turns a structured benchmark into an RSI system that joins diagnosis, content-isolated practice, policy updates, and re-evaluation. • Stage Bank. The asynchronous Stage Bank pipeline synthesizes leakage-checked tasks and exposes targeted, mixed-stage, and E2E surrogate training states. • BiCuRL. BiCuRL couples an outer stage curriculum and checkpoint fallback with inner rubric-verified GRPO updates. • BaT Agents. BaT-4B and BaT-9B more than double their Qwen Instruct baselines, and BaT-9B Agent reaches 79.6 Overall on AutoMedBench-Lite. Figure 2: BaT-9B Agent leads AutoMedBench-Lite and trails the leaders by 9.3 points on ABRA and 14.8 points on MedXpertQA-Text. It scores 79.6 on AutoMedBench-Lite, 70.6 on ABRA, and 50.2 on MedXpertQA-Text. The strongest non-BaT systems score 77.5, 79.9, and 65.0, respectively. The legend shortens BaT-4B Agent and BaT-9B Agent to BaT-4B and BaT-9B. Each panel uses its benchmark’s own protocol, and each system keeps the execution setting shown in the legend (15; 20; 54; 38; 39; 9; 8; 32; 1). 2 Benchmark-as-Teacher Benchmark-as-Teacher is an RSI system with two coupled components. The asynchronous Stage Bank pipeline prepares content-isolated practice states. BiCuRL uses benchmark diagnostics to choose among those states, update the policy, retain checkpoints, and return the policy to evaluation. Figure 1 shows this closed loop, and Figure 3 shows the Stage Bank pools. 2.1 Stage Bank Data Factory. Stage Bank builds practice data from public workflow descriptions and benchmark stage contracts. We write a staged benchmark as ℬ=(,,),B=(S,C,V), (1) where S contains the ordered public stages and their boundaries, C contains rubric and evidence contracts, and V is a fixed held-out evaluation. For policy π, the evaluation returns (π)=(M,)V(π)=(M,e), where M is Overall and e contains one score per stage. AutoMedBench-Lite defines five stages: Plan, Setup, Validate, Inference, and Submit (15). Each stage and its rubric provide a shorter learning objective within the complete workflow. The Data Factory synthesizes and validates all practice content outside the policy-update loop. It asks teacher models to write fictional medical-imaging tasks and complete them as multi-turn trajectories. Public workflow descriptions ground the task templates, while Self-Instruct and agent trajectory synthesis provide the data-generation pattern (16; 45; 47). A leakage preflight rejects held-out identifiers, paths, reports, traces, answers, and evaluation-derived metadata before a row enters training. Aggregate recording statistics can guide synthesis, but raw evaluation content never enters a Stage Bank prompt or row. SFT rows. Stage Bank turns each accepted teacher trajectory into single-response slices for the SFT cold start. One row contains the task, stage skill, prior agent turns, and tool observations as context, followed by one teacher response as the training target. This format keeps the multi-turn history while applying loss only to the selected response (47; 5). The resulting SFT data produce the initial policy θ0 _0. RL rows. Stage Bank builds RL data as executable sandboxes for full multi-turn rollouts (34; 18). It reconstructs a fictional task at a public stage boundary and stores the resulting files, tools, and intermediate artifacts as the initial state. A stage sandbox begins at one workflow boundary and carries that stage’s goal, execution procedure, recovery steps, and rubric. An End-to-End (E2E) surrogate chains all five stages in a smaller workflow that supports repeated rollouts. We store SFT slices and RL sandbox rows separately; each RL row attaches an execution environment and a reward contract. Appendix A.1 gives the data construction and E2E surrogate details, and Appendix A.2 gives the multi-turn RL objective. Each verified Stage Bank row has the form zi=(pi,si,ci,i,κi,mi),z_i=(p_i,s_i,c_i,G_i, _i,m_i), (2) where pip_i is the sandbox state, sis_i names a stage or E2E, and cic_i contains rubric items. iG_i contains evidence requirements, κi _i is the stage skill, and mim_i records provenance. A stable state_id links rollouts from the same row. The bank indexes rows by task type, stage, outcome label, and reward source. BiCuRL draws each round from three Stage Bank pools. S-target contains states for the selected weak stage. S-mix contains states from the remaining stages and limits forgetting. E2E contains complete surrogate workflows and preserves cross-stage coordination. The ablation changes only which pools enter training. Table 1: SFT data (a) and RL Stage Bank data (b) across the five workflow stages and the E2E surrogate. SFT rows select one response from a multi-turn teacher trajectory, while RL rows initialize executable sandboxes for multi-turn rollouts (47; 5; 34; 18). We report average turns, average response tokens, and stored-row counts. (a) SFT S-target and S-mix End-to-End Statistic S1 Plan S2 Setup S3 Validate S4 Infer S5 Submit E2E surrogate Turns 1.0 1.0 1.0 1.0 1.0 1.0 Tokens (K) 0.4 1.4 0.3 0.3 0.3 0.4 Count (K) 2.0 2.0 2.0 2.0 2.0 2.0 (b) RL S-target and S-mix End-to-End Statistic S1 Plan S2 Setup S3 Validate S4 Infer S5 Submit E2E surrogate Turns 25.6 40.5 36.7 37.8 28.5 45.5 Tokens (K) 8.4 11.0 10.4 10.8 8.5 12.1 Count (K) 3.7 6.9 5.5 2.9 1.4 1.2 Figure 3: Each BaT round combines all three sandboxes built from clean synthetic Stage Bank. The Data Factory fills the Stage Bank, and the report for the next round only picks the weak stage: E2E trains on the whole unsegmented trajectory, S-target applies the weak-stage rubric, and S-mix blends stage sandboxes from the other stages. No evaluation task content enters training rows. 2.2 Bilevel Curriculum Reinforcement Learning (BiCuRL) BiCuRL is the self-improving post-training method inside BaT. Its outer loop chooses what to practice from held-out stage scores, and its inner loop updates the policy on that curriculum with GRPO (42). At round r, the fixed controller evaluation returns an Overall score MrM_r and stage scores r=(er,s)s∈e_r=(e_r,s)_s . Over the Stage Bank mixture family SBQ_SB, BiCuRL targets the bilevel objective max∈SB _q _SB ℳ(θ^()), ( θ(q) ), (3) s.t. .t. θ^()∈argmaxθGRPO(θ,). θ(q)∈ _θJ_GRPO(θ;q). BiCuRL approximates this objective through alternating updates. Each round runs a finite GRPO block, evaluates the candidate, updates the curriculum, and starts the next block from the retained checkpoint. The controller evaluation and final evaluation use disjoint runs. Only five stage scores and one Overall score leave the controller evaluation; the controller scores and discards its rollouts. 2.2.1 Inner Loop: Rubric-Verified Agentic RL Given curriculum rq_r, BiCuRL samples Stage Bank rows ziz_i and draws K rollouts from policy πθ¯r _ θ_r in sandbox state pip_i. An LLM rubric verifier scores rollout yi,ky_i,k against rubric items cic_i using execution records and artifact evidence xi,kx_i,k (52). The verifier also reports evidence completeness ηi,k∈[0,1] _i,k∈[0,1], the fraction of required evidence confirmed by the rollout and its artifacts: vi,k,ℓ v_i,k, =Verify(ℓ,pi,yi,k,xi,k)∈0,1, =Verify( ;p_i,y_i,k,x_i,k)∈\0,1\, (4) ri,k r_i,k =ηi,k1|ci|∑ℓ∈civi,k,ℓ, = _i,k 1|c_i| _ ∈ c_iv_i,k, , Ai,k A_i,k =ri,k−K−1∑j=1Kri,jstdj(ri,j)+ϵadv. = r_i,k-K^-1 _j=1^Kr_i,jstd_j(r_i,j)+ _adv. Binary rubric decisions avoid free-form score calibration, while evidence completeness lowers rewards for unsupported success claims. Group normalization compares rollouts that share one state and reward contract. GRPO uses the resulting advantages to produce candidate checkpoint θ~r+1 θ_r+1. 2.2.2 Outer Loop: Stage Routing and Checkpoint Fallback The curriculum router reads re_r and writes an auditable report that selects target stage sr⋆s_r . The round curriculum mixes three Stage Bank pools with fixed proportions ρ: qr(z)= q_r(z)= ρtargetqtarget(z∣sr⋆)+ρmixqmix(z∣sr⋆) _target\,q_target(z s_r )+ _mix\,q_mix(z s_r ) (5) +ρE2EqE2E(z). + _E2E\,q_E2E(z). S-target changes with the selected stage, while S-mix and E2E preserve the rest of the workflow. The router also controls fallback to the best retained checkpoint θ⋆θ . A counter crc_r records consecutive score drops, and fallback fires after three drops or a policy shift above threshold τ: cr+1 c_r+1 =cr+1,Mr+1<Mr,0,otherwise, = casesc_r+1,&M_r+1<M_r,\\ 0,&otherwise, cases (6) θ¯r+1 θ_r+1 =θ⋆,cr+1≥3orDKL(πθ~r+1∥πθ⋆)>τ,θ~r+1,otherwise. = casesθ ,&c_r+1≥ 3\;or\;D_KL\! ( _ θ_r+1\,\|\, _θ )>τ,\\ θ_r+1,&otherwise. cases BaT records each round’s Stage Bank states, pool mixture, reward version, controller scores, and retained checkpoint. The next round starts only after every selected state passes the leakage preflight. 2.3 Agent A BaT Agent combines a BiCuRL-trained policy with a fixed OpenHands execution environment (44). This environment includes public stage skills distilled from the training-time LLM prescriptions. The skills restate each stage’s goal, checks, and recovery steps without carrying evaluation content. Inside OpenHands, the agent can edit files, run commands, inspect results, and submit artifacts. We keep this engineering layer fixed across BaT Agent comparisons and exclude it from BiCuRL: the policy is the only part that training changes. 3 Experimental Setting 3.1 Benchmark and Metrics Medical benchmarks. AutoMedBench is a long-horizon benchmark for medical-AI research agents (15). It scores five workflow stages across two difficulty tiers. We use the AutoMedBench-Lite tier, which provides more task-brief support while keeping the same workflow and scoring structure. Our diagnostic evaluation suite contains 7 long-horizon task tracks. For each track, we repeat the task 10 times, giving 70 runs per evaluated system. The tested agents average 33 interaction turns per run. ABRA tests radiology agents inside an OHIF viewer and an Orthanc DICOM server (20). Its 655 tasks span three difficulty tiers and eight task types, and agents use 21 tools for image navigation, annotation, and reporting. MedXpertQA tests expert medical knowledge and reasoning across 17 specialties and 11 body systems (54). We use its Text subset, which contains text-only medical questions. AutoMedBench-Lite and ABRA evaluate full agent systems, while MedXpertQA-Text evaluates text responses. Figure 2 keeps the system setting shown in its legend and supports comparisons within each benchmark panel. Its values are the recorded evaluation aggregates for those system settings under each benchmark’s native scorer. The cited benchmark papers define the tasks and scorers; our evaluations supply the plotted model scores. AutoMedBench-Lite metrics. AutoMedBench-Lite measures both process and outcome. Task scores the submitted result. Agentic scores completion of Plan, Setup, Validate, Inference, and Submit. Overall gives Task and Agentic equal weight and uses their unrounded values. We use Overall as the main summary because a long-horizon agent must complete the workflow and produce a valid result. We report Task and Agentic separately to show whether a change comes from the process or the outcome. We report all three scores on a 0–100 scale with one decimal. We compute score differences from unrounded values and then round each difference to one decimal. The statistical unit is the task track. We average the ten repeats within each track and then average the seven track means. This two-level aggregation avoids treating all 70 runs as independent. Table 2 carries over the source reported uncertainty half-widths across the seven track-level means. 3.2 Models and Baselines We train the Instruct versions of Qwen3.5-4B and Qwen3.5-9B (38; 39). At each size, Figure 4 compares the Qwen Instruct Baseline, supervised fine-tuning (SFT), GRPO, and BiCuRL inside the full BaT loop (33; 42). Baseline always denotes the corresponding Qwen Instruct checkpoint before post-training. SFT reports the cold-start checkpoint alone, GRPO applies group-relative policy optimization with a single final task reward on E2E data, and BiCuRL applies stage-guided post-training after the same SFT cold start. Figure 2 compares systems on AutoMedBench-Lite, ABRA, and MedXpertQA-Text (15; 20; 54). The comparison includes Qwen3.5-4B and Qwen3.5-9B with Claude Code, Gemini 3.1 Pro with Claude Code, and GLM-5 with Claude Code (38; 39; 9; 8). It also includes GPT-5.5 with Codex and Claude Opus 4.6 with Claude Code (32; 1). Each system keeps the execution setting named in the figure. Figure 4 reports policy-level post-training runs, while Figure 2 reports full agent-system runs. 3.3 Training Data and Checks Stage Bank contains 20,299 candidate prompt states that support rubric scoring. A separate content-isolated synthetic E2E source pool contains 1,008 rows. Stage Bank projects that source pool into E2E, S-target, and S-mix sandboxes without copying evaluation content. The 4B and 9B SFT starts use 4,608 and 4,608 rows. Each GRPO group samples 4 continuations from one state. We prepared a 275-row matched ablation pool. Generation-time rules normalize required fields and block known task and path markers. A hard preflight covers every training and validation file, and the rubric judge scores fresh continuations against each row’s rubric. We keep the diagnostic evaluation fixed across rounds and apply the leakage rules described above before every update. We will release the source manifests, synthesis prompts, checker settings, pool mixtures, optimizer settings, and leakage rules with the training data. 4 Results The results connect the BaT system to three claims. The Overall scores of BaT-4B and BaT-9B more than double their Qwen Instruct baselines, BaT-9B Agent leads AutoMedBench-Lite and approaches the leaders on ABRA and MedXpertQA-Text, and external transfer depends on model size. Figure 4 scores trained policies, while Figure 2 compares medical benchmark results. We report these levels separately and avoid direct comparisons between them. 4.1 BiCuRL More Than Doubles Baselines Table 2 and Figure 4 report the completed BaT-4B and BaT-9B policy runs. BaT-4B has 22.9 Overall, compared with 6.1 for the Instruct baseline. BaT-9B has 53.4 Overall, compared with 19.9 for the Instruct baseline. The 9B score is also 21.5 points above GRPO after rounding the displayed means. The figure keeps the recorded protocol for each completed run, so these differences describe the observed scores instead of a matched training ablation. Table 2: BiCuRL more than doubles the Qwen Instruct AutoMedBench-Lite Overall score at both model sizes. Each score is the mean over the seven track-level means, with ± showing the source draft’s reported uncertainty half-width. Bold and underline mark the best and second-best result within each model size. Baseline denotes Qwen3.5 Instruct (38; 39). SFT follows supervised instruction tuning (33), and GRPO follows group-relative policy optimization (42). Ovl. denotes Overall, and Agt. denotes Agentic. Qwen3.5-4B (38) Qwen3.5-9B (39) Training Ovl. Agt. Task Ovl. Agt. Task Baseline 6.1 ± 1.7 12.1 ± 2.0 0.0 ± 1.1 19.9 ± 2.3 11.3 ± 2.0 28.4 ± 2.6 SFT (33) 18.1 ± 2.3 10.8 ± 1.9 25.4 ± 2.5 12.9 ± 2.1 21.6 ± 2.4 4.1 ± 1.5 GRPO (42) 11.4 ± 2.0 15.9 ± 2.2 6.9 ± 1.7 31.9 ± 2.7 39.2 ± 2.8 24.5 ± 2.5 BiCuRL 22.9 ± 2.4 28.8 ± 2.6 17.0 ± 2.2 53.4 ± 3.2 64.7 ± 3.1 42.1 ± 3.2 Figure 4: BaT keeps improving across rounds, while GRPO with one final reward saturates early. Solid step lines track the best BaT checkpoint so far and stars mark the selected checkpoints; dashed curves show GRPO in the matching model color. Takeaway. BaT-4B and BaT-9B reach 22.9 and 53.4 Overall, more than twice their corresponding Instruct baselines. 4.2 BiCuRL Performance across Training Rounds BiCuRL repeats evaluation, stage selection, and post-training over several rounds. Figure 1(c) reports how the Overall score changes across rounds for both model sizes. At both sizes, the retained BiCuRL checkpoint passes the corresponding GRPO score early in training. The best-so-far staircases preserve each accepted gain through round ten even when a later candidate scores lower. The raw round curves fluctuate, which motivates checkpoint retention and fallback in the outer loop. Takeaway. Some candidates score lower than their predecessors; checkpoint retention preserves the best observed Overall score across later rounds. 4.3 BiCuRL Ablation Study Figure 5 compares the S-target, S-mix, and E2E pools defined in Section 2.1. The full three-pool run leads with 53.4 Overall, matching the BaT-9B row in Figure 4. The strongest partial mix, E2E alone, reaches 31.9, a gap of 21.5 points. Among the three runs that each omit one pool, every point estimate trails the full mix by at least 26 points. Figure 5: The full S-target, S-mix, and E2E sandbox mix leads every partial mix. The dot matrix on the left marks which pools enter training (columns 1–3: S-target, S-mix, End-to-End; dark included, pale omitted). Green bars show Agentic, red bars show Task, and gray circles show Overall. Rows from top to bottom: full BaT, E2E+S-mix, E2E+S-target, S-target+S-mix, E2E, S-mix, S-target, and the Qwen3.5-9B Instruct baseline. All scores use a 0–100 scale and one decimal. Takeaway. The full sandbox mix leads every partial mix by at least 21.5 Overall points and each drop-one-pool run by at least 26 points. 4.4 BaT-9B Ranks First among Local LLMs Given the critical privacy requirements in medical research workflows, local deployment is highly desirable. Tiny models, defined as those with fewer than 12B parameters, offer a cost-effective and performant alternative, prompting us to download and test a range of representative open-weight models. Figure 6 compares tiny local LLMs with the same default OpenHands runner and benchmark protocol (44). BaT-4B raises its Qwen3.5-4B backbone from 6.1 to 22.9 Overall. BaT-9B ranks first at 53.4 Overall, leading 2x compared to the second place Gemma 12B (7). For further comparison with top-tier open-source local LLMs, see Table 6. Figure 6: BaT-9B ranks first, and BaT-4B ranks third among 10 tiny local LLMs under the same default runner. Here tiny local LLM is defined as parameter size no more than 12B. Bars report AutoMedBench-Lite Overall scores. Takeaway. BaT moves both Qwen backbones upward, and the 4B policy trails only one larger-total-parameter LLM. 4.5 BaT-9B Leads One Medical Benchmark and Approaches Two Leaders Figure 2 compares systems on three medical benchmarks (15; 20; 54). Each BaT Agent pairs its BiCuRL-trained policy with the fixed OpenHands environment described under Agent. On AutoMedBench-Lite, BaT-9B Agent reaches 79.6, 2.1 points above Claude Opus 4.6 with Claude Code at 77.5 (1). For AutoMedBench-Lite per-track details, Figure 7 in Appendix A.6 presents a preliminary per-track comparison of BaT-4B Agent, BaT-9B Agent, and Claude Opus 4.6 with Claude Code. On ABRA, it reaches 70.6, compared with 79.9 for GPT-5.5 with Codex, a 9.3-point gap (32). On MedXpertQA-Text, it reaches 50.2, compared with 65.0 for Gemini 3.1 Pro with Claude Code, a 14.8-point gap (9). The 4B BaT Agent scores 45.8, 39.1, and 19.2 on the same three benchmarks. At both sizes, each BaT Agent outperforms Qwen3.5 with Claude Code on all three benchmarks (38; 39). Takeaway. BaT-9B Agent leads AutoMedBench-Lite and trails the leaders by 9.3 points on ABRA and 14.8 points on MedXpertQA-Text. 4.6 External Benchmarks Show Scale-Dependent Retention Strong medical results could come with weaker general reasoning or tool use. Table 3 compares each BaT policy with its Instruct baseline on three short-turn reasoning tasks and five long-horizon tasks (21; 41; 2; 35; 24; 30; 22). The 9B policy remains within 3.4–5.8 points of its baseline on the three reasoning benchmarks and improves τ2τ^2-Bench by 5.4 points. It also improves SWE-bench Verified and Terminal Bench 2.0, while BFCL-Parity and GAIA remain below the baseline. The 4B policy scores below its baseline on all eight benchmarks. The pattern links transfer to model size and task type. Table 3: BaT-9B retains most reasoning accuracy and improves three of five long-horizon scores, while BaT-4B declines on all eight benchmarks. Base denotes each Qwen3.5 Instruct checkpoint (38; 39). Delta is BaT minus Base in percentage points; green marks gains and red marks drops. Each pair uses the same benchmark protocol. Qwen3.5-4B (38) Qwen3.5-9B (39) Benchmark Base BaT Base BaT short-turn reasoning tasks AIME 2025 (21) 85.0 56.7 -28.3 88.3 83.6 -4.7 AIME 2026 (21) 89.7 53.3 -36.4 92.5 86.7 -5.8 GPQA-Diamond (41) 76.2 58.1 -18.1 81.7 78.3 -3.4 long-horizon tasks τ2τ^2-Bench (2) 79.9 29.1 -50.8 79.1 84.5 +5.4 BFCL-Parity (35) 62.6 10.6 -52.0 79.4 68.3 -11.1 GAIA (24) 15.2 2.4 -12.8 24.9 21.2 -3.7 SWE-bench Verified (30) 39.4 4.8 -34.6 49.0 51.8 +2.8 Terminal Bench 2.0 (22) 11.2 1.1 -10.1 19.9 21.3 +1.4 Takeaway. BaT-9B stays close to its short-turn reasoning baseline and improves three long-horizon scores; BaT-4B scores lower on all eight tests. 5 Discussion 5.1 Why the Loop Matters BaT uses a benchmark in two linked roles. AutoMedBench-Lite measures the current agent, and its aggregate diagnostics control what the model practices next. Its task content stays outside training rows. Its scores set later training mixtures, which makes the suite a diagnostic evaluation and rules out treating it as an untouched final test. External benchmarks provide a separate transfer check, although their protocols differ. BaT requires benchmarks that expose stable stages, stage-level scores, and rubrics or checks that can score a continuation. The separation between Stage Bank and BiCuRL matters operationally: synthesis and validation can run asynchronously, while the policy-update loop reads only versioned, executable states. 5.2 Limits The current evidence has six limits. First, AutoMedBench-Lite supplies aggregate diagnostic signals, so the suite participates in adaptation even though task content stays outside training rows. This setup provides content isolation, while a separate untouched final test and a measured semantic leakage audit remain future work. Second, the external table combines retained local runs with published baselines, and its local aggregates lack complete per-run manifests. Third, BaT-4B scores below its baseline on all eight external benchmarks, while BaT-9B gains on three of five long-horizon tasks. Fourth, the top-tier comparison evaluates complete agent systems with their named execution settings, so it supports only a system-level claim. 6 Related Work 6.1 Medical and Long-Horizon Agent Benchmarks Long-horizon benchmarks test agents through tools, files, and many-step tasks. Terminal Bench 2.0 tests command-line work in isolated environments (22). The Berkeley Function Calling Leaderboard (BFCL) focuses on function calling (35). IFBench tests unseen instruction rules (36). In healthcare, MedAgentBench evaluates physician-authored EHR tasks in a FHIR environment, HealthAgentBench spans realistic agentic healthcare settings, and AutoMedBench adds stage scores for medical AI development (12; 16; 15). These benchmarks measure agent performance, while the stage structure in AutoMedBench also supports diagnosis. BaT uses public stage structure to build content-isolated training sandboxes and choose the focus of the next round. 6.2 Agent Post-Training GRPO compares samples from the same group and avoids a learned value model (42). ReAct joins reasoning with tool actions (49). Process supervision assigns feedback to intermediate reasoning steps, while curriculum learning schedules training examples by difficulty or structure (14; 3). Self-Refine uses model feedback to revise outputs during inference (19). Agent Lightning converts multi-step agent traces into RL transitions with credit assignment (18). Self-Rewarding Language Models and SPIN use iterative model-generated feedback or self-play to improve later checkpoints (50; 6). Frontis-MA1 develops recursive self-improvement for machine-learning engineering through OpenMLE-Gym, OpenMLE-RL, and OpenMLE-Evo (48). BaT instead centers a structured held-out benchmark in the control loop. Public stages define practice targets, public rubrics score rollouts, and aggregate diagnostics change the next round’s Stage Bank mixture. 6.3 Execution Environments and Synthetic Task Generation OpenHands provides a workspace for agents that edit files, run programs, and inspect results (44). SWE-World studies software-agent training without full Docker execution (43). SWE-Gym supplies executable software-engineering environments for training agents and verifiers, while VerlTool provides modular tool-use RL infrastructure with asynchronous rollout support (34; 11). On the data side, Self-Instruct bootstraps instruction data from model-written tasks, and later agent pipelines scale this recipe to tool-use trajectories (45). These lines supply the runtime and the raw material for agent training, but they leave open which experience the agent should practice next. BaT combines both: Stage Bank writes leakage-checked synthetic tasks in the Self-Instruct spirit, executable sandboxes support training, and a fixed OpenHands layer runs the trained policy. 7 Conclusion Benchmark-as-Teacher turns public benchmark stages and checks into an RSI system. Its asynchronous Stage Bank pipeline builds content-isolated E2E, S-target, and S-mix states, while BiCuRL uses aggregate diagnostics to select a weak stage and update the policy. The data boundary keeps task-specific evaluation content outside training while aggregate scores guide routing. BaT-4B and BaT-9B more than double their corresponding Qwen Instruct Overall scores on AutoMedBench-Lite. BaT-9B Agent reaches 79.6 Overall and exceeds Claude Opus 4.6 with Claude Code by 2.1 points. On external benchmarks, the 9B policy stays within 3.4–5.8 points on AIME and GPQA-Diamond and gains on three of five long-horizon tasks; the 4B policy scores lower on all eight tests. Together, Stage Bank, BiCuRL, and the fixed BaT Agent execution layer show how a structured benchmark can serve as both evaluator and post-training teacher. Appendix A Technical Details A.1 Stage Bank Construction Row schema. Stage Bank is a versioned set Z of rows. Each row is a tuple z=(p,s,c,,κ,m),z=(p,\,s,\,c,\,G,\,κ,\,m), (7) where s∈∪E2Es ∪\E2E\ keys the row to a stage or the end-to-end pool, and p stores the executable sandbox state. The public benchmark rubric supplies item set c, and G lists the evidence requirements used to compute ηi,k _i,k (15). κ stores the attached stage skill, and m records row provenance. The three pools in Equation 5 are target(sr⋆)=z:s=sr⋆Z_target(s_r )=\z:s=s_r \, mix(sr⋆)=z:s∈∖sr⋆Z_mix(s_r )=\z:s \s_r \\, and E2E=z:s=E2EZ_E2E=\z:s=E2E\. Teacher trajectories and SFT slices. For fictional task ξi _i, a teacher model produces a checked multi-turn trajectory τiT=((,,,,,))t=1Ti, _i^T= ((a_i,t^T,o_i,t^T,s_i,t) )_t=1^T_i, (8) where ai,tTa_i,t^T is one teacher response, oi,tTo_i,t^T is the environment observation that follows it, and si,ts_i,t is the public workflow stage. The history hi,tTh_i,t^T contains the task, stage skill, and all earlier responses and observations. Stage Bank converts the trajectory into single-response rows SFT=(hi,tT,ai,tT):1≤t≤Ti,ξiandτiTpass validation.D_SFT= \ (h_i,t^T,a_i,t^T ):1≤ t≤ T_i,\; _i\ and\ _i^T\ pass validation \. (9) Each row keeps the multi-turn history but applies loss only to the selected teacher response (47; 5): ℒSFT(θ)=−(h,a)∼SFT[∑j=1|a|logπθ(aj∣h,a<j)].L_SFT(θ)=-E_(h,a) _SFT [ _j=1^|a| _θ(a_j h,a_<j) ]. (10) Stage Bank excludes tool observations and other environment text from the loss. Stage sandbox construction. Let bi,sb_i,s denote the turn where stage s begins in τiT _i^T. A stage sandbox for s replays the trajectory prefix inside the execution environment: pi,s=Φξi((ai,tT,oi,tT)t<bi,s),p_i,s= _ _i ((a_i,t^T,o_i,t^T)_t<b_i,s ), (11) where Φξi _ _i executes the prefix and materializes the resulting files, environment, and intermediate artifacts as the row’s start state. Training therefore begins at the stage boundary with an upstream context, while the row rubric scores only the work of stage s. E2E surrogate construction. Stage Bank first applies a reduction operator to the fictional task: ξ~i=R(ξi),=(λcase,λinput,λvalidate). ξ_i=R_ λ( _i), λ=( _case, _input, _validate). (12) R_ λ retains λcase∈1,…,Ni _case∈\1,…,N_i\ of the task’s NiN_i cases, caps each input at a fraction λinput∈(0,1] _input∈(0,1] of its source size, and permits at most λvalidate∈ℕ+ _validate _+ validation passes. The Data Factory chooses λ from a versioned task-specific grid and records it in provenance mim_i. The reduction keeps the five-stage order, task semantics, and output schema. The teacher then executes the reduced task and produces a checked trajectory τ~iT=((,,,,,))q=1T~i, τ_i^T= (( u_i,q^T, o_i,q^T, s_i,q) )_q=1 T_i, (13) where T~i T_i is the number of interaction turns after reduction. The E2E sandbox starts before the first stage, piE2E=Φξ~i(∅)p_i^E2E= _ ξ_i( ), and attaches the ordered skills (κi,s)s∈( _i,s)_s . Its rubric and evidence contracts combine the stage contracts: ciE2E=⋃s∈ci,s,iE2E=⋃s∈i,s.c_i^E2E= _s c_i,s, _i^E2E= _s G_i,s. (14) Stage Bank accepts the surrogate only when the sandbox starts, the checked teacher trajectory fits the rollout limit H, every stage contract passes, and the leakage scan returns zero: Accept(ziE2E)= (z_i^E2E)= [Boot(piE2E)=1][T~i≤H] [Boot(p_i^E2E)=1]I[ T_i≤ H] (15) ×∏s∈[Checks(τ~iT)=1] × _s I[Check_s( τ_i^T)=1] ×[Leak(ziE2E)=0]. ×I[Leak(z_i^E2E)=0]. The accepted row preserves the full workflow while keeping each rollout small enough for repeated training. A.2 Multi-Turn Supervision in BiCuRL Each rollout yi,ky_i,k is a multi-turn interaction. At turn q, the policy emits response ui,k,qu_i,k,q conditioned on the sandbox state, stage skill, and interaction history, after which the sandbox returns observation oi,k,qo_i,k,q. The rubric verifier scores the completed rollout, so one rollout-level reward must reach every token that produced it. For Qi,kQ_i,k interaction turns, the policy and environment generate Pθ(yi,k,oi,k∣zi)=∏q=1Qi,kπθ(ui,k,q∣hi,k,q)Pℰ(oi,k,q∣hi,k,q,ui,k,q).P_θ(y_i,k,o_i,k z_i)= _q=1^Q_i,k _θ(u_i,k,q h_i,k,q)P_E(o_i,k,q h_i,k,q,u_i,k,q). (16) After tokenizing and joining the Qi,kQ_i,k policy responses, we write yi,k=(ai,k,1,…,ai,k,Ti,k)y_i,k=(a_i,k,1,…,a_i,k,T_i,k) for the generated tokens and exclude observation tokens from the loss. Let hi,k,ttokh^tok_i,k,t be the full token context before ai,k,ta_i,k,t; it contains pip_i, κi _i, earlier generated tokens, and every observation returned before that token. For each generated token, the importance ratio compares the updated policy with the policy that produced the rollout: wi,k,t=πθ(ai,k,t∣hi,k,ttok)πθ¯r(ai,k,t∣hi,k,ttok).w_i,k,t= _θ(a_i,k,t h^tok_i,k,t) _ θ_r(a_i,k,t h^tok_i,k,t). (17) Each token receives rollout advantage Ai,kA_i,k from Equation 4, clipped within ϵε: gi,k,t=min(wi,k,tAi,k,clip(wi,k,t,−ϵ,+ϵ)Ai,k).g_i,k,t= \! (w_i,k,tA_i,k,\;clip (w_i,k,t,1\!-\!ε,1\!+\!ε )A_i,k ). (18) The inner objective of Equation 3 averages these token terms over each rollout and group, with a KL penalty toward the current policy (42): GRPO(θ,r)=z∼r[1K∑k=1K1Ti,k∑t=1Ti,kgi,k,t]−βDKL(πθ∥πθ¯r). splitJ_GRPO(θ;q_r)&=E_z _r [ 1K _k=1^K 1T_i,k _t=1^T_i,kg_i,k,t ]\\ & -β D_KL\! ( _θ\,\|\, _ θ_r ). split (19) Each generated token of rollout yi,ky_i,k receives Ai,kA_i,k, so one group-normalized and evidence-discounted rubric decision supervises every turn. The curriculum rq_r decides which stage those trajectories practice. A.3 Migration to Other Staged Benchmarks BiCuRL uses the staged benchmark ℬ=(,,)B=(S,C,V) defined in Section 2.1. A compatible benchmark supplies ordered stage boundaries in S, public rubric and evidence contracts in C, and fixed evaluation (π)=(M,)V(π)=(M,e). Proposition 1. Suppose Stage Bank can materialize executable states for every stage in S. If policy πθ _θ exposes the token likelihoods required by GRPO and policy KL, the BiCuRL update is well-defined for ℬB. Argument. The router reads only re_r and selects sr⋆∈s_r . Equation 5 partitions Stage Bank by stage key, with S-mix drawing the remaining stages. Stage Bank construction uses the assumed stage boundaries, executable states, and public rubric contracts. Equations 4 and 19 use rubric items and evidence requirements regardless of the number or meaning of stages. Equation 6 uses the scalar score sequence and the assumed policy KL. The likelihood assumption also defines the GRPO ratios in Equation 17. These inputs define every term in the update. □ This proposition establishes procedural compatibility. Whether a migrated BaT system improves a particular benchmark remains an empirical question, consistent with the bounded transfer results in the main text. A.4 Post-Training and Runtime Details Cold-start SFT. We train the Qwen3.5-9B cold-start policy with full-parameter supervised fine-tuning on 4,608 SFT rows (39; 33). Training uses eight GPUs with PyTorch Fully Sharded Data Parallel (FSDP) full_shard and fused AdamW (51; 17). Table 4 reports the key configuration. BiCuRL uses the final archived SFT checkpoint as its initializer. Parameter Qwen3.5-9B cold SFT Parameter update Full parameters, without adapters Parallelism 8 GPUs, FSDP full_shard Per-GPU microbatch 1 Gradient accumulation 8 Effective global batch 64 Learning rate 2×10−52× 10^-5 Schedule / warmup Cosine / 3% Optimizer Fused AdamW Weight decay / max grad norm 0 / 1 Target training budget 200 steps Checkpoint interval 5 steps SFT rows / evaluation 4,608 / disabled Seed 42 Table 4: Qwen3.5-9B cold-start SFT configuration. We use full-parameter training with PyTorch FSDP and AdamW (39; 51; 17). Pool weights and sampling. Each round draws rows from the three Stage Bank pools with proportions (ρtarget,ρmix,ρE2E)=(1/2,1/4,1/4)( _target, _mix, _E2E)=(1/2,1/4,1/4). Within E2E, Stage Bank draws surrogate rows uniformly. Sampling is without replacement inside one round and resets between rounds. Matched ablation runs draw from the 275-row pool described in Section 3 and renormalize proportions after omitting a pool. Fallback KL threshold. The policy-shift trigger in Equation 6 uses threshold τ=0.1τ=0.1. We estimate it as the mean per-token KL divergence between candidate policy πθ~r+1 _ θ_r+1 and retained policy πθ⋆ _θ over 256 Stage Bank prompts held out from training. The patience trigger uses three consecutive score drops. Serving stack and throughput. We run training and agent rollouts on eight NVIDIA A100 GPUs with 80GB memory per GPU (25). The serving layer supports SGLang and vLLM (53; 13). We use SGLang by default for agentic tasks and set the serving context limit to 256K tokens. Matched ablations cap each training sequence at 12,288 tokens, as Table 8 records. The source run summary records one node-level throughput value per model on the eight-GPU node: 320 tokens/s for Qwen3.5-4B and 220 tokens/s for Qwen3.5-9B. The summary combines prefill and decoding and omits the remaining serving controls, so we treat both rates as descriptive measurements. Runtime setting Qwen3.5-4B Qwen3.5-9B GPUs 8 × NVIDIA A100 80GB Default engine SGLang Supported engine vLLM Context limit 256K tokens Recorded throughput 320 tokens/s 220 tokens/s Table 5: Serving configuration and recorded throughput. The stack supports SGLang and vLLM, with SGLang as the default engine for agentic tasks (53; 13). Each recorded rate uses an eight-GPU NVIDIA A100 80GB node (25). The source run summary reports an unsplit token rate and omits the remaining serving controls. A.5 Local LLM Comparison Local deployment helps keep medical data within the research environment. We define local LLMs as models with fewer than 40B parameters that can run on a single NVIDIA A100. We downloaded and deployed 19 representative open-weight models, then evaluated them with the same default runner and benchmark protocol (44). The default runner alone produces these results; the BaT agent harness plays no role. Table 6 shows that BaT-9B achieves 53.4 Overall and ranks third. The two higher-ranked models have 27B and 35B total parameters, which suggests that model scale contributes to the remaining gap. Future work will study how BaT scales to larger LLM backbones. Model Overall Agentic Task Qwen3.5-4B (38) 6.1 12.1 0.0 Meissa-4B (4) 7.6 15.2 0.0 Gemma 4 E2B (7) 9.0 18.0 0.0 Nemotron 3 Nano 4B (28) 9.1 18.2 0.0 GPT-OSS-20B (31) 17.6 25.3 9.9 Nemotron 3 Nano 30B-A3B (26) 18.0 22.5 13.5 Nemotron Nano 9B v2 (27) 18.7 24.6 12.8 Qwen3.5-9B (39) 19.9 11.3 28.4 Gemma 4 E4B (7) 21.9 24.2 19.6 BaT-4B 22.9 28.8 17.0 Gemma 4 12B (7) 23.1 20.2 26.0 ClinSeek-35B (46) 26.0 39.7 12.3 Gemma 4 31B (7) 30.4 29.0 31.8 Gemma 4 26B-A4B (7) 33.9 33.8 34.0 Muse-Glimmer-30B (23) 36.5 44.0 29.0 Nemotron 3.5 Lightning 30B-A3B (29) 39.7 47.0 32.4 BaT-9B 53.4 64.7 42.1 Qwen3.5-35B-A3B (37) 57.9 68.8 47.0 Qwen3.6-27B (40) 65.3 73.3 57.3 Table 6: Full local LLM comparison with the default runner. Scores use the AutoMedBench-Lite evaluation (15) and a 0–100 scale, ranked by Overall score. The two shaded rows are BaT policies. Overall equals the arithmetic mean of Agentic and Task scores. A.6 AutoMedBench-Lite Protocol AutoMedBench-Lite applies the five-stage workflow to seven held-out medical AI tracks (15). Table 7 distinguishes each track’s public case pool from the ten separate agent executions used to evaluate a system on that track. Track Held-out task Cases Runs Classification Skin-lesion classification 100 10 Detection Wrist radiograph detection 100 10 Enhancement Low-dose CT denoising 20 10 Report Chest X-ray reporting 100 10 Segmentation Multi-organ CT segmentation 40 10 Synthesis Pancreas CT synthesis 20 10 VQA Multimodal medical VQA 2,005 10 Table 7: AutoMedBench-Lite spans seven held-out medical AI tracks and 70 runs per system. Cases report the public evaluation subsets, and each system runs every track ten times. Agentic weights Plan, Setup, Validate, Inference, and Submit by 0.25/0.15/0.35/0.15/0.10; Overall averages Task and Agentic (15). Each run yields a Task score for its submitted result and stage scores for Plan, Setup, Validate, Inference, and Submit. Agentic combines the five stage scores with the weights in the table caption, and Overall gives Task and Agentic equal weight. Aggregation and uncertainty. The statistical unit is the track. For each system, we first average each metric over the ten repeats of a track and then report the mean of the seven track scores. We treat the seven track means as the independent units because runs within a track share task content. We pair comparisons between two systems at the track level. The controller evaluation during training and the final evaluation share these seven tracks but use disjoint runs; controller-round scores never enter the reported tables. Figure 7: Per-track Overall comparison of the BaT Agents and Claude Opus 4.6 with Claude Code on AutoMedBench-Lite. Bars show the mean Overall score per track on a 0–100 scale. We order tracks left to right by ascending average output tokens per run, from the lightest question-answering tracks to the heaviest imaging pipelines. We estimate the Opus enhancement and report values from its published aggregate scores and measure the remaining values. A.7 Matched Ablation Contract The matched pool-ablation contract changes one variable: which of S-target, S-mix, and E2E enters GRPO. Table 8 records the controls for the 275-row matched pool. Parameter Recorded setting Initializer Qwen3.5-9B SFT checkpoint Train minibatch 8 / 8 Rollouts per prompt 4 Actor learning rate 1×10−61× 10^-6 Actor KL coefficient 1×10−31× 10^-3 Rollout temperature 0.7 Prompt / response limit 8,192 / 2,048 tokens Model context / max sequences 12,288 / 16 Training seed 42 Optimizer between rounds Reset Adaptive schedule 10 rounds × 50 steps Final evaluation 7 tracks × 10 repeats Table 8: Recorded controls for the matched Qwen3.5-9B pool-ablation contract. A.8 External Benchmark Protocol We evaluate the Instruct baselines and BiCuRL policies on eight external benchmarks. AIME 2025 and AIME 2026 test competition-math reasoning with integer-answer problems (21). GPQA-Diamond tests graduate-level biology, physics, and chemistry questions (41). τ2τ^2-Bench tests multi-turn agents in environments where the agent and user can both take actions (2). BFCL-Parity tests function calling, GAIA tests general assistants, SWE-bench Verified tests software issue resolution, and Terminal Bench 2.0 tests command-line work (35; 24; 30; 22). Within each row, Base and BiCuRL use the same evaluation setting. We compute deltas as BiCuRL minus Base in percentage points. References Anthropic (2026) Anthropic Claude opus 4.6 system card. Note: Official model system card External Links: Link Cited by: Figure 2, §3.2, §4.5. Barres et al. (2025) V. Barres, H. Dong, S. Ray, X. Si, and K. Narasimhan τ2τ^2-Bench: evaluating conversational agents in a dual-control environment. External Links: 2506.07982, Link Cited by: §A.8, §4.6, Table 3. Bengio et al. (2009) Y. Bengio, J. Louradour, R. Collobert, and J. Weston Curriculum learning. In Proceedings of the 26th Annual International Conference on Machine Learning, p. 41–48. External Links: Document Cited by: §6.2. Chen et al. (2026) Y. Chen, X. Bai, Y. Pan, Z. Zhou, and A. Yuille Meissa: multi-modal medical agentic intelligence. arXiv preprint arXiv:2603.09018. External Links: Link Cited by: Table 6. Chen and Yuille (2026) Y. Chen and A. Yuille Agentic-DPO: from imitation to agentic policy optimization on expert trajectories. External Links: 2607.10601, Link Cited by: §A.1, §2.1, Table 1. Chen et al. (2024) Z. Chen, Y. Deng, H. Yuan, K. Ji, and Q. Gu Self-play fine-tuning converts weak language models to strong language models. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, p. 6621–6642. External Links: Link Cited by: §1, §6.2. Gemma Team (2026) Gemma Team Gemma 4 Technical Report. arXiv preprint arXiv:2607.02770. External Links: Link Cited by: Table 6, Table 6, Table 6, Table 6, Table 6, §4.4. GLM-5 Team (2026) GLM-5 Team GLM-5: from vibe coding to agentic engineering. External Links: 2602.15763, Link Cited by: Figure 2, §3.2. Google DeepMind (2026) Google DeepMind Gemini 3.1 Pro model card. Note: Official model card External Links: Link Cited by: Figure 2, §3.2, §4.5. Gunjal et al. (2026) A. Gunjal, A. Wang, E. Lau, V. Nath, Y. He, B. Liu, and S. M. Hendryx Rubrics as rewards: reinforcement learning beyond verifiable domains. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: Figure 1. Jiang et al. (2025a) D. Jiang, Y. Lu, Z. Li, Z. Lyu, P. Nie, H. Wang, A. Su, H. Chen, K. Zou, C. Du, et al. VerlTool: towards holistic agentic reinforcement learning with tool use. External Links: 2509.01055, Link Cited by: §6.3. Jiang et al. (2025b) Y. Jiang, K. C. Black, G. Geng, D. Park, J. Zou, A. Y. Ng, and J. H. Chen MedAgentBench: a realistic virtual EHR environment to benchmark medical LLM agents. External Links: 2501.14654, Link Cited by: §1, §6.1. Kwon et al. (2023) W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. E. Gonzalez, H. Zhang, and I. Stoica Efficient memory management for large language model serving with PagedAttention. In Proceedings of the 29th Symposium on Operating Systems Principles, p. 611–626. External Links: Document, Link Cited by: §A.4, Table 5. Lightman et al. (2023) H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe Let’s verify step by step. External Links: 2305.20050, Link Cited by: §6.2. Liu et al. (2026a) J. Liu, S. Song, Y. Wang, J. Mao, H. Chen, X. Huang, T. Qi, P. Guo, Y. Tang, Y. He, C. Zhao, A. Myronenko, D. Yang, D. Xu, and Y. Zhou AutoMedBench: towards medical autoresearch with agentic ai models. External Links: 2606.01961, Link Cited by: §A.1, §A.6, Table 6, Table 7, Figure 2, §1, §1, §2.1, §3.1, §3.2, §4.5, §6.1. Liu et al. (2026b) Q. Liu, S. Zhang, G. Qin, J. M. J. Valanarasu, M. Rokuss, M. Lu, T. Ossowski, J. M. Z. Chaves, C. Wong, P. Argaw, Y. Hasija, M. Wei, W. Yim, Q. Liu, Z. Jing, J. Entenmann, N. Usuyama, T. Naumann, and H. Poon HealthAgentBench: a unified benchmark suite of realistic agentic healthcare environments for challenging frontier AI agents. External Links: 2606.31179, Link Cited by: §1, §2.1, §6.1. Loshchilov and Hutter (2019) I. Loshchilov and F. Hutter Decoupled weight decay regularization. In International Conference on Learning Representations, External Links: Link Cited by: §A.4, Table 4. Luo et al. (2025) X. Luo, Y. Zhang, Z. He, Z. Wang, S. Zhao, D. Li, L. K. Qiu, and Y. Yang Agent lightning: train ANY AI agents with reinforcement learning. External Links: 2508.03680, Link Cited by: §2.1, Table 1, §6.2. Madaan et al. (2023) A. Madaan, N. Tandon, P. Gupta, S. Hallinan, L. Gao, S. Wiegreffe, U. Alon, N. Dziri, S. Prabhumoye, Y. Yang, S. Gupta, B. P. Majumder, K. Hermann, S. Welleck, A. Yazdanbakhsh, and P. Clark Self-refine: iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, Vol. 36. External Links: Link Cited by: §6.2. Maksudov et al. (2026) B. Maksudov, V. Kurenkov, K. M. Curran, and A. Mileo ABRA: agent benchmark for radiology applications. External Links: 2605.11224, Link Cited by: Figure 2, §1, §3.1, §3.2, §4.5. Mathematical Association of America (2026) Mathematical Association of America MAA invitational competitions: american invitational mathematics examination. Note: Official competition documentation External Links: Link Cited by: §A.8, §4.6, Table 3, Table 3. Merrill et al. (2026) M. A. Merrill, A. G. Shaw, N. Carlini, B. Li, H. Raj, I. Bercovich, L. Shi, et al. Terminal-bench: benchmarking agents on hard, realistic tasks in command line interfaces. External Links: 2601.11868, Link Cited by: §A.8, §4.6, Table 3, §6.1. Meta (2026) Meta Muse-Glimmer-30B. Note: Official model card External Links: Link Cited by: Table 6. Mialon et al. (2024) G. Mialon, C. Fourrier, C. Swift, T. Wolf, Y. LeCun, and T. Scialom GAIA: a benchmark for general AI assistants. In International Conference on Learning Representations, External Links: Link Cited by: §A.8, §4.6, Table 3. NVIDIA (2022) NVIDIA NVIDIA A100 Tensor Core GPU data sheet. Note: Product data sheet External Links: Link Cited by: §A.4, Table 5. NVIDIA (2025a) NVIDIA Nemotron 3 Nano: open, efficient mixture-of-experts hybrid mamba-transformer model for agentic reasoning. Note: Technical report External Links: Link Cited by: Table 6. NVIDIA (2025b) NVIDIA NVIDIA-Nemotron-Nano-9B-v2. Note: Official model card External Links: Link Cited by: Table 6. NVIDIA (2026a) NVIDIA NVIDIA-Nemotron-3-Nano-4B-BF16. Note: Official model card External Links: Link Cited by: Table 6. NVIDIA (2026b) NVIDIA NVIDIA-Nemotron-3.5-Lightning-30B-A3B-BF16. Note: Official model card External Links: Link Cited by: Table 6. OpenAI (2024) OpenAI Introducing SWE-bench Verified. Note: Benchmark release External Links: Link Cited by: §A.8, §4.6, Table 3. OpenAI (2025) OpenAI Gpt-oss-120b & gpt-oss-20b model card. External Links: 2508.10925, Link Cited by: Table 6. OpenAI (2026) OpenAI Introducing GPT-5.5. Note: Official model release External Links: Link Cited by: Figure 2, §3.2, §4.5. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. L. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, External Links: Link Cited by: §A.4, §3.2, Table 2, Table 2. Pan et al. (2024) J. Pan, X. Wang, G. Neubig, N. Jaitly, H. Ji, A. Suhr, and Y. Zhang Training software engineering agents and verifiers with SWE-Gym. External Links: 2412.21139, Link Cited by: §2.1, Table 1, §6.3. Patil et al. (2025) S. G. Patil, H. Mao, F. Yan, C. C. Ji, V. Suresh, I. Stoica, and J. E. Gonzalez The berkeley function calling leaderboard (BFCL): from tool use to agentic evaluation of large language models. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 48371–48392. External Links: Link Cited by: §A.8, §4.6, Table 3, §6.1. Pyatkin et al. (2025) V. Pyatkin, S. Malik, V. Graf, H. Ivison, S. Huang, P. Dasigi, N. Lambert, and H. Hajishirzi Generalizing verifiable instruction following. External Links: 2507.02833, Link Cited by: §6.1. Qwen Team (2026a) Qwen Team Qwen3.5-35B-A3B. Note: Official model card External Links: Link Cited by: Table 6. Qwen Team (2026b) Qwen Team Qwen3.5-4B. Note: Official model card External Links: Link Cited by: Table 6, Figure 2, §3.2, §3.2, §4.5, Table 2, Table 2, Table 3, Table 3. Qwen Team (2026c) Qwen Team Qwen3.5-9B. Note: Official model card External Links: Link Cited by: §A.4, Table 4, Table 6, Figure 2, §3.2, §3.2, §4.5, Table 2, Table 2, Table 3, Table 3. Qwen Team (2026d) Qwen Team Qwen3.6-27B: flagship-level coding in a 27b dense model. Note: Official model card External Links: Link Cited by: Table 6. Rein et al. (2023) D. Rein, B. L. Hou, A. C. Stickland, J. Petty, R. Y. Pang, J. Dirani, J. Michael, and S. R. Bowman GPQA: a graduate-level google-proof Q&A benchmark. External Links: 2311.12022, Link Cited by: §A.8, §4.6, Table 3. Shao et al. (2024) Z. Shao, P. Wang, Q. Zhu, R. Xu, J. Song, X. Bi, H. Zhang, M. Zhang, Y. K. Li, Y. Wu, and D. Guo DeepSeekMath: pushing the limits of mathematical reasoning in open language models. External Links: 2402.03300, Link Cited by: §A.2, Figure 1, §1, §2.2, §3.2, Table 2, Table 2, §6.2. Sun et al. (2026) S. Sun, H. Song, L. Huang, J. Jiang, R. Le, Z. Lv, Z. Chen, Y. Hu, W. Luo, W. X. Zhao, Y. Song, H. Xu, T. Zhang, and J. Wen SWE-world: building software engineering agents in docker-free environments. External Links: 2602.03419, Link Cited by: §6.3. Wang et al. (2025) X. Wang, B. Li, Y. Song, F. F. Xu, X. Tang, M. Zhuge, J. Pan, Y. Song, B. Li, J. Singh, H. H. Tran, F. Li, R. Ma, M. Zheng, B. Qian, Y. Shao, N. Muennighoff, Y. Zhang, B. Hui, J. Lin, R. Brennan, H. Peng, H. Ji, and G. Neubig OpenHands: an open platform for AI software developers as generalist agents. In International Conference on Learning Representations, External Links: Link Cited by: §A.5, §2.3, §4.4, §6.3. Wang et al. (2023) Y. Wang, Y. Kordi, S. Mishra, A. Liu, N. A. Smith, D. Khashabi, and H. Hajishirzi Self-instruct: aligning language models with self-generated instructions. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 13484–13508. External Links: Document, Link Cited by: §2.1, §6.3. Wu et al. (2026) J. Wu, L. Zhang, Y. Wang, H. Tu, H. Chen, Z. Wang, C. Xie, and Y. Zhou ClinSeekAgent: automating multimodal evidence seeking for agentic clinical reasoning. arXiv preprint arXiv:2605.20176. External Links: Link Cited by: Table 6. Xu et al. (2025) Y. Xu, D. Lu, Z. Shen, J. Wang, Z. Wang, Y. Mao, C. Xiong, and T. Yu AgentTrek: agent trajectory synthesis via guiding replay with web tutorials. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §A.1, §2.1, §2.1, Table 1. Yang et al. (2026) J. Yang, C. Jiang, Y. Fu, T. Luo, C. Ren, W. Wang, K. Zhao, H. Liu, Y. Zuo, Y. Wang, Y. Fan, K. Tian, Z. Yuan, X. Lin, L. Sheng, R. Qiang, G. Jia, X. Lv, E. Hua, D. Lei, Y. Sun, N. Ding, B. Zhou, and K. Zhang Frontis-MA1: training an AI4AI model towards recursive self-improvement in machine learning engineering. External Links: 2607.28568, Link Cited by: §6.2. Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations, External Links: Link Cited by: §6.2. Yuan et al. (2024) W. Yuan, R. Y. Pang, K. Cho, X. Li, S. Sukhbaatar, J. Xu, and J. E. Weston Self-rewarding language models. In Proceedings of the 41st International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 235, p. 57905–57923. External Links: Link Cited by: §1, §6.2. Zhao et al. (2023) Y. Zhao, A. Gu, R. Varma, L. Luo, C. Huang, M. Xu, L. Wright, H. Shojanazeri, M. Ott, S. Shleifer, A. Desmaison, C. Balioglu, P. Damania, B. Nguyen, G. Chauhan, Y. Hao, A. Mathews, and S. Li PyTorch FSDP: experiences on scaling fully sharded data parallel. Proceedings of the VLDB Endowment 16 (12), p. 3848–3860. External Links: Document, Link Cited by: §A.4, Table 4. Zheng et al. (2023) L. Zheng, W. Chiang, Y. Sheng, S. Zhuang, Z. Wu, Y. Zhuang, Z. Lin, Z. Li, D. Li, E. P. Xing, H. Zhang, J. E. Gonzalez, and I. Stoica Judging LLM-as-a-judge with MT-Bench and chatbot arena. External Links: 2306.05685, Link Cited by: §2.2.1. Zheng et al. (2024) L. Zheng, L. Yin, Z. Xie, C. Sun, J. Huang, C. H. Yu, S. Cao, C. Kozyrakis, I. Stoica, J. E. Gonzalez, C. Barrett, and Y. Sheng SGLang: efficient execution of structured language model programs. In Advances in Neural Information Processing Systems, Vol. 37. External Links: Document, Link Cited by: §A.4, Table 5. Zuo et al. (2025) Y. Zuo, S. Qu, Y. Li, Z. Chen, X. Zhu, E. Hua, K. Zhang, N. Ding, and B. Zhou MedXpertQA: benchmarking expert-level medical reasoning and understanding. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 80961–80990. External Links: Link Cited by: Figure 2, §1, §3.1, §3.2, §4.5.