Paper deep dive
TACO: Tool-Augmented Credit Optimization for Agentic Tool Use
Mingkuan Feng, Jinyang Wu, Hao Gu, Fangrui Lv, Ruihan Jin, Chuyuan Zhang, Zhengqi Wen, Jianhua Tao
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 97%
Last extracted: 7/5/2026, 1:30:25 AM
Summary
TACO (Tool-Augmented Credit Optimization) is a GRPO variant designed for agentic multimodal models that use code-based tools. It addresses the limitations of outcome-only rewards by using two coupled advantage channels: Differential Answer-Probe Reward (DAPR), which uses self-supervised probe tokens to measure a tool's specific contribution to correctness, and Outcome-Gated Advantage Routing (OGAR), which routes the final-answer advantage to the specific tokens responsible for the outcome. This prevents redundant or misleading tool calls from receiving undue credit or blame, improving accuracy in perception and reasoning tasks.
Entities (6)
Relation Signals (5)
TACO → incorporates → DAPR
confidence 100% · TACO... built on two coupled advantage channels. The first, Differential Answer-Probe Reward (DAPR)
TACO → incorporates → OGAR
confidence 100% · The second is the outcome advantage from the final answer, distributed by Outcome-Gated Advantage Routing (OGAR)
TACO → isavariantof → GRPO
confidence 100% · TACO, a GRPO variant for code-tool agents
Qwen2.5-VL-7B → istrainedwith → TACO
confidence 100% · We build on Qwen2.5-VL-7B... with 2 epochs of SFT then 1 epoch of GRPO [using TACO]
DAPR → uses → Probe Tokens
confidence 95% · Probe tokens inserted into the model's reasoning elicit its predictions with and without the tool
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Agentic multimodal models perform diverse operations on an image via code and reason over the returned view, an effective paradigm for fine-grained visual question answering. However, code operations can be useful, redundant, or misleading. Outcome-only rewards cannot precisely distinguish these cases, and existing process rewards either fail to attribute final correctness to individual tool calls, or require an external judge model. To address this, we introduce Tool-Augmented Credit Optimization (TACO), a GRPO variant for code-tool agents built on two coupled advantage channels. The first, Differential Answer-Probe Reward (DAPR), is a self-supervised, judge-free tool-contribution advantage that credits each tool call by its own effect on answering correctly. Probe tokens inserted into the model's reasoning elicit its predictions with and without the tool, and the difference in outcome reward is taken as the call's value: positive for a useful call, negative for a misleading one, and zero for one that changes nothing. This reuses the existing answer checker with no auxiliary judge, and, being a difference rather than an absolute probe score, is naturally robust to probe-hacking. The second is the outcome advantage from the final answer, distributed by Outcome-Gated Advantage Routing (OGAR): a parameter-free rule that, conditioned on the call's outcome, delivers this credit only to the responsible segments, suppressing wasted tool calls without any cost term. We train TACO through a two-stage SFT+RL pipeline. Extensive experiments across perception, reasoning, and general multimodal benchmarks show that it yields consistent accuracy gains and learns to invoke its tools only when they help.
Tags
Links
- Source: https://arxiv.org/abs/2606.30251v1
- Canonical: https://arxiv.org/abs/2606.30251v1
Trouble viewing inline? Open PDF directly →
Full Text
82,552 characters extracted from source content.
Expand or collapse full text
TACO: Tool-Augmented Credit Optimization for Agentic Tool Use Mingkuan Feng 1, * , Jinyang Wu 1, * ,† , Hao Gu 2 , Fangrui Lv 1 , Ruihan Jin 1 Chuyuan Zhang 1 , Zhengqi Wen 1 , Jianhua Tao 1 1 Department of Automation, BNRist, Tsinghua University 2 Institute of Automation, Chinese Academy of Sciences Corresponding to: fmk24@mails.tsinghua.edu.cn, wu-jy23@mails.tsinghua.edu.cn Abstract Agentic multimodal models perform diverse operations on an image via code and reason over the returned view, an effective paradigm for fine-grained visual question answer- ing. However, code operations can be useful, redundant, or misleading. Outcome-only rewards cannot precisely distin- guish these cases, and existing process rewards either fail to attribute final correctness to individual tool calls, or require an external judge model. To address this, we introduce Tool- Augmented Credit Optimization (TACO), a GRPO variant for code-tool agents built on two coupled advantage chan- nels. The first, Differential Answer-Probe Reward (DAPR), is a self-supervised, judge-free tool-contribution advantage that credits each tool call by its own effect on answering correctly. Probe tokens inserted into the model’s reasoning elicit its predictions with and without the tool, and the dif- ference in outcome reward is taken as the call’s value: posi- tive for a useful call, negative for a misleading one, and zero for one that changes nothing. This reuses the existing answer checker with no auxiliary judge, and, being a difference rather than an absolute probe score, is naturally robust to probe- hacking. The second is the outcome advantage from the fi- nal answer, distributed by Outcome-Gated Advantage Rout- ing (OGAR): a parameter-free rule that, conditioned on the call’s outcome, delivers this credit only to the responsible seg- ments, suppressing wasted tool calls without any cost term. We train TACO through a two-stage SFT+RL pipeline. Ex- tensive experiments across perception, reasoning, and general multimodal benchmarks show that it yields consistent accu- racy gains and learns to invoke its tools only when they help. 1 Introduction Recent vision–language models go beyond text-only reason- ing to “think with images,” an ability popularized by Ope- nAI o3 (OpenAI 2025). They write and execute code that crops or zooms an image, runs computations, or otherwise transforms the input, then reason over the result (Zhang et al. 2026; Zheng et al. 2026; Hong et al. 2026; Wang et al. 2025a; Lai et al. 2026; Zhao et al. 2026; Wang et al. 2025b; Li et al. 2026a; Yan et al. 2026; Liu, Feng, and Chen 2026). When the decisive detail is too small to read, such a code-tool agent can act to obtain a sharper obser- vation rather than guess (Qi et al. 2026). These agents are * Equal Contribution † Project Leader Example ①:What color is the mounting cap of the ball hanging from the ceiling? It is black. Code: Useful Answer: Right The train is white. Code: Misleading Answer: Wrong The mounting cap is above the ball. I'l now write code to get its color. The train is white and red, I'l now write code to confirm it. Example ②:What color is the train? Figure 1: A visual tool call can help or hurt. Example 1: a crop turns a wrong answer right (useful); Example 2: a crop flips a would-be-correct answer to wrong (misleading). trained with reinforcement learning from verifiable rewards (RLVR), the standard recipe for eliciting reasoning in LLMs and VLMs (Guo et al. 2025; Team et al. 2025, 2026; Wu et al. 2026d). However, code operations do not always help (Ma et al. 2026; Hou et al. 2026). The same crop can turn a wrong an- swer right, leave it unchanged, or turn a right answer wrong (Figure 1). A tool call is therefore useful, inconclusive, or misleading. To teach an agent to do a code operation when it helps, we need a reward that scores each call by its own con- tribution, positive for a useful one and negative for a harm- ful one, and delivers that signal to the tokens that issued the call (Wei et al. 2025; Li et al. 2026b). The difficulty is that most RLVR optimizes the final answer alone, so its reward attaches to the whole trajectory rather than to the call, and may struggle to separate a helpful operation from a wasted or a harmful one (Yoon et al. 2025; Hu et al. 2026). Worse, the signal is confounded: a recent analysis finds that the ac- curacy gains of crop-zoom tool-use RL are driven mostly by the model’s intrinsic improvement rather than by the tool itself (Ma et al. 2026), so a higher final score does not cer- tify that the call helped. Existing process rewards fall short arXiv:2606.30251v1 [cs.MA] 29 Jun 2026 for two different reasons. Step-wise rewards for single-chain text reasoning cannot isolate a tool call’s contribution or flag a harmful one (Wang et al. 2026; Yoon et al. 2025; Wu et al. 2026a); while those defined on a tool’s output need an exter- nal judge model and never ask whether the call changed the answer (Hou et al. 2026). This raises the question we study: can a self-supervised, judge-free signal score the tool call by its own effect on final correctness and deliver that credit only to the tokens responsible for it? We answer this with Tool-Augmented Credit Optimiza- tion (TACO), a GRPO variant for code-tool visual agents built on two ingredients (Figure 2): (i) a tool-call value reward, Differential Answer-Probe Reward (DAPR), and (i) Outcome-Gated Advantage Routing (OGAR) of the final-answer advantage. Our key observation is that a tool call splits the trajec- tory into a clean before and after: the increment between them is the entire tool branch—the code, its observation, and the post-tool reasoning it triggers. We insert two lightweight probes (Figure 2) that read out the agent’s answer just be- fore the tool call (tool-off, a 1 ) and the answer it commits to after the tool call has returned and been reasoned over (tool- on, a 2 ) (Zhang et al. 2026). Scoring both with a rule-based answer checker, the DAPR of the call is their difference: positive for a useful call, negative for a misleading one, and zero when the call changes nothing. Because the two an- swers share the same question, image, and pre-tool reason- ing, this difference cancels what the model “already knew” before invoking the tool and credits the call by how much taking the tool branch changes the answer. DAPR reuses the existing answer checker with no auxiliary judge, and tends to resist probe-hacking. A scalar tool-value is not enough: it must reach the right tokens. A naive implementation lets the final-answer advan- tage land on every token, so a code block that merely co- occurs with a correct answer is rewarded even when it is re- dundant, and a correct chain of pre-tool reasoning is blamed for an answer the tool later spoiled. OGAR fixes this with one principle: a token segment receives the final-answer ad- vantage only when it is responsible for the answer, and re- sponsibility is decided by the call’s outcome. This sorts every call into four regimes (Figure 2): a useful call and a mislead- ing one are credited or penalized on the tool branch (the code andT 2 ); a right-but-redundant call (already correct) has its undue credit withheld so the wasted call is suppressed; and a necessary-but-failed call (still wrong) has its blame with- held so a warranted attempt on a hard item is not discour- aged. The gate is parameter-free and needs no tool-call cost term. We instantiate TACO with a two-stage SFT-then-RL recipe and evaluate across perception, reasoning, and gen- eral multimodal benchmarks, where it delivers consistent ac- curacy gains while learning to invoke its tools only when they help. We summarize our main contributions as follows: • TACO. We introduce TACO, a GRPO variant for code- tool visual agents that couples DAPR and OGAR into a single objective. • Differential Answer-Probe Reward (DAPR). A self- supervised, judge-free, tool-call reward that scores a call by a tool-off/tool-on comparison and assigns negative value to misleading calls, unlike per-text-step gains, at zero API cost. • Outcome-Gated Advantage Routing (OGAR). A parameter-free, token-level rule that routes the final- answer advantage by the call’s outcome, suppressing wasted calls without any tool-call cost term. • Probe-hacking: diagnosis and defense. We expose a failure mode of generative probes, show that the be- fore/after difference is more resistant to it, and verify this in the training reward dynamics. 2 Related Work Thinking with images. Recent work lets multimodal models “think with images,” emitting code that crops, zooms, or transforms the input and reasoning over the returned view. DeepEyes (Zheng et al. 2026), Pixel- Reasoner (Wang et al. 2025a), and Mini-o3 (Lai et al. 2026) incentivize pixel-space operations with RL; PyVision (Zhao et al. 2026), Thyme (Zhang et al. 2026), and Deep- EyesV2 (Hong et al. 2026) run general image-processing code in a sandbox; MathCoder-VL (Wang et al. 2025b) ex- tends this to math; and Agent0-VL (Liu et al. 2025) pushes toward self-evolving tool use. Trained almost entirely from outcome rewards that credit a whole trajectory rather than an individual call, these agents tend to over-call their tools (Yan et al. 2026). MED (Ma et al. 2026) shows the apparent gain in crop-and-zoom RL is largely confounded by the model’s own improvement, while Zoom-Consistency (Kim and Che- likavada 2026) and RTWI (Li et al. 2026a) read intermedi- ate signals as test-time reliability cues. These observations expose what an outcome reward leaves implicit: whether a given call actually helped. TACO measures this directly, scoring each call by its own effect on the answer as a train- ing reward rather than a test-time signal. A code operation splits the trajectory into a clean before and after that TACO exploits to credit the tool branch itself. Process rewards and credit assignment. A parallel line densifies RL with process rewards on intermediate steps. For single-chain text reasoning, MIG (Wang et al. 2026) uses a watermarked per-step marginal gain, PACR (Yoon et al. 2025) rewards progressively ascending confidence, and SPAE (Wu et al. 2026a) estimates step advantages from in- termediate confidence and correctness. Defined on a tex- tual chain in log-probability space, none isolates an external tool observation or separates a helpful step from a harm- ful one. For multi-turn LLM agents, SIOP (Hu et al. 2026) gives a verifier-free turn-level potential (our soundness an- chor) and HISR (Lu et al. 2026) modulates segmental re- wards with hindsight, but neither targets a single visual call; CodeV (Hou et al. 2026) scores visual tool use but via an ex- ternal GPT-4o judge, adding API cost and inheriting its bi- ases. In contrast, TACO scores credit from the agent’s own outcome reward on two probe answers: the pre-tool probe gives a “what would you answer without the tool” baseline these methods lack, and its signed before/after difference can penalize misleading calls a marginal-gain signal cannot. Image(퐼) Q Question(푞) Policy MLLM Rollout × N Sandbox ... Concate Concate FMT 푅 푓푚푡 Advantage Estimate 퐴 1 퐿 푇퐴퐶푂 =훼 1 ∙퐿 퐺푅푃푂 푚⨀퐴 1 +훼 2 ∙퐿 퐺푅푃푂 푔∙퐴 2 Probe token: </think> <answer> Answer (푎 1 ) 푅(푎 1 ) 푅(푎 2 ) 푟 표푢푡 Answer (푎 2 ) 푟 표푢푡 ∆=푅푎 2 −푅(푎 1 ) Advantage Estimate 퐴 2 표 1 표 2 표 푛 Tool use(Code) Output+Think( ) Think( ) ① ② Final Answer ( 퐴 푓 ) ③ Think ① 푎 1 푐표푟푒푐푡 ∆=0 Right-but-Redundant Ans(퐴 푓 ) ③ Tool ② 푎 2 푐표푟푒푐푡 퐴 1 Think ① 푎 1 푤푟표푛푔 ∆>0 Useful Ans(퐴 푓 ) ③ Tool ② 푎 2 푐표푟푒푐푡 퐴 1 Outcome-Gated Advantage Routing(OGAR) Think ① 푎 1 푤푟표푛푔 ∆=0 Necessary-but-failed Ans(퐴 푓 ) ③ Tool ② 푎 2 푤푟표푛푔 퐴 1 Think ① 푎 1 푐표푟푒푐푡 ∆<0 Misleading Ans(퐴 푓 ) ③ Tool ② 푎 2 푤푟표푛푔 퐴 1 푅(퐴 푓 ) 푟 표푢푡 ∆≥0 Figure 2: Overview of TACO. (a) The accuracy channel A 1 scores the final answer; the process channel A 2 is the before/after probe difference (the tool-call value), gated into the loss only when ∆≥0. (b) OGAR sends A 1 to a segment only when it is responsible for the answer (solid box), gating it out otherwise (dashed box). The unit of credit is thus an action with a real observation, not a token span. 3 Method TACO augments Group Relative Policy Optimization (GRPO) (Shao et al. 2024) for code-tool visual agents with two coupled components (Figure 2). Differential Answer- Probe Reward (DAPR, Sec. 3.2) scores a tool call by the change in outcome reward just before versus just after it. Outcome-Gated Advantage Routing (OGAR, Sec. 3.3) then routes the final-answer advantage to the responsible to- kens through an outcome-conditioned gate. We train in two stages: an SFT cold-start, then GRPO with this gated dual- channel advantage (Sec. 3.4). 3.1 Setting and Notation A code-tool visual agent receives a question q and image I and produces a trajectory that interleaves reasoning with a tool call: it first reasons, then emits code that is executed in a sandbox and returns a visual observation IMG (a crop or zoom of I ), and finally reasons over the result and an- swers (Figure 2). The trajectory is delimited by three spe- cial tokens: <think> for free-form reasoning, <code> for a Python program executed on I , and <answer> for the final answer that ends the trajectory. We write r out (·) ∈ −1, 0, +1 for the verifiable outcome reward, a rule-based answer checker (e.g., string matching against the ground truth y ∗ ): +1 for a correct answer,−1 for an incorrect one, and 0 when no answer is produced. We split each trajectory into three token segments used throughout: the pre-tool rea- soningT 1 (which produces a 1 ), the code tokensC of the call, and the post-tool reasoning with the final answerT 2 (which produces a f ). 3.2 Differential Answer-Probe Reward (DAPR) Around the tool call we insert two lightweight probes that prefill the answer header </think> <answer> and greedily decode a short answer (Figure 2): • Pre-tool probe (tool-off): taken after Think 1 but be- fore the code runs; with context (q,I,Think 1 ), it yields the answer a 1 the agent would give without invoking the tool. • Post-tool probe (tool-on): taken after the tool call has returned and been reasoned over in Think 2 ; the con- text additionally contains the tool view IMG, yielding a 2 . With multiple calls this probe is read after the final call, so a 2 is the answer the agent commits to once its tool branch is complete. Scoring both with r out gives the tool-value r out (a 1 )∈−1, 0, +1, r out (a 2 )∈−1, 0, +1, (1) ∆ = r out (a 2 )− r out (a 1 ).(2) ∆ > 0 marks a useful call (Figure 1, Example 1), ∆ < 0 a misleading one (Example 2), and ∆ = 0 a call that does not change the outcome: either an easy item already correct without the tool or a hard item wrong regardless (OGAR handles these two cases in Sec. 3.3). When the agent emits no code and answers directly, the process chan- nel is empty and only the accuracy channel applies. Comput- ing ∆ needs only two short probe decodes and no API call, unlike CodeV’s per-step GPT-4o judge (Hou et al. 2026). Differencing cancels the pre-tool baseline. a 1 and a 2 share the same question, image, and pre-tool reasoning Think 1 ; the increment between them is the entire tool branch—the codeC, its observation IMG, and the post-tool reasoning Think 2 it triggers. Subtracting therefore cancels what the model “already knew” before invoking the tool (in- cluding any answer it pre-committed to in Think 1 ) and credits the call by how much taking the tool branch changes the answer. By removing this pre-tool baseline, the very term an outcome-only reward leaves in, ∆ directly targets the tool-gain confound noted by MED (Ma et al. 2026). Robustness to probe-hacking. The same cancellation de- fends against a failure mode of generative probes: a probe that truncates Think and appends <answer> can be ex- ploited if the model writes its conclusion early into Think, since the probe simply copies it. But both probes read out from the same Think 1 , so pre-writing inflates r out (a 1 ) as much as r out (a 2 ) and the two gains cancel in ∆: it lifts the agent’s own baseline but leaves ∆ unchanged. A call earns positive ∆ only when taking the tool branch turns a wrong pre-tool answer right. 3.3 Outcome-Gated Advantage Routing (OGAR) OGAR routes the final-answer advantage to the right to- kens, using the tool-value ∆ as a gate and optimized jointly with the process channel (Figure 2), under one principle: each segment is credited only by the outcome it controls. Accuracy channel. The final answer a f is scored by the rule-based answer checker r out (a f ) ∈ −1, 0, +1, plus a format term: R acc = r out (a f ) + 0.5R fmt .(3) Here R fmt rewards output that follows the required <think>/<code>/<answer> structure. Over the group of G rollouts, the accuracy advantage is the standard GRPO normalization of R acc : A (i) 1 = R (i) acc − mean R (j) acc G j=1 std R (j) acc G j=1 .(4) Process channel. The tool-value ∆ yields a single trajectory-level advantage, normalized over the group in the same GRPO fashion as the accuracy channel: A (i) 2 = ∆ (i) − mean ∆ (j) G j=1 std ∆ (j) G j=1 .(5) The process channel acts on the whole sequence, but only for non-misleading calls: for a misleading call (∆<0) we switch it off through a trajectory-level gate g (i) that equals 1 when ∆ (i) ≥0 and 0 otherwise. The penalty is then carried entirely by the gated accuracy channel on the code and T 2 (outcome gate below), which keeps A 1 off T 1 when ∆<0, so the correct pre-tool reasoning T 1 stays unpenalized by both channels. Outcome gate. The final-answer advantage A 1 reaches a segment only when it is responsible for a f , which the call’s outcome ∆ decides. This sorts every call into four regimes (Figure 2): • Right-but-redundant (∆=0, a 1 already correct): the item is answered correctly with or without the call, so A 1 is withheld from the tool branch (the code andT 2 ), and the pre-tool reasoningT 1 that already solved the item keeps the credit. • Misleading (∆<0): the call misleads the correct pre-tool reasoningT 1 , so the A 1 blame falls on the code andT 2 , not onT 1 (Figure 1, Example 2). • Necessary-but-failed (∆=0, a 1 wrong): the A 1 blame is withheld from the whole tool branch (the code and T 2 ), encouraging exploration of tool use on hard items. • Useful (∆>0): the call positively aids the reasoning, so the whole trajectory (T 1 , code, and T 2 ) receives the A 1 credit. We treat the code and the post-tool reasoningT 2 as a single tool branch: whenever the call is responsible for a f they are gated together. We write this per-segment routing of A 1 as the gate m[t] = 1 t∈T 1 and ∆≥ 0, 1 t∈C∪T 2 and ∆̸= 0, 0 otherwise. (6) which masks A 1 to its responsible tokens while A 2 cov- ers the whole sequence whenever it is active (∆≥0). In the ∆=0 case the masked tool-branch tokens still receive A 2 and the format reward, so the answer credit shifts from a non-responsible tool branch to the pre-tool reasoning with- out dropping the signal that produces the answer. Because the two channels act on different tokens and at different scales, they are optimized as separate clipped-GRPO losses (Sec. 3.4) and summed, with m the outcome gate above and g the per-trajectory process gate (g=1 when ∆≥0 and 0 oth- erwise) that switches A 2 off on misleading calls, L TACO = α 1 L GRPO (m⊙ A 1 ) + α 2 L GRPO (g A 2 ). (7) 3.4 Training We optimize TACO in two stages. Stage 1: SFT cold-start. Instruction-tuned VLMs rarely invoke code productively out of the box, and RL from such a start collapses to text-only reasoning (Hou et al. 2026). We therefore supervise-fine-tune on trajectories that interleave reasoning, code, and tool outputs: this teaches the Think– Code–Answer format and provides the reference policy π ref for Stage 2. Stage 2: RL with gated advantages. For each (q,I) we sample a group of G on-policy rollouts and form the gated accuracy advantage m⊙ A 1 and the process advantage A 2 (Sec. 3.3). The importance ratio is r t = π θ (a t | s t ) π θ old (a t | s t ) ,(8) Table 1: Main results across perception, reasoning, and general multimodal benchmarks. All entries are accuracy in percent. Within the code-tool / visual-agent group (including Ours), best per column in bold, second best underlined. † Instruct variant. Model PerceptionReasoningGeneral Avg. HR-4K HR-8K MME-RW V ∗ MVision MVista MVerse WeMath LVista M ∗ ChartQA BLINK Closed-source models GPT-4o65.059.662.867.536.563.435.344.253.2 65.785.763.358.5 GPT-575.374.168.672.561.581.466.777.169.3 73.076.869.572.2 Gemini-2.5-Pro83.981.558.379.139.880.976.978.073.8 73.683.673.573.6 Open-source MLLMs (no visual tool) LLaVA-OneVision-7B63.059.857.475.417.658.619.320.933.3 61.780.048.249.6 Qwen2.5-VL-7B † 68.865.358.376.427.068.235.234.339.8 64.783.756.456.5 InternVL3-8B70.069.361.370.226.370.429.231.745.6 68.585.955.557.0 Qwen2.5-VL-32B † 73.470.461.081.235.272.240.047.154.4 69.181.163.662.4 Qwen3-VL-8B † 78.974.660.386.453.977.262.136.954.9 70.988.169.167.8 Code-tool / visual-agent models MathCoder-VL-8B (ACL’25)73.870.652.977.526.173.646.552.141.6 62.578.851.759.0 Pixel-Reasoner-7B (NIPS’25) 74.066.964.484.326.371.246.936.846.4 64.776.256.659.6 DeepEyes-7B (ICLR’26)75.172.664.185.626.670.147.338.947.7 65.369.457.560.0 Mini-o3-7B (ICLR’26)77.573.365.5 88.225.767.245.434.945.7 63.885.555.260.7 Thyme-7B (ICLR’26)77.072.064.882.227.670.039.139.349.0 65.986.156.160.8 DeepEyes-v2-7B (ICLR’26)77.973.864.981.828.971.952.738.148.7 66.172.257.861.2 CodeV-RL-7B (CVPR’26)76.171.364.584.833.671.849.240.548.7 67.683.558.262.5 PyVision-RL-7B (Arxiv’26)78.174.359.688.728.771.455.847.749.265.686.858.763.7 Ours-7B83.881.666.289.635.876.357.253.155.6 69.386.761.668.1 and the clipped-GRPO surrogate of a per-token advantage A (with A[t] its value at token t) is L GRPO (A) =E τ,t min r t A[t], clip(r t , 1−ε, 1+ε)A[t] . (9) TACO maximizes L TACO (Eq. 7) minus the usual KL penalty βE[D KL (π θ ∥π ref )]. 4 Experiments We describe our data and implementation below, then report the main comparison (Sec. 4.2), an efficiency comparison (Sec. 4.3), component ablations (Sec. 4.4), generalization across base models (Sec. 4.5), and training dynamics with the probe-hacking analysis (Sec. 4.6). 4.1 Experimental Setup SFT data curation. Our SFT data is built on the Thyme SFT corpus (Zhang et al. 2026), whose trajectories inter- leave reasoning, code, and tool outputs. We re-curate it with three filters. (i) Execution validity: we re-run every code block in our sandbox and discard trajectories with execu- tion errors or tool observations/answers inconsistent with the actual output, which otherwise teach the model to hal- lucinate observations. (i) Tool necessity: we drop samples that Qwen2.5-VL-7B (Bai et al. 2025b) already solves with- out tools (pass@8=1), keeping only trajectories where a tool call is genuinely needed. (i) Quality: Gemini-3-Pro scores each trajectory for reasoning coherence and tool-use ratio- nale, and low-quality or blind-tool-use traces are removed. RL data curation. For RL we follow CodeV (Hou et al. 2026), building on its open-source prompt data and adopting its data-cleaning recipe. Keeping only questions with ver- ifiable ground-truth answers, we clean them in two ways. (i) Environmental fidelity: each prompt is passed through Gemini-3-Pro to check image quality, question clarity, and image-text consistency, and prompts with corrupted images or severe ambiguity are removed so the policy does not fit to noise. (i) Difficulty calibration: prompts that our SFT checkpoint already solves on all G=8 rollouts are trivially easy and yield zero-variance accuracy rewards (and thus no GRPO advantage), so we remove them. Implementation details. Following Thyme, we build on Qwen2.5-VL-7B (Bai et al. 2025b), with 2 epochs of SFT followed by 1 epoch of GRPO. The prompt template spec- ifies the <think>, <code>, <answer>, and sandbox- output tokens. We set α 1 =1.0 and α 2 =0.15 (Sec. 3.3), use no KL penalty (β=0), and sample G=8 rollouts per prompt at temperature 1.0, with a total batch size of 128 and learn- ing rate 1×10 −6 . Training runs on a single node of 8×80 GB A100 GPUs. For evaluation we use VLMEvalKit (Duan et al. 2024) with its default protocol across all benchmarks. More details are provided in the Appendix. Table 2: Accuracy vs. end-to-end inference efficiency (7B). For each benchmark we report accuracy (%) and per-question latency in seconds (batch 1 on one A100). The latency gap reflects how many tool/sandbox rounds a policy emits at inference (CodeV’s GPT-4o judge is training-time only). Best accuracy and lowest latency per benchmark in bold. Model V ∗ HR-Bench-4KHR-Bench-8KMathVisionMMStar Acc↑ Latency(s)↓ Acc↑ Latency(s)↓ Acc↑ Latency(s)↓ Acc↑ Latency(s)↓ Acc↑ Latency(s)↓ Thyme-7B (ICLR’26)82.23.077.03.772.04.027.62.865.92.5 DeepEyes-7B (ICLR’26)85.62.875.15.072.65.726.63.765.33.4 DeepEyes-v2-7B (ICLR’26)81.83.177.95.873.85.928.93.466.13.1 CodeV-RL-7B (CVPR’26)84.83.476.15.371.36.133.62.967.62.5 PyVision-RL-7B88.73.678.15.574.36.228.72.965.62.6 Ours-7B89.62.383.83.281.63.535.82.469.32.0 Benchmarks and baselines. We evaluate on twelve benchmarks in three groups: perception (HR-Bench- 4K/8K (Wang et al. 2025c), MME-RealWorld (Zhang et al. 2025), V ∗ (Wu and Xie 2024)), reasoning (MathVi- sion (Wang et al. 2024), MathVista (Lu et al. 2024), Math- Verse (Zhang et al. 2024), WeMath (Qiao et al. 2025), Log- icVista (Xiao et al. 2024)), and general (MMStar (Chen et al. 2024), ChartQA (Masry et al. 2022), BLINK (Fu et al. 2024)), reporting per-benchmark accuracy and the macro-average. Baselines span closed-source models (GPT- 4o, Gemini-2.5-Pro), open-source MLLMs (Qwen2.5- VL, Qwen2.5-VL-32B-Instruct (Bai et al. 2025b), In- ternVL3 (Zhu et al. 2025), LLaVA-OneVision (Li et al. 2024), Qwen3-VL (Bai et al. 2025a)), and 7–8B code- tool agents: Thyme (Zhang et al. 2026), DeepEyes (Zheng et al. 2026), DeepEyesV2 (Hong et al. 2026), Pixel- Reasoner (Wang et al. 2025a), Mini-o3 (Lai et al. 2026), MathCoder-VL (Wang et al. 2025b), CodeV (Hou et al. 2026), and PyVision (Zhao et al. 2026). More details are provided in the Appendix. 4.2 Performance of TACO Table 1 reports accuracy across the three groups. TACO (“Ours”) reaches an average of 68.1. The most telling com- parison is with the other code-tool / visual-agent models, which share our recipe of acting on the image through code: TACO clears the best of them (PyVision, 63.7) by 4.4 points, and improves on Thyme-7B (60.8), DeepEyes-7B (60.0), DeepEyes-v2-7B (61.2), and CodeV-7B-RL (62.5) by 5.6 to 8.1 points on average, surpassing CodeV’s judge- based process reward without any external judge. Against closed-source models, as a 7B model it substantially outper- forms the proprietary GPT-4o, 68.1 vs. 58.5 (+9.6). The gains concentrate where a code operation can expose a decisive detail. On all four perception benchmarks TACO leads every code-tool agent, raising HR-Bench-8K to 81.6 and V ∗ to 89.6, and it is strongest on the fine-grained reason- ing tasks LogicVista (55.6) and WeMath (53.1). Crucially, unlike simply calling the tool more often, it obtains these gains through appropriate tool use (Table 2). We attribute this to the tool-call credit assignment: DAPR and OGAR reward a call only when it changes a wrong answer to a right one and withhold credit otherwise, teaching the policy Table 3: Component ablations (accuracy %, 7B). Best per column in bold; Avg. is over the five-benchmark subset shown. w/o DAPR is the additive-probe variant: it keeps the tool-value channel but rewards the sum r out (a 1 )+r out (a 2 ) instead of the before/after difference. w/o OGAR keeps DAPR but disables gated routing, so the final-answer advan- tage A 1 lands on every token rather than only the outcome- selected segments. MethodV ∗ HR-4K HR-8K MathV MMStar Avg. Ours (full)-7B 89.683.881.635.869.372.0 w/o DAPR85.177.278.031.165.967.5 w/o OGAR87.481.680.233.467.470.0 Table 4: TACO generalizes across base models (accuracy %, subset of Table 1 benchmarks); Avg. is over this five- benchmark subset. Base / MethodV ∗ HR-4K HR-8K MathV MMStar Avg. Qwen2.5-VL-7B 76.468.865.327.064.760.4 + TACO89.683.881.635.869.372.0 Qwen3-VL-8B86.478.974.653.970.972.9 + TACO92.985.384.159.672.178.8 to crop precisely where it helps rather than over-call. 4.3 Efficiency and Tool Use TACO is simultaneously the most accurate and the fastest agent (Table 2): on all five benchmarks it reaches the high- est accuracy at the lowest end-to-end latency, e.g. 89.6% at 2.3 s on V ∗ versus 88.7% at 3.6 s for PyVision. The speed- up follows directly from the tool-use behavior of Figure 3b: by invoking the tool only when it helps, TACO emits fewer tool and sandbox rounds per question, so appropriate crop- ping improves both accuracy and latency at once. This real- izes the goal stated in the abstract, an agent that uses its tools when they help and abstains when they do not. 025050075010001250 Training step 0.3 0.4 0.5 0.6 0.7 0.8 VQA reward VQA reward Standard GRPO Additive-probe TGPO (Ours) (a) Accuracy reward 025050075010001250 Training step 625 650 675 700 725 750 Completion length Completion length (b) Completion length 025050075010001250 Training step 0.1 0.2 0.3 0.4 0.5 Policy entropy (nats) Policy entropy (c) Policy entropy Figure 3: Training dynamics. (a) Accuracy reward: TACO stays highest, while the additive-probe variant rises fastest early (probe hacking pays off) but plateaus and ends below standard GRPO. (b) Completion length: TACO steadily shortens com- pletions, ending well below the others, so fewer tool/sandbox rounds mirror its latency advantage (Table 2). (c) Policy entropy: all three decline smoothly to a healthy non-zero band without collapsing; TACO keeps the highest entropy and the additive- probe variant the lowest, matching its premature convergence. 4.4 Ablation Study Table 3 removes one component at a time; the full TACO objective reaches 72.0, and both components contribute. w/o DAPR drops the average to 67.5 (−4.5): rewarding r out (a 1 ) directly reopens the probe-hacking that differencing pre- vents (analysis below). w/o OGAR drops it to 70.0 (−2.0): spreading the final-answer advantage over every token blurs the credit that gated routing keeps on the responsible seg- ments. The two ingredients are complementary, and each is needed for the full gain. 4.5 Generalization across base models TACO is not tied to a single backbone. On a representa- tive five-benchmark subset (Table 4), it lifts Qwen2.5-VL- 7B from 60.4 to 72.0 (+11.6; the main comparison of Ta- ble 1) and the much stronger Qwen3-VL-8B from 72.9 to 78.8 (+5.9). The gains are consistent on every benchmark and match what TACO rewards: largest on high-resolution perception, where deciding when and where to crop mat- ters most (HR-8K +16.3/+9.5, HR-4K +15.0/+6.4, V ∗ +13.2/+6.5), and smaller but never negative on reasoning and general (MathVision +8.8/+5.7, MMStar +4.6/+1.2). The smaller Qwen3-VL-8B lift is expected, as a stronger base leaves less perceptual headroom; that TACO still adds nearly six points shows the gains come from the credit- assignment mechanism and compound with, rather than sub- stitute for, a more capable backbone. 4.6 Training Dynamics Figure 3 compares TACO against two baselines. Standard GRPO trains on the accuracy reward alone, with no tool- value channel and no gating. The additive-probe variant (w/o DAPR) adds a tool-value channel but rewards the two probe scores additively, r out (a 1 )+r out (a 2 ), instead of taking their difference. Probe-hacking is real, and differencing defends against it. On the accuracy reward (panel a), the additive-probe variant climbs fastest at first, when writing the answer early into the reasoning inflates its probe score, but it then plateaus and is overtaken, ending even below standard GRPO: the early spike was reward, not capability. TACO’s before/after difference cannot be inflated this way and sustains the high- est reward throughout. The agent learns to crop only when needed. Completion length (panel b) falls steadily under TACO (from about 720 to 640), while the additive-probe variant stays longest and standard GRPO barely moves. We read this as the policy converging to more economical tool use—it learns to issue a crop only when it expects to help rather than reflexively, which shortens the average response. The same converged policy is the one we evaluate, so this learned economy is consistent with its lower end-to-end latency in Table 2. Exploration is preserved, not collapsed. Policy entropy (panel c) tells the same story from the other side: all three variants start from a similar level and decline smoothly with- out collapsing, but the additive-probe variant falls fastest as it commits to the shortcut, while TACO keeps the highest entropy. Because DAPR rewards a genuine change in the answer rather than a gameable probe score, the policy has no shortcut to collapse onto and keeps exploring, which sus- tains its late reward growth in panel a. 5 Conclusion WepresentedTool-AugmentedCreditOptimization (TACO), a GRPO variant that gives code-tool visual agents a tool-call learning signal. It couples DAPR, a judge-free, probe-hacking-robust reward that scores each call by the before/after difference of the model’s own answer-probe outcomes, with OGAR, which routes the final-answer advantage only to the responsible segments, reinforcing useful calls and suppressing wasted ones. Across twelve benchmarks, TACO achieves the best average among prior code-tool agents, and transfers to a stronger Qwen3-VL backbone, showing the gains come from the mechanism, not the base or data. The bottleneck is learning which tool calls are worth making, which TACO scores at the tool-call level. More broadly, this value can be read from the agent’s own outcome reward by differencing a before/after answer probe, with no auxiliary judge or cost term. References Bai, S.; Cai, Y.; Chen, R.; Chen, K.; Chen, X.; Cheng, Z.; Deng, L.; Ding, W.; Gao, C.; Ge, C.; et al. 2025a. Qwen3-vl technical report. arXiv preprint arXiv:2511.21631. Bai, S.; Chen, K.; Liu, X.; Wang, J.; Ge, W.; Song, S.; Dang, K.; Wang, P.; Wang, S.; Tang, J.; Zhong, H.; Zhu, Y.; Yang, M.; Li, Z.; Wan, J.; Wang, P.; Ding, W.; Fu, Z.; Xu, Y.; Ye, J.; Zhang, X.; Xie, T.; Cheng, Z.; Zhang, H.; Yang, Z.; Xu, H.; and Lin, J. 2025b. Qwen2.5-VL Technical Report. arXiv:2502.13923. Chen, L.; Li, J.; Dong, X.; Zhang, P.; Zang, Y.; Chen, Z.; Duan, H.; Wang, J.; Qiao, Y.; Lin, D.; et al. 2024. Are we on the right way for evaluating large vision-language mod- els? Advances in Neural Information Processing Systems, 37: 27056–27087. Duan, H.; Yang, J.; Qiao, Y.; Fang, X.; Chen, L.; Liu, Y.; Dong, X.; Zang, Y.; Zhang, P.; Wang, J.; Lin, D.; and Chen, K. 2024. VLMEvalKit: An Open-Source Toolkit for Evalu- ating Large Multi-Modality Models. In Proceedings of the 32nd ACM International Conference on Multimedia, 11198– 11201. Feng, M.; Wu, J.; Liu, S.; Zhang, S.; Fang, H.; Jin, R.; Che, F.; Shao, P.; Wen, Z.; and Tao, J. 2026. Two-stage regularization-based structured pruning for llms. In Pro- ceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2996– 3012. Feng, M.; Wu, J.; Zhang, S.; Shao, P.; Jin, R.; Wen, Z.; Tao, J.; and Che, F. 2025. Dress: Data-driven regularized struc- tured streamlining for large language models. arXiv preprint arXiv:2501.17905. Fu, X.; Hu, Y.; Li, B.; Feng, Y.; Wang, H.; Lin, X.; Roth, D.; Smith, N. A.; Ma, W.-C.; and Krishna, R. 2024. Blink: Multimodal large language models can see but not per- ceive. In European Conference on Computer Vision, 148– 166. Springer. Guo, D.; Yang, D.; Zhang, H.; Song, J.; Wang, P.; Zhu, Q.; Xu, R.; Zhang, R.; Ma, S.; Bi, X.; et al. 2025. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning. arXiv preprint arXiv:2501.12948. Hong, J.; Zhao, C.; Zhu, C.; Lu, W.; Xu, G.; and XingYu. 2026. DeepEyesV2: Toward Agentic Multimodal Model. In The Fourteenth International Conference on Learning Rep- resentations. Hou, X.; Xu, S.; Biyani, M.; Li, M.; Liu, J.; Hollon, T. C.; and Wang, B. 2026. Codev: Code with images for faithful visual reasoning via tool-aware policy optimization. In Pro- ceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 21500–21510. Hu, S.; Dai, Y.; Han, X.; Fang, Z.; Zhao, Y.; Kwong, S. T. W.; and Fang, Y. 2026. Self-Induced Outcome Potential: Turn-Level Credit Assignment for Agents without Verifiers. arXiv preprint arXiv:2605.04984. Jin, R.; Shao, P.; Wen, Z.; Wu, J.; Feng, M.; Yang, S.; Zhang, C. Y.; and Tao, J. 2026. Exploring Knowledge Purification in Multi-Teacher Knowledge Distillation for LLMs. arXiv preprint arXiv:2602.01064. Jin, R.; Shao, P.; Wen, Z.; Wu, J.; Feng, M.; Zhang, S.; and Tao, J. 2025. Radialrouter: Structured representation for efficient and robust large language models routing. arXiv preprint arXiv:2506.03880. Kim, K.; and Chelikavada, K. 2026. Zoom Consistency: A Free Confidence Signal in Multi-Step Visual Grounding Pipelines. arXiv preprint arXiv:2604.15376. Lai, X.; Li, J.; Li, W.; Liu, T.; Li, T.; and Zhao, H. 2026. Mini-o3: Scaling Up Reasoning Patterns and Interaction Turns for Visual Search. In The Fourteenth International Conference on Learning Representations. Li, B.; Zhang, Y.; Guo, D.; Zhang, R.; Li, F.; Zhang, H.; Zhang, K.; Zhang, P.; Li, Y.; Liu, Z.; et al. 2024. Llava-onevision: Easy visual task transfer. arXiv preprint arXiv:2408.03326. Li, H.; Yang, Y.; Lin, Y.; Dai, X.; Yang, M.; and Peng, X. 2026a. Reliable Thinking with Images. In International Conference on Machine Learning. Li, X.; Jiao, W.; Jin, J.; Dong, G.; Jin, J.; Wang, Y.; Wang, H.; Zhu, Y.; Wen, J.-R.; Lu, Y.; and Dou, Z. 2026b. Deep- Agent: A General Reasoning Agent with Scalable Toolsets. arXiv:2510.21618. Liu, J.; Feng, M.; and Chen, L. 2026.Better, stronger, faster: Tackling the trilemma in mllm-based segmentation with simultaneous textual mask prediction. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pat- tern Recognition, 33121–33130. Liu, J.; Xiong, K.; Xia, P.; Zhou, Y.; Ji, H.; Feng, L.; Han, S.; Ding, M.; and Yao, H. 2025. Agent0-VL: Exploring Self- Evolving Agent for Tool-Integrated Vision-Language Rea- soning. arXiv preprint arXiv:2511.19900. Lu, P.; Bansal, H.; Xia, T.; Liu, J.; Li, C.; Hajishirzi, H.; Cheng, H.; Chang, K.-W.; Galley, M.; and Gao, J. 2024. Mathvista: Evaluating mathematical reasoning of founda- tion models in visual contexts. In International Conference on Learning Representations, volume 2024, 23439–23554. Lu, Z.; Lin, Z.; Jia, W.; Tian, C.; Ye, D.; Li, P.; Jin, L.; Liu, N.; Xu, G.; and Feng, W. 2026.HISR: Hind- sight Information Modulated Segmental Process Rewards for Multi-turn Agentic Reinforcement Learning.arXiv preprint arXiv:2603.18683. Ma, Y.; Zhang, W.; Li, T.; Du, L.; Shen, X.; and Liu, P. 2026. What Does Vision Tool-Use Reinforcement Learning Really Learn? Disentangling Tool-Induced and Intrinsic Effects for Crop-and-Zoom. arXiv preprint arXiv:2602.01334. ICML 2026. Masry, A.; Do, X. L.; Tan, J. Q.; Joty, S.; and Hoque, E. 2022. Chartqa: A benchmark for question answering about charts with visual and logical reasoning. In Findings of the association for computational linguistics: ACL 2022, 2263– 2279. Ng, A. Y.; Harada, D.; and Russell, S. 1999. Policy Invari- ance Under Reward Transformations: Theory and Applica- tion to Reward Shaping. In International Conference on Ma- chine Learning (ICML), 278–287. OpenAI. 2025. Thinking with Images. https://openai.com/ index/thinking-with-images/. Qi, Y.; Fu, P.; Li, H.; Liu, Y.; Jiang, C.; Qin, B.; Luo, Z.; and Luan, J. 2026. Patchcue: Enhancing vision-language model reasoning with patch-based visual cues.arXiv preprint arXiv:2603.05869. Qiao, R.; Tan, Q.; Dong, G.; MinhuiWu, M.; Sun, C.; Song, X.; Wang, J.; Gongque, Z.; Lei, S.; Zhang, Y.; et al. 2025. We-math: Does your large multimodal model achieve human-like mathematical reasoning? In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 20023–20070. Shao, Z.; Wang, P.; Zhu, Q.; Xu, R.; Song, J.; Bi, X.; Zhang, H.; Zhang, M.; Li, Y. K.; Wu, Y.; and Guo, D. 2024. DeepSeekMath: Pushing the Limits of Mathemati- cal Reasoning in Open Language Models. arXiv preprint arXiv:2402.03300. Team, K.; Bai, T.; Bai, Y.; Bao, Y.; Cai, S.; Cao, Y.; Charles, Y.; Che, H.; Chen, C.; Chen, G.; et al. 2026. Kimi K2. 5: Vi- sual Agentic Intelligence. arXiv preprint arXiv:2602.02276. Team, K.; Du, A.; Gao, B.; Xing, B.; Jiang, C.; Chen, C.; Li, C.; Xiao, C.; Du, C.; Liao, C.; et al. 2025. Kimi k1. 5: Scaling reinforcement learning with llms. arXiv preprint arXiv:2501.12599. Wang, H.; Su, A.; Ren, W.; Lin, F.; and Chen, W. 2025a.Pixel Reasoner: Incentivizing Pixel-Space Rea- soning with Curiosity-Driven Reinforcement Learning. arXiv:2505.15966. Wang, K.; Pan, J.; Shi, W.; Lu, Z.; Ren, H.; Zhou, A.; Zhan, M.; and Li, H. 2024. Measuring multimodal mathematical reasoning with math-vision dataset. Advances in Neural In- formation Processing Systems, 37: 95095–95169. Wang, K.; Pan, J.; Wei, L.; Zhou, A.; Shi, W.; Lu, Z.; Xiao, H.; Yang, Y.; Ren, H.; Zhan, M.; et al. 2025b. Mathcoder-vl: Bridging vision and code for enhanced multimodal mathe- matical reasoning. In Findings of the Association for Com- putational Linguistics: ACL 2025, 2505–2534. Wang, W.; Ding, L.; Zeng, M.; Zhou, X.; Shen, L.; Luo, Y.; Yu, W.; and Tao, D. 2025c. Divide, conquer and combine: A training-free framework for high-resolution image percep- tion in multimodal large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, 7907–7915. Wang, X.; Wang, W.; Chen, K.; Nimalsiri, N.; and Halgamuge, S. 2026.Discovering Process-Outcome Credit in Multi-Step LLM Reasoning.arXiv preprint arXiv:2602.01034. Wei, Q.; Zeng, S.; Li, C.; Brown, W.; Frunza, O.; Deng, W.; Schneider, A.; Nevmyvaka, Y.; Zhao, Y. K.; Garcia, A.; and Hong, M. 2025. Reinforcing Multi-Turn Reasoning in LLM Agents via Turn-Level Reward Design. arXiv preprint arXiv:2505.11821. Wu, F.; Zhang, Z.; Chang, Q.; Zhang, J.; Liu, Q.; and Du, J. 2026a. Step Potential Advantage Estimation: Harnessing In- termediate Confidence and Correctness for Efficient Mathe- matical Reasoning. arXiv preprint arXiv:2601.03823. Wu, J.; Feng, M.; Zhai, G.; Zhang, S.; Lian, Z.; Lv, F.; Shao, P.; Jin, R.; Wen, Z.; and Tao, J. 2026b. Astar: Boosting multimodal reasoning with automated structured thinking. In Proceedings of the AAAI Conference on Artificial Intelli- gence, volume 40, 33926–33934. Wu, J.; Feng, M.; Zhang, S.; Che, F.; Wen, Z.; Liao, C.; Yang, L.; Luo, H.; Lian, Z.; and Tao, J. 2026c. Beyond Ex- amples: Towards Automated Thought-level In-Context Rea- soning for Large Language Models. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2955–2995. Wu, J.; Liao, C.; Feng, M.; Zhang, S.; Wen, Z.; Luo, H.; Yang, L.; Xu, H.; and Tao, J. 2026d. TemplateRL: Struc- tured Template-Guided Reinforcement Learning for LLM Reasoning. arXiv preprint arXiv:2505.15692. Wu, J.; Liao, C.; Feng, M.; Zhang, S.; Wen, Z.; Shao, P.; Xu, H.; and Tao, J. 2025a. Thought-augmented policy opti- mization: Bridging external guidance and internal capabili- ties. arXiv preprint arXiv:2505.15692, 1(8): 10. Wu, J.; Zhang, S.; Che, F.; Feng, M.; Shao, P.; and Tao, J. 2025b. Pandora’s box or aladdin’s lamp: A comprehensive analysis revealing the role of rag noise in large language models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 5019–5039. Wu, P.; and Xie, S. 2024. V?: Guided visual search as a core mechanism in multimodal llms. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 13084–13094. Xiao, Y.; Sun, E.; Liu, T.; and Wang, W. 2024. Logicvista: Multimodal llm logical reasoning benchmark in visual con- texts. arXiv preprint arXiv:2407.04973. Yan, S.; Tong, J.; Xue, H.; Tang, X.; Wang, Y.; Shi, K.; Zhang, G.; Li, R.; and Zou, Y. 2026. Act Wisely: Cultivat- ing Meta-Cognitive Tool Use in Agentic Multimodal Mod- els. arXiv preprint arXiv:2604.08545. Yoon, E.; Yoon, H. S.; Jang, J.; Eom, S.; Dai, Q.; Luo, C.; Hasegawa-Johnson, M. A.; and Yoo, C. D. 2025. PACR: Progressively Ascending Confidence Reward for LLM Rea- soning. arXiv preprint arXiv:2510.22255. Zhang, R.; Jiang, D.; Zhang, Y.; Lin, H.; Guo, Z.; Qiu, P.; Zhou, A.; Lu, P.; Chang, K.-W.; Qiao, Y.; et al. 2024. Math- verse: Does your multi-modal llm truly see the diagrams in visual math problems? In European Conference on Com- puter Vision, 169–186. Springer. Zhang, Y.; Lu, X.; Yin, S.; Fu, C.; Chen, W.; Hu, X.; Wen, B.; Jiang, K.; Liu, C.; Zhang, T.; fan, H.; Chen, K.; Chen, J.; Ding, H.; Tang, K.; Zhang, Z.; Wang, L.; Yang, F.; Gao, T.; and Zhou, G. 2026. Thyme: Think Beyond Images. In The Fourteenth International Conference on Learning Rep- resentations. Zhang, Y.; Zhang, H.; Tian, H.; Fu, C.; Zhang, S.; Wu, J.; Li, F.; Wang, K.; Wen, Q.; Zhang, Z.; et al. 2025. Mme-realworld: Could your multimodal llm challenge high- resolution real-world scenarios that are difficult for humans? In International Conference on Learning Representations, volume 2025, 89655–89701. Zhao, S.; Lin, S.; Li, M.; Zhang, H.; Peng, W.; Zhang, K.; and Wei, C. 2026. PyVision-RL: Forging Open Agentic Vi- sion Models via RL. arXiv:2602.20739. Zheng, Z.; Yang, M.; Hong, J.; Zhao, C.; Xu, G.; Yang, L.; Shen, C.; and XingYu. 2026. DeepEyes: Incentivizing ”Thinking with Images” via Reinforcement Learning. In The Fourteenth International Conference on Learning Rep- resentations. Zhu, J.; Wang, W.; Chen, Z.; Liu, Z.; Ye, S.; Gu, L.; Tian, H.; Duan, Y.; Su, W.; Shao, J.; et al. 2025. Internvl3: Explor- ing advanced training and test-time recipes for open-source multimodal models. arXiv preprint arXiv:2504.10479. Technical Appendix of TACO: Tool-Augmented Credit Optimization for Agentic Tool Use This supplementary material provides additional details on TACO, covering the training algorithm, a theoretical motivation, the full experimental setup (benchmarks, baselines, and data-curation pipelines), supplementary results, qualitative case studies, and a probe-hacking analysis. All notation follows the main paper. The appendix is organized as follows: A. TL;DR: Main Contributions and Takeaways B. Training Algorithm and Edge Cases B.1. Training Algorithm B.2. Edge Cases C. Theoretical Motivation C.1. Background: GRPO and Uniform Credit Assignment C.2. DAPR as a Counterfactual Baseline C.3. Potential-Outcomes Interpretation of ∆ C.4. Robustness to Probe-Hacking, Formally C.5. A Potential-Based-Shaping View of the Process Channel C.6. OGAR as Conservative Advantage Masking C.7. The Two-Channel Objective C.8. Scope and Assumptions D. Experimental Setup Details D.1. Benchmarks D.2. Baselines D.3. Implementation Details D.4. SFT Data Curation D.5. RL Data Curation D.6. Channel-Weight Sensitivity E. Show Cases F. Limitations and Future Work A TL;DR: Main Contributions and Takeaways Contributions. • TACO, a GRPO variant for code-tool visual agents that turns a single per-call signal into both a reward and a routing rule, coupling DAPR and OGAR into one objective. • DAPR, a self-supervised, judge-free per-tool-call reward: it scores a call by the before/after difference of the agent’s own answer-probe outcomes, reusing the existing answer checker with no auxiliary model and at near-zero added cost. • OGAR, a parameter-free, token-level rule that routes the final-answer advantage only to the segments responsible for the outcome, suppressing wasted calls without any explicit cost term. Takeaways. 1. Credit the call, not the trajectory. Tying each tool call’s reward to its own measurable effect on the answer—rather than to the whole trajectory’s outcome—is what lets the policy learn when a crop helps and abstain when it does not. 2. Differencing beats absolute probes. Because DAPR subtracts the pre-tool baseline, it cancels what the model “already knew” and is naturally robust to probe-hacking, where an absolute probe score can be inflated by writing the answer early. 3. Routing matters as much as the reward. A scalar tool value is not enough; OGAR must deliver the final-answer advantage to the responsible tokens, otherwise redundant calls are over-credited and correct pre-tool reasoning is wrongly blamed. 4. Accuracy and efficiency together. TACO attains the best average among open-source models while invoking tools only when they help, so it is simultaneously the most accurate and the lowest-latency code-tool agent. 5. Backbone-agnostic. The gains transfer from Qwen2.5-VL-7B to the stronger Qwen3-VL-8B, indicating they come from the credit-assignment mechanism rather than from the base model or data. Algorithm 1 TACO: two-stage training. Require: base VLM π θ ; SFT setD sft , RL promptsD rl ; checker r out ; group size G; weights α 1 ,α 2 ; clip ε, lr η 1: Stage 1 (SFT). fine-tune π θ onD sft ; set π ref ←π θ , π θ old ←π θ 2: Stage 2 (RL). 3: for each mini-batch of prompts (q,I)∈D rl do 4:sample a group of G rolloutsτ (i) G i=1 ∼ π θ old (·| q,I) 5:for i = 1,...,G do 6:parse τ (i) = (T 1 ,C,T 2 ,a f ); executeC in the sandbox→ observation IMG 7:pre-tool probe: prefill </think><answer> afterT 1 , greedy-decode a 1 from (q,I,T 1 ) 8:post-tool probe: greedy-decode a 2 from (q,I,T 1 ,C, IMG,T 2 ) 9:∆ (i) ←r out (a 2 )−r out (a 1 ); R (i) acc ←r out (a f )+0.5R (i) fmt 10:end for 11: A (i) 1 ← (R (i) acc −μ R )/σ R , A (i) 2 ← (∆ (i) −μ ∆ )/σ ∆ // group mean/std over i=1:G 12:for i = 1,...,G, token t∈ τ (i) do 13:set m (i) [t] = 1 if t∈T 2 , or t∈C with ∆ (i) ̸= 0, or t∈T 1 with ∆ (i) ≥ 0; else m (i) [t] = 0 14:end for 15:set g (i) = 1 if ∆ (i) ≥ 0, else g (i) = 0// process channel off on misleading calls 16: L←α 1 L GRPO (m⊙A 1 ) + α 2 L GRPO (g A 2 )// clipped surrogate, Eq. (11) 17: θ←θ− η∇ θ L; periodically π θ old ←π θ // KL weight β = 0 18: end for Ensure: trained policy π θ B Training Algorithm and Edge Cases This section provides the implementation-level details omitted from the main paper: pseudocode for the training loop and the edge cases the gate must handle. The notation (T 1 ,C,T 2 , a 1 ,a 2 , ∆,A 1 ,A 2 ,m,g) is that of the main paper. B.1 Training Algorithm TACO is the two-stage pipeline of Algorithm 1: an SFT cold-start that establishes the Think–Code–Answer format and fixes the reference policy π ref , followed by group-relative RL with the gated dual-channel advantage. Per prompt, each rollout is parsed into its three segments, the tool code is executed, the two probes are decoded to score the call, and the accuracy and tool-value advantages are group-normalized; the per-token gate m and the trajectory gate g then route the two channels before the clipped-GRPO update. B.2 Edge Cases No tool call. If a rollout emits no <code> block and answers directly, the tool branch is empty: there is no ∆ and the process channel is inactive, so only the (gated) accuracy channel applies, with A 1 on the answer tokens. This lets the policy abstain from tools on items it already solves, at no penalty. Multiple tool calls. When a trajectory contains more than one call, the post-tool probe is read after the final call, so ∆ measures the value of the entire tool branch (all calls, their observations, and the reasoning they trigger) against the same pre-tool baseline a 1 , andC is taken as the union of the code blocks. Our experiments operate almost entirely in the single-call regime, for which the before/after split is cleanest; finer per-call attribution within a multi-call branch is left to future work. Probe parse failure. A probe whose decoded string cannot be parsed into a valid answer is scored r out = 0, so it neither rewards nor penalizes the call; this keeps malformed probe decodes from injecting a spurious ∆. C Theoretical Motivation This section motivates TACO from first principles. We start from the GRPO objective and its credit-assignment limitation (C.1), recast DAPR as a counterfactual difference (C.2), give a potential-based-shaping reading of the process channel (C.5), show that OGAR is a conservative masking of the outcome advantage (C.6), and finally state precisely what is and is not claimed (C.8). C.1 Background: GRPO and Uniform Credit Assignment For a prompt x = (q,I), GRPO (Shao et al. 2024) samples a group of G trajectories τ (i) G i=1 ∼ π θ old , scores each with a scalar reward R (i) , and forms the group-relative advantage A (i) = R (i) − μ R σ R , μ R = 1 G X j R (j) , σ R = std R (j) .(10) Writing ρ (i) t = π θ (o (i) t | x,o (i) <t )/π θ old (·) for the per-token importance ratio, the clipped surrogate is L GRPO (A) =E i,t h min ρ (i) t A (i) , clip(ρ (i) t , 1−ε, 1+ε)A (i) i .(11) The defining property of (11) is that A (i) is constant across all tokens t of τ (i) : the outcome reward is broadcast uniformly. Decompose a code-tool trajectory into three segments, the pre-tool reasoningT 1 , the codeC, and the post-tool reasoning with the final answer T 2 . Uniform broadcasting gives the code tokens the gradient A (i) P t∈C ∇ θ logπ θ (o (i) t ), so C is reinforced whenever the trajectory ends correctly, regardless of whether the tool call contributed; symmetrically, a correctT 1 is penalized whenever a later tool call spoils the answer. Outcome-level credit thus cannot separate a call’s contribution from the trajectory’s overall correctness. TACO addresses this with a per-call reward (DAPR) and a per-segment gate (OGAR). C.2DAPR as a Counterfactual Baseline Let c 1 = (q,I,T 1 ) be the context just before the call and c 2 = (q,I,T 1 ,C, IMG,T 2 ) the context just after it. Each probe prefills the answer header and greedily decodes a = arg max a π θ (a | c), scored by the verifiable checker r out (·) ∈ −1, 0, +1. The tool value is the difference ∆ = r out (a 2 )− r out (a 1 ).(12) Equation (12) is a counterfactual estimate of the value of the tool branch: r out (a 2 ) is the outcome with the branch and r out (a 1 ) is the outcome on the same trajectory without it. Equivalently, r out (a 1 ) serves as a trajectory-specific reference for the post-tool outcome. Because c 1 is a prefix of c 2 , the subtraction removes the component of the outcome explained by the shared pre-tool context (in particular any answer the model has already committed to inT 1 ), and isolates the marginal effect of extending the context with the tool branch (C, IMG,T 2 ). This is exactly the term a uniform outcome reward conflates with the call: ∆ > 0 only when the branch turns a wrong pre-tool answer right, ∆ < 0 when it spoils a right one, and ∆ = 0 when it does not move the outcome. We deliberately do not claim that ∆ isolates the visual observation IMG alone. The marginal increment bundles IMG with the extra reasoningT 2 that the observation triggers, andT 2 exists only on the post-tool side, so it does not cancel. ∆ therefore estimates the value of taking the tool branch, which is the quantity the policy actually controls when it decides to call a tool. C.3 Potential-Outcomes Interpretation of ∆ It is useful to read (12) through the lens of potential outcomes. Treat invoking the tool branch as a binary treatment T ∈0, 1 applied to a fixed pre-tool state c 1 , and define the two potential outcomes Y (1) = r out (answer| branch taken), Y (0) = r out (answer| branch withheld).(13) The post-tool probe realizes Y (1) (it answers from c 2 ) and the pre-tool probe realizes Y (0) (it answers from c 1 ), on the same trajectory, so ∆ = Y (1)− Y (0) reads off both potential outcomes directly rather than estimating a missing counterfactual. Averaged over the group and the data, ∆ is the average realized effect of the tool branches the policy produces. Because both outcomes are read from the same pre-tool context c 1 , the comparison is not confounded by the pre-tool state—the obstacle for outcome-only credit, where a tool’s effect and the trajectory’s prior correctness are entangled. Consistent with C.2, the quantity is the effect of the whole branch (includingT 2 ), not of the observation in isolation. C.4 Robustness to Probe-Hacking, Formally A generative probe is vulnerable to answer leakage: the policy could write its answer early intoT 1 so that the prefilled-header decode merely copies it. We formalize why differencing neutralizes this. Proposition 1 (Invariance to common-mode shifts). Suppose a change toT 1 shifts both probe scores by a common amount, r out (a 1 )7→ r out (a 1 ) + δ and r out (a 2 )7→ r out (a 2 ) + δ. Then ∆ is unchanged. In particular, if the change makes both probes copy the same pre-committed answer ˆy (so a 1 = a 2 = ˆy), then ∆ = 0. Proof. ∆ = r out (a 2 )− r out (a 1 ); adding δ to both terms leaves the difference unchanged. If a 1 = a 2 = ˆy then r out (a 2 ) = r out (a 1 ) = r out (ˆy), so ∆ = 0.□ Answer leakage is exactly a common-mode shift of the pre-tool baseline (it raises r out (a 1 ) and r out (a 2 ) together), so by Proposition 1 the process channel grants it no advantage: the only way to earn ∆ > 0 is to make the post-tool answer correct when the pre-tool answer was not, which requires the tool branch to change the outcome. An absolute-probe reward such as r out (a 1 ) + r out (a 2 ) lacks this invariance—under the same shift it increases by 2δ—which is precisely the mechanism behind the probe-hacking we observe for the additive-probe variant. C.5 A Potential-Based-Shaping View of the Process Channel Let s 1 and s 2 denote the reasoning states reached after T 1 and after the tool branch, and define the potential Φ(s) = r out (probe(s)) as the probe correctness at a state. Then (12) is a potential difference, ∆ = Φ(s 2 )− Φ(s 1 ).(14) Potential-based reward shaping adds to the reward a term F (s,s ′ ) = γ Φ(s ′ )− Φ(s) and preserves the set of optimal poli- cies (Ng, Harada, and Russell 1999); with γ = 1 the process signal ∆ has precisely this form. Proposition 2 (Policy invariance of potential shaping). In an episodic MDP with reward r and Φ(terminal) = 0, replacing r by r ′ (s,a,s ′ ) = r(s,a,s ′ ) + γΦ(s ′ )− Φ(s) leaves the optimal policy set unchanged, and the optimal action values satisfy Q ′∗ (s,a) = Q ∗ (s,a)− Φ(s). Proof. Along any trajectory the shaping terms telescope: P t≥0 γ t γΦ(s t+1 )− Φ(s t ) =−Φ(s 0 ), using Φ(terminal) = 0. The shaped return thus differs from the original by the constant−Φ(s 0 ), which does not depend on the policy; hence arg max over policies is preserved and Q ′∗ (s,a) = Q ∗ (s,a)− Φ(s).□ The process channel is the γ = 1 instance with Φ(s) = r out (probe(s)). Proposition 2 is what licenses adding ∆ as an auxiliary signal: it changes how fast and where credit flows, not the optimum implied by the outcome reward. We use this for the shaping form: in practice the channel is applied at the group/trajectory level under the GRPO normalization of (10) rather than as an exact per-step MDP potential, and the probe only approximates the state value, so we treat it as a soundness argument rather than an end-to-end invariance guarantee. C.6OGAR as Conservative Advantage Masking Standard GRPO sets the per-token advantage to A[t] = A 1 for every t. OGAR replaces this with A[t] = m[t]A 1 , where m[t] ∈ 0, 1 is the outcome-conditioned gate of the main paper. Since m[t] only zeroes (never sign-flips) the advantage, the gated gradient ∇ θ L = X t m[t]A 1 ∇ θ logπ θ (o t )(15) is a sub-sum of the original GRPO gradient: credit is withheld, never inverted. Proposition 3 (Conservativeness of gating). For m[t] ∈ 0, 1, the gated gradient is the sum of a subset of the GRPO gradient’s per-token terms. Gating never reverses the sign of any token’s contribution; it only removes it. Consequently OGAR can withhold credit but cannot convert outcome-justified reinforcement into penalization, or vice versa. Proof. Multiplying a term by m[t]∈0, 1 either keeps it (m[t] = 1) or sets it to0 (m[t] = 0); neither operation flips its sign. The gated gradient is P t:m[t]=1 A 1 ∇ θ logπ θ (o t ), a subset of the ungated sum.□ The update therefore never pushes a token in the direction opposite to its outcome-justified one; it only declines to assign credit on segments that the call’s outcome shows are not responsible. Two consequences follow directly from the gate definition. On a right-but-redundant call the code’s advantage is set to 0 rather than positive, removing the gradient that would otherwise reward an unnecessary call and thereby discouraging over-calling at no extra cost term. On a misleading call the blame is kept off the correct pre-tool reasoningT 1 , so a single spoiled tool call does not teach the model to distrust reasoning that was already right. C.7 The Two-Channel Objective TACO combines the gated accuracy channel and the process channel into L TACO = α 1 L GRPO (m⊙ A 1 ) + α 2 L GRPO (g A 2 ),(16) where A 1 is the group-normalized accuracy advantage (10) of R acc = r out (a f ) + 0.5R fmt , the mask m restricts it to outcome- responsible tokens, A 2 is the group-normalized advantage of ∆, and the trajectory gate g (equal to 1 when ∆ ≥ 0 and 0 otherwise) disables the process channel on misleading calls. The two channels act on different supports: m⊙A 1 lives on a subset of tokens (C.6), while g A 2 , when active, applies to the whole sequence. This separation is deliberate. On a misleading call the penalty is already carried by the gated accuracy channel on C and T 2 ; setting g = 0 there avoids double-counting it through the process channel, and, since m keeps A 1 off T 1 , the correct pre-tool reasoning is left unpenalized by both channels. The coefficients trade outcome correctness (α 1 ) against tool-value shaping (α 2 ); the strong asymmetry we use (α 1 =1.0, α 2 =0.15) keeps the verifiable outcome as the primary objective and treats ∆ as an auxiliary shaping signal (C.5). Table 5: Evaluation benchmarks grouped by category, with the number of questions in each benchmark. BenchmarkCategory#Samples V ∗ (Wu and Xie 2024) Perception 191 HR-Bench-4K (Wang et al. 2025c) 800 HR-Bench-8K (Wang et al. 2025c) 800 MME-RealWorld (Zhang et al. 2025) 29,429 MathVision (Wang et al. 2024) Reasoning 3,040 MathVista (Lu et al. 2024) 6,141 MathVerse (Zhang et al. 2024) 2,612 WeMath (Qiao et al. 2025) 6,500 LogicVista (Xiao et al. 2024) 448 MMStar (Chen et al. 2024) General 1,500 ChartQA (Masry et al. 2022) 2,500 BLINK (Fu et al. 2024) 3,807 C.8 Scope and Assumptions The analysis above relies on three assumptions, which we state plainly. (i) Faithful probing: the prefilled-header greedy decode reflects the answer the model would commit to from the given context. (i) Verifiable outcomes: r out is a rule-based checker, so ∆ ∈ −2,−1, 0, 1, 2 and the method applies most directly to tasks with checkable answers. (i) Single decisive call: the before/after split is cleanest when a trajectory contains one tool branch (the regime our setting targets). Finally, we make no variance-reduction claim: differencing two probe scores can increase per-sample variance relative to a single outcome reward, and we rely on the group normalization in (10) to control it; the empirical training-reward dynamics, not a variance inequality, are our evidence that the signal is well-behaved. D Experimental Setup Details D.1 Benchmarks We evaluate on twelve benchmarks spanning three groups (Table 5). For each we report accuracy (%) on the standard evaluation split, and the macro-average is taken over all twelve. Answers are extracted and scored with the default Duan et al. (2024) protocol, identical for every model. Perception. These benchmarks stress fine-grained recognition, often on high-resolution images where a decisive detail is small and easy to miss — exactly the regime a crop/zoom tool targets. • HR-Bench-4K / HR-Bench-8K (Wang et al. 2025c) evaluate perception on high-resolution images at 4K and 8K, each with 800 samples covering fine-grained single-instance and cross-instance perception (attributes, positions, and relations of small objects). We report the two resolutions as separate columns. • MME-RealWorld (Zhang et al. 2025) is a large-scale, manually annotated real-world benchmark of high-resolution images with deliberately challenging perception and reasoning questions. • V ∗ (Wu and Xie 2024) is a visual-search benchmark of 191 items that requires locating a small target in a high-resolution scene, with attribute-recognition and spatial-relationship subtasks. Reasoning. These benchmarks test multimodal mathematical and logical reasoning over diagrams, figures, and charts. • MathVision (Wang et al. 2024) contains 3,040 competition-level visual math problems across 16 disciplines and five diffi- culty levels. • MathVista (Lu et al. 2024) aggregates 6,141 examples of mathematical reasoning in visual contexts from 28 source datasets plus three newly collected ones (IQTest, FunctionQA, PaperQA). • MathVerse (Zhang et al. 2024) provides 2,612 diagram-based math problems, each rendered in several versions that shift information between the text and the diagram to test genuine visual understanding. • WeMath (Qiao et al. 2025) organizes visual math problems into a hierarchy of 67 knowledge concepts, probing reasoning beyond end-to-end accuracy. • LogicVista (Xiao et al. 2024) comprises 448 multiple-choice questions evaluating logical reasoning (inductive, deductive, spatial, and more) in visual contexts. General. These benchmarks measure broad multimodal understanding. • MMStar (Chen et al. 2024) is a 1,500-sample, human-curated benchmark of vision-indispensable questions spanning six core capabilities and 18 axes, built to reduce text-only solvability and data leakage. • ChartQA (Masry et al. 2022) tests question answering over charts that requires both visual reading and logical/arithmetic reasoning, combining human-written and machine-generated questions. • BLINK (Fu et al. 2024) contains 3,807 multiple-choice questions over 14 classic visual-perception tasks that are easy for humans but remain hard for current MLLMs. D.2 Baselines We compare against three families. Closed-source models: GPT-4o and Gemini-2.5-Pro. Open-source MLLMs without visual tools: Qwen2.5-VL, Qwen2.5-VL-32B (Bai et al. 2025b), InternVL3 (Zhu et al. 2025), LLaVA-OneVision (Li et al. 2024), and Qwen3-VL (Bai et al. 2025a). The most relevant family is the 7–8B code-tool / visual-agent models, which, like TACO, act on the image through code or visual operations and then reason over the result: • Thyme (Zhang et al. 2026) (Think Beyond Images) goes beyond simple cropping to a broad space of image-processing operations expressed as code (zoom, rotation, contrast, and general computation), trained with a two-stage SFT-then-RL recipe. It is the source of our SFT corpus and our base recipe. • DeepEyes (Zheng et al. 2026) incentivizes “thinking with images” through end-to-end reinforcement learning, where a zoom-in tool-use behavior emerges natively without pre-collected reasoning data or an external tool model. • DeepEyesV2 (Hong et al. 2026) builds an agentic multimodal model with a cold-start stage followed by RL, observing that RL alone fails to induce robust tool use; it exhibits task-adaptive invocation, using image operations for perception and numerical computation for reasoning. • Pixel-Reasoner (Wang et al. 2025a) equips a VLM with pixel-space operations (zoom-in, select-frame) and uses curiosity- driven RL to escape the “learning trap” where the model falls back on text-only reasoning and neglects the visual operations. • Mini-o3 (Lai et al. 2026) scales up multi-turn visual search to tens of interaction turns, using a visual-probe dataset and an over-turn masking strategy that avoids penalizing long trajectories during RL. • MathCoder-VL (Wang et al. 2025b) bridges vision and code for multimodal mathematical reasoning, aligning visual con- tent with executable code to improve diagram-grounded math problem solving. • CodeV (Hou et al. 2026) represents visual tools as executable Python code and trains with Tool-Aware Policy Optimization, a process-level RL framework that assigns dense rewards directly on tool inputs and outputs (via an external judge) to encourage faithful, evidence-consistent tool use. • PyVision (Zhao et al. 2026) is an agentic framework in which the MLLM autonomously generates, executes, and refines task-specific Python tools at inference, enabling flexible and interpretable visual problem solving. D.3 Implementation Details Following Thyme, we build on Qwen2.5-VL-7B with 2 epochs of SFT followed by 1 epoch of GRPO. We set α 1 = 1.0 and α 2 = 0.15, use no KL penalty (β = 0; i.e. KL regularization is disabled), and sample G = 8 rollouts per prompt at temperature 1.0, with a total batch size of 128 and learning rate 1× 10 −6 . Training runs on a single node of 8×80 GB A100 GPUs. We report the sensitivity to the channel weights α 1 ,α 2 in Table 8. Prompt templates. Both stages use the same system prompt (Table 6), inherited from Thyme (Zhang et al. 2026): it instructs the agent to reason step by step and, optionally, to emit sandboxed Python for image manipulation, returning the processed image or result for further reasoning. The per-example user prompt (Table 7) supplies the image together with the question and the image path and size, and fixes the required <think>/<answer> output format. Both are applied to every (q,I) under the Qwen2.5-VL chat template. D.4 SFT Data Curation Our SFT data is built on the Thyme SFT corpus (Zhang et al. 2026), re-curated with three filters. (i) Execution validity: we re-run every code block in our sandbox and discard trajectories with execution errors or with tool observations/answers inconsistent with the actual output, which would otherwise teach the model to hallucinate observations. (i) Tool necessity: we drop samples that Qwen2.5-VL-7B (Bai et al. 2025b) already solves without tools (pass@8 = 1), keeping only trajectories where a tool call is genuinely needed. (i) Quality: Gemini-3-Pro scores each trajectory for reasoning coherence and tool-use rationale, and low-quality or blind-tool-use traces are removed. Table 6: System prompt used for the SFT cold-start (and reused unchanged in RL). Tag and code tokens are shown in typewriter. System Prompt (SFT & RL) You are a helpful assistant. Solve the following problem step by step, and optionally write Python code for image manipulation to enhance your rea- soning process. The Python code will be executed by an external sandbox, and the processed image or result (wrapped in <sandboxoutput></sandboxoutput>) can be returned to aid your reasoning and help you arrive at the final answer. Reasoning & Image Manipulation (Optional but Encouraged): • You have the capability to write executable Python code to perform image manipulations (e.g., cropping to a Region of Interest (ROI), resizing, rotation, adjusting contrast) or perform calculation for better reasoning. • The code will be executed in a secure sandbox, and its output will be provided back to you for further analysis. • All Python code snippets must be wrapped as follows: <code> ```python # your code. ``` </code> • At the end of the code, print the path of the processed image (processed path) or the result for further processing in a sandbox environment. Table 7: Per-example user prompt used for the SFT cold-start (and reused unchanged in RL). [·] placeholders are filled per sample; <image> is the visual token consumed by the Qwen2.5-VL chat template. Tag tokens are shown in typewriter. User Prompt (SFT & RL) <image> User’s Question: [User Question] User Image Path: [Image Path] User Image Size: [Image Size] Output Format (strict adherence required): <think>Your detailed reasoning process, including any code, should go here.</think> <answer>Your final answer to the user’s question goes here.</answer> D.5 RL Data Curation For RL we follow CodeV (Hou et al. 2026), building on its open-source prompt data and adopting its data-cleaning recipe. Keeping only questions with verifiable ground-truth answers, we clean them in two ways. (i) Environmental fidelity: each prompt is passed through Gemini-3-Pro to check image quality, question clarity, and image–text consistency, and prompts with corrupted images or severe ambiguity are removed. (i) Difficulty calibration: prompts that our SFT checkpoint already solves on all G = 8 rollouts yield zero-variance accuracy rewards (no GRPO advantage), so we remove them. D.6 Channel-Weight Sensitivity We fix the accuracy weight α 1 =1.0 and sweep the tool-value weight α 2 , reporting the macro-average over all twelve bench- marks (Table 8). The setting α 1 =1.0,α 2 =0.15 is the configuration used throughout the paper. Small α 2 keeps the verifiable outcome as the dominant objective while still letting the tool-value channel shape exploration; very large α 2 would over-weight the auxiliary signal relative to final correctness. E Show Cases Figures 4–8 present five real trajectories from a TACO-trained agent, spanning crop/zoom perception, scientific-figure reading, image rotation, and chart-grounded math. Each panel shows the question, the agent’s reasoning, the Python tool call and the view it returns, and the final answer. A show case of an agent trained with TGPO Question: What bank is advertised on the board behind the player? Model Responses: <think> The question asks for the sponsor name on the board behind the player. In the full frame the wall does carry large lettering, but the characters are small relative to the whole image and, worse, the player's head, torso and racket arm occlude the middle of the wordmark, so I can only make out "BNP" on the left and a fragment ending in "...RIBAS" on the right. The lower slogan line is also partly hidden. This is exactly a case where a closer look helps: I will crop the top banner band that carries the wordmark and upscale it so the letters become legible. <code> ```python from PIL import Image img = Image.open("/data/Thyme/dataset/case/01_useful_in.png") # size (500, 333) # Region of Interest: the top band of the backdrop holding the sponsor wordmark roi = img.crop((0, 25, 500, 120)) # (left, upper, right, lower) roi = roi.resize((roi.width * 2, roi.height * 2)) # 2x zoom for legibility processed_path = "/data/Thyme/dataset/case/01_useful_crop.png" roi.save(processed_path) print(processed_path) <.code> <sandbox_output> <subimage_sign> <.sandbox_output> With the band enlarged the wordmark is now readable: the bold green capitals spell "BNP P A R I B A S" across the wall, with the player only breaking the middle "A". The corporate slogan underneath, "The bank for a changing world," is the registered tagline of BNP Paribas, which confirms the reading. The advertised bank is therefore BNP Paribas. <.think> <answer>BNP Paribas<.answer> Code Tool: crop & zoom in Figure 4: Reading an occluded sponsor wordmark: the agent crops the backdrop banner and zooms 2× to recover “BNP Paribas”. A show case of an agent trained with TGPO Question: What are the three labels printed above the white connector on the red circuit board? Model Responses: <think> This is a high-resolution photo of an electronics bench: a blue microcontroller board on the left, a red PCB shield on top with a small black RF module, SMA connectors, and a tan board on the right. The question asks about the small silk-screen labels printed above a white 3-pin connector on the red board. At the full 1600x1103 scale that text is only a few pixels tall and I cannot read it reliably — it looks like "...V ... V" but I can't be sure. This is a fine-grained detail in a large image, so I will crop the connector area and zoom in. <code> ```python from PIL import Image img = Image.open("/data/Thyme/dataset/case/07_hires_in.png") # size (1600, 1103) # Region of Interest: the white connector and the silk-screen labels beneath it roi = img.crop((395, 300, 690, 400)) # (left, upper, right, lower) roi = roi.resize((roi.width * 3, roi.height * 3)) # 3x zoom for legibility processed_path = "/data/Thyme/dataset/case/07_hires_crop.png" roi.save(processed_path) print(processed_path) <.code> <sandbox_output> <subimage_sign> <.sandbox_output> In the zoomed crop the white printing under the 3-pin connector is now clearly legible: from left to right it reads "-15V", "GND", "+15V" (a bipolar power header, also marked "PWR"). So the three labels are -15V, GND and +15V. <.think> <answer>-15V, GND, +15V<.answer> Code Tool: crop & zoom in Figure 5: High-resolution perception: the agent crops and zooms a circuit board to read the three small labels printed above a connector. A show case of an agent trained with TGPO Question: In panel A (missForest, 60% overall missingness), which method first reaches precision ≥ 0.9 as π increases? Model Responses: <think> This is a 3x2 grid of precision-vs-π plots (panels A--F) comparing six methods (BART, RF, XGB, LASSO, Stepwise, CRF), each drawn as a thin dashed line with a small marker. I need panel A (top-left, missForest 60% missingness) and the method whose curve first crosses precision = 0.9 as π grows. At the full image scale the panel is small and the dashed curves near the top are bunched together, so I cannot match a curve to its legend marker reliably. Let me crop panel A and zoom in. <code> ```python from PIL import Image img = Image.open("/data/Thyme/dataset/case/04_failed_in.png") # size (2016, 1860) roi = img.crop((120, 55, 846, 409)) # panel A, upper-left region roi = roi.resize((int(roi.width*1.6), int(roi.height*1.6))) processed_path = "/data/Thyme/dataset/case/04_failed_crop.png" roi.save(processed_path) print(processed_path) <-code> <sandbox_output> <subimage_sign> <-sandbox_output> Even enlarged, the top of panel A is crowded: three curves rise steeply around π ≈ 0.1 and all sit in the 0.9--1.0 band almost on top of one another. Their markers -- an open circle (BART), a plus (RF) and a filled triangle (CRF) -- overlap so closely there that I cannot confidently tell which one crosses 0.9 at the smaller π. The plus (RF) looks marginally highest at π ≈ 0.1, so I will go with RF, though I am not certain. <-think> <answer>RF<-answer> Code Tool: crop & zoom in Figure 6: Scientific-figure reading: the agent isolates one panel of a six-method precision plot to extract the requested value. A show case of an agent trained with TGPO Question: What route number is shown on the front destination board of the red bus? Model Responses: <think> This photo was clearly uploaded sideways: the three double-decker buses are lying on their side, with the ground running down the right edge and the sky down the left. In this orientation any text on the buses — including the red bus's destination board — is rotated 90 degrees and I cannot read the route number reliably. The right operation here is not cropping but rotating the image upright first, then looking at the front board of the red bus. <code> ```python from PIL import Image img = Image.open("/data/Thyme/dataset/case/08_rotate_in.png") # size (375, 500), sideways upright = img.rotate(90, expand=True) # rotate to the correct orientation # the red bus is on the left; zoom its front destination board board = upright.crop((40, 110, 175, 235)).resize((405, 375)) processed_path = "/data/Thyme/dataset/case/08_rotate_crop.png" board.save(processed_path) print(processed_path) </code> <sandbox_output> <subimage_sign> </sandbox_output> After rotating the image upright and zooming the front of the red bus, the destination box at the top is now readable: it shows a white-on-black route number "339" next to the destination text. So the route number on the red bus is 339. </think> <answer>339</answer> Code Tool: rotate & crop & zoom in Figure 7: Image rotation: the photo was uploaded sideways, so the agent rotates it before reading the route number on the bus’s destination board. A show case of an agent trained with TGPO Question: Sophia owns six dogs and monitors how much food they eat in a month. What fraction of the food was eaten by Scout? Simplify your answer. Model Responses: <think> The image is clear: a table "Dog" / "Food (kg)" with six dogs and their monthly food in kilograms: Buddy 45, Scout 16, Max 38, Bella 41, Lucy 36, Cooper 21. No image processing is needed to read it. The fraction eaten by Scout is Scout's food divided by the total of all six dogs. That means summing six numbers and then reducing the resulting fraction to lowest terms — reducing fractions by hand is exactly where mistakes happen, so I will do the arithmetic exactly with Python's Fraction, which auto-simplifies. <code> ```python ```python from fractions import Fraction food = "Buddy": 45, "Scout": 16, "Max": 38, "Bella": 41, "Lucy": 36, "Cooper": 21 total = sum(food.values()) frac = Fraction(food["Scout"], total) # Fraction reduces to lowest terms automatically print(f"Scout = food['Scout'] kg, total = total kg") print(f"fraction = frac.numerator/frac.denominator") </code> The six amounts sum to 197 kg, and Scout ate 16 kg, so Scout's share is 16/197. Since 197 is prime, 16/197 is already in lowest terms. </think> <sandbox_output> fraction = 16/197 </sandbox_output> <answer> 16/197 </answer> Code Tool: computation Figure 8: Chart-grounded math: the agent crops the relevant chart region and computes the requested fraction. Table 8: Sensitivity to the channel weights α 1 ,α 2 : macro-average accuracy over the twelve benchmarks. The row used in the paper is in bold. α 1 α 2 Avg. (12 benchmarks) 1.0 0.0567.8 1.0 0.1068.0 1.00.1568.1 1.0 0.3067.9 1.0 0.5067.7 F Limitations and Future Work TACO relies on a rule-based outcome checker, so it applies most directly to tasks with verifiable answers, and its probes assume the tool’s effect is observable in the answer. The single-call scoping exploits a clean before/after split; extending the probe-difference signal to multi-call trajectories, open-ended generation, and richer tool spaces is left to future work. Several further directions are promising: integrating TACO with model-compression methods for efficient deployment (Feng et al. 2025, 2026), coupling it with thought-augmented reasoning paradigms (Wu et al. 2026c, 2025a, 2026b, 2025b), and pairing it with model-routing methods (Jin et al. 2025, 2026).