Paper deep dive

Learning after Execution with Action-Specific Credit

Yufeng Tian, Jiahao Tang, Minghao Wu

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 112

Intelligence

Status: not_run | Model: - | Prompt: - | Confidence: 0%

Entities (0)

No extracted entities yet.

Relation Signals (0)

No relation signals yet.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Execution is often used as terminal feedback: it scores a generated program, but the resulting state is not used for another learning action. Learning after execution instead uses the execution result as an intermediate state that supports an additional learning interaction for the same policy. This creates a credit-assignment problem because the two actions share the same final objective but play different roles in the trajectory. Accordingly, action-specific credit assigns the initial action credit from the final trajectory outcome and the corrective action credit for improvement over the intermediate state. Image-to-TikZ serves as an executable visual testbed for this setting. On 945 held-out examples, learning after execution strengthens the shared policy under single-call inference and also produces a post-execution corrective capability. Single-call render success reaches 92.70\%, compared with 88.25\% for continued single-stage GRPO after the same number of additional training iterations. This corrective capability is not reproduced by continued single-stage training and depends on the intermediate state. Alternative credit definitions also induce different group-relative preferences over the same trajectories. Together, these results suggest that executable tasks contain useful learning structure beyond a terminal reward. Code is available at this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

108,408 characters extracted from source content.

Expand or collapse full text

Learning after Execution with Action-Specific Credit Yufeng Tian 1 , Jiahao Tang 1,2 , Minghao Wu 1 1 The Chinese University of Hong Kong, Shenzhen, China 2 Central South University, Changsha, China tian_yufeng@outlook.com Abstract Execution is often used as terminal feedback: it scores a generated program, but the resulting state is not used for another learning action. Learning after execution instead uses the execution result as an intermediate state that supports an additional learning interaction for the same policy. This creates a credit-assignment problem because the two actions share the same final objective but play different roles in the trajectory. Accordingly, action-specific credit assigns the initial action credit from the final trajectory outcome and the corrective action credit for improvement over the intermediate state. Image-to-TikZ serves as an executable visual testbed for this setting. On 945 held-out examples, learning after execution strengthens the shared policy under single-call inference and also produces a post-execution corrective capability. Single-call render success reaches 92.70%, compared with 88.25% for continued single-stage GRPO after the same number of additional training iterations. This corrective capability is not reproduced by continued single-stage training and depends on the intermediate state. Alternative credit definitions also induce different group-relative preferences over the same trajectories. Together, these results suggest that executable tasks contain useful learning structure beyond a terminal reward. Code is available atGitHub. 1 Introduction Executing a generated program exposes the result of the model’s prediction. This result can be scored to train the policy, but it also provides a state on which the policy can act again [1,2,3]. In a single-stage interaction, execution marks the end of the trajectory. The executed outcome provides a training signal, but the state produced by execution is not used for another learning action. Rather than ending the interaction at this point, we ask whether the execution state can support an additional learning interaction, and whether learning from this interaction strengthens the shared policy itself. This setting introduces a credit-assignment problem because the two actions share the same final objective but serve different roles in the trajectory [4,5]. The initial action generates the intermediate output and determines the state from which the corrective action begins, while the corrective action inherits that state and seeks to improve it. The final trajectory outcome is relevant to both actions, but the corrective action must also be evaluated relative to the state it receives. Training should therefore account for both where the trajectory ends and what changes after execution. We study learning after execution by using the execution state of the policy’s own prediction to create an additional learning interaction from the same training target. For each target, the shared policy first generates and executes an initial program, then revises or repairs it from the resulting state. We assign action-specific credit to the two actions. The initial action receives credit from the final trajectory outcome, while the corrective action receives credit for improvement over the intermediate result. Both actions jointly update the same policy. Preprint. arXiv:2610.14573v1 [cs.AI] 10 Oct 2026 TikZ CodeTikZ CodeTikZ CodeTikZ Code Render ImageRender Image Failure Compile Failure Compile Render Result Failure Compile Render Result Initial Action Credit G 0 =S 1 Failure S 0 =0 Valid Render S 0 =RSim(I*,I 0 ) Execution Repair Revise Execution Corrective Action Credit G 1 =S 1 -S 0 I * I 1 I 0 e 0 e 1 C 1 C 0 π θ Shared Policy Terminal Learning Signal From S 0 Score Valid Render S 1 =RSim(I*,I 1 ) Failure S 1 =0 π θ Shared Policy Execution as a State Enables Additional Learning Learn after Execution Score Valid Render ✓ Failure ✘ Target ImageTarget ImageTarget ImageTarget Image Image-to-TikZ Testbed π θ π θ Execution Figure 1: Overview of learning after execution with action-specific credit. Execution first provides a score for the initial action; learning after execution additionally uses the resulting render or failure state for a corrective action by the same policy. The initial action receives final-outcome credit, while the corrective action receives improvement-relative credit. The right panel illustrates the Image-to-TikZ instantiation with valid and failed executions. We use Image-to-TikZ as an executable visual testbed for this problem [6,7]. Image-to-TikZ reconstructs a target image as an executable TikZ program and belongs to the broader family of image-to-code [8,9,10,11,12]. A generated program can be executed to reveal either a rendered image or an execution failure. A valid render makes the visual difference from the target observable, while a failed execution identifies a program that still requires repair. The result before and after correction can therefore be evaluated under the same execution and visual criteria. This makes Image-to-TikZ suitable for studying both how a policy learns from an execution state and what changes when it acts on that state. Using Image-to-TikZ, we study learning after execution through a series of controlled comparisons. We separate improvement of the shared policy from the effect of invoking a corrective action at inference time, and distinguish learned post-execution correction from behavior that may arise from continued single-stage training alone. We further examine how alternative credit definitions shape the group-relative learning signals assigned to the two actions. Together, these analyses characterize whether learning after execution strengthens the shared policy, produces a distinct corrective capability, and how its learning is shaped by action-specific credit. Our main contributions are: •We introduce learning after execution as a training setting in which execution is used not only to score a prediction, but also to produce an intermediate state that provides additional learning experience for the same policy. •We formulate action-specific credit for this setting, preserving a shared final-quality objective while matching credit to the different roles of initial generation and correction. •We show that learning from post-execution interaction strengthens the shared policy even un- der single-call inference, while also producing a corrective capability that is not reproduced by continued single-stage training alone. 2 Preliminaries: Code Supervision and Execution Feedback Before introducing learning after execution, we define the program generation, execution score, and single-stage training setting used throughout the paper. In our Image-to-TikZ instantiation, a target imageI ∗ is mapped to an executable TikZ programC. ExecutingCeither produces a rendered image or results in execution failure. These outcomes define the execution score used by Direct-GRPO and by our method in Section 3.1. 2 2.1 Supervised Image-to-TikZ Given paired training dataD =(I ∗ i ,C ∗ i ) N i=1 , whereI ∗ i is a target image andC ∗ i is its reference TikZ program, supervised fine-tuning trains the model to generateCconditioned onI ∗ i . The supervision is defined on the reference program rather than on the executed outcome of a program sampled from the policy. Different TikZ programs can produce similar rendered figures, while small code changes can alter the visual result or break execution. The rendered consequence of the model’s own prediction therefore does not directly participate in the supervised training objective. 2.2 Direct Reinforcement Learning with Execution Feedback Execution exposes the consequence of a generated program. For each target image, we sample a group ofKprogramsC i ∼ π θ old (·| I ∗ ), fori = 1,...,K, and evaluate their executed outputs. We define the execution score S(C;I ∗ ) = ( RSim(E (C),I ∗ ), if C produces a valid render, 0,otherwise. (1) E denotes the L A T E X execution and rasterization pipeline. Following TikZilla, we use RSim as the visual similarity reward, which compares patch-level representations from the DeTikZify-V2 image encoder using Earth Mover’s Distance [13,6,14]. The score combines executability and visual fidelity: failed executions receive zero, while valid renders are scored by their similarity to the target. For Direct-GRPO, each sampled program receivesS i = S(C i ;I ∗ ). Within each target-image group, we mean-center these scores to obtain group-relative advantages and optimize them with the clipped GRPO objective [15]. Implementation details are given in Appendix B. We refer to this single-stage setting as Direct-GRPO. Direct-GRPO learns from the executed outcomes of programs sampled from the current policy. Its optimization signal reflects both executability and rendered visual fidelity. Execution, however, still marks the end of the interaction: the result is used to score the sampled program, but the policy does not observe the rendered output or failure state as input for another action. The next section uses this execution result as an intermediate state on which the same policy can act again. 3 Learning after Execution with Action-Specific Credit Direct-GRPO uses execution as terminal feedback. We instead use the execution result as an intermediate state on which the same policy can act again. The policy first generates and executes a program, then revises or repairs it based on the observed execution result. The initial and corrective actions are jointly trained with action-specific credit. 3.1 Execution-Conditioned Correction LetI ⋆ denote the target image. At training iterationk, the same policyπ θ k is called twice within each trajectory. The first call directly generates an initial TikZ program, C 0,i ∼ π θ k (·| I ⋆ ),(2) whereiindexes one sampled trajectory. We executeC 0,i using the same L A T E X rendering pipeline introduced in Section 2.2 and compute S t,i = S(C t,i ;I ⋆ ), t∈0, 1,(3) using the execution score defined in Section 2.2. Thus,S 0,i evaluates the intermediate program produced by the first call. The score is used for credit computation; it is not itself the observation provided to the second call. The execution result determines the context for the corrective action. IfC 0,i produces a valid rendered imageI 0,i , the model receives the target image, the current render, and the current TikZ program. It 3 can therefore compare the target with what its own program actually produced. If no valid render is produced, the model instead receives the target image, the current TikZ attempt, and a compact execution status e 0,i . The corrective action is sampled as C 1,i ∼ ( π θ k (·| I ⋆ ,I 0,i ,C 0,i ),valid Stage 0 render, π θ k (·| I ⋆ ,C 0,i ,e 0,i ), otherwise. (4) The first branch performs visual revision. The second branch repairs, completes, or replaces an unsuccessful program. We then execute C 1,i and obtain the final score S 1,i using Eq. 3. Stage 0 and Stage 1 are two calls to the same policyπ θ k under different contexts. The first call observes only the target image. The second call observes the execution state produced by the first action. No parameter update occurs between the two calls. Both actions are sampled before the shared policy is updated. 3.2 Action-Specific Credit from Incremental Rewards Both actions serve the same final-quality objective,J (θ) =E τ∼π θ [S 1 ]. With two actions and no discounting, we decompose the execution score into incremental rewards and use the corresponding returns-to-go as action-specific credit: r 0,i = S 0,i ,r 1,i = S 1,i − S 0,i , G 0,i = r 0,i + r 1,i = S 1,i ,G 1,i = r 1,i = S 1,i − S 0,i . (5) The incremental rewards telescope to the final score,r 0,i + r 1,i = S 1,i . The initial action therefore receives the return of the complete trajectory, while the corrective action receives the improvement relative to the intermediate state it inherits. For the initial action, assigning onlyS 1,i −S 0,i would omit its immediate rewardS 0,i and, for a fixed final score, favor a lower intermediate score. We therefore use the complete returnG 0,i = S 1,i and evaluate the initial action by the final quality reached by the trajectory. For the corrective action,G 1,i = S 1,i − S 0,i directly measures the score change from the state it receives. Its raw credit is positive when the execution score increases and negative when the score decreases. Because unsuccessful executions receive zero, the same definition also covers execution repair and failure: ifS 0,i = 0, thenG 1,i = S 1,i ≥ 0; ifS 1,i = 0, thenG 1,i = −S 0,i ≤ 0. Visual revision and execution repair are therefore handled by the same credit definition. 3.3 Group-Relative Optimization For each target image, we sample K complete trajectories, with K = 8 in all experiments using the corrective action. Each trajectoryτ i = (C 0,i ,S 0,i ,C 1,i ,S 1,i )contains an independently sampled initial action followed by a corrective action conditioned on its own execution state. Thus, a group containsKindependently sampled trajectories rather than one shared Stage 0 program followed by multiple Stage 1 candidates. The action-specific returns in Eq. 5 are centered separately within the same target-image group. With the implementation scale λ = 10, A t,i = λ   G t,i − 1 K K X j=1 G t,j   , t∈0, 1.(6) We subtract only the group mean and do not normalize by the group standard deviation.A 0,i is assigned to the completion tokens of C 0,i , while A 1,i is assigned to those of C 1,i . The distinction between raw credit and group-relative advantage is important. A positiveG 1,i means that the corrective action increases the execution score, but it does not necessarily implyA 1,i > 0; the latter also depends on the mean improvement within the sampled group. 4 For the corrective action,S 0,i is determined beforeC 1,i is sampled and acts as a state-dependent baseline. Appendix E further compares improvement-relative and absolute final-score credit. The group-relative objective is applied separately to the two action sets, L = 0.5L GRPO (C 0 ,A 0 ) + 0.5L GRPO (C 1 ,A 1 ).(7) Both terms optimize the same policy parameters. All Stage 0 and Stage 1 actions in a training iteration are sampled from the same policyπ θ k before any parameter update. The corresponding losses for the two stages are optimized sequentially over action microbatches using the shared policy. 4 Experiments The experiments examine how using execution during training affects both initial generation and subsequent correction. We first evaluate overall performance under direct and post-execution infer- ence. We then study the corrective action, the role of action-specific credit, and how correction varies across executed intermediate states. 4.1 Experimental Setup Models and Training Data. Qwen3-VL-8B-Instruct [16] is used as the base model throughout the experiments. All supervised and reinforcement learning data are drawn from DaTikZ-V4 [14]. Direct-SFT is trained on approximately 4K Image-to-TikZ pairs. Direct-GRPO is initialized from the final SFT checkpoint and trained on 1K samples for 125 training iterations. Our method is initialized from the final Direct-GRPO checkpoint and trained on the same 1K samples for 125 training iterations. Evaluation Data. A temporally held-out evaluation set is constructed using the same source collection procedure as DaTikZ-V4, restricted to data available from 2026 onward. Samples are collected from arXiv, GitHub, and TeX StackExchange. The 1,095 examples are divided into two non-overlapping subsets. Dev-150 contains 150 examples for checkpoint selection, ablations, and behavioral analysis. Final-945 contains the 945 examples and is reserved for final evaluation. Evaluation Metrics.Render rate measures whether the generated TikZ program executes success- fully. Visual fidelity is evaluated with RSim [14], DINO [17], DreamSim [18], and CLIPImg [19]. RSim is also used as the reinforcement learning reward. Unless stated otherwise, visual metrics use the fixed-zero convention, where failed executions receive a score of zero. Inference Protocol. All models are evaluated with temperature1.0, top-p 0.9, repetition penalty 1.0, and at most 8192 generated tokens. Generation stops at the first complete document. Stage 0 denotes the initial output produced from the target image alone, while Stage 1 denotes the raw output produced after observing the Stage 0 execution state. We additionally evaluate an execution-aware selection rule over the two outputs already produced. If one stage fails to render, the valid output is retained. If both render, the output with the higher RSim score is selected, with ties assigned to Stage 0. No additional model generation is required by the selection rule. 4.2 Overall Results We first compare the main models on Final-945. Our initializer is the final Direct-GRPO checkpoint, evaluated with the same Stage 0/Stage 1 interaction before further training. Because decoding is stochastic, its Stage 0 result and the separately sampled Direct-GRPO baseline can differ slightly. As an additional control, we continue Direct-GRPO from the same checkpoint on the same training set for another 100 training iterations. Table 1 shows that SFT does not improve performance on this held-out set, whereas Direct-GRPO reaches a render rate of81.90%, together with consistent improvements across all four visual metrics. Optimizing executed outcomes provides a substantially stronger signal than reference code supervision alone. 5 Table 1: Main results on Final-945. Visual metrics use the fixed-zero convention. Direct denotes single-call generation. Stage 0 is the initial single-call output of our policy, while Stage 1 is produced after execution-conditioned correction. Selected chooses between the two outputs using execution success and RSim. Direct-GRPO (+100) continues single-stage GRPO for 100 additional training iterations from the same Direct-GRPO checkpoint. Best results are shown in bold and second-best results are underlined. MethodOutput Render Success ↑ RSim↑DINO↑DreamSim↑CLIPImg↑ Base modelDirect71.64%0.40330.60250.55450.6435 SFTDirect64.66%0.35540.54270.49600.5781 Direct-GRPODirect81.90%0.47090.70460.64680.7410 Direct-GRPO (+100)Direct88.25%0.51350.76290.70190.7996 Ours (initializer)Stage 083.07%0.47660.71290.65440.7489 Ours (initializer)Stage 183.17%0.47810.71420.65560.7501 OursStage 092.70%0.53820.80140.73300.8372 OursStage 192.80% 0.54460.80460.74010.8409 OursSelected94.18%0.56490.82210.75930.8583 Continuing Direct-GRPO for another 100 training iterations provides further gains, raising render success to88.25%. Starting from the same Direct-GRPO checkpoint and after the same number of additional training iterations, our Stage 0 output reaches92.70%and is higher under all four visual metrics. Continued single-stage training therefore does not reproduce the gain obtained with our training. Importantly, this gain is already present in Stage 0, which uses the same single-call generation setting as the direct baselines. Simply adding a second execution-conditioned call is insufficient. At initialization, Stage 0 and Stage 1 differ little. After training, raw Stage 1 reaches92.80%render success and achieves slightly higher visual similarity than Stage 0 across all four metrics. Execution-aware selection further raises render success to 94.18% and improves all four visual metrics. These results separate two effects. Training strengthens the shared policy itself, as shown by the improved Stage 0 output, while the second action provides an additional opportunity to act on the execution state. We examine the latter behavior next. 4.3 Learning Dynamics of Generation and Correction Since Stage 0 already improves substantially, we isolate the corrective action itself. For visual correction, we consider only examples where both stages render successfully, termed the paired success subset. For a visual similarity metric M , we define ∆M = M (I 1 ,I ⋆ )− M (I 0 ,I ⋆ ),(8) whereI 0 andI 1 are the rendered outputs before and after correction, andI ⋆ is the target image. This comparison excludes gains from execution recovery and isolates the visual change produced by Stage 1 on already executable outputs. For the correction-outcome composition, we use a small neutral region:∆RSim > 0.005is counted as improvement,|∆RSim| ≤ 0.005as neutral, and ∆RSim <−0.005 as degradation. Figure 2 shows that the initializer does not yet act as a reliable visual corrector. On paired-success examples, its average Stage 1 change is negative under all four visual metrics. Strong direct generation therefore does not automatically provide the ability to revise an executed result. Further single-stage optimization shows the same pattern: when Continued Direct-GRPO is evaluated with the same execution-conditioned second call, its paired visual changes remain close to zero. This behavior changes during training. At step 75, the average Stage 1 change is positive under RSim, DINO, DreamSim, and CLIPImg. RSim is used for training. The concurrent gains under the other three metrics show that the visual improvement is not confined to the reward metric. Correction quality is not monotonic across training, even though render rate improves much more consistently. 6 Success→Success Stage 0 1.0 0.9 0.8 Init255075100125 Stage 1 Render Rate -0.01 0.00 0.01 Init255075100125 ΔM Stage1-Stage0 RSimDINODreamSimCLIPImg Init255075100125 ImproveNeutralDegrade 0 50% 100% Correction Outcome 0 50% 100% Execution Transition Success→Failure Init255075100125 IterationsIterationsIterationsIterations Failure→SuccessFailure→Failure Figure 2: Learning dynamics of our method on Dev-150. The figure tracks Stage 0 and Stage 1 render rates, paired visual changes∆M, the distribution of correction outcomes, and execution transitions across training checkpoints. This shows that initial generation quality and the effect of the corrective action can evolve differently. We return to this difference in Section 4.5. The correction-outcome distribution also changes substantially. The fraction of paired-success examples improved by Stage 1 rises from12.2%for the initializer to42.5%at step 75 and46.8% at step 125. Most successful Stage 0 executions remain executable after Stage 1, while failed-to- successful recovery remains relatively rare on Dev-150. Together, these results show that training changes both the initial action and the behavior of the subsequent corrective action. Checkpoint 100 is selected on Dev-150 for subsequent behavioral analyses and final evaluation on Final-945, while training dynamics are examined across the full checkpoint trajectory. 4.4 Credit Assignment in Post-Execution Learning Table 2: Factorized ablation of action-specific credit assignment. Rows vary Stage 0 credit and columns vary Stage 1 credit. Stage 1 credit Stage 0 credit λ(S 1 − S 0 ) λS 1 λS 1 A (Ours)C λS 0 BD We factorize the action-specific credit design into a2× 2ablation. Stage 0 credit is either the final trajectory scoreS 1 or the local intermediate score S 0 , while Stage 1 credit is either the improvement S 1 −S 0 or the absolute final scoreS 1 . This yields the four variants in Table 2, with A corresponding to our method. Downstream versus local credit for the initial action. The A–B comparison isolates Stage 0 credit. At step 50, B already achieves a slightly higher Stage 0 render rate than A, yet its subsequent Stage 1 correction is nearly neutral. A produces a substantially larger correction at the same checkpoint and remains positive at step 75, whereas B becomes slightly negative. Thus, optimizing the intermediate output itself does not produce the same subsequent correction behavior. Using downstream creditS 1 instead favors initial actions according to the quality reached after the corrective action. Improvement-relative versus absolute credit for the corrective action. The A–C comparison isolates Stage 1 credit. We recompute absoluteS 1 -based Stage 1 advantages on the 8,000 trajectories collected from the completed A run. The difference is substantial at both trajectory and group levels: replacingS 1 − S 0 withS 1 flips the sign of the group-relative advantage for37.2%of trajectories and changes the highest-advantage trajectory in74.5%of target-image groups. The two credit definitions therefore induce substantially different group-relative preferences over corrective actions. Optimization behavior of the credit variants. Variants C and D share absolute Stage 1 credit and enter the same failure regime from step 26 despite using different Stage 0 credits. Their Stage 1 generations become substantially longer and normal document termination deteriorates, causing most outputs to fail execution. For D, the average Stage 1 completion approaches 7K tokens and86%of completions reach the generation limit. Once both stage scores collapse to zero within a group, reward variation disappears and the group- relative advantages, policy loss, and gradient norm become zero. The zero learning signal therefore follows the generation failure rather than preceding it. In contrast, A and B remain executable and retain nonzero training signals on the same groups. C and D differ in Stage 0 credit but share the same 7 255075 0.92 0.88 0.84 Stage0 Render RateCorrection Gain 0.010 0.005 0.000 255075 BBAA Iterations BA Iterations BBAA Iterations BA Iterations Stage0 Credit 0.5 0 -0.5 A 0 (S 1 ) 0.50-0.5 A 0 (S 0 ) 21.4%SameSameFlipFlip A 0 (S 0 ) 21.4%SameFlip 0.5 0 -1 -101 -0.5 1 A 1 (S 1 - S 0 ) Stage1 Credit A 1 (S 1 ) 37.2%SameSameFlipFlip A 1 (S 1 ) 37.2%SameFlip Init255075100125 B C D A Training Signal Iterations Figure 3: Effect of action-specific credit assignment. The figure compares Stage 0 quality and correc- tion gain under alternative credit definitions, measures how the resulting group-relative preferences differ, and tracks the training signal of all four ablation variants. absolute Stage 1 credit, making this the common factor associated with their early failure. Together, the fixed-trajectory preference analysis and observed optimization behavior show thatS 1 andS 1 −S 0 are not interchangeable credit signals for the corrective action. 4.5 Correction Outcomes Across Intermediate States The previous analysis shows that the effect of the corrective action is not constant across training. We therefore examine how Stage 1 outcomes vary with the quality of the executed Stage 0 state. Using the Dev-selected checkpoint at step 100, Figure 4 shows a clear pattern across intermediate-state quality. Stage 1 provides the largest average gain when the intermediate render is weak. For Stage 0 RSim below0.4, it improves over Stage 0 in76%of paired examples. As Stage 0 quality increases, both the average correction gain and the probability of improvement generally decrease. In the strongest group, the average change becomes negative and Stage 1 improves over Stage 0 in only23%of cases. Additional qualitative examples covering successful revision, execution repair, over-revision, preservation, and unresolved cases are provided in Appendix A. Mean Stage1 - Stage0( Δ RSim) 0.04 0.02 0.00 -0.02 100% 50% 0% 0.0260 0.0029 0.0021 -0.0026 0.7647 0.5385 0.2333 0.5517 0.5385 -0.0235 P(Stage1>Stage0) Stage0 RSim Mean Stage1-Stage0(ΔRSim) P(Stage1>Stage0) Correction Depends on States 1739302926 Paired Success: Over Revise Failure Compile Failure Compile Stage 0 Stage 1I * Stage 0 Stage 1I * Failure Compile Stage 0 Stage 1I * Stage 0 Stage 1I * Repair Failure Compile Failure Compile Failure Compile Failure Compile Stage 0 Stage 1 I * Stage 0 Stage 1 I * Failure Compile Failure Compile Stage 0 Stage 1 I * Stage 0 Stage 1 I * Successful Revise Stage 0 Stage 1 I * Stage 0 Stage 1 I * Stage 0 Stage 1 I * Stage 0 Stage 1 I * Figure 4: Correction behavior as a function of Stage 0 quality. We report the mean Stage 1 change in RSim and the frequency with which Stage 1 achieves higher RSim than Stage 0 across Stage 0 quality ranges, together with qualitative examples of beneficial and harmful corrections. This pattern helps explain the non-monotonic correction curves in Figure 2. Weak intermediate renders leave larger discrepancies for Stage 1 to address. As Stage 0 becomes stronger, the remaining errors become smaller and further modification carries a greater risk of degrading an already good reconstruction. Thus, a stronger Stage 0 policy can reduce the average marginal value of correction even when Stage 1 remains capable of improving weak states. 8 Execution-Aware Selection.The asymmetric effect of correction motivates selecting between the two outputs already produced by the policy. We use the execution-aware rule defined in Section 4.1, which requires no additional model generation. On Dev-150, checkpoint 100 gives the strongest selected result. Selection raises render success to97.3%, compared with96.7%for Stage 0 and 94.7%for raw Stage 1, while RSim-FZ reaches0.5646. DINO, DreamSim, and CLIPImg also improve although they are not used by the selector. The checkpoint and selection rule are then frozen before evaluation on Final-945. As shown in Table 1, selection raises render success from92.80%for raw Stage 1 to94.18%and improves all four fixed-zero visual metrics. Stage 0 can therefore be used directly as the single-call output, while the post-execution action and selection provide an optional additional refinement mode. 5 Related Work Rendering- and execution-aware reinforcement learning.RLRF [20] optimizes vector graphics with rewards from rendered SVG outputs, while TikZilla [14] applies GRPO to Text-to-TikZ using rendered visual similarity. In code synthesis, RLEF [21] and related work [22,23] use execution feedback to improve generated programs. These methods establish execution as an effective learning signal. We focus on the additional learning and credit-assignment problem that arises when the execution result becomes an intermediate state for another action. Iterative correction and multi-turn policy learning. Iterative improvement has been explored through search, self-correction, and multi-turn reinforcement learning [24,25,26,27,28]. DeTikZ- ify refines programs with MCTS, while Sketch2Diagram selects among independently generated candidates. Recent agentic RL methods study finer-grained credit across interaction steps, including GiGPO [29] and turn-level policy optimization [30]. Our setting instead uses the execution state produced by the policy’s own initial program as the context for a subsequent action, and assigns credit to the initial and corrective actions according to their different roles in the trajectory. Graphics program synthesis. Recent work increasingly treats scientific figure reconstruction as graphics program synthesis rather than raster image generation [31,32,33,34]. DeTikZ- ify [6] generates TikZ programs from figures and sketches with inference-time refinement, while Sketch2Diagram [7] studies sketch-to-TikZ generation with data augmentation and multi-candidate inference. Omni-I2C [8] further highlights the difficulty of high-fidelity image-to-code generation across diverse domains. We use Image-to-TikZ as an executable visual testbed for studying learning after execution. 6 Conclusion and Limitations The study shows that execution can provide more than a terminal learning signal. It can expose an intermediate state from which additional learning becomes possible. Using the same training targets, shared policy, and execution score, our method turns this state into a second learning interaction and assigns credit according to the distinct roles of initial generation and correction. This simple change strengthens the policy even under single-call inference, while also producing an additional post-execution corrective capability. More broadly, our results suggest that executable tasks may contain useful learning structure beyond the scalar reward obtained at the end of generation, and that carefully assigning credit can expose this structure without requiring a more complex reward design. At the same time, our experiments provide an initial demonstration of this principle rather than a complete characterization of its scope. Our study is limited to one 8B backbone, one executable visual program domain, and a relatively small training set. Correction is not always beneficial for strong intermediate states, motivating learned stopping or routing, while the instability of absolute corrective credit remains specific to our current RSim-based setting. References [1] Tianyu Zheng, Ge Zhang, Tianhao Shen, Xueling Liu, Bill Yuchen Lin, Jie Fu, Wenhu Chen, and Xiang Yue. Opencodeinterpreter: Integrating code generation with execution and refinement. 9 In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, volume ACL 2024 of Findings of ACL, pages 12834–12859. Association for Computational Linguistics, 2024. [2] Qingyao Li, Xinyi Dai, Xiangyang Li, Weinan Zhang, Yasheng Wang, Ruiming Tang, and Yong Yu. Codeprm: Execution feedback-enhanced process reward model for code generation. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025, volume ACL 2025 of Findings of ACL, pages 8169–8182. Association for Computational Linguistics, 2025. [3]Zhangqian Bi, Yao Wan, Zheng Wang, Hongyu Zhang, Batu Guan, Fangxin Lu, Zili Zhang, Yulei Sui, Hai Jin, and Xuanhua Shi. Iterative refinement of project-level code context for precise code generation with compiler feedback. In Lun-Wei Ku, Andre Martins, and Vivek Srikumar, editors, Findings of the Association for Computational Linguistics, ACL 2024, Bangkok, Thailand and virtual meeting, August 11-16, 2024, volume ACL 2024 of Findings of ACL, pages 2336–2353. Association for Computational Linguistics, 2024. [4] Changle Qu, Sunhao Dai, Hengyi Cai, Jun Xu, Shuaiqiang Wang, and Dawei Yin. Matchtir: Fine-grained supervision for tool-integrated reasoning via bipartite matching. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens, editors, Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2026, San Diego, California, United States, July 2-7, 2026, pages 11953–11968. Association for Computational Linguistics, 2026. [5]Linxuan Du, Guangquan Xue, Xiaobo Liang, Qipeng Huang, Yuyang Ding, Xinyu Shi, Zhang Yijun, Ji Qi, Wenpeng Zhu, Juntao Li, and Min Zhang. Escaping the echo trap: On credit assignment failure in multi-turn LLM self-reflection. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens, editors, Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2026, San Diego, California, United States, July 2-7, 2026, pages 35393–35405. Association for Computational Linguistics, 2026. [6]Jonas Belouadi, Simone Paolo Ponzetto, and Steffen Eger. Detikzify: Synthesizing graphics programs for scientific figures and sketches with tikz. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 37: Annual Conference on Neural Infor- mation Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. [7]Itsumi Saito, Haruto Yoshida, and Keisuke Sakaguchi. Sketch2diagram: Generating vector diagrams from hand-drawn sketches. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. [8] Jiawei Zhou, Chi Zhang, Xiang Feng, Qiming Zhang, Haibo Qiu, Lihuo He, Dengpan Ye, Xinbo Gao, and Jing Zhang. Omni-i2c: A holistic benchmark for high-fidelity image-to-code generation. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens, editors, Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2026, San Diego, California, United States, July 2-7, 2026, pages 37761–37798. Association for Computational Linguistics, 2026. [9] Cheng Yang, Chufan Shi, Yaxin Liu, Bo Shui, Junjie Wang, Mohan Jing, Linran Xu, Xinyu Zhu, Siheng Li, Yuxiang Zhang, Gongye Liu, Xiaomei Nie, Deng Cai, and Yujiu Yang. Chartmimic: Evaluating lmm’s cross-modal reasoning capability via chart-to-code generation. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. [10]Chengyue Wu, Zhixuan Liang, Yixiao Ge, Qiushan Guo, Zeyu Lu, Jiahao Wang, Ying Shan, and Ping Luo. Plot2code: A comprehensive benchmark for evaluating multi-modal large language models in code generation from scientific plots. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Findings of the Association for Computational Linguistics: NAACL 2025, Albuquerque, 10 New Mexico, USA, April 29 - May 4, 2025, volume NAACL 2025 of Findings of ACL, pages 3006–3028. Association for Computational Linguistics, 2025. [11]Chenglei Si, Yanzhe Zhang, Ryan Li, Zhengyuan Yang, Ruibo Liu, and Diyi Yang. Design2code: Benchmarking multimodal code generation for automated front-end engineering. In Luis Chiruzzo, Alan Ritter, and Lu Wang, editors, Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL 2025 - Volume 1: Long Papers, Albuquerque, New Mexico, USA, April 29 - May 4, 2025, pages 3956–3974. Association for Computational Linguistics, 2025. [12]Josselin Somerville Roberts, Tony Lee, Chi Heem Wong, Michihiro Yasunaga, Yifan Mai, and Percy Liang. Image2struct: Benchmarking structure extraction for vision-language models. In Amir Globersons, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang, editors, Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. [13]Matt J. Kusner, Yu Sun, Nicholas I. Kolkin, and Kilian Q. Weinberger. From word embeddings to document distances. In Francis R. Bach and David M. Blei, editors, Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, volume 37 of JMLR Workshop and Conference Proceedings, pages 957–966. JMLR.org, 2015. [14]Christian Greisinger and Steffen Eger. Tikzilla: Scaling text-to-tikz with high-quality data and reinforcement learning. CoRR, abs/2603.03072, 2026. [15]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, Y. K. Li, Y. Wu, and Daya Guo. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. CoRR, abs/2402.03300, 2024. [16] Shuai Bai, Yuxuan Cai, Ruizhe Chen, Keqin Chen, Xionghui Chen, Zesen Cheng, Lianghao Deng, Wei Ding, Chang Gao, Chunjiang Ge, Wenbin Ge, Zhifang Guo, Qidong Huang, Jie Huang, Fei Huang, Binyuan Hui, Shutong Jiang, Zhaohai Li, Mingsheng Li, Mei Li, Kaixin Li, Zicheng Lin, Junyang Lin, Xuejing Liu, Jiawei Liu, Chenglong Liu, Yang Liu, Dayiheng Liu, Shixuan Liu, Dunjie Lu, Ruilin Luo, Chenxu Lv, Rui Men, Lingchen Meng, Xuancheng Ren, Xingzhang Ren, Sibo Song, Yuchong Sun, Jun Tang, Jianhong Tu, Jianqiang Wan, Peng Wang, Pengfei Wang, Qiuyue Wang, Yuxuan Wang, Tianbao Xie, Yiheng Xu, Haiyang Xu, Jin Xu, Zhibo Yang, Mingkun Yang, Jianxin Yang, An Yang, Bowen Yu, Fei Zhang, Hang Zhang, Xi Zhang, Bo Zheng, Humen Zhong, Jingren Zhou, Fan Zhou, Jing Zhou, Yuanzhi Zhu, and Ke Zhu. Qwen3-vl technical report. CoRR, abs/2511.21631, 2025. [17]Mathilde Caron, Hugo Touvron, Ishan Misra, Hervé Jégou, Julien Mairal, Piotr Bojanowski, and Armand Joulin. Emerging properties in self-supervised vision transformers. In 2021 IEEE/CVF International Conference on Computer Vision, ICCV 2021, Montreal, QC, Canada, October 10-17, 2021, pages 9630–9640. IEEE, 2021. [18] Stephanie Fu, Netanel Tamir, Shobhita Sundaram, Lucy Chai, Richard Zhang, Tali Dekel, and Phillip Isola. Dreamsim: Learning new dimensions of human visual similarity using synthetic data. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. [19]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agar- wal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In Marina Meila and Tong Zhang, editors, Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings of Machine Learning Research, pages 8748–8763. PMLR, 2021. [20]Juan A. Rodríguez, Haotian Zhang, Abhay Puri, Rishav Pramanik, Aarash Feizi, Pascal Wich- mann, Arnab Kumar Mondal, Mohammad Reza Samsami, Rabiul Awal, Perouz Taslakian, Spandana Gella, Sai Rajeswar Mudumba, David Vázquez, Chris Pal, and Marco Pedersoli. 11 Rendering-aware reinforcement learning for vector graphics generation. In Danielle Belgrave, Cheng Zhang, Laura N. Montoya, Hsuan-Tien Lin, Razvan Pascanu, Piotr Koniusz, Marzyeh Ghassemi, Nancy Chen, Iván Vladimir Meza Ruíz, and Arturo Loaiza-Bonilla, editors, Ad- vances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, 2025. [21]Jonas Gehring, Kunhao Zheng, Jade Copet, Vegard Mella, Taco Cohen, and Gabriel Synnaeve. RLEF: grounding code llms in execution feedback with reinforcement learning. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu, editors, Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 of Proceedings of Machine Learning Research. PMLR / OpenReview.net, 2025. [22]Hung Le, Yue Wang, Akhilesh Deepak Gotmare, Silvio Savarese, and Steven Chu-Hong Hoi. Coderl: Mastering code generation through pretrained models and deep reinforcement learning. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh, editors, Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9, 2022, 2022. [23] Jiate Liu, Yiqin Zhu, Kaiwen Xiao, Qiang Fu, Xiao Han, Wei Yang, and Deheng Ye. RLTF: reinforcement learning from unit test feedback. Trans. Mach. Learn. Res., 2023, 2023. [24]Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, and Peter Clark. Self-refine: Iterative refinement with self-feedback. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. [25] Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: language agents with verbal reinforcement learning. In Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine, editors, Advances in Neural Information Processing Systems 36: Annual Conference on Neural Information Processing Systems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, 2023. [26]Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC: large language models can self-correct with tool-interactive critiquing. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. [27]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik R. Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. [28]Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. [29]Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An. Group-in-group policy optimization for LLM agent training. In Danielle Belgrave, Cheng Zhang, Laura N. Montoya, Hsuan-Tien Lin, Razvan Pascanu, Piotr Koniusz, Marzyeh Ghassemi, Nancy Chen, Iván Vladimir Meza Ruíz, and Arturo Loaiza-Bonilla, editors, Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, 2025. [30]Yifeng Ding, Hung Le, Songyang Han, Kangrui Ruan, Zhenghui Jin, Varun Kumar, Zijian Wang, and Anoop Deoras. Empowering multi-turn tool-integrated agentic reasoning with group turn policy optimization. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David 12 Jurgens, editors, Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2026, San Diego, California, United States, July 2-7, 2026, pages 42409–42423. Association for Computational Linguistics, 2026. [31]Jonas Belouadi, Eddy Ilg, Margret Keuper, Hideki Tanaka, Masao Utiyama, Raj Dabre, Steffen Eger, and Simone Paolo Ponzetto. Tikzero: Zero-shot text-guided graphics program synthesis. In IEEE/CVF International Conference on Computer Vision, ICCV 2025, Honolulu, HI, USA, October 19-25, 2025, pages 17793–17806. IEEE, 2025. [32]Juan A. Rodríguez, Abhay Puri, Shubham Agarwal, Issam H. Laradji, Sai Rajeswar, David Vázquez, Christopher Pal, and Marco Pedersoli. Starvector: Generating scalable vector graphics code from images and text. In Toby Walsh, Julie Shah, and Zico Kolter, editors, Thirty- Ninth AAAI Conference on Artificial Intelligence, Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence, Fifteenth Symposium on Educational Advances in Arti- ficial Intelligence, AAAI 2025, Philadelphia, PA, USA, February 25 - March 4, 2025, pages 29691–29693. AAAI Press, 2025. [33]Jonas Belouadi, Anne Lauscher, and Steffen Eger. Automatikz: Text-guided synthesis of scientific vector graphics with tikz. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. [34]Ronghuan Wu, Wanchao Su, Kede Ma, and Jing Liao. Iconshop: Text-guided vector icon synthesis with autoregressive transformers. ACM Trans. Graph., 42(6):230:1–230:14, 2023. 13 Appendix Appendix Overview §A Additional Qualitative Results and Failure Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 §A.1 Qualitative Progression across Learning Settings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 §A.2 Successful Visual Revision . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 15 §A.3 Execution Repair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16 §A.4 Over-Revision of Strong Intermediate States . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 §A.5 Preservation and Unresolved Cases . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 §B Implementation Details . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18 §B.1 Full Optimization Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 §B.2 Prompt and Routing Contract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 §B.3 Execution Routing and State Construction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20 §B.4 Training Configuration . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21 §B.5 Execution and Reward Environment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 §C Data and Evaluation Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 §C.1 Training Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23 §C.2 Held-Out Evaluation Data . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 §C.3 Evaluation Metrics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 24 §C.4 Inference and Rendering Protocol . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 §C.5 Checkpoint Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 §D Extended Learning Dynamics and Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25 §D.1 Complete Learning Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 §D.2 Paired-Success Correction Dynamics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 §D.3 Continued Direct-GRPO Controls . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27 §D.4 Paired Bootstrap on Final-945 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 §E Credit Assignment Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 §E.1 Factorized Credit Variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 §E.2 Downstream versus Local Stage 0 Credit . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 28 §E.3 Direct Audit of Credit-Induced Preferences . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29 §E.4 Optimization Behavior of the Credit Variants . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30 §E.5 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 §F State-Dependent Correction and Post-Execution Behavior . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 §F.1 Full State-Dependent Correction Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31 §F.2 State Dependence across Checkpoints . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 §F.3 Revision and Repair Routing during Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32 §F.4 Execution Transitions and Repair . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 §F.5 Execution-Aware Selection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33 §F.6 Summary . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34 14 A Additional Qualitative Results and Failure Cases Stage 0Stage 1Target ImageStage 0Stage 1Target Image Figure 5: Additional qualitative examples of our method from Final-945. This section provides additional qualitative examples from Final-945. All examples from our method use the checkpoint selected on Dev-150 at step 100 and the same inference protocol as the quantitative evaluation. The examples complement the quantitative analyses by illustrating changes across learning settings, successful visual revision, execution repair, over-revision, preservation, and unresolved cases. A.1 Qualitative Progression across Learning Settings Figure 6 compares outputs across the main learning settings studied in the paper. For each target image, we show predictions from the base model, Direct-SFT, Direct-GRPO, and Stage 0 and Stage 1 of our trained policy. The examples complement the aggregate results in Table 1. Execution-feedback training can substantially change the direct prediction relative to supervised code learning, while our Stage 0 output shows that the shared policy itself improves further. Stage 1 then provides an additional opportunity to act on discrepancies revealed by execution. Stage 1 does not necessarily introduce a visible change. When Stage 0 already closely matches the target, the Stage 1 render can remain effectively unchanged. These cases are consistent with the state-dependent analysis in Appendix F.1, where the marginal value of correction decreases as the intermediate state becomes stronger. A.2 Successful Visual Revision Figure 7 shows cases in which Stage 0 already produces a valid program but its rendered result still differs visibly from the target. Stage 1 receives the target image, the Stage 0 render, and the corresponding TikZ program, and revises the program from this executed state. The examples show that execution-conditioned correction is not restricted to small cosmetic edits. In some cases, Stage 1 substantially reorganizes an incorrect visual configuration, for example by changing geometric structure or restoring a missing layout component. In others, the correction is more localized, such as modifying the shape, placement, or appearance of a visual element. The second action is conditioned on the specific program and render produced by Stage 0 rather than independently resampling from the target image. The examples therefore illustrate how the policy can use the observed execution result to modify its own previous program. The cross-metric improvements reported in Section 4.3 and Appendix F.1 show that these changes are not confined to RSim. 15 BaseDirect-SFTTarget ImageStage 0Stage 1Direct-GRPO Failure Compile Failure Compile Failure Compile Failure Compile Failure Compile Failure Compile Figure 6: Qualitative progression across the main learning settings on Final-945. Base model, Direct- SFT, and Direct-GRPO show single-call predictions. Ours Stage 0 is also a single-call prediction from the trained policy, while Stage 1 is produced after observing the Stage 0 execution state. The examples include visible corrective changes and cases where an already strong Stage 0 result is largely preserved. A.3 Execution Repair The corrective action is also used for execution repair. When Stage 0 fails, no rendered image is available. Stage 1 instead receives the target image, the unsuccessful TikZ program, and a compact execution status. Figure 8 shows examples in which this route turns an invalid Stage 0 attempt into a valid program. The selected examples include complete Stage 0 generations that fail during compilation, after which Stage 1 modifies the existing program into an executable one. The corrective action therefore operates on an unsuccessful program state rather than simply appending missing tokens. Repair and visual revision receive different observations but use the same improvement-relative credit. For an invalid Stage 0 execution,S 0 = 0, so a successful repair with positive visual similarity receives positive corrective credit. Repair trajectories are less frequent than visual-revision trajectories, as quantified in Appendix F.3, while their execution outcomes are analyzed in Appendix F.4. 16 Stage 0 Stage 1✓ Target ImageStage 0 Stage 1✓ Target Image Figure 7: Successful visual revisions from Final-945 at checkpoint 100. Stage 0 already renders successfully, and Stage 1 revises the executed state to better match the target. The examples illustrate both large structural corrections and more localized changes to geometry, visual elements, or layout. Failure Compile Failure Compile Failure Compile Failure Compile Stage 0 Stage 1✓ Target ImageStage 0 Stage 1✓ Target Image Figure 8: Successful execution repair from Final-945. Stage 0 does not produce a valid render and therefore receives zero execution score. Stage 1 conditions on the failed program and its execution status and produces a valid TikZ program. The illustrated repairs include cases in which Stage 0 has already produced a complete document that nevertheless fails compilation. A.4 Over-Revision of Strong Intermediate States Correction is not always beneficial. Figure 9 shows examples in which Stage 0 already provides a strong reconstruction and Stage 1 modifies the program in a way that reduces visual fidelity. Some Stage 1 outputs introduce visible changes that are unsupported by the target, such as additional graphical elements, distorted annotations, or large rendering artifacts. These examples provide a qualitative counterpart to the negative average correction observed for the strongest Stage 0 bucket in Appendix F.1. Together with the preservation cases in Figure 6, they illustrate the asymmetric value of the corrective action: substantial modification can help when the executed state contains clear discrepancies, while an already strong state may be better left largely unchanged. When correction-enabled inference is used, execution-aware selection can retain Stage 0 when the subsequent correction is worse. The rule does not generate a new candidate; it only chooses between the two outputs already produced. A.5 Preservation and Unresolved Cases Stage 1 does not always substantially change the state produced by Stage 0. Figure 10 shows both useful preservation and cases where important errors remain unresolved. When Stage 0 already provides a strong reconstruction, preserving the existing result can avoid unnecessary modification. In other cases, however, Stage 0 still contains visible errors and Stage 1 changes little. 17 Stage 0 ✓ Stage 1Target Image Stage 0 ✓ Stage 1Target Image Figure 9: Examples of over-revision at checkpoint 100. Stage 0 already provides a useful reconstruc- tion, while the Stage 1 modification reduces RSim or introduces visible artifacts. The execution-aware selection rule retains Stage 0 in these cases. Stage 0 ✓Stage 1 ✓ Target Image Stage 0 ✓Stage 1 ✓ Target Image Figure 10: Examples of preservation and unresolved cases on Final-945. Stage 1 may preserve an already strong Stage 0 result, or leave an imperfect reconstruction largely unchanged. Some unsuccessful Stage 0 programs also remain non-executable after the repair action. Together with the successful revisions, execution repairs, and over-revisions shown above, these examples illustrate the range of behaviors of the corrective action: it can improve, degrade, or largely preserve the execution state it receives. B Implementation Details This section provides implementation details omitted from the main text. We first give the complete optimization procedure, followed by the prompt templates and execution-dependent routing used for the two actions. We then summarize the training configuration and execution environment. 18 B.1 Full Optimization Algorithm Algorithm 1 summarizes one training iteration of our method. For each target image, we sample K = 8independent complete trajectories. Each trajectory contains an initial action followed by a corrective action conditioned on the execution state produced by that initial action. Both actions are sampled from the same pre-update policy, and no parameter update occurs during rollout collection. Algorithm 1 One training iteration of our method Require: Policy π θ k , target-image batchB, group size K = 8, credit scale λ = 10 1: for each target image I ⋆ ∈B do 2:for i = 1,...,K do 3:Sample initial program C 0,i ∼ π θ k (·| I ⋆ ) 4:Execute C 0,i and obtain Stage 0 score S 0,i = RSim(I 0,i ,I ⋆ ), if execution succeeds, 0,otherwise 5:if C 0,i produces a valid render I 0,i then 6:Construct a visual-revision state (I ⋆ ,I 0,i ,C 0,i ) 7:Sample C 1,i ∼ π θ k (·| I ⋆ ,I 0,i ,C 0,i ) 8:else 9:Construct a repair state (I ⋆ ,C 0,i ,e 0,i ) 10:Sample C 1,i ∼ π θ k (·| I ⋆ ,C 0,i ,e 0,i ) 11:end if 12:Execute C 1,i and obtain S 1,i = RSim(I 1,i ,I ⋆ ), if execution succeeds, 0,otherwise 13:Compute action-specific returns G 0,i = S 1,i , G 1,i = S 1,i − S 0,i 14:end for 15:Mean-center and scale the two return sets independently A 0,i = λ   G 0,i − 1 K K X j=1 G 0,j   , A 1,i = λ   G 1,i − 1 K K X j=1 G 1,j   16: end for 17: Optimize the Stage 0 loss over action microbatches with weight 0.5 18: Optimize the Stage 1 loss over action microbatches with weight 0.5 Equation 7 in the main text summarizes the relative weighting of the two action losses. In the implementation, the two losses are optimized sequentially over action microbatches rather than accumulated into one optimizer update. All Stage 0 and Stage 1 trajectories in an iteration are generated before this optimization begins. Both Direct-GRPO and our method use the same clipped token-level loss form and Dr.GRPO-style fixed-length normalization. For our method, letM a,b denote action microbatchbof stagea∈0, 1, and letθ a,b denote the current policy parameters when that microbatch is processed. For each completion token, the implementation first computes a detached reference log probability from the current policy, q ref a,i,t = sg logπ θ a,b (c a,i,t | x a,i ,c a,i,<t ) ,(9) wheresg[·]denotes stop-gradient andx a,i is the action-specific context. A differentiable forward pass of the same current policy is then used to form r a,i,t (θ) = exp logπ θ (c a,i,t | x a,i ,c a,i,<t )− q ref a,i,t ,(10) with the differentiable forward evaluated at the current parametersθ = θ a,b before the corresponding update. 19 The clipped token-level surrogate is ℓ a,i,t (θ) =− min [r a,i,t (θ)A a,i , clip (r a,i,t (θ), 1− ε low , 1 + ε high )A a,i ].(11) For an action microbatchM a,b , the optimized loss is L a,b = w a N a T max X i∈M a,b T max X t=1 m a,i,t ℓ a,i,t (θ), w 0 = w 1 = 0.5,(12) whereN a is the number of local actions for stageain the training iteration,T max is the maximum completion length, andm a,i,t masks padding and non-completion positions. The denominator remains fixed at the stage level when the actions are split into microbatches. Completions that reach the generation limit are masked from the policy loss. We useε low = 0.20andε high = 0.28. No group-standard-deviation normalization is applied, and no KL term is added (β = 0). Only completion tokens contribute to the policy loss. Because the detached reference log probabilities are recomputed from the current policy immediately before each differentiable forward pass, they are not a frozen copy of the policy that generated the rollout. Consistent with this implementation, the recorded clipping ratio is zero throughout the formal 8B run, so the clipping branch is inactive; the advantage-weighted token loss and its gradients remain active. For the formal 8B run, training uses four ranks with two target images per rank, yielding eight target-image groups per training iteration. Each group contains eight complete trajectories, and each rank receives 16 actions per stage for optimization. With an action microbatch size of two, each stage is processed in eight action microbatches. Under the executed ZeRO-2 configuration, each such microbatch reaches an AdamW update boundary, giving eight updates for Stage 0 followed by eight updates for Stage 1, or 16 AdamW updates per training iteration. Checkpoint labels such as 25, 50, 75, 100, and 125 therefore refer to training iterations rather than individual optimizer updates. B.2 Prompt and Routing Contract Three prompt templates are used: direct generation, visual revision, and execution repair. The direct-generation prompt is shared with Direct-SFT and Direct-GRPO and is used for Stage 0 of our method. Stage 1 uses either the revision or repair prompt according to the Stage 0 execution result. The templates below reproduce the prompts used in the formal experiments; bracketed placeholders denote dynamically inserted state information. B.2.1 Direct Generation Prompt The direct prompt receives only the target image. B.2.2 Visual Revision Prompt When Stage 0 renders successfully, Stage 1 receives the target image, the current Stage 0 render, and the current TikZ source. The source is inserted at <CURRENT TIKZ PROGRAM>. B.2.3 Execution Repair Prompt When Stage 0 does not produce a valid render, Stage 1 receives the target image, the unsuccessful TikZ attempt, and a compact execution status instead of a current render. B.3 Execution Routing and State Construction Routing is determined solely by the execution result of Stage 0. A successful render routes the trajectory to visual revision; otherwise it is routed to repair or completion. The routing decision does not use RSim or any learned threshold. For the revision route, the Stage 1 multimodal context is ordered as target image → current rendered image → current TikZ source. The rendered image and TikZ source are produced by the same Stage 0 action, so the second call observes the visual consequence of the exact program it is asked to modify. 20 You are an expert Image-to-TikZ model and LaTeX/TikZ developer. Study the provided target image carefully and reconstruct it as a high-quality TikZ figure. Focus on the visible objects, geometry, topology, text and mathematical labels, colors, line and arrow styles, relative sizes, alignment, and overall layout. Choose the native TikZ, PGFPlots, circuitikz, tikz-cd, or other LaTeX drawing constructs that best reproduce the image. Do not invent or omit important visible content when it can be represented faithfully. OUTPUT REQUIREMENTS - Return only raw LaTeX source code. Do not output Markdown fences, explanations, review prose, or any text outside the document. - Output exactly one complete standalone document. Start with [tikz]standalone; only when a small margin is needed to prevent visible clipping, use border=1pt, border=2pt, or border=3pt in the same document class. - Include every package and TikZ library required by the generated code in the preamble. The document must be self-contained, use native TikZ/PGF drawing, and compile successfully with pdflatex with shell escape disabled. Do not load external files or assets. - Reconstruct the target image as faithfully as possible while preserving compilability and document completeness. - Include exactly one document and one matching document. The final non- whitespace content must be document; stop immediately after it. Direct Generation Prompt Figure 11: Direct generation prompt. For the repair route, the context contains target image → compact execution status → current TikZ attempt. The status is intentionally compact rather than a full compiler log. Depending on the failure, it can include the tagshit_max_tokens,document_incomplete, and the leading category of the renderer/compiler error; render_failed is used as a fallback. A generated program is considered complete at the first document. Generation terminates independently for each sequence once this string is produced, subject to a maximum of 8192 completion tokens for either action. B.4 Training Configuration The training subsets and initialization sequence are summarized in Appendix C.1. Here we report the optimization and runtime configuration used for Direct-SFT, Direct-GRPO, and our method. Direct-SFT uses a global batch size of 32 (two GPUs, eight examples per GPU, and two gradient- accumulation steps). Direct-GRPO uses two target prompts per device with gradient accumulation of eight andK = 8sampled completions per target. Across four devices, this corresponds to eight target-image groups per training iteration and one AdamW update per iteration. Our method also uses two target images per rank on four ranks, yielding eight target-image groups per training iteration; each group contains eight independent complete trajectories. The policy loss is evaluated in action microbatches of two. Each rank therefore processes 16 actions per stage as eight action microbatches, which gives 16 AdamW updates per training iteration under the executed ZeRO-2 configuration, as detailed in Appendix B.1. The main run uses random seed 20260823. For our method, rollout generation uses temperature1.0, top-p 0.9, repetition penalty1.0, and a maximum of 8192 tokens for each action. Stage 0 and Stage 1 generation use microbatch sizes of six and three, respectively. Training uses AdamW with weight decay10 −2 , a constant learning-rate schedule, no warmup, and gradient clipping at norm 1.0. 21 Visual Revision Prompt You are an expert Image-to-TikZ revision model and LaTeX/TikZ developer. You are given three aligned inputs describing one revision state: 1. TARGET / REFERENCE IMAGE: the first image is the visual ground truth that the final TikZ should reproduce. 2. CURRENT RENDERED IMAGE: the second image is the rendering produced by the current TikZ program below. 3. CURRENT TIKZ PROGRAM: the editable source code that produced the current rendered image. The current rendered image and current TikZ program describe the same current state. Compare that state against the target image and revise the current TikZ program so that the new rendered result matches the target image as faithfully as possible. Use the target image as the authority for visible content. Use the current rendered image as execution evidence for what the current program actually draws, and use the current TikZ as the program to edit. Focus on visible objects, geometry, topology, text and mathematical labels, colors, line and arrow styles, relative sizes, alignment, spacing, and overall layout. Preserve correct parts when appropriate, while fixing missing, extra, misplaced, malformed, or visually incorrect elements. Do not add details that are not visually supported by the target image merely because they appear in the current code. CURRENT TIKZ PROGRAM CURRENT_TIKZ_PLACEHOLDER OUTPUT REQUIREMENTS - Return only the complete revised raw LaTeX source code. Do not output Markdown fences, explanations, review prose, analysis, or any text outside the document. - Output exactly one complete standalone document. Start with [tikz]standalone; only when a small margin is needed to prevent visible clipping, use border=1pt, border=2pt, or border=3pt in the same document class. - Include every package and TikZ library required by the revised code in the preamble. The document must be self-contained, use native TikZ/PGF drawing, and compile successfully with pdflatex with shell escape disabled. Do not load external files or assets. - The revised program may keep, modify, reorganize, or replace parts of the current program as needed to improve visual fidelity to the target image. Return the full document, not a patch or diff. - Include exactly one document and one matching document. The final non- whitespace content must be document; stop immediately after it. Figure 12: Visual revision prompt. B.5 Execution and Reward Environment Direct-GRPO and our method use the RSim-based execution score defined in Eq. 1 during reinforce- ment learning. For our method, execution uses TeX Live 2026 and attemptspdflatex,lualatex, andxelatex, with shell escape disabled. Compilation has a 30-second timeout, and rasterization uses 150 DPI before conversion to the448× 448model input resolution. Failed compilation or rendering receivesS t = 0. The same execution score is used for both actions; their learning signals differ through the action-specific returns defined in Section 3.2. C Data and Evaluation Protocol This section provides additional details on the training subsets, held-out evaluation data, evaluation metrics, inference protocol, and checkpoint selection used throughout the experiments. 22 Execution Repair Prompt You are an expert Image-to-TikZ model and LaTeX/TikZ developer. The first-stage TikZ attempt did not produce a valid rendered image. Use the target image as visual ground truth and the current TikZ attempt as editable state. Complete or repair the program into one complete, self-contained, compilable TikZ document that matches the target image as faithfully as possible. STAGE-0 STATUS FAILURE_STATUS_PLACEHOLDER CURRENT TIKZ ATTEMPT CURRENT_TIKZ_PLACEHOLDER OUTPUT REQUIREMENTS - Return only raw LaTeX source code. Do not output Markdown fences, explanations, review prose, analysis, or text outside the document. - Return one complete standalone document, not a patch or diff. - Preserve useful existing content when appropriate, but freely repair, complete, reorganize, or replace the current attempt when needed. - The document must compile with shell escape disabled and must not depend on external assets. - Include exactly one document and one matching document. The final non- whitespace content must be document; stop immediately after it. Figure 13: Execution repair prompt. Table 3: Optimization and runtime configuration for Direct-SFT, Direct-GRPO, and our method. The final column reports the configuration used by our method in the main experiments. Direct-SFTDirect-GRPOOurs Training duration3 epochs1 epoch / 125 iterations 1 epoch / 125 iterations Learning rate1× 10 −4 5× 10 −6 5× 10 −6 Weight decay–10 −2 10 −2 LoRAr = 256, α = 512continuedcontinued Group size K–88 GRPO loss–Dr.GRPODr.GRPO β–00 Clip range–[0.80, 1.28][0.80, 1.28] Reward normalization–nonenone Credit scale λ–10 Action loss weights–0.5/0.5 Sampling temperature–1.01.0 Top-p–0.90.9 Maximum completion2,0488,1928,192 / action Image resolution448× 448448× 448448× 448 PrecisionBF16BF16BF16 Gradient checkpointing✓ Distributed trainingDDPZeRO-2ZeRO-2 A100-80GB GPUs244 C.1 Training Data All supervised and reinforcement-learning data are drawn from DaTikZ-V4 [14]. Direct-SFT is trained on 3,851 eligible image–TikZ pairs obtained from an approximately 4K candidate subset after preprocessing. Direct-GRPO uses 1,000 examples sampled from this supervised-training pool. Our method is initialized from the final Direct-GRPO policy and reuses the same 1,000 target images. The corrective action is generated from execution states produced by the policy’s own predictions, so no additional training examples or paired program annotations are introduced. 23 Table 4: Training data used by the main learning settings. The Direct-GRPO subset is contained in the supervised-training pool, and the same 1K subset is reused by our method. MethodTraining ExamplesInitializationTraining Signal Direct-SFT3,851BaseReference TikZ code Direct-GRPO1,000Direct-SFTExecuted outcome S Ours1,000Direct-GRPO S 1 / S 1 − S 0 For the continued Direct-GRPO controls in Section 4.2, we continue Direct-GRPO from the same final Direct-GRPO checkpoint on the same 1K target images while retaining the single-stage Direct-GRPO objective. We evaluate controls after 100 additional training iterations, matching the training-iteration budget and target-image exposure at the Dev-selected checkpoint of our method, and after 125 additional training iterations as an endpoint comparison. The controls do not match the number of low-level optimizer updates performed by our implementation; the corresponding update schedule is described in Appendix B. C.2 Held-Out Evaluation Data We construct the evaluation set independently from the training subsets using the same broad source families as DaTikZ-V4: arXiv, GitHub, and TeX StackExchange. Evaluation examples are restricted to source records from 2026 onward; the examples used in this work span January to mid July 2026. Each retained item contains an executable reference TikZ program and its corresponding rendered target image, and all examples undergo manual review during dataset construction. The paper uses 1,095 held-out examples. We fix 150 examples as Dev-150 for checkpoint selection, ablations, and behavioral analysis, and reserve the remaining 945 examples as Final-945 for final evaluation. Final-945 is not used for checkpoint selection or method development. Table 5: Composition of the held-out evaluation set. All samples are drawn from source records dated in 2026. SplitarXivGitHubTeX.SETotal Dev-150824325150 Final-945458309178945 All5403522031,095 The two splits contain distinct evaluation examples and serve different purposes throughout the paper: all checkpoint-dependent analysis is performed on Dev-150, while Final-945 is evaluated only after the checkpoint and inference procedure have been fixed. C.3 Evaluation Metrics We evaluate both executability and visual fidelity. Render success indicates whether a generated program can be successfully compiled and rasterized. For a valid renderI, visual fidelity to the target imageI ⋆ is measured using four complementary image-similarity metrics: RSim [14], DINO [17], DreamSim [18], and CLIPImg [19]. RSim uses the DeTikZify-V2 vision encoder and serves as the only visual optimization reward in Direct-GRPO and our method. DINO, DreamSim, and CLIPImg are used only for evaluation and do not participate in training or execution-aware selection. We report DreamSim similarity as one minus the perceptual distance returned by the official DreamSim implementation. Under the fixed-zero convention, unsuccessful executions receive zero for every visual metric and successful executions retain their measured similarity. This convention is used for the main aggregate results because it reflects both executability and visual quality. When analyzing the visual effect of the corrective action independently of execution recovery, we instead restrict to the paired-success subset in which both Stage 0 and Stage 1 render successfully, as defined in Section 4.3. 24 Table 6: Evaluation metrics and their roles. All main visual results use the fixed-zero convention. MetricRole in TrainingMain Evaluation Convention Render successExecution signalSuccess / failure RSimRL reward + evaluationFailure→ 0 DINOEvaluation onlyFailure→ 0 DreamSimEvaluation onlyFailure→ 0 CLIPImgEvaluation onlyFailure→ 0 C.4 Inference and Rendering Protocol All paper-active evaluations use stochastic decoding with temperature1.0, top-p 0.9, repetition penalty1.0, a maximum of 8192 newly generated tokens, and early termination at the first complete document. Direct models produce a single output from the target image. For our method, Stage 0 is generated from the target image alone and is reported as the single-call inference output. In correction-enabled evaluation, the same policy is called again after Stage 0 execution to produce Stage 1 according to the routing procedure in Appendix B.3. The final Selected output is chosen only from the two programs already produced: • if only Stage 0 renders, Stage 0 is retained; • if only Stage 1 renders, Stage 1 is retained; • if both render, the program with higher RSim is retained; • ties are assigned to Stage 0; • if neither renders, the selected output is unsuccessful. No additional generation is performed by the selection procedure. Generation parameters and evaluation metrics are unified across the paper-active evaluations. The Direct Final-945 evaluation uses a 120-second compilation timeout, whereas the correction-enabled renderer uses a 30-second timeout; all successful Direct predictions in Final-945 execute within 30 seconds. Both pipelines normalize rendered images to 448× 448 before visual evaluation. C.5 Checkpoint Selection Checkpoints of our method are evaluated on Dev-150 at the initializer and after 25, 50, 75, 100, and 125 training iterations. For consistency with the training logs and figures, we retain the checkpoint labels Step 25 through Step 125 throughout the paper. For each checkpoint, we evaluate Stage 0, raw Stage 1, and the execution-aware selected output using the same inference and metric protocol. Table 7 reports the quantities used to compare the checkpoints. Among the evaluated checkpoints, checkpoint 100 achieves the highest selected render rate as well as the highest selected RSim, DINO, DreamSim, and CLIPImg scores. We therefore fix checkpoint 100 for subsequent single-policy behavioral analyses and for the final evaluation on Final-945. Training-dynamics analyses instead report the full checkpoint trajectory. The selected checkpoint is fixed before Final-945 evaluation. Final-945 is therefore used only to report final performance rather than to choose between training checkpoints or analysis settings. D Extended Learning Dynamics and Controls This section provides the complete numerical results behind the learning dynamics reported in the main text, compares our method with continued single-stage Direct-GRPO under matched training- iteration budgets, and quantifies sample-level uncertainty for selected Final-945 comparisons. All Dev-150 checkpoint results follow the evaluation protocol described in Appendix C. 25 Table 7: Checkpoint evaluation for our method on Dev-150. Visual metrics are computed on the execution-aware selected output using the fixed-zero convention. Checkpoint 100 gives the highest selected render rate and all four selected visual metrics. Checkpoint Stage 0 Render Stage 1 Render Selected RenderRSimDINODreamSim CLIPImg Initializer82.00%82.67%82.67%0.4736 0.70880.64970.7468 Step 2587.33%88.00%89.33%0.5161 0.76670.70760.8116 Step 5089.33%88.00%90.67%0.5223 0.78050.71600.8196 Step 7591.33%90.00%92.00%0.5454 0.79750.73860.8416 Step 10096.67%94.67%97.33%0.5646 0.83260.76770.8774 Step 12595.33%92.67%95.33%0.5549 0.81170.75380.8666 D.1 Complete Learning Dynamics We evaluate the initializer and checkpoints at steps 25, 50, 75, 100, and 125 on Dev-150. Table 8 reports the execution behavior of both actions together with the four possible execution transitions. We denote successful and failed executions bySandF, respectively. For example,FSdenotes a Stage 0 failure followed by a successful Stage 1 repair. Table 8: Complete execution dynamics of our method on Dev-150. Each row contains 150 examples. S, SF, FS, and F denote the execution outcomes of Stage 0 and Stage 1, respectively. CheckpointStage 0 RenderStage 1 RenderSSSFFSFF Initializer82.00%82.67%1230126 Step 2587.33%88.00%1292316 Step 5089.33%88.00%1304214 Step 7591.33%90.00%1343112 Step 10096.67%94.67%141414 Step 12595.33%92.67%139407 The execution rates improve substantially over training. Stage 0 rises from82.00%for the initializer to96.67%at step 100, while Stage 1 rises from82.67%to94.67%. Most of this change comes from the reduction ofFFtrajectories. As Stage 0 improves, failed initial executions also become less frequent, so the repair route is exercised less often on later Dev-150 checkpoints. Appendix F analyzes routing and repair in more detail. D.2 Paired-Success Correction Dynamics To isolate visual correction from execution recovery, we use the paired-success subset defined in Section 4.3. For each visual metric M , the reported quantity is ∆M = M (I 1 ,I ⋆ )− M (I 0 ,I ⋆ ). Positive values indicate that Stage 1 improves the already executable Stage 0 state. Table 9 reports the complete paired-success visual dynamics. The correction-outcome categories follow the definition used in the main text:∆RSim > 0.005is an improvement,|∆RSim|≤ 0.005 is neutral, and ∆RSim <−0.005 is a degradation. The complete trajectory highlights the distinction between initial generation quality and corrective behavior. The initializer has negative average Stage 1 changes under all four visual metrics. By steps 50–75, correction becomes more effective, and all four mean changes are positive at step 75. The trend is not monotonic: at step 100, Stage 0 reaches its highest render rate, while the average paired-success change becomes slightly negative. At step 125, RSim, DINO, and DreamSim again show positive average changes. Checkpoint selection is therefore based on the complete inference result rather than the paired-success correction gain alone. Step 100 provides the strongest selected result on Dev-150 (Appendix C.5), while Appendix F.1 shows that its corrective behavior varies with the quality of the intermediate state. 26 Table 9: Visual correction dynamics on the paired-success subset of Dev-150. The four∆columns report the mean Stage 1 minus Stage 0 change. The last three columns show the composition of correction outcomes under the±0.005 RSim neutral region used in the main text. Checkpoint Paired n∆RSim∆DINO ∆DreamSim ∆CLIPImg ImproveNeutral Degrade Initializer123 −0.0031 −0.0027 −0.0039 −0.001712.20% 65.04%22.76% Step 25129 +0.0013 −0.0014+0.0043+0.004228.68% 48.06%23.26% Step 50130 +0.0098 +0.0053+0.0077+0.001643.85% 26.92%29.23% Step 75134 +0.0121 +0.0058+0.0112+0.006342.54% 30.60%26.87% Step 100141 −0.0012 −0.0072 −0.0047 −0.000043.26% 19.86%36.88% Step 125139 +0.0065 +0.0057+0.0025 −0.001346.76% 19.42%33.81% D.3 Continued Direct-GRPO Controls We compare our method with continued single-stage Direct-GRPO from the same Direct-GRPO initializer and on the same 1K training images. The+100comparison matches the additional training- iteration budget and target-image exposure at the Dev-selected checkpoint used for final evaluation. The+125comparison uses the corresponding training endpoint. The controls match initialization, training data, training-iteration count, and target-image exposure, but not the number of low-level optimizer updates; our method additionally samples the execution-conditioned corrective action during training. Table 10 compares single-call outputs on Final-945. For our method, this is the Stage 0 output generated directly from the target image without invoking correction at inference time. Table 10: Continued Direct-GRPO single-call results on Final-945. Both methods start from the same Direct-GRPO checkpoint and use the same 1K training images. The+100row corresponds to the Dev-selected checkpoint of our method; +125 reports the training endpoint. Additional IterationsMethodRenderRSimDINODreamSimCLIPImg +100Continued Direct-GRPO88.25%0.51350.76290.70190.7996 +100Ours (Stage 0)92.70%0.53820.80140.73300.8372 +125Continued Direct-GRPO87.20%0.51190.75570.69840.7952 +125Ours (Stage 0)93.54%0.53560.80110.73460.8431 Continued Direct-GRPO improves over the original Direct-GRPO checkpoint, but our Stage 0 output is higher under render success and all four visual metrics at both corresponding training endpoints. The gain from training is therefore already present in the initial action and does not require the second call at inference time. We also evaluate the continued Direct-GRPO policies with the same execution-conditioned second call. Table 11 reports paired-success visual changes on Final-945. At both corresponding endpoints, the continued single-stage policies produce changes close to zero, whereas our corrective action yields larger positive mean changes across the reported metrics. Table 11: Continued Direct-GRPO paired-success correction on Final-945.∆Mdenotes Stage 1 minus Stage 0 on examples where both outputs render successfully. Additional IterationsMethodPaired n∆RSim∆DINO∆DreamSim∆CLIPImg +100Continued Direct-GRPO828+0.0002+0.0019+0.0020 −0.0007 +100Ours863+0.0073+0.0034+0.0074+0.0031 +125Continued Direct-GRPO815+0.0017+0.0013+0.0009+0.0008 +125Ours869+0.0079+0.0046+0.0068+0.0035 For the+125continued Direct-GRPO correction analysis, paired statistics use the Stage 0 records from the correction-enabled evaluation run. The canonical single-call Direct result is reported separately in Table 10. 27 D.4 Paired Bootstrap on Final-945 We quantify uncertainty over the fixed Final-945 evaluation set using a paired bootstrap. For each comparison, we resample the same 945 example indices with replacement for both methods and compute the difference in their mean performance. We use 10,000 paired resamples with a fixed random seed of 20260914 and report percentile95%intervals. Positive differences indicate better performance for the second method. Table 12 reports three fixed comparisons: from supervised learning to direct execution feedback, from Direct-GRPO to the Stage 1 output of our trained policy, and from raw Stage 1 to execution-aware selection. Table 12: Paired bootstrap uncertainty on the fixed Final-945 evaluation set (N = 945, 10,000 paired resamples). Each entry reports the observed difference with its95%bootstrap interval. Positive values favor the second method. Render differences are in percentage points; visual metrics use fixed-zero scores. Intervals quantify uncertainty over the fixed evaluation samples rather than variation across training seeds. MetricSFT→ Direct-GRPODirect-GRPO→ Ours Stage 1Stage 1→ Selected ∆ Render Success +17.25 p [+13.86, +20.63] +10.90 p [+8.25, +13.54] +1.38 p [+0.74, +2.12] ∆ RSim +0.1154 [+0.0962, +0.1345] +0.0738 [+0.0580, +0.0892] +0.0203 [+0.0162, +0.0249] ∆ DINO +0.1619 [+0.1334, +0.1905] +0.1000 [+0.0765, +0.1238] +0.0175 [+0.0113, +0.0243] ∆ DreamSim +0.1508 [+0.1244, +0.1773] +0.0932 [+0.0721, +0.1149] +0.0192 [+0.0137, +0.0254] ∆ CLIPImg +0.1629 [+0.1333, +0.1939] +0.0999 [+0.0755, +0.1245] +0.0174 [+0.0111, +0.0246] The intervals exclude zero for all five metrics in each comparison. Direct execution feedback provides a large improvement over supervised code learning, the trained policy provides a further positive shift over the preceding Direct-GRPO checkpoint, and execution-aware selection adds a smaller but consistent improvement over raw Stage 1. These intervals quantify uncertainty over the fixed Final-945 sample set. They do not measure variation across independent training runs. E Credit Assignment Analysis This section expands the credit-assignment analysis in Section 4.4. We first report the controlled comparison between downstream and local credit for the initial action. We then compare the group-relative preferences induced by alternative credit definitions on the same sampled trajectories, followed by the optimization behavior of the four factorized variants. E.1 Factorized Credit Variants The ablation varies the credit assigned to the initial and corrective actions. Stage 0 either receives the final trajectory scoreS 1 or its local intermediate scoreS 0 . Stage 1 either receives its improvement over the intermediate state,S 1 − S 0 , or the absolute final scoreS 1 . All variants use the same group centering and scale λ = 10 defined in Section 3.3. A versus B isolates the Stage 0 credit while keeping improvement-relative Stage 1 credit fixed. A versus C isolates the Stage 1 credit while keeping downstream Stage 0 credit fixed. Variants C and D both use absolute Stage 1 credit with different Stage 0 definitions. E.2 Downstream versus Local Stage 0 Credit Table 14 gives the complete Dev-150 comparison between variants A and B at the checkpoints used in the main analysis. The paired visual changes are computed only on examples for which both stages render successfully. 28 Table 13: Factorized credit-assignment variants. Variant A is our method. The table reports the raw credit before group centering and scaling. VariantStage 0 creditStage 1 credit A (Ours)S 1 S 1 − S 0 BS 0 S 1 − S 0 CS 1 S 1 DS 0 S 1 Table 14: Comparison of downstream Stage 0 credit (A) and local Stage 0 credit (B) on Dev-150. ∆M denotes the mean Stage 1 minus Stage 0 change on the paired-success subset. A and B use the same improvement-relative Stage 1 credit and differ only in whether Stage 0 receives S 1 or S 0 . Checkpoint Variant Stage 0 Render Stage 1 Render∆RSim∆DINO ∆DreamSim ∆CLIPImg Step 25A87.33%88.00%+0.0013 −0.0014+0.0043+0.0042 B84.67%86.00%+0.0012 −0.0015+0.0005 −0.0007 Step 50A89.33%88.00%+0.0098 +0.0053+0.0077+0.0016 B90.00%89.33%+0.0002 +0.0023+0.0038+0.0001 Step 75A91.33%90.00%+0.0121 +0.0058+0.0112+0.0063 B88.00%88.67% −0.0008 −0.0023 −0.0037 −0.0014 The comparison separates Stage 0 quality from the behavior of the subsequent corrective action. At step 50, B has a slightly higher Stage 0 render rate than A (90.00%versus89.33%), and its Stage 0 RSim-FZ is also higher (0.5095versus0.4960). Its paired Stage 1 correction, however, is nearly neutral:∆RSim = +0.0002, compared with+0.0098for A. By step 75, A retains a positive correction gain of +0.0121, whereas B becomes slightly negative at−0.0008. Thus, a stronger intermediate output at one checkpoint does not imply a stronger subsequent correction. UsingS 1 rather thanS 0 for Stage 0 changes which initial actions are favored according to the quality reached after the corrective action. E.3 Direct Audit of Credit-Induced Preferences The trained ablations compare the behavior produced by different credit definitions. We additionally inspect the credit signals while holding the sampled trajectories fixed. For this analysis, we use the completed A training run, containing 1,000 target-image groups and eight independently sampled complete trajectories per group, for a total of 8,000 trajectories. For Stage 0, we compare the downstream advantage A 0 (S 1 ) = S 1 −S 1 with the local advantage A 0 (S 0 ) = S 0 − S 0 . For Stage 1, we compare the improvement-relative advantage A 1 (S 1 − S 0 ) = (S 1 − S 0 )− (S 1 − S 0 ) with the absolute-score advantage A 1 (S 1 ) = S 1 −S 1 . The common factor λ = 10 is omitted because it does not affect correlation, sign, or ranking. For Stage 0, the two definitions remain moderately correlated but produce different group-relative preferences. The advantage sign changes for1,715/8,000 = 21.4%of trajectories, and the highest- advantage trajectory changes in 461/1,000 = 46.1% of target-image groups. The distinction is larger for Stage 1. Improvement-relative and absolute credit have a Pearson correlation of0.3848and a Spearman correlation of0.3146. The advantage sign changes for 2,974/8,000 = 37.2%of trajectories, and the highest-advantage corrective trajectory changes in 745/1,000 = 74.5% of groups. 29 Table 15: Comparison of group-relative advantages induced by alternative credit definitions on the same 8,000 trajectories from 1,000 target-image groups. Sign flips are computed over all trajectories, with exact-zero advantages counted as neither positive nor negative. Top-1 change denotes the fraction of groups whose highest-advantage trajectory changes under the alternative definition. Advantage ComparisonPearsonSpearmanSign FlipGroup Top-1 Change Pairwise Ordering Change A 0 (S 1 ) vs. A 0 (S 0 )0.74080.708321.4%46.1%– A 1 (S 1 − S 0 ) vs. A 1 (S 1 )0.38480.314637.2%74.5%41.76% We additionally compare every unordered pair of trajectories within each group. For a pair(i,j), we compare the sign of A 1,i (S 1 − S 0 )− A 1,j (S 1 − S 0 ) with the sign of A 1,i (S 1 )− A 1,j (S 1 ). Ties form a separate ordering category. Of the28,000within-group trajectory pairs,11,693change their relative ordering, corresponding to41.76%. Thus,S 1 − S 0 andS 1 do not merely rescale the same corrective preference; they change the relative ordering of corrective actions within a group. E.4 Optimization Behavior of the Credit Variants The fixed-trajectory audit above compares the learning signals directly. The corresponding training runs also differ in optimization behavior. Table 16 summarizes the four variants. Table 16: Optimization behavior observed for the four factorized credit variants. VariantStable UntilFirst Abnormal StepObserved Behavior A125–Completed normally B∼ 8182Late length/termination pathology C2425–26Early zero-signal collapse D2526Early zero-signal collapse Variant A completes all 125 training iterations without the length or termination failure observed in the alternative runs. Variant B behaves normally through the checkpoints used for the A–B comparison but develops a late-training length pathology. Its mean Stage 1 completion length is 482 tokens at step 80, 965 tokens at step 81, and 3,663 tokens at step 82. At step 83 it increases to 8,067 tokens, with 98.4%of Stage 1 completions reaching the generation limit and no Stage 1 completion terminating normally at document. Variants C and D show an earlier shared failure regime. Both use absolute Stage 1 creditS 1 but different Stage 0 credits. Their execution behavior collapses around step 26. Table 17 shows the transition around this point. Table 17: Training diagnostics around the C/D collapse. “Hit Max” is the fraction of Stage 1 genera- tions reaching the token limit, “End Doc” is the fraction terminating normally at document, andA 0 /A 1 nonzero counts the trajectories receiving nonzero group-relative advantages. Each training iteration contains 64 complete trajectories. VariantStep Stage 1 TokensHit MaxEnd Doc Stage 0 Compile Stage 1 CompileFF A 0 Nonzero A 1 Nonzero C25272331%45%1011524848 C26563969%0%006400 C27742591%0%006400 D25131311%80%3839236457 D26704186%0%006400 D29–318192100%0%006400 30 From step 26 onward, both C and D enter an all-zero execution regime. All 64 trajectories in the observed training iteration fail at both stages, so the fixed-zero execution score gives identical zero returns within the group. The resulting group-relative advantages, policy loss, and gradient norm are therefore zero, while Stage 1 generations become substantially longer and normal document termination disappears. At the corresponding steps, A and B remain executable and retain nonzero training signals on the same sample groups. C and D use different Stage 0 credits but share the absolute Stage 1 definition, making it the common credit choice associated with the observed early failure regime. E.5 Summary The ablations and fixed-trajectory audit show that the two credit choices affect different parts of learning. Using downstream Stage 0 credit changes which initial programs are favored according to the quality reached after the corrective action. Using improvement-relative Stage 1 credit produces substantially different corrective preferences from absolute final-score credit, changing advantage signs and rankings for a large fraction of the same sampled trajectories. These differences also appear in training behavior. Variant A combines downstream Stage 0 credit with improvement-relative Stage 1 credit and completes the full training run, while the alternative variants exhibit distinct correction behavior or optimization failures. F State-Dependent Correction and Post-Execution Behavior This section expands the state-dependent correction analysis in Section 4.5. We first report the complete statistics behind Figure 4, then examine the same behavior across nearby checkpoints. We finally analyze revision and repair routing, execution transitions, and the optional selection rule applied when both outputs are available. F.1 Full State-Dependent Correction Analysis Figure 4 analyzes the Dev-selected checkpoint at step 100. As in the main text, we restrict this analysis to paired-success examples for which both Stage 0 and Stage 1 produce valid renders. We partition examples according to their Stage 0 RSim and define ∆M = M (I 1 ,I ⋆ )− M (I 0 ,I ⋆ ). We separately report P (S 1 > S 0 ), which denotes the fraction of examples with a strictly higher Stage 1 RSim. This quantity uses the threshold∆RSim > 0, whereas the correction-outcome categories used in Section 4.3 use a±0.005 neutral region. Table 18: Complete state-dependent correction statistics for our method at checkpoint 100 on Dev-150. The analysis is restricted to paired-success examples.∆values denote Stage 1 minus Stage 0. The 95%intervals are bootstrap intervals over examples within each bucket. Improve, neutral, and degrade use the±0.005RSim threshold from the main learning-dynamics analysis, whereasP (S 1 > S 0 ) uses the strict ∆RSim > 0 criterion. Stage 0 RSim n∆RSim95% CIP (S 1 > S 0 )ImproveNeutralDegrade [0.0, 0.4)17+0.0260[+0.0103, +0.0442]76.5%58.8%29.4%11.8% [0.4, 0.5)29+0.0029[−0.0103, +0.0168]55.2%41.4%20.7%37.9% [0.5, 0.6)39+0.0021[−0.0087, +0.0128]53.8%48.7%7.7%43.6% [0.6, 0.7)26 −0.0026[−0.0287, +0.0194]53.8%53.8%15.4%30.8% [0.7, 1.0]30 −0.0235[−0.0593, +0.0026]23.3%20.0%33.3%46.7% The full statistics reinforce the pattern shown in the main figure. The weakest intermediate states receive the largest average correction: for Stage 0 RSim below0.4, Stage 1 improves RSim by0.0260 on average and achieves a higher RSim in76.5%of paired examples. The average gain decreases as 31 Stage 0 quality increases. For the highest-quality bucket, the mean change becomes−0.0235, and only 23.3% of examples obtain a strictly higher Stage 1 RSim. The same contrast is visible under metrics that are not used to train the policy. Table 19 reports the corresponding DINO, DreamSim, and CLIPImg changes. All four metrics improve on average in the weakest bucket, whereas all four decrease in the highest-quality bucket. Table 19: Cross-metric Stage 1 changes at checkpoint 100. RSim is the optimization reward; DINO, DreamSim, and CLIPImg are evaluation-only metrics. Stage 0 RSim∆RSim∆DINO∆DreamSim∆CLIPImg [0.0, 0.4)+0.0260+0.0195+0.0352+0.0191 [0.4, 0.5)+0.0029+0.0002 −0.0001+0.0015 [0.5, 0.6)+0.0021 −0.0166 −0.0093 −0.0033 [0.6, 0.7) −0.0026 −0.0222 −0.0189+0.0030 [0.7, 1.0] −0.0235 −0.0044 −0.0133 −0.0109 The intermediate buckets are less uniform across metrics, while the endpoint contrast is consistent: weak intermediate states leave more room for correction, whereas additional modification of already strong renders is more likely to degrade the result. F.2 State Dependence across Checkpoints The main analysis uses step 100 because it is the checkpoint selected on Dev-150 for final evaluation. We repeat the same analysis at steps 75 and 125 using identical RSim buckets and definitions. Table 20 summarizes the weakest and strongest buckets. Across all three later checkpoints, weak Stage 0 states receive a larger average gain and are more likely to obtain a higher Stage 1 RSim than strong Stage 0 states. Table 20: State-dependent correction across later checkpoints of our method. “Weak” denotes Stage 0 RSim below0.4and “strong” denotes Stage 0 RSim in[0.7, 1.0]. All statistics use paired-success examples and the same fixed bucket boundaries. CheckpointWeak ∆RSimWeak P (S 1 > S 0 )Strong ∆RSimStrong P (S 1 > S 0 ) Step 75+0.041258.3%+0.004528.0% Step 100+0.026076.5%−0.023523.3% Step 125+0.033781.3%−0.015828.0% The exact magnitude changes over training, but the separation between weak and strong intermediate states is preserved. At step 75, the strongest bucket still has a small positive average∆RSim, but Stage 1 achieves a higher RSim in only28.0%of those examples. At steps 100 and 125, the strongest bucket has a negative average change. F.3 Revision and Repair Routing during Training The Stage 1 route is determined independently for every trajectory by its own Stage 0 execution result. A valid Stage 0 render enters visual revision; an unsuccessful execution enters repair. Across the completed training run, the 1,000 target-image groups contain 8,000 complete trajectories. Of these, 7,447 trajectories (93.1%) enter visual revision and 553 (6.9%) enter repair. The distribution across groups is shown in Table 21. The 259 mixed groups also show that a training group contains independently sampled Stage 0 actions rather than one shared Stage 0 state followed by multiple corrective candidates. Each Stage 1 action operates on the execution state produced by its own Stage 0 program. 32 Table 21: Routing composition of the 1,000 training groups. Each group contains eight independently sampled complete trajectories. A mixed group contains at least one revision trajectory and at least one repair trajectory. Group compositionGroupsFraction Revision only73873.8% Mixed revision / repair25925.9% Repair only30.3% All groups1,000100% F.4 Execution Transitions and Repair We next examine how the corrective action changes execution status. We denote the four possible transitions bySS,SF,FS, andFF. TheFStransition corresponds to execution repair, whereasSF denotes an initially valid program that becomes invalid after correction. Table 22: Execution transitions for the initializer and the Dev-selected checkpoint of our method. “Repair success” is P (S 1 succeeds| S 0 fails). SplitPolicySSSFFSFFStage 0 failuresRepair success Dev-150Initializer1230126273.7% Dev-150Step 100141414520.0% Final-945Initializer780561541603.8% Final-945Step 1008631314556920.3% Training substantially improves Stage 0 executability, reducing Final-945 Stage 0 failures from 160 for the initializer to 69 at checkpoint 100. Among the remaining failed states, Stage 1 repairs 14 of 69 programs, compared with 6 of 160 for the initializer. The improvement is therefore already visible before correction is invoked, while the second action adds repair capability for the failed states that remain. The oppositeSFtransition also remains possible. On Final-945, 13 initially successful Stage 0 programs fail after Stage 1. This motivates retaining the initial output when the corrective action makes execution worse. F.5 Execution-Aware Selection For correction-enabled inference, both Stage 0 and Stage 1 are available. Because the effect of Stage 1 depends on the intermediate state, we additionally evaluate the selection rule defined in Appendix C.4. The rule chooses only between the two programs already produced and requires no additional model generation. Table 23 compares Stage 0, raw Stage 1, and the selected output at the fixed checkpoint 100. Table 23: Stage 0, raw Stage 1, and execution-aware selection for our method at checkpoint 100. Visual metrics use the fixed-zero convention. SplitOutputRenderRSimDINODreamSimCLIPImg Dev-150Stage 096.67%0.54790.82250.75540.8668 Stage 194.67%0.53580.79940.73480.8485 Selected97.33%0.56460.83260.76770.8774 Final-945Stage 092.70%0.53820.80140.73300.8372 Stage 192.80%0.54460.80460.74010.8409 Selected94.18%0.56490.82210.75930.8583 On Dev-150, Stage 0 is already strong at checkpoint 100, while raw Stage 1 is lower on the aggregate metrics. The state-dependent analysis nevertheless shows that the corrective action remains useful for 33 weaker intermediate states. Selection retains those gains while keeping Stage 0 when it is preferable, raising selected RSim-FZ to 0.5646. On Final-945, raw Stage 1 is slightly higher than Stage 0 under all four visual metrics, and selection provides a further improvement: render success rises from92.80%for raw Stage 1 to94.18%, while RSim-FZ increases from0.5446to0.5649. As reported in Appendix D.4, the paired bootstrap interval for each Stage 1-to-Selected improvement excludes zero on the fixed Final-945 sample set. Stage 0 remains the single-call output of the trained policy. The corrective action and selection provide an optional additional inference mode when execution and a second model call are available. F.6 Summary The extended analysis supports three observations. First, correction is state dependent: weak intermediate renders receive larger gains, while strong renders are more susceptible to degradation after further modification. Second, the same separation appears across nearby checkpoints rather than only at the Dev-selected checkpoint. Third, training improves the initial action while also producing visual revision and execution-repair behavior in the corrective action; this second action can be used optionally at inference time. 34

Instant research discovery