Paper deep dive

Function-Level Execution Feedback for Code Preference Optimization

Idris Nechnech, Sehwan Kim, Jimin Seo, Yeongoon Kim, Minhae Oh, Sangwoo Hong, Jungwoo Lee

Year: 2026Venue: arXiv preprintArea: cs.AIType: PreprintEmbeddings: 79

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 93%

Last extracted: 8/26/2026, 3:58:19 AM

Summary

The paper introduces STEP-KTODER, a framework for code preference optimization that applies stepwise Kahneman-Tversky Optimization (KTO) at the function level. It decomposes programs into module-level functions, generates automatic unit tests for each function to create binary correctness labels, and combines this function-level process supervision with outcome-level feedback. Experiments on benchmarks like HumanEval, MBPP, BigCodeBench, and LiveCodeBench demonstrate that STEP-KTODER outperforms outcome-only KTO and DPO, particularly on larger models. The study also highlights that execution-based labels are critical, as LLM-as-a-judge annotations degrade performance by over-predicting failures.

Entities (11)

STEP-KTODER · method · 100%BigCodeBench · dataset · 95%DPO · method · 95%HumanEval · dataset · 95%KTO · method · 95%LiveCodeBench · dataset · 95%MBPP · dataset · 95%Deepseek Coder · model · 90%process supervision · concept · 90%Qwen2.5-Coder · model · 90%LLM-as-a-judge · method · 85%

Relation Signals (9)

STEP-KTODER → evaluatedon → MBPP

confidence 95% · We evaluate on HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench

STEP-KTODER → evaluatedon → LiveCodeBench

confidence 95% · We evaluate on HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench

STEP-KTODER → evaluatedon → BigCodeBench

confidence 95% · We evaluate on HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench

STEP-KTODER → evaluatedon → HumanEval

confidence 95% · We evaluate on HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench

STEP-KTODER → uses → KTO

confidence 95% · Our method provides a code-specific instantiation of stepwise KTO

LLM-as-a-judge → degradesperformanceof → STEP-KTODER

confidence 90% · LLM-as-a-judge annotations systematically over-predict function failures... and degrade downstream preference optimization.

Qwen2.5-Coder → improvedby → STEP-KTODER

confidence 90% · Even for already instruction-tuned Qwen2.5-Coder models, Step-KTOder achieves substantial improvements

STEP-KTODER → outperforms → DPO

confidence 90% · showing that STEP-KTODER improves over outcome-only KTO and DPO.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Process supervision has improved mathematical reasoning, where intermediate steps are naturally expressed as chains of thought. In code generation, however, process supervision remains underexplored because there is no standard notion of a step. Supervision can target lines, reasoning traces, or program states, making it unclear what to label and optimize. We propose STEP-KTODER, a framework for code preference optimization that defines steps as module-level functions in decomposed multi-function programs and assigns binary correctness labels via automatically generated unit tests. Our method provides a code-specific instantiation of stepwise KTO, combining function-level process supervision with outcome-level feedback on the full program. We evaluate on HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench, showing that STEP-KTODER improves over outcome-only KTO and DPO. Further analysis shows that execution-based labels are essential: LLM-as-a-judge annotations systematically over-predict function failures, corrupt positive step labels, and degrade downstream preference optimization. Code is available at: this https URL.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

76,909 characters extracted from source content.

Expand or collapse full text

Function-Level Execution Feedback for Code Preference Optimization Idris Nechnech Affiliation: Department of Electrical and Computer Engineering, Seoul National University Sehwan Kim Affiliation: Department of Electrical and Computer Engineering, Seoul National University Jimin Seo Affiliation: Department of Electrical and Computer Engineering, Seoul National University Yeongoon Kim Affiliation: Department of Electrical and Computer Engineering, Seoul National University Minhae Oh Affiliation: Department of Electrical and Computer Engineering, Seoul National University Sangwoo Hong Affiliation: Department of Computer Science and Engineering, Konkuk Universityinechnech,junglee@snu.ac.kr Jungwoo Lee Abstract Process supervision has improved mathematical reasoning, where intermediate steps are naturally expressed as chains of thought. In code generation, however, process supervision remains underexplored because there is no standard notion of a step. Supervision can target lines, reasoning traces, or program states, making it unclear what to label and optimize. We propose Step-KTOder, a framework for code preference optimization that defines steps as module-level functions in decomposed multi-function programs and assigns binary correctness labels via automatically generated unit tests. Our method provides a code-specific instantiation of stepwise KTO, combining function-level process supervision with outcome-level feedback on the full program. We evaluate on HumanEval(+), MBPP(+), BigCodeBench, and LiveCodeBench, showing that Step-KTOder improves over outcome-only KTO and DPO. Further analysis shows that execution-based labels are essential: LLM-as-a-judge annotations systematically over-predict function failures, corrupt positive step labels, and degrade downstream preference optimization. Code is available at: https://github.com/inechnech/STEP-KTODER. $ $$ $footnotetext: Corresponding author. 1 Introduction Code generation has become a major application of Large Language Models (LLMs), with recent code-specialized models achieving strong performance on a wide range of programming tasks (8; 11). These models are typically improved through two stages: supervised fine-tuning (SFT) (30) on high-quality code data, followed by an alignment stage that adapts the model toward better outputs (24). Among alignment methods, Direct Preference Optimization (DPO) (26) is widely used to learn from paired preference data, whereas Kahneman–Tversky Optimization (KTO) (7) learns from binary desirability labels on individual outputs. However, applying these techniques to code generation is nontrivial. These methods are typically instantiated at the level of the entire program, treating a full solution as a single unit of feedback. This provides only a coarse training signal: a program can be mostly correct and fail only on a narrow edge case, but outcome-level supervision still marks the entire output as undesirable. Such supervision does not reveal which component is correct and which is responsible for the failure, motivating finer-grained supervision for code generation. In mathematical reasoning, process supervision—providing feedback on intermediate reasoning steps—has proven highly effective (17; 21; 29), with process reward models (PRMs) underpinning state-of-the-art reasoning systems. Building on this success, 18 combined stepwise binary feedback with KTO to jointly optimize intermediate and final-answer quality for mathematical problem solving. Transferring this idea to code generation, however, remains an open challenge: code does not naturally decompose into a single standardized sequence of intermediate logical steps, making the very definition of a “step” ambiguous. Figure 1: Overview of the Step-KTOder framework. A reference solution y⋆y is decomposed into module-level functions s1,…,sns_1,…,s_n, and function-level unit tests are generated for each function. Candidate completions are evaluated against these tests to obtain step labels ziz_i indicating local correctness, alongside an outcome label o from the dataset-provided test suite. The resulting labeled data is used to train the model with a joint objective that combines outcome-level KTO (ℒoutL_out) and function-level supervision (ℒstepL_step), reinforcing locally correct functions while penalizing locally incorrect ones. We address this ambiguity by defining steps as module-level functions in decomposed programs, and propose Step-KTOder, a framework that brings function-level process supervision to code generation. Our key idea is that decomposed programs provide an executable step structure: when a solution is written as multiple functions, each can be tested independently using automatically generated unit tests, yielding binary labels of local correctness. Step-KTOder uses these labels to extend outcome-level KTO with function-level supervision, guiding training by both global program success and local correctness, as indicated in Figure 1. Beyond the main empirical gains, we investigate two design choices that are important to Step-KTOder’s effectiveness. First, programs that pass dataset-provided tests but contain a locally incorrect function provide a form of supervision that outcome-only training cannot express. We find that preserving these local/global mismatches strengthens the benefit of step-level supervision, whereas masking such mismatches substantially weakens the signal. Second, we assess whether automatically generated unit tests are necessary for reliable step supervision by replacing execution-based step labels with LLM-as-a-judge annotations. Although these annotations show moderate agreement with execution-based labels, they systematically over-predict step failures, and when used for Step-KTOder training, they degrade performance. Even for already instruction-tuned Qwen2.5-Coder models, Step-KTOder achieves substantial improvements: it improves over the base model by up to +26.7% on BigCodeBench Hard (35) and +27.0% on LiveCodeBench (12). Our contributions are as follows: • We propose Step-KTOder, a code-specific instantiation of stepwise KTO that combines function-level process supervision with outcome-level feedback. • We introduce an automatic data construction pipeline that decomposes solutions into functions, generates function-level unit tests, and derives execution-based step labels for training. • Experiments across widely used benchmarks show that Step-KTOder improves post-trained code models. We further demonstrate that replacing execution-based step labels with LLM-as-a-judge annotations weakens the step-level signal and degrades performance. 2 Related Work 2.1 Process Supervision for Reasoning Process supervision has been highly effective for mathematical reasoning, where step-level feedback improves over outcome-only supervision (5; 28; 17). Subsequent work has scaled this supervision: 29 introduced automatic step-level annotation via Monte Carlo estimation, and 21 proposed a divide-and-conquer Monte Carlo Tree Search framework for large-scale process supervision data. 18 introduced Step-KTO, which combines KTO with PRM-labeled stepwise feedback to jointly optimize intermediate reasoning quality and final response correctness. Our work builds on this idea but reformulates it for code: instead of using PRM-labeled reasoning steps, we define steps as concrete, independently testable functions and supervise them directly through execution-based unit test feedback. This perspective is consistent with recent work on code PRMs showing that execution feedback improves the reliability of process supervision for code (15). In contrast to PRM training, we study how execution-based process labels can be incorporated directly into offline preference optimization. 2.2 Preference Optimization for Code Preference optimization has become a standard post-training technique for code models. DPO (26) learns from paired preferences, while KTO (7) removes the requirement for paired data using binary desirability labels grounded in prospect theory. Recent work has explored localized and fine-grained feedback for code: 34 concentrated the DPO loss on error-prone token spans, while 31 introduced Target-DPO, a focal preference alignment framework that localizes preference updates to targeted code regions. 6 trained a PRM that provides dense line-level feedback during code generation via reinforcement learning. 33 introduced CodeDPO, which constructs self-generated preference pairs to align code models for both correctness and efficiency. These approaches target different granularities—tokens, code blocks, lines, or whole programs—but to our knowledge, none of these approaches defines steps as semantically meaningful, independently testable functions and integrates them into a binary preference optimization objective. 2.3 Solution Decomposition and Automated Testing Fill-in-the-middle objectives (2) train models to complete code given surrounding context, encouraging modular structure. 27 extend this idea to alignment, splitting code into AST-based blocks to construct more diverse preference pairs for DPO. Prompting-based approaches have also emphasized modular decomposition for code generation: 25 break complex programming problems into smaller reasoning modules through hierarchical prompting. 19 proposed generating both code and unit tests simultaneously, then using self-validation to enhance generation quality. Automated unit test generation has matured significantly (3; 22); our pipeline leverages this capability in a function-level setting, generating tests for individual functions and using execution outcomes as binary step labels for training. 2.4 Execution-Grounded Inference-Time Methods Recent execution-grounded methods instead improve code at inference time. S∗ combines parallel sampling with iterative debugging and selects among candidates using adaptively synthesized inputs and their execution results (14), while ORPS explores a tree of reasoning and code trajectories guided by execution outcomes and self-critique (32). Unlike these per-query search and selection methods, Step-KTOder uses function-level execution during offline data construction to produce labels that update the model policy. 3 Method 3.1 Problem Formulation Given a prompt x and a reference solution y⋆y , we first rewrite y⋆y into a decomposed multi-function program y~=(s1,s2,…,sn), y=(s_1,s_2,…,s_n), using a strong code language model, where each sis_i denotes one function in the decomposed program and n is the total number of such functions. We treat each function sis_i as a step. Training samples are then generated from this decomposition: the target model fills the function skeleton defined by y~ y to produce a candidate solution y. Each training sample receives two forms of binary supervision: • An outcome label o∈0,1o∈\0,1\, indicating whether y passes the dataset-provided test suite for x. • A sequence of step labels =(z1,…,zn)z=(z_1,…,z_n), where each zi∈0,1,∅z_i∈\0,1, \ indicates whether the implementation of sis_i in y passes its function-level unit tests. A label of ∅ indicates that no valid unit test is available for that step. This formulation distinguishes local correctness from global correctness, providing finer-grained supervision than outcome-only evaluation. Rather than treating partially correct programs as uniformly desirable or undesirable, it enables the objective to reinforce correct functions while penalizing those responsible for failure. 3.2 KTO Background We build on Kahneman–Tversky Optimization (KTO) (7), which aligns a policy πθ _θ from binary feedback using a Kahneman–Tversky-inspired value function over the log-ratio between the policy and a frozen reference πref _ref: rθ(x,y)=log⁡πθ(y∣x)πref(y∣x).r_θ(x,y)= _θ(y x) _ref(y x). The outcome-level reference point is z0out=KL⁡(πθ(y′∣x)∥πref(y′∣x)),z_0^out=KL\! ( _θ(y x)\; \|\; _ref(y x) ), where y′y denotes an output sequence used to estimate the divergence between the policy and reference. Given a binary desirability label o∈0,1o∈\0,1\, the outcome-level value function is vout=λDσ(βout(rθ−z0out)),o=1,λUσ(βout(z0out−rθ)),o=0,v_out= cases _D\,σ\! ( _out(r_θ-z_0^out) ),&o=1,\\[2.0pt] _U\,σ\! ( _out(z_0^out-r_θ) ),&o=0, cases where rθ≡rθ(x,y)r_θ≡ r_θ(x,y) and σ⁡(⋅)σ(·) denotes the sigmoid function. βout>0 _out>0 controls the sensitivity of the outcome-level value function, and λD _D and λU _U weight desirable and undesirable samples, respectively. The outcome-level KTO loss is then ℒout(πθ,πref)=(x,y,o)∼[λo−vout],L_out( _θ, _ref)=E_(x,y,o) [ _o-v_out ], where λo=λD _o= _D if o=1o=1 and λo=λU _o= _U if o=0o=0. 3.3 Step-KTO Objective Following 18, we instantiate stepwise KTO over the function-level steps defined in Section 3.1 and introduce reliability masking for functions without validated local tests. For each step sis_i, we compute a step-local log-ratio over the tokens of that function: ri=∑t∈silog⁡πθ(yt∣x,y<t)πref(yt∣x,y<t).r_i= _t∈ s_i _θ(y_t x,y_<t) _ref(y_t x,y_<t). Each step has a label ziz_i as defined above and a mask mi∈0,1m_i∈\0,1\; only steps with mi=1m_i=1 and zi∈0,1z_i∈\0,1\ contribute to the step-level loss. Similarly, we define a step-level reference point z0step=KL⁡(πθ(yi′∣x,s<i)∥πref(yi′∣x,s<i)),z_0^step=KL\! ( _θ(y _i x,s_<i)\; \|\; _ref(y _i x,s_<i) ), where yi′y _i denotes the subsequence of generated tokens belonging to the i-th function. The step-level value function is vi=λD,stepσ(βstep(ri−z0step)),zi=1,λU,stepσ(βstep(z0step−ri)),zi=0.v_i= cases _D,step\,σ\! ( _step(r_i-z_0^step) ),&z_i=1,\\[2.0pt] _U,step\,σ\! ( _step(z_0^step-r_i) ),&z_i=0. cases Here, βstep _step, λD,step _D,step, and λU,step _U,step are the step-level counterparts of the outcome-level KTO parameters. Let ℳτ=i:mi=1,zi∈0,1M_τ=\i:m_i=1,\ z_i∈\0,1\\ denote the supervised steps for sample τ, and let M=|ℳτ|M=|M_τ|. Let λzi,step _z_i,step denote the step-level target coefficient, with λzi,step=λD,step _z_i,step= _D,step when zi=1z_i=1 and λzi,step=λU,step _z_i,step= _U,step when zi=0z_i=0. The masked step-level loss is ℒstep=τ∼[1n∑i∈ℳτ(λzi,step−vi)],L_step=E_τ [ 1n _i _τ ( _z_i,step-v_i ) ], where the sum is zero when M=0M=0, so samples without supervised steps fall back to outcome-only KTO. Combining with the outcome-level loss yields the final Step-KTOder objective: ℒStep-KTOder=ℒout+λstepℒstep.L_Step-KTOder=L_out+ _step\,L_step. By jointly optimizing outcome-level and function-level feedback, Step-KTOder reinforces locally correct functions and penalizes locally incorrect ones, providing a more localized learning signal than outcome-level supervision alone. Figure 2 illustrates this difference on a failing candidate: outcome-only KTO treats the full program as undesirable, whereas Step-KTOder assigns negative step-level signal only to the faulty function. Figure 2: Training signal comparison for a failing candidate. The wrong candidate differs from the decomposed reference solution only in the highlighted line, yielding locally correct functions z1,z2,z3=1z_1,z_2,z_3=1, a faulty final function z4=0z_4=0, and outcome label o=0o=0. Outcome-only KTO penalizes all function spans, whereas Step-KTOder preserves the locally correct functions and penalizes only the faulty one. 3.4 Data Construction Pipeline Our pipeline consists of four stages, illustrated in Figure 1. (1) Reference decomposition. We use a strong instruction-tuned code model to rewrite each ground-truth solution into an interface-preserving multi-function program, retaining only rewrites that pass the original test suite. (2) Function-level unit test generation. For each decomposed function, a strong code model generates several targeted unit tests for its interface and edge cases. We retain only tests that pass on the reference implementation, invoke the target function, and contain nontrivial assertions whose pass condition depends on the function’s output. Functions without valid tests receive a null step label (zi=∅z_i= ) and are masked from the step loss. The prompts are provided in Appendix O. (3) On-policy candidate generation. From each decomposition, we construct a skeleton by replacing function bodies with pass while preserving signatures, docstrings, imports, and module-level context. Then, for each skeleton we sample k=8k=8 candidate completions from the model being post-trained, using temperature T=0.4T=0.4 and top-p sampling with p=0.95p=0.95; this keeps the preference data close to the model’s own generation distribution. (4) Labeling and dataset construction. Each candidate is evaluated against the dataset-provided test suite to obtain an outcome label o. Step labels ziz_i are obtained by executing each candidate function against its generated unit tests. By default, we use local step labels: each function is tested after being inserted into the decomposed reference program, with all other functions kept fixed to their reference implementations. This isolates function-level correctness from downstream composition effects. Across datasets, 83.2% of testable decomposed functions retain at least one validated test. We refer to this quantity as validated-test availability; the corresponding per-problem distribution is shown in Figure 3. For each task, we retain up to one passing and one failing candidate, preferring candidates with richer step-level supervision. Rows with fewer than two supervised steps fall back to outcome-only KTO, avoiding single-step supervision that largely duplicates the outcome-level signal. Figure 3: Distribution of per-problem validated-test availability. Most problems achieve near-full test availability. 3.5 Conflict-Preserving Step Labels A key design question is how to handle examples where outcome-level and step-level supervision disagree. Two conflict patterns arise: passing programs with negative step labels, where the full program passes dataset tests but at least one supervised function fails its function-level tests; and failing programs with all-positive step labels, where the full program fails dataset tests even though all supervised functions pass their unit tests. These examples reflect a mismatch between local and end-to-end correctness. Rather than removing them, we compare masked and conflict-preserving variants to test whether such cases provide useful training signal beyond outcome-only supervision and ordinary partial-correctness patterns. 4 Experimental Setup 4.1 Training Configuration All target models are instruction-tuned code models that have already undergone substantial post-training. We fine-tune each model for 1 epoch using LoRA (10) with global batch size 16. KTO and Step-KTOder use a learning rate of 1×10−61× 10^-6, while DPO uses 5×10−75× 10^-7. We set λstep=1.0 _step=1.0 by default. Full training details are provided in Appendix B. 4.2 Datasets Training data is drawn from the training splits of TACO (16) and APPS (9). We construct training samples using the pipeline described in Section 3: reference solutions are decomposed with Qwen2.5-Coder-32B-Instruct (11), and each target model generates on-policy candidate completions from the resulting function skeletons. For each problem, we retain up to one passing and one failing candidate using a ranking procedure that prioritizes positives whose supervised functions all pass their unit tests and negatives with both passing and failing function-level labels. Such negatives localize which components remain correct and which fail (Appendix A). Table 1 summarizes the resulting training sets. For same-family training with Qwen2.5-Coder-1.5B-Instruct and Qwen2.5-Coder-3B-Instruct, we include decomposed reference solutions as additional positive anchors. For cross-family training with DeepSeek-Coder-6.7B-Instruct, we use only on-policy generated candidates, since adding off-policy samples degrades performance (Appendix M). Rows with fewer than two supervised steps fall back to outcome-only KTO. As a result, 82% of Qwen2.5-Coder-1.5B-Instruct rows, 83% of Qwen2.5-Coder-3B-Instruct rows, and 78% of DeepSeek-Coder-6.7B-Instruct rows carry active step-level supervision. We justify this threshold in Section 6.1. 4.3 Evaluation Benchmarks and Baselines Qwen-1.5B Qwen-3B DeepSeek Dataset composition Total rows 11,397 11,440 6,573 GT rows 5,018 5,018 0 Generated rows 6,379 6,422 6,573 Step supervision Step-supervised 9,376 9,455 5,157 Fallback KTO 2,021 1,985 1,416 Avg. steps / active row 2.66 2.66 2.68 Conflict analysis Conflicts (o=1,zi=0o=1,z_i=0) 1,194 1,397 1,428 Structure quality AST alignment 84.2% 84.7% 80.3% Table 1: Training dataset statistics. Step-supervised rows contain at least two supervised function-level labels and activate the step-level loss; fallback KTO rows use outcome-only KTO. Avg. steps / active row is computed over step-supervised rows. AST alignment measures exact agreement with the decomposition skeleton. We evaluate on seven code generation benchmarks: HumanEval (4) and HumanEval+ (20), MBPP (1) and MBPP+ (20), BigCodeBench Full and Hard (35), and LiveCodeBench (12) release v4_v5. All evaluations use greedy decoding with vLLM (13). We compare Step-KTOder against four baselines: the original instruction-tuned model, DPO (26), KTO (7), and Target-DPO (31). Baseline construction details are provided in Appendix D. 5 Results 5.1 Main Results Method HumanEval MBPP BigCodeBench LiveCodeBench Base Plus Base Plus Full Hard Qwen2.5-Coder-1.5B-Instruct 0.689 0.604 0.664 0.556 0.240 0.054 0.052 w/ DPO 0.695 0.610 0.664 0.563 0.245 0.054 0.056 w/ Target-DPO 0.683 0.628 0.664 0.558 0.245 0.068 0.019 w/ KTO 0.701 0.610 0.648 0.542 0.245 0.061 0.052 w/ Step-KTOder 0.701 0.616 0.656 0.548 0.248 0.068 0.056 Relative improvement over KTO 0.0% +1.0% +1.2% +1.1% +1.2% +11.5% +7.7% Qwen2.5-Coder-3B-Instruct 0.860 0.799 0.741 0.619 0.356 0.101 0.100 w/ DPO 0.854 0.811 0.741 0.630 0.360 0.115 0.112 w/ Target-DPO 0.854 0.805 0.746 0.630 0.351 0.108 0.112 w/ KTO 0.878 0.835 0.741 0.624 0.367 0.115 0.116 w/ Step-KTOder 0.878 0.835 0.746 0.630 0.367 0.128 0.127 Relative improvement over KTO 0.0% 0.0% +0.7% +1.0% 0.0% +11.3% +9.5% DeepSeek-Coder-6.7B-Instruct 0.799 0.713 0.746 0.640 0.342 0.108 0.127 w/ DPO 0.799 0.726 0.749 0.643 0.347 0.115 0.127 w/ Target-DPO 0.799 0.738 0.725 0.624 0.346 0.108 0.112 w/ KTO 0.793 0.726 0.749 0.640 0.347 0.122 0.127 w/ Step-KTOder 0.799 0.732 0.754 0.648 0.347 0.128 0.131 Relative improvement over KTO +0.8% +0.8% +0.7% +1.2% 0.0% +4.9% +3.2% Table 2: Pass rate across model families and scales. Relative improvements are computed over outcome-only KTO within each model; gains are largest on harder benchmarks. The best results are highlighted in bold. The experimental results are reported in Table 2. Since all target models are already instruction-tuned code models, easier benchmarks such as HumanEval and MBPP leave limited headroom, and the clearest gains appear on harder benchmarks. On Qwen2.5-Coder-1.5B-Instruct, Step-KTOder improves over KTO by +11.5% on BigCodeBench Hard and +7.7% on LiveCodeBench. On Qwen2.5-Coder-3B-Instruct, the gains are similarly concentrated on the hardest benchmarks, with +11.3% on BigCodeBench Hard and +9.5% on LiveCodeBench, while matching or slightly improving KTO elsewhere. DeepSeek-Coder-6.7B-Instruct shows the same qualitative pattern: BigCodeBench Hard improves monotonically from DPO to KTO to Step-KTOder, with Step-KTOder achieving a +4.9% gain over KTO, and LiveCodeBench improving by +3.2%. On the easier benchmarks, where headroom is limited, Step-KTOder remains competitive, obtaining the best or tied-best result in most cases, with Target-DPO leading slightly on HumanEval+ for DeepSeek-Coder-6.7B-Instruct. The gains persist across seeds: Step-KTOder exceeds KTO on both BigCodeBench Hard and LiveCodeBench across three seeds (Appendix E). Both Target-DPO and Step-KTOder target localized code failures, but they optimize different supervision signals: Target-DPO derives preference pairs from debugging traces and localizes loss to changed token regions, whereas Step-KTOder labels functions directly via unit tests. Step-KTOder is consistently stronger on the harder benchmarks. For Qwen2.5-Coder-1.5B-Instruct, Target-DPO is competitive on easier benchmarks and ties on BigCodeBench Hard, but Step-KTOder is stronger on LiveCodeBench. For Qwen2.5-Coder-3B-Instruct and DeepSeek-Coder-6.7B-Instruct, Step-KTOder improves over Target-DPO on both BigCodeBench Hard (+18.5% for both) and LiveCodeBench (+13.4% and +17.0%, respectively). On easier benchmarks, Step-KTOder ties or improves over Target-DPO in most cases. Notably, Step-KTOder uses at most 11,440 training samples, compared with Target-DPO’s 59,000 preference pairs. 5.2 Ablations Table 3 reports three core ablations on Qwen2.5-Coder-3B-Instruct. First, conflict preservation is essential: masking conflicts reduces Step-KTOder to KTO-level performance on BigCodeBench Hard and decreases on LiveCodeBench as well. This shows that apparent disagreements between outcome-level and function-level labels can provide useful training signal rather than noise to be removed. Second, λstep=1.0 _step=1.0 gives the best trade-off (λstep=0 _step=0 corresponds to outcome-only KTO), reported above. Lower step weights underuse the function-level signal, while larger weights begin to regress easier benchmarks without improving the hardest ones. Third, requiring at least two supervised steps outperforms the looser threshold of one, consistent with the redundancy of single-step supervision discussed in Section 6.1. 6 Analysis Method HumanEval MBPP BigCodeBench LiveCodeBench Base Plus Base Plus Full Hard Conflict-handling policy KTO (no step loss) 0.878 0.835 0.741 0.624 0.367 0.115 0.116 Step-KTOder + mask conflicts 0.878 0.835 0.741 0.624 0.367 0.115 0.119 Step-KTOder + keep conflicts (default) 0.878 0.835 0.746 0.630 0.367 0.128 0.127 Step loss weight λstep _step 0.5 0.872 0.829 0.746 0.624 0.364 0.122 0.127 1.0 (default) 0.878 0.835 0.746 0.630 0.367 0.128 0.127 2.0 0.872 0.829 0.746 0.622 0.363 0.122 0.127 Min. supervised steps threshold 1 0.878 0.835 0.741 0.627 0.363 0.115 0.116 2 (default) 0.878 0.835 0.746 0.630 0.367 0.128 0.127 Table 3: Core ablations on Qwen2.5-Coder-3B-Instruct. We vary conflict handling, the step-loss weight λstep _step, and the minimum number of supervised steps required to activate the step loss. 6.1 Why Function-Level Supervision Helps Both DPO and KTO operate at the program level, weighting the policy–reference log-ratio by a single binary outcome label. They therefore cannot distinguish which parts of a generated program contributed to success or failure: when correct and incorrect functions coexist, outcome-level methods reinforce or penalize all components together. Step-KTOder targets this missing information by assigning labels to individual functions, allowing the model to reinforce locally correct functions and penalize locally incorrect ones within the same program. This explains why conflict preservation matters. When zi=oz_i=o for all supervised steps, the step loss is directionally aligned with the outcome loss and mainly changes where the gradient is applied. In contrast, when zi≠oz_i≠ o, the step loss provides a direction that outcome-level supervision cannot produce, such as penalizing a specific function even when the full program passes. Such conflicts can arise when the decomposed function interface exposes edge cases not exercised by the original test suite. Masking these conflicts removes this directional disagreement, making the remaining step signal largely redundant with outcome-level supervision. The same reasoning explains the minimum-supervised-steps threshold. For single-function rows, the step span covers the full answer, so r1=routr_1=r_out. When z1=oz_1=o, the step loss adds no information beyond a rescaled outcome loss. Requiring at least two supervised steps filters out these redundant rows. 6.2 Function-Level Labels Localize Failures To test whether negative step labels identify functions that contribute to end-to-end failure, we perform a repair intervention on failing candidates whose supervised functions include both positive and negative labels. We replace the locally negative functions with their decomposed reference implementations and evaluate the repaired candidates using the original dataset-provided test suite. As controls, we replace either the same number of locally positive functions or the same number of randomly selected observed functions. As shown in Table 4, replacing locally negative functions repairs 74.9% of failing candidates, substantially outperforming both random and locally positive replacement. This suggests that execution-based step labels localize repair-relevant faults, rather than merely correlating with outcome correctness. Replacement target Repair rate Local-negative (zi=0z_i=0) 74.9% Random observed 32.9% Local-positive (zi=1z_i=1) 1.2% Table 4: Repair intervention on failing candidates. 6.3 Execution-Based Labels Are Essential A natural question is whether the execution-based unit tests at the core of Step-KTOder are necessary, or whether an LLM-as-a-judge can provide comparable step-level supervision. We test this by replacing execution-based step labels with judgments from GPT-5.4 mini (23), and we retrain Qwen2.5-Coder-3B-Instruct with identical settings. The LLM shows moderate overall agreement with execution-based labels (73.2%73.2\%), but this aggregate score hides a strong asymmetry. It agrees with execution on 94.1%94.1\% of failing functions, but on only 52.2%52.2\% of passing functions, systematically over-predicting failure. This bias corrupts the positive step labels needed for Step-KTOder: training with LLM-as-a-judge labels instead of execution-based labels degrades performance, especially on BigCodeBench Hard and LiveCodeBench. Thus, execution-based labels are essential not only because they provide step-level supervision: they also provide reliable step-level supervision. Full agreement statistics and benchmark results across all seven benchmarks are provided in Appendix N. 7 Conclusion We propose Step-KTOder, a framework for code preference optimization that defines process-supervision steps as module-level functions in decomposed programs. By combining outcome-level KTO with execution-based function labels, Step-KTOder provides localized feedback that reinforces correct functions while penalizing incorrect ones. Experiments on post-trained code models show consistent gains over outcome-only KTO and DPO, with ablations confirming that preserving local/global label conflicts and using execution-based labels are central to these gains. These results suggest that function-level execution feedback offers a practical path toward process supervision for code generation. 8 Limitations Model and data scope. Step-KTOder is most natural when solutions admit a meaningful decomposition into independently testable functions, and is most reliable when candidate solutions are generated on-policy by the target model, which adds engineering cost for each new model. Our experiments focus on instruction-tuned code models that have already undergone substantial post-training; applying function-level supervision earlier, during SFT or on larger base-model training runs, remains a promising direction. Difficulty distribution. Our filtering pipeline requires problems to have a clear function-style structure or to admit a meaningful decomposition. As a result, the training data is biased toward problems whose solutions can be decomposed into testable functions, while harder competition problems are less represented. Extending the pipeline to more complex I/O formats and harder problem regimes is a natural next step. Unit-test quality. Step labels depend on automatically generated unit tests, which we validate by execution against decomposed reference solutions but do not formally verify. A mutation-sensitivity audit (Appendix H) shows that the retained unit tests reject 92.6% of semantically perturbed reference implementations, and our repair intervention (Section 6.2) shows that negative step labels identify functions responsible for end-to-end failure. Our LLM-as-a-judge comparison further shows that execution-based labels are substantially more reliable than LLM judgments. Still, stronger automated test-generation or verification methods could further improve label quality. Ethical Considerations The models used in this paper, Qwen2.5-Coder (11) and DeepSeek-Coder (8), are licensed for academic research purposes. The training datasets, TACO (16) and APPS (9), and all evaluation benchmarks (4; 20; 1; 35; 12) are publicly available and distributed for research use. Acknowledgments This work was supported in part by the National Research Foundation of Korea (NRF) grant funded by the Ministry of Science and ICT (MSIT) (RS-2024-00451435, 20%; RS-2024-00413957, 20%), the Institute of Information & Communications Technology Planning & Evaluation (IITP) grant funded by the MSIT (RS-2025-02305453, 15%; RS-2025-02273157, 15%; RS-2025-25442149, 15%; RS-2021-I211343, 15%), the Institute of New Media and Communications (INMAC), the BK21 FOUR program funded by the Ministry of Education, the Artificial Intelligence Graduate School Program (Seoul National University), and the Research Program for Future ICT Pioneers at Seoul National University in 2026. References Austin et al. (2021) J. Austin, A. Odena, M. Nye, M. Bosma, H. Michalewski, D. Dohan, E. Jiang, C. Cai, M. Terry, Q. Le, and C. Sutton Program synthesis with large language models. External Links: 2108.07732, Link Cited by: §4.3, Ethical Considerations. Bavarian et al. (2022) M. Bavarian, H. Jun, N. Tezak, J. Schulman, C. McLeavey, J. Tworek, and M. Chen Efficient training of language models to fill in the middle. External Links: 2207.14255, Link Cited by: §2.3. Chen et al. (2023) B. Chen, F. Zhang, A. Nguyen, D. Zan, Z. Lin, J. Lou, and W. Chen CodeT: code generation with generated tests. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: §2.3. Chen et al. (2021) M. Chen, J. Tworek, H. Jun, Q. Yuan, H. P. de Oliveira Pinto, J. Kaplan, H. Edwards, Y. Burda, N. Joseph, G. Brockman, A. Ray, R. Puri, G. Krueger, M. Petrov, H. Khlaaf, G. Sastry, P. Mishkin, B. Chan, S. Gray, N. Ryder, M. Pavlov, A. Power, L. Kaiser, M. Bavarian, C. Winter, P. Tillet, F. P. Such, D. Cummings, M. Plappert, F. Chantzis, E. Barnes, A. Herbert-Voss, W. H. Guss, A. Nichol, A. Paino, N. Tezak, J. Tang, I. Babuschkin, S. Balaji, S. Jain, W. Saunders, C. Hesse, A. N. Carr, J. Leike, J. Achiam, V. Misra, E. Morikawa, A. Radford, M. Knight, M. Brundage, M. Murati, K. Mayer, P. Welinder, B. McGrew, D. Amodei, S. McCandlish, I. Sutskever, and W. Zaremba Evaluating large language models trained on code. External Links: 2107.03374, Link Cited by: §4.3, Ethical Considerations. Cobbe et al. (2021) K. Cobbe, V. Kosaraju, M. Bavarian, M. Chen, H. Jun, L. Kaiser, M. Plappert, J. Tworek, J. Hilton, R. Nakano, C. Hesse, and J. Schulman Training verifiers to solve math word problems. External Links: 2110.14168, Link Cited by: §2.1. Dai et al. (2025) N. Dai, Z. Wu, R. Zheng, Z. Wei, W. Shi, X. Jin, G. Liu, C. Dun, L. Huang, and L. Yan Process supervision-guided policy optimization for code generation. External Links: 2410.17621, Link Cited by: §2.2. Ethayarajh et al. (2024) K. Ethayarajh, W. Xu, N. Muennighoff, D. Jurafsky, and D. Kiela Model alignment as prospect theoretic optimization. In Forty-first International Conference on Machine Learning, External Links: Link Cited by: §1, §2.2, §3.2, §4.3. Guo et al. (2024) D. Guo, Q. Zhu, D. Yang, Z. Xie, K. Dong, W. Zhang, G. Chen, X. Bi, Y. Wu, Y. K. Li, F. Luo, Y. Xiong, and W. Liang DeepSeek-coder: when the large language model meets programming – the rise of code intelligence. External Links: 2401.14196, Link Cited by: §1, Ethical Considerations. Hendrycks et al. (2021) D. Hendrycks, S. Basart, S. Kadavath, M. Mazeika, A. Arora, E. Guo, C. Burns, S. Puranik, H. He, D. Song, and J. Steinhardt Measuring coding challenge competence with APPS. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2), External Links: Link Cited by: §4.2, Ethical Considerations. Hu et al. (2022) E. J. Hu, Y. Shen, P. Wallis, Z. Allen-Zhu, Y. Li, S. Wang, L. Wang, and W. Chen LoRA: low-rank adaptation of large language models. In International Conference on Learning Representations, External Links: Link Cited by: Appendix B, §4.1. Hui et al. (2024) B. Hui, J. Yang, Z. Cui, J. Yang, D. Liu, L. Zhang, T. Liu, J. Zhang, B. Yu, K. Lu, K. Dang, Y. Fan, Y. Zhang, A. Yang, R. Men, F. Huang, B. Zheng, Y. Miao, S. Quan, Y. Feng, X. Ren, X. Ren, J. Zhou, and J. Lin Qwen2.5-coder technical report. External Links: 2409.12186, Link Cited by: §1, §4.2, Ethical Considerations. Jain et al. (2025) N. Jain, K. Han, A. Gu, W. Li, F. Yan, T. Zhang, S. Wang, A. Solar-Lezama, K. Sen, and I. Stoica LiveCodeBench: holistic and contamination free evaluation of large language models for code. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §4.3, Ethical Considerations. Kwon et al. (2023) W. Kwon, Z. Li, S. Zhuang, Y. Sheng, L. Zheng, C. H. Yu, J. Gonzalez, H. Zhang, and I. Stoica Efficient memory management for large language model serving with pagedattention. In Proceedings of the 29th Symposium on Operating Systems Principles, SOSP ’23, New York, NY, USA, p. 611–626. External Links: ISBN 9798400702297, Link, Document Cited by: §4.3. Li et al. (2025a) D. Li, S. Cao, C. Cao, X. Li, S. Tan, K. Keutzer, J. Xing, J. E. Gonzalez, and I. Stoica S*: test time scaling for code generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, p. 15964–15978. External Links: Link, Document, ISBN 979-8-89176-335-7 Cited by: §2.4. Li et al. (2025b) Q. Li, X. Dai, X. Li, W. Zhang, Y. Wang, R. Tang, and Y. Yu CodePRM: execution feedback-enhanced process reward model for code generation. In Findings of the Association for Computational Linguistics: ACL 2025, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, p. 8169–8182. External Links: Link, Document Cited by: §2.1. Li et al. (2023) R. Li, J. Fu, B. Zhang, T. Huang, Z. Sun, C. Lyu, G. Liu, Z. Jin, and G. Li TACO: topics in algorithmic code generation dataset. External Links: 2312.14852, Link Cited by: §4.2, Ethical Considerations. Lightman et al. (2024) H. Lightman, V. Kosaraju, Y. Burda, H. Edwards, B. Baker, T. Lee, J. Leike, J. Schulman, I. Sutskever, and K. Cobbe Let’s verify step by step. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: §1, §2.1. Lin et al. (2025a) Y. Lin, D. Jin, T. Xu, T. Wu, S. Sukhbaatar, C. Zhu, Y. He, Y. Chen, J. E. Weston, Y. Tian, A. Rahnama, S. Wang, H. Ma, and H. Fang Step-KTO: optimizing mathematical reasoning through stepwise binary feedback. In Proceedings of The 3rd Workshop on Mathematical Natural Language Processing (MathNLP 2025), M. Valentino, D. Ferreira, M. Thayaparan, L. Ranaldi, and A. Freitas (Eds.), Suzhou, China, p. 15–33. External Links: Link, Document Cited by: §1, §2.1, §3.3. Lin et al. (2025b) Z. Lin, S. Shen, J. Shang, J. E. Weston, and Y. Nie Learning to solve and verify: a self-play framework for mutually improving code and test generation. In NeurIPS 2025 Fourth Workshop on Deep Learning for Code, External Links: Link Cited by: §2.3. Liu et al. (2023) J. Liu, C. S. Xia, Y. Wang, and L. Zhang Is your code generated by chatGPT really correct? rigorous evaluation of large language models for code generation. In Thirty-seventh Conference on Neural Information Processing Systems, External Links: Link Cited by: §4.3, Ethical Considerations. Luo et al. (2024) L. Luo, Y. Liu, R. Liu, S. Phatale, M. Guo, H. Lara, Y. Li, L. Shu, Y. Zhu, L. Meng, J. Sun, and A. Rastogi Improve mathematical reasoning in language models by automated process supervision. External Links: 2406.06592, Link Cited by: §1, §2.1. Ma et al. (2025) Z. Ma, X. Zhang, J. Zhang, J. Yu, S. Luo, and J. Tang Dynamic scaling of unit tests for code reward modeling. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, p. 6917–6935. External Links: Link, Document, ISBN 979-8-89176-251-0 Cited by: §2.3. OpenAI (2026) OpenAI Introducing GPT-5.4 mini and nano. Note: https://openai.com/index/introducing-gpt-5-4-mini-and-nano/Accessed: 2026-04-28 Cited by: §6.3. Ouyang et al. (2022) L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Gray, J. Schulman, J. Hilton, F. Kelton, L. Miller, M. Simens, A. Askell, P. Welinder, P. Christiano, J. Leike, and R. Lowe Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho (Eds.), External Links: Link Cited by: §1. Pan and Zhang (2025) R. Pan and H. Zhang Modularization is better: effective code generation with modular prompting. External Links: 2503.12483, Link Cited by: §2.3. Rafailov et al. (2023) R. Rafailov, A. Sharma, E. Mitchell, C. D. Manning, S. Ermon, and C. Finn Direct preference optimization: your language model is secretly a reward model. In Thirty-seventh Conference on Neural Information Processing Systems, External Links: Link Cited by: Appendix D, §1, §2.2, §4.3. Ren et al. (2025) H. Ren, Z. Lu, W. Shi, H. Hou, Y. Yang, K. Wang, A. Zhou, J. Pan, M. Zhan, and H. Li Alignment with fill-in-the-middle for enhancing code generation. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, p. 8304–8320. External Links: Link, Document, ISBN 979-8-89176-332-6 Cited by: §2.3. Uesato et al. (2022) J. Uesato, N. Kushman, R. Kumar, F. Song, N. Siegel, L. Wang, A. Creswell, G. Irving, and I. Higgins Solving math word problems with process- and outcome-based feedback. External Links: 2211.14275, Link Cited by: §2.1. Wang et al. (2024) P. Wang, L. Li, Z. Shao, R. Xu, D. Dai, Y. Li, D. Chen, Y. Wu, and Z. Sui Math-shepherd: verify and reinforce LLMs step-by-step without human annotations. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 9426–9439. External Links: Link, Document Cited by: §1, §2.1. Wei et al. (2022) J. Wei, M. Bosma, V. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le Finetuned language models are zero-shot learners. In International Conference on Learning Representations, External Links: Link Cited by: §1. Wu et al. (2025) J. Wu, H. Li, X. Zhang, X. Liu, Y. Huang, J. Luo, Y. Zhang, Z. Li, R. Chu, Y. Yang, and S. Li Teaching your models to understand code via focal preference alignment. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, C. Christodoulopoulos, T. Chakraborty, C. Rose, and V. Peng (Eds.), Suzhou, China, p. 14003–14023. External Links: Link, Document, ISBN 979-8-89176-332-6 Cited by: Appendix D, §2.2, §4.3. Yu et al. (2025) Z. Yu, W. Gu, Y. Wang, X. Jiang, Z. Zeng, J. Wang, W. Ye, and S. Zhang Reasoning through execution: unifying process and outcome rewards for code generation. In Forty-second International Conference on Machine Learning, External Links: Link Cited by: §2.4. Zhang et al. (2025a) K. Zhang, G. Li, Y. Dong, J. Xu, J. Zhang, J. Su, Y. Liu, and Z. Jin CodeDPO: aligning code models with self generated and verified source code. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, p. 15854–15871. External Links: Link, Document, ISBN 979-8-89176-251-0 Cited by: §2.2. Zhang et al. (2025b) K. Zhang, G. Li, J. Li, Y. Dong, J. Li, and Z. Jin Focused-DPO: enhancing code generation through focused preference optimization on error-prone points. In Findings of the Association for Computational Linguistics: ACL 2025, W. Che, J. Nabende, E. Shutova, and M. T. Pilehvar (Eds.), Vienna, Austria, p. 9578–9591. External Links: Link, Document Cited by: §2.2. Zhuo et al. (2025) T. Y. Zhuo, V. M. Chien, J. Chim, H. Hu, W. Yu, R. Widyasari, I. N. B. Yusuf, H. Zhan, J. He, I. Paul, S. Brunner, C. Gong, J. Hoang, A. R. Zebaze, X. Hong, W. Li, J. Kaddour, M. Xu, Z. Zhang, P. Yadav, N. Jain, A. Gu, Z. Cheng, J. Liu, Q. Liu, Z. Wang, D. Lo, B. Hui, N. Muennighoff, D. Fried, X. Du, H. de Vries, and L. V. Werra BigCodeBench: benchmarking code generation with diverse function calls and complex instructions. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: §1, §4.3, Ethical Considerations. Appendix A Preference Pair Selection For each problem we generate 8 candidate completions from the target model and select up to one positive and one negative candidate for the training set, based on the outcome label o and the step labels. Positive candidates. Among candidates with o=1o=1, we select the one with the cleanest step-level signal: we prioritize candidates whose supervised functions all pass their unit tests, followed by candidates with passing labels and some unknown steps, then candidates containing both passing and failing step labels, and finally candidates without active step supervision. Within each category, ties are broken by preferring candidates with more locally passing functions, fewer locally failing functions, valid parsing, and non-empty outputs. Negative candidates. Among candidates with o=0o=0, we select the one with the richest local supervision. We first prioritize candidates that contain both passing and failing supervised functions, since they identify which components remain correct and which are responsible for the failure. If none are available, we fall back to candidates with at least one locally failing function. As a secondary fallback, we use generated-context labels, obtained by executing each generated function within the full generated module rather than inserting it into the decomposed reference program. We prioritize candidates with both passing and failing generated-context labels, then candidates with at least one generated-context failure, and finally candidates without informative step signal. Reference-context and generated-context agreement. We compare our default local labels, computed in the reference context, with generated-context labels on selected training rows. They agree on 89.4%89.4\% of 43,45843,458 comparable supervised steps, consistently across models (89.289.2–89.6%89.6\%), suggesting that local labels closely align with generated-context behavior while isolating function-level correctness. Pairing. Tasks with only a positive candidate contribute an unpaired positive row, and tasks with only a negative candidate contribute an unpaired negative row; both are usable by KTO but not DPO. Tasks with both contribute one positive and one negative row. Appendix B Training Details We fine-tune all target models with LoRA (10) using rank r=32r=32, scaling factor α=64α=64, dropout 0.050.05, maximum sequence length 2048, and global batch size 16. We use AdamW with (β1,β2)=(0.9,0.95)( _1, _2)=(0.9,0.95) and a cosine learning-rate schedule with 50 warmup steps and minimum ratio 0.10.1. We use learning rate 1×10−61× 10^-6 for KTO and Step-KTOder, and 5×10−75× 10^-7 for DPO. For DPO and KTO, we set β=0.1β=0.1. For Step-KTOder, we use βout=βstep=0.1 _out= _step=0.1, set λstep=1.0 _step=1.0, and use unit weights for all KTO value-function coefficients: λD=λU=λD,step=λU,step=1.0 _D= _U= _D,step= _U,step=1.0. Sensitivity to these coefficients is analyzed in Appendix K. All experiments were run on 2 NVIDIA RTX A5000 GPUs with 24 GB of memory. Appendix C Data Construction Cost For Qwen2.5-Coder-3B-Instruct, data construction required approximately 78 GPU-hours: 24 GPU-hours for reference decomposition, 43 for function-level unit-test generation, and 11 for sampling k=8k=8 candidate completions. Decompositions, skeletons, and tests are generated once and reused across target models; only candidate sampling and CPU-based execution and labeling are model-specific. Method HumanEval MBPP BigCodeBench LiveCodeBench Base Plus Base Plus Full Hard KTO 0.872±0.0060.872_± 0.006 0.827±0.0070.827_± 0.007 0.743±0.0030.743_± 0.003 0.628±0.0030.628_± 0.003 0.364±0.0040.364_± 0.004 0.113±0.0040.113_± 0.004 0.109±0.0060.109_± 0.006 Step-KTOder 0.872±0.0060.872_± 0.006 0.829±0.0060.829_± 0.006 0.743±0.0040.743_± 0.004 0.628±0.0030.628_± 0.003 0.365±0.0030.365_± 0.003 0.128±0.0000.128_± 0.000 0.123±0.0060.123_± 0.006 Table 5: Mean pass rate ± standard deviation across three training seeds. Bold denotes the best mean within each benchmark. Method HumanEval MBPP BigCodeBench LiveCodeBench Base Plus Base Plus Full Hard Qwen2.5-7B-Instruct 0.811 0.768 0.839 0.704 0.366 0.135 0.134 w/ DPO 0.817 0.768 0.844 0.706 0.362 0.149 0.142 w/ Target-DPO 0.799 0.744 0.796 0.677 0.376 0.108 0.138 w/ KTO 0.823 0.780 0.847 0.709 0.361 0.155 0.138 w/ Step-KTOder 0.829 0.787 0.844 0.717 0.362 0.162 0.142 Table 6: Pass rates for the general-purpose Qwen2.5-7B-Instruct model. Appendix D DPO and Baseline Construction Details DPO (26) trains from paired preferences. Given a prompt x, a preferred solution y+y^+, and a dispreferred solution y−y^-, its loss is ℒDPO=−⁡[log⁡σ⁡(β⁡(rθ(x,y+)−rθ(x,y−)))],L_DPO=-E [ σ (β (r_θ(x,y^+)-r_θ(x,y^-) ) ) ], where rθ(x,y)=log⁡πθ(y∣x)πref(y∣x).r_θ(x,y)= _θ(y x) _ref(y x). Here, πθ _θ is the optimized policy, πref _ref is the frozen reference policy, σ⁡(⋅)σ(·) is the sigmoid function, and β controls the KL regularization strength. In our experiments, DPO pairs are constructed using the same candidate pool as KTO and Step-KTOder: a passing candidate is used as y+y^+ and a failing candidate as y−y^-. Unlike DPO, KTO and Step-KTOder operate on individually labeled candidates, allowing them to use unpaired positives or negatives. KTO samples are created using the same candidate rows and outcome labels as Step-KTOder, but we discard all function-level step labels, reducing training to outcome-only preference optimization. We train Target-DPO (31) on its dataset of 59,000 preference pairs. For comparability, we match our DPO LoRA setup: r=32r=32, α=64α=64, dropout 0.050.05, global batch size 16, and learning rate 5×10−75× 10^-7. Appendix E Multi-Seed Robustness To assess seed sensitivity, we train Qwen2.5-Coder-3B-Instruct with both KTO and Step-KTOder across three random seeds, holding all other settings fixed. Table 5 reports the resulting mean pass rate and standard deviation. Step-KTOder exceeds seed-matched KTO in all three runs on both BigCodeBench Hard and LiveCodeBench. The corresponding mean gains are 0.015 and 0.014, respectively; on the other five benchmarks, the difference between the method means is at most 0.002. Appendix F General-Purpose Model To test whether this pattern persists for a general-purpose instruction-tuned model, we additionally train Qwen2.5-7B-Instruct using on-policy candidates together with the validated ground-truth anchors used in our Qwen experiments. Qwen2.5-7B-Instruct starts from a stronger base than our other target models on MBPP, BigCodeBench, and LiveCodeBench. Step-KTOder exceeds KTO on six of seven benchmarks, with the largest gains on BigCodeBench Hard and LiveCodeBench; see Table 6 for full results. Appendix G Dataset Statistics This section reports difficulty distributions across the three stages of our data construction pipeline—the raw TACO and APPS training splits, the problems retained after ground-truth decomposition validation, and the final per-source training rows used by each target model—together with quality statistics for the function-level unit tests generated in Stage 2. Raw datasets. Table 7 reports the difficulty composition of the raw TACO and APPS training splits as released. Both datasets contain difficulty annotations covering a wide range from introductory exercises to competition-level problems. Our pipeline retains only problems with an explicit function-style entry point—identified by a non-empty fn_name field in the dataset’s annotations—and discards problems with other interfaces, which typically contain monolithic algorithmic solutions resistant to function-level decomposition. As a result, higher-difficulty buckets contribute few or no problems to our training set: in TACO, only easy, medium, and medium_hard problems contribute, while hard, very_hard, and unknown_difficulty problems are filtered out entirely. In APPS, only introductory and interview problems remain; competition problems are excluded. Source Difficulty # Problems % TACO easy 8,904 35.0 medium 3,244 12.7 medium_hard 2,745 10.8 hard 3,162 12.4 very_hard 2,374 9.3 unknown_difficulty 5,014 19.7 Total 25,443 100.0 APPS introductory 2,353 84.0 interview 450 16.0 Total 2,803 100.0 Table 7: Difficulty distribution of the raw TACO and APPS training splits before any filtering. Ground-truth decomposition validation. Table 8 reports per-difficulty pass rates after Stage 1, where ground-truth solutions are decomposed with Qwen2.5-Coder-32B-Instruct and validated by execution against the original test suite. In the table, Total denotes the number of problems entering Stage 1 with a clear function-style entry point, and Validated denotes the number whose decomposed solution still passes the original tests. Only validated problems proceed to candidate generation. Validation is much more permissive on APPS (88.9%88.9\%) than on TACO (43.9%43.9\%). The TACO drop is concentrated in the harder buckets: easy validates at 52.1%52.1\%, medium_hard at 37.1%37.1\%, and medium at only 18.8%18.8\%, reflecting the difficulty of producing semantically equivalent decompositions for problems with complex global state. Source Difficulty Total Validated Pass rate TACO easy 4,112 2,141 52.1% medium 1,200 225 18.8% medium_hard 520 193 37.1% Total 5,832 2,559 43.9% APPS introductory 2,325 2,069 89.0% interview 442 390 88.2% Total 2,767 2,459 88.9% Table 8: Ground-truth decomposition validation rates by difficulty. Qwen2.5-Coder family DeepSeek-Coder Source Difficulty Ground truth anchors 1.5B gen. 3B gen. 6.7B gen. TACO easy 2,141 2,671 2,695 2,748 medium 225 271 278 301 medium_hard 193 245 237 260 APPS introductory 2,069 2,643 2,657 2,661 interview 390 549 555 603 Total rows 5,018 6,379 6,422 6,573 Table 9: Difficulty distribution of the final training sets for each target model. Qwen2.5-Coder-1.5B-Instruct and Qwen2.5-Coder-3B-Instruct use the same validated decomposed reference solutions as positive anchors, but differ in their on-policy generated candidates. DeepSeek-Coder-6.7B-Instruct uses only on-policy generated candidates. Final training sets. Table 9 reports the difficulty distribution of the rows actually used to train each target model. For Qwen2.5-Coder-3B-Instruct (same-family), we retain all 5,018 validated ground-truth decompositions as additional positive anchors and add 6,422 generated rows produced by the target model itself. For DeepSeek-Coder-6.7B-Instruct (cross-family), ground-truth rows are discarded (Appendix M) and only on-policy generated candidates are retained. The final training sets are skewed toward easier problems, reflecting both the natural distribution of decomposition-friendly problems and the per-difficulty validation rates from Table 8. We discuss the implications in the Difficulty distribution paragraph of Section 8. Appendix H Unit Test Generation Quality Generation and validation protocol. Function-level unit tests are produced by Qwen2.5-Coder-32B-Instruct (Section 3.4, Stage 2) and validated by execution against the decomposed reference implementation. The generator outputs structured JSON test cases with named inputs, which are converted into a unittest class that calls the target function on each input and compares the result to the reference output. Tests are validated by execution: only test classes that parse, execute, and pass against the ground-truth code are retained. Functions for which no valid test remains receive a null step label (zi=∅z_i= ) and fall back to outcome-only KTO supervision during training. Table 10 summarizes unit-test generation quality, and Figure 3 shows the full validated-test-availability distribution. TACO APPS Decomposition scale Validated problems 2,559 2,459 Total functions 7,199 6,926 Testable functions 7,144 6,920 Unit-test generation First-pass parse success 98.8% 99.6% Testable functions w/ ≥1≥ 1 valid test 82.0% 84.5% Mean valid tests / covered function 5.46 5.59 Mean per-problem test availability 83.4% 85.4% Excluded functions Skipped functions (I/O glue) 55 6 Table 10: Function-level unit test generation quality. Validated-test availability is over testable functions. Configuration HumanEval MBPP BigCodeBench LiveCodeBench Base Plus Base Plus Full Hard λU=λU,step=1.0 _U= _U,step=1.0 (default) 0.878 0.835 0.746 0.630 0.367 0.128 0.127 λU,step=2.5 _U,step=2.5 0.878 0.835 0.746 0.624 0.365 0.122 0.127 λU=2.5 _U=2.5 0.878 0.835 0.746 0.624 0.365 0.122 0.049 Table 11: Effect of rebalancing outcome- and step-level λU _U on Qwen2.5-Coder-3B-Instruct. Method HumanEval MBPP BigCodeBench LiveCodeBench Base Plus Base Plus Full Hard KTO only 0.878 0.835 0.741 0.624 0.367 0.115 0.116 Step only 0.866 0.823 0.741 0.630 0.360 0.100 0.112 Step-KTOder (default) 0.878 0.835 0.746 0.630 0.367 0.128 0.127 Table 12: Step-only objective ablation on Qwen2.5-Coder-3B-Instruct. As shown in Table 10, first-pass parsing succeeds in over 98%98\% of calls, and covered functions have around 5.55.5 valid tests on average, providing multiple independent assertions per supervised step. Overall, 83.2%83.2\% of testable decomposed functions retain at least one validated test, while mean per-problem validated-test availability is 84.4%84.4\%; the latter is visualized in Figure 3. Mutation-sensitivity audit. As an additional sanity check, we evaluate whether the generated function-level tests detect controlled perturbations rather than merely executing successfully on the reference implementation. We audit all 11,705 functions from validated decomposed programs that retained at least one valid generated unit test across TACO and APPS, and apply syntax-preserving mutations to the reference function, including comparison flips, Boolean-operator flips, arithmetic-operator changes, constant perturbations, and default-return replacements. Across 27,958 valid mutants, the generated tests reject 25,880 mutants, yielding a mutation kill rate of 92.6%. This suggests that the retained tests reliably detect local behavioral changes rather than only validating executability on the reference solution. Appendix I Representative Function-Level Label Cases Incomplete local test. For a Pair of Shoes task (TACO 11007), the candidate checks whether the concatenated left and right-shoe sizes are unique, rather than comparing the two size multisets. Because the retained test for pair_of_shoes covers only the empty input, both local labels are positive although the full program is incorrect. This illustrates why Step-KTOder retains the outcome-level term. Useful local/global conflict. In a Task Scheduler problem (APPS 171), the candidate passes the dataset-provided test suite, but its decomposed step calculate_min_intervals omits len(tasks) from the maximum, understating the required number of intervals on inputs where the task count exceeds the frequency-based bound. Its negative local label exposes a hidden defect that outcome-only KTO cannot represent. Appendix J Repair Intervention Details Beyond the aggregate repair rates reported in Section 6.2, we also examine how the intervention behaves across failure types. The analysis is run on originally failing Qwen2.5-Coder-3B-Instruct selected training candidates whose supervised functions include both positive and negative step labels. For each candidate, we replace local-negative functions with their decomposed reference implementations and rerun the dataset-provided tests. As controls, we replace either the same number of local-positive functions or the same number of randomly selected observed functions. The repair effect is strongest for wrong-answer failures: replacing local-negative functions repairs 78.6% of such candidates on the combined APPS and TACO subset. This suggests that local-negative labels are especially effective at identifying semantic errors, rather than merely capturing parsing or execution artifacts. Appendix K Sensitivity to KTO Value-Function Weights The default configuration uses unit weights for all KTO value-function coefficients (λD=λU=λD,step=λU,step=1.0 _D= _U= _D,step= _U,step=1.0). The KTO authors recommend rebalancing λU _U when the positive/negative sample ratio is skewed; our Qwen2.5-Coder-3B-Instruct training set has a step-level positive/negative ratio of approximately 2.5:12.5:1, motivating a check on whether upweighting undesirable samples improves performance. When one coefficient is varied, all other value-function coefficients remain fixed at 1.01.0. Table 11 reports two rebalancing runs against the default. Both deviations from uniform weights regress BigCodeBench Hard. The outcome-level λU=2.5 _U=2.5 run additionally collapses LiveCodeBench performance, suggesting that outcome-level rebalancing is unstable in this setting. One possible explanation is that the observed positive/negative ratio reflects the natural pass distribution of a strong instruction-tuned 3B model rather than simple dataset imbalance; reweighting failures too aggressively may over-allocate gradient budget toward avoiding failures at the cost of reinforcing successful behavior, hurting generalization to harder benchmarks. Appendix L Additional Objective Ablations Our default objective combines outcome-level and function-level supervision: ℒStep-KTOder=ℒout+λstepℒstep.L_Step-KTOder=L_out+ _stepL_step. To better understand the role of the outcome-level term, we evaluate a step-only variant on Qwen2.5-Coder-3B-Instruct. This variant removes the outcome-level KTO loss and optimizes only the stepwise loss, allowing us to assess how much signal is provided by local execution feedback alone. As shown in Table 12, the step-only variant retains useful signal from function-level execution feedback, matching KTO on MBPP and matching the full Step-KTOder objective on MBPP+. However, removing the outcome-level KTO term weakens performance on HumanEval, BigCodeBench, and LiveCodeBench. The Step-KTOder objective performs best overall, with the clearest advantage on BigCodeBench Hard. These results support our formulation of Step-KTOder as a joint objective: the step loss provides localized credit assignment, while the outcome-level term preserves the end-to-end correctness signal needed for robust program-level performance. Appendix M Off-Policy Training Data in Cross-Family Training To test whether on-policy candidate generation is necessary, we train DeepSeek-Coder-6.7B-Instruct on candidates generated by Qwen2.5-Coder-3B-Instruct on the same set of decomposed problems used in our main DeepSeek run. Both training sets share the same decomposed function skeletons and unit tests, generated by Qwen2.5-Coder-32B-Instruct; the only difference is the model that produced the candidate completions. The Qwen-generated candidates are fully off-policy with respect to the DeepSeek reference model. Training data Method HumanEval MBPP BigCodeBench LiveCodeBench Base Plus Base Plus Full Hard On-policy (default) KTO 0.793 0.726 0.749 0.640 0.347 0.122 0.127 On-policy (default) Step-KTOder 0.799 0.732 0.754 0.648 0.347 0.128 0.131 Off-policy (Qwen 3B-generated) KTO 0.805 0.735 0.749 0.640 0.345 0.115 0.112 Off-policy (Qwen 3B-generated) Step-KTOder 0.805 0.735 0.751 0.648 0.347 0.122 0.127 Table 13: Effect of training with off-policy data on DeepSeek-Coder-6.7B-Instruct. As shown in Table 13, off-policy training degrades both methods on the hardest benchmark (BigCodeBench Hard: KTO 0.122 to 0.115, Step-KTOder 0.128 to 0.122). The effect on LiveCodeBench is asymmetric: KTO regresses substantially (0.127 to 0.112) while Step-KTOder regresses marginally (0.131 to 0.127), suggesting that function-level supervision is more robust to policy mismatch than outcome-only supervision. Easier benchmarks (HumanEval, HumanEval+) show small improvements under off-policy training, likely reflecting the broader problem distribution covered by the Qwen model’s candidate pool. We attribute the BigCodeBench Hard regression to noisier log-ratio rewards: when the candidate distribution diverges from the reference model’s, the ratio log⁡πθ/πref _θ/ _ref is poorly calibrated, weakening the KTO objective. The function-level signal partially compensates because it operates on shorter token spans, where calibration noise has less cumulative effect. We therefore use the on-policy configuration as our default setting. Appendix N LLM-as-a-Judge vs. Execution-Based Step Labels To evaluate execution-based step labeling against a common alternative, we compare Step-KTOder’s unit-test-based function labels with annotations from GPT-5.4 mini, applied to the same candidate functions in the Qwen2.5-Coder-3B-Instruct training pipeline. The comparison has two components: (i) label-level agreement statistics on the full candidate pool, and (i) a downstream training experiment in which execution-based labels are replaced by LLM-as-a-judge labels in the actual Step-KTOder training set. Setup. For agreement analysis, we sample N=88,469N=88,469 supervised functions from the Qwen2.5-Coder-3B-Instruct candidate pool, stratified by the (o,z)(o,z) outcome–step pair, and ask GPT-5.4 mini to predict whether each function would pass its associated unit tests. The judge sees the function source code and the unit tests, and returns a binary label with a one-sentence rationale; the system prompt is reproduced in Appendix O. For the downstream experiment, we apply the same procedure only to the supervised functions in the final training rows, replace execution-based labels with LLM-as-a-judge labels where available, and retrain with the same settings as the main result. (a) Agreement by (o,z)(o,z) stratum. (b) Row-normalized confusion matrix. Figure 4: LLM-as-a-judge agreement with execution-based labels is high for failing functions (z=0z=0) but much lower for passing functions (z=1z=1), revealing a bias toward predicting failure. Step labels HumanEval MBPP BigCodeBench LiveCodeBench Base Plus Base Plus Full Hard KTO baseline 0.878 0.835 0.741 0.624 0.367 0.115 0.116 Step-KTOder + execution (default) 0.878 0.835 0.746 0.630 0.367 0.128 0.127 Step-KTOder + LLM-as-a-judge (GPT-5.4 mini) 0.860 0.817 0.746 0.635 0.354 0.095 0.112 Relative change vs. execution -2.1% -2.2% +0.0% +0.8% -3.5% -25.8% -11.8% Table 14: Effect of replacing execution-based step labels with GPT-5.4 mini judgments on Qwen2.5-Coder-3B-Instruct. Agreement is asymmetric. Figure 4(a) reports agreement by (o,z)(o,z) stratum, and Figure 4(b) shows the row-normalized confusion matrix. Overall agreement is 73.2%73.2\%, but this aggregate score hides a strong asymmetry. The LLM agrees with execution on 94.1%94.1\% of functions labeled as failing by execution (z=0z=0), but on only 52.2%52.2\% of functions labeled as passing by execution (z=1z=1). At the stratum level, agreement is high on both failing-function strata (o=1,z=0o=1,z=0: 92.4%92.4\%; o=0,z=0o=0,z=0: 94.8%94.8\%), but only around 52%52\% on both passing-function strata. This pattern indicates that the LLM judge has a strong false-negative bias: it often flags true failures, but also frequently hallucinates issues in correct implementations. This preserves many negative conflict labels while corrupting the much larger pool of clean positive step labels. Downstream impact. Table 14 compares Step-KTOder trained with execution-based labels versus LLM-as-a-judge labels on Qwen2.5-Coder-3B-Instruct. Replacing execution labels with GPT-5.4 mini judgments substantially degrades the hardest benchmarks, with relative drops of 25.8%25.8\% on BigCodeBench Hard and 11.8%11.8\% on LiveCodeBench. Notably, BigCodeBench Hard falls below the outcome-only KTO baseline, confirming that moderate label-level agreement can still hide systematic bias that harms downstream training. Discussion. Moderate aggregate agreement does not translate into useful supervision because the LLM judge’s errors are highly asymmetric: its false-negative bias corrupts passing step labels, which Step-KTOder needs to stabilize function-level training. Within the operating regime of our framework, execution-based labels are not merely a convenient choice but a necessary one. Appendix O Prompts Here, we outline the key prompts used in our Step-KTOder data construction pipeline. Figure 5 shows the prompts used to decompose reference solutions into multi-function programs. Figure 6 gives the prompt used for generating function-level unit tests, and Figure 7 shows the prompt used to sample on-policy candidate completions from function skeletons. Finally, Figure 8 gives the prompt used by the LLM-as-a-judge in our labeling-method comparison (Appendix N). Prompt for Function Decomposition SYSTEM PROMPT You are an expert Python refactoring assistant. Your task is to decompose a Python solution into semantically meaningful top-level helper functions while preserving exact behavior and the exact required interface. When the solution contains distinct logical stages, decompose it into multiple helper functions rather than leaving everything inside one large function. Do not invent trivial helpers or change the algorithm unnecessarily. Follow the requested XML schema exactly. Return ONLY valid XML. No explanations. USER PROMPT Refactor the following Python solution into structured XML with multiple <function> blocks. Goal: - Decompose into helper functions where useful. - Preserve behavior exactly. - Keep the required entrypoint unchanged (same name and signature). Required entrypoint (must appear EXACTLY in your code): required_entrypoint_line Rules: 1. Do NOT introduce class Solution (this is a plain function problem). 2. Do NOT read from stdin or print output. 3. Do NOT include any top-level execution (no main(), no __starting_point(), no if __name__ == "__main__": ...). 4. Put helper functions ABOVE the required entrypoint. 5. Each function definition must be in its own <function> block. 6. The <code> section should contain exactly ONE def statement (or one class). 7. Copy all original imports at the very top of the FIRST <code> block. 8. Do NOT add unit tests yet: keep <tests> as placeholder comments. 9. Do NOT define nested functions inside another function or method. Every helper must be top-level and placed in its own <function> block. 10. When all <code> blocks are concatenated in order, the result must be a valid standalone Python solution with exactly the same behavior as the original. 11. When the solution contains distinct logical stages, decompose it into multiple semantically meaningful helper functions. 12. Avoid returning a single large function unless decomposition is genuinely unnecessary. 13. Do not invent trivial helpers just to increase the number of functions. XML format: <function name="..."> <docstring>...</docstring> <tests> tests_placeholder </tests> <code> ... code here ... </code> </function> Problem: problem_description Original solution: ground_truth_solution Refactored XML format (ONLY XML): Figure 5: System and user prompts used to decompose reference solutions into behavior-preserving multi-function programs. Prompt for Unit Test Generation Output must be a JSON object with this exact schema: "cases": [ "name": "short_name", "args": [...], "kwargs": ..., ... ] Rules: - args must be a JSON array; kwargs must be a JSON object (use if none). - Use only JSON-serializable values: null, true/false, numbers, strings, lists, objects. - Keep inputs physically SMALL (e.g., arrays under 5 items, integers between -50 and 50) to avoid execution timeouts. - The test suite MUST include at least one or two ADVERSARIAL EDGE CASES (e.g., empty lists [], empty strings "", zero 0, or negative numbers -1). - Do NOT make every case an edge case. Provide a balanced mix of typical and boundary inputs. - Do NOT include expected outputs. - Provide 4 to 6 diverse cases. Figure 6: User prompt schema and constraints for generating function-level unit tests. The model returns only test inputs; expected outputs are derived by executing the reference implementation. Prompt for Candidate Generation Solve the following programming problem. You MUST implement the provided skeleton by replacing each pass with a correct implementation. Rules: - Keep all function/class names and signatures unchanged. - Do not remove any definitions. - Do not add new functions, methods, or classes. - Only fill the bodies of the provided definitions. - Return a single complete Python module. ### problem question ### skeleton skeleton_code Figure 7: User prompt for candidate generation. The target model fills the function skeleton (with pass as each function body) constructed in Stage 3 of our pipeline. Prompt for LLM-as-a-Judge You are an expert Python programmer acting as a code reviewer. You will be given a Python function and a set of unit tests for that function. Your job is to decide whether the function implementation would pass all the provided unit tests when executed. Respond in strict JSON of the form: "label": 0 or 1, "reason": "<one short sentence>" - label = 1 means you believe the function would pass all provided tests. - label = 0 means you believe at least one test would fail. Do not include any other text outside of the JSON object. Figure 8: System prompt for the GPT-5.4 mini judge in the LLM-as-a-judge experiment (Appendix N). The judge sees the function’s source code and its generated unit tests, and predicts whether the function passes them.

Instant research discovery

Function-Level Execution Feedback for Code Preference Optimization

Intelligence

Summary

Entities (11)

Relation Signals (9)

Abstract

Tags

Links

PDF

Full Text

Paper	Published	Area	Tags	Intel	Citations