Paper deep dive

Steerable Instruction Following Coding Data Synthesis with Actor-Parametric Schema Co-Evolution

Tinglin Huang, Bo Chen, Xiao Zhang, Kai Shen, Rex Ying

Year: 2026Venue: arXiv preprintArea: cs.SEType: PreprintEmbeddings: 83

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/20/2026, 6:19:08 AM

Summary

The paper introduces IFCodeEvolve, a framework for synthesizing large-scale instruction-following coding data using an actor-schema co-evolution paradigm. It employs parametric function schemas and Monte Carlo Tree Search (MCTS) to navigate the instruction space, ensuring logical compatibility and solvability. The framework iteratively evolves both the actor model and the schema library to generate increasingly challenging problems, resulting in significant performance boosts for base models, with a 32B model achieving parity with proprietary SOTA models. The authors also release IFCodeBench, a human-verified benchmark with AST-based verification.

Entities (8)

IFCodeEvolve · framework · 98%IFCodeBench · datasetbenchmark · 97%Monte Carlo Tree Search · algorithm · 95%Parametric Instruction Schema · concept · 92%Actor Model · model-component · 90%Qwen2.5-Coder-32B · model · 90%AST-based Verification · method · 88%32B Model · model · 85%

Relation Signals (6)

IFCodeEvolve → produces → IFCodeBench

confidence 97% · we contribute IFCodeBench, a comprehensive human-verified benchmark

IFCodeEvolve → uses → Monte Carlo Tree Search

confidence 95% · Monte Carlo Tree Search (MCTS) sampler is applied to efficiently navigate this space

IFCodeEvolve → utilizes → Parametric Instruction Schema

confidence 92% · By representing instructions as parametric function schema

IFCodeEvolve → employs → Actor Model

confidence 90% · utilizing actor model feedback as a dynamic termination signal

IFCodeEvolve → uses → AST-based Verification

confidence 88% · robust AST-based verification

IFCodeEvolve → improves → 32B Model

confidence 85% · our 32B model achieving parity with proprietary SOTA models

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Interpreting and following human instructions is a critical capability of large language models (LLMs) in automatic programming. However, synthesizing large-scale instruction-paired coding data remains largely unexplored and is particularly challenging when ensuring logical compatibility among multiple constraints. In this study, we propose IFCodeEvolve, an actor-schema co-evolution framework for instruction following coding data generation. By representing instructions as parametric function schema, we construct a library that covers the vast instruction space via dynamic constraint instantiation. Building upon this, Monte Carlo Tree Search (MCTS) sampler is applied to efficiently navigate this space, utilizing actor model feedback as a dynamic termination signal. Furthermore, to progressively explore challenging problems, we introduce a co-evolving paradigm that iteratively advances both the actor model and the schema library, via schema composition and mutation, based on sampler statistics. Empirical results demonstrate that IFCodeEvolve significantly boosts base model performance, with our 32B model achieving parity with proprietary SOTA models. Additionally, we contribute IFCodeBench, a comprehensive human-verified benchmark equipped with solutions and robust AST-based verification.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

81,475 characters extracted from source content.

Expand or collapse full text

Steerable Instruction Following Coding Data Synthesis with Actor-Parametric Schema Co-Evolution Tinglin Huang 1,∗ , Bo Chen 2,∗,† , Xiao Zhang 2 , Kai Shen 2 , Rex Ying 1 1 Yale University, 2 ByteDance Seed ∗ Equal Contribution, † Corresponding authors Abstract Interpreting and following human instructions is a critical capability of large language models (LLMs) in automatic programming. However, synthesizing large-scale instruction-paired coding data remains largely unexplored and is particularly challenging when ensuring logical compatibility among multiple constraints. In this study, we propose IFCodeEvolve, an actor-schema co- evolution framework for instruction following coding data generation. By representing instructions as parametric function schema, we construct a library that covers the vast instruction space via dynamic constraint instantiation. Building upon this, Monte Carlo Tree Search (MCTS) sampler is applied to efficiently navigate this space, utilizing actor model feedback as a dynamic termination signal. Furthermore, to progressively explore challenging problems, we introduce a co-evolving paradigm that iteratively advances both the actor model and the schema library, via schema composition and mutation, based on sampler statistics. Empirical results demonstrate that IFCodeEvolve significantly boosts base model performance, with our 32B model achieving parity with proprietary SOTA models. Additionally, we contribute IFCodeBench, a comprehensive human-verified benchmark equipped with solutions and robust AST-based verification. Date: April 21, 2026 Correspondence: chenbo.1015@bytedance.com 1 Introduction Large language models (LLMs) have recently achieved remarkable success in automatic programming, spanning domains from competitive programming [1,16,23] to software engineering [9,14,36]. Beyond fundamental coding proficiency, an important factor leading to these advancements is the instruction following (IF) capability [21]. IF serves as a bridge between human intent expressed in natural language and executable logic, as illustrated in Figure 1. Question Description: Write a Python function to calculate the n-th Fibonacci number Instruction: 1. Use recursion 2. Use snake_case for naming Instruction: 1. Use a for loop in solution 2. Add type hints to function Instruction: 1. Use functools.lru_cache 2. Don't use temporary variables def fib_recursive(n): if n <= 1: return n return fib_recursive(n-1) \ + fib_recursive(n-2) def fibIterative(n: int) -> int: a, b = 0, 1 for _ in range(n): a, b = b, a + b return a from functools import lru_cache @lru_cache(maxsize=None) def fib(n): if n < 2: return n return fib(n-1) + fib(n-2) Figure 1 Instruction-driven code generation. Improving the IF capability of LLMs necessitates large- scale coding data paired with high-quality instructions, which currently depends heavily on manual curation. While recent approaches have explored automatic data synthesis using static brute-force sampling [5,31] or adversarial self-evolving frameworks [12,40], they remain limited by two challenges: (1) Validity and verification bottleneck: Directly prompting LLMs to generate multi- 1 arXiv:2604.16322v1 [cs.SE] 27 Feb 2026 Actor MCTS Sampler Evolving IF Coding Dataset MCTS SamplerMCTS Sampler Evolving ActorActor : Schema : Sampled Schema IF Coding Dataset IF Coding Dataset Figure 2 Illustration of actor-schema co-evolution paradigm for IF coding data generation. constraint coding problems struggles to guarantee logical consistency and solvability [32,34]. Furthermore, generating accurate verification logic (e.g., unit tests) is itself error-prone. (2) Unstructured hard-sample exploration: Static frameworks rely primarily on the generator’s intrinsic priors, resulting in repetitive data patterns and limited coverage of out-of-distribution samples. In contrast, without carefully engineered and highly intricate reward designs, the generator of adversarial strategies can easily hack the reward by producing unsolvable or ill-defined problems. Motivated by this, we propose IFCodeEvolve, a framework that optimizes logical validity and problem complexity by unifying progressive proof-by-construction and adversarial actor-schema co-evolution. We apply parametric instructions, which represents a vast instruction space within a compact library of schema. This formulation transforms data synthesis into a steerable and sequential sampling process. Specifically, we frame synthesis as MCTS trajectory sampling integrated with proof-by-construction validation strategy: at each step, the sampler navigates the combinatorial schema space to incrementally impose constraints, which are immediately instantiated and verified via code adaptation to ensure strict solvability. Furthermore, this structured representation facilitates the exploration of high-complexity problems through actor-schema co-evolution: in parallel with the actor iteratively improves via post-training, the schema library autonomously evolves by merging synergistic primitives and mutating under-performing constraints. Unlike methods that improve data distribution by explicitly training generator, we enhance data synthesis in a steerable manner, explicitly optimizing structural primitives to continuously challenge the evolving model. The overall paradigm is illustrated in Figure 2, where the sampler constructs IF coding data with iteratively improved actor and instruction schema. We evaluate IFCodeEvolve on two public benchmarks across five LLMs with varying parameter sizes (1.3B to 32B). Empirical results demonstrate that our framework drives consistent performance gains to the actor models across evolving procedures. Notably, the larger models trained on our synthesized data achieve a substantial improvement, achieving performance comparable to proprietary SOTA reasoning models. To complement these general baselines with fine-grained assessment, we introduce IFCodeBench, a curated benchmark equipped with executable tests and AST-based verification protocols. Finally, we posit that this paradigm, which extends the scope of evolution from the target model to the core components of the synthesis framework itself, offers a generalizable methodology for constructing self-adapting curricula in other reasoning-intensive domains, such as mathematical problem solving [24, 30] and scientific discovery [7, 15]. 2 Related Work Instruction Tuning Instruction-following (IF) is a foundational capability of LLMs, requiring the model to accurately interpret complex instructions and synthesize valid responses within an open-ended search space [18]. To address the scarcity of high-quality instruction data, prior studies have resorted to manual annotation [3,21] or leveraged LLMs for large-scale data generation based on seed datasets [5,27,31], typically necessitating dedicated data filtering procedures. Concurrently, numerous benchmarks have been proposed to evaluate these capabilities across diverse domains, including single-turn QA [13,22,41], multi-turn conversation [4], and 2 code generation [32,34]. Different from a static pipeline, our work introduces a co-evolutionary framework for IF coding data generation, which dynamically adapts its core synthesis modules to iteratively refine the data distribution. Self-Evolving Agents As LLMs demonstrate increasingly reasoning capabilities, a promising line of research investigates their potential to self-optimize agentic systems [8]. Existing studies generally fall into four categories in terms of applications: (1) prompt tuning, where LLMs function as meta-optimizers to iteratively rewrite instructions and in-context exemplars, reducing reliance on manual configurations [33,35]; (2) workflow construction; which involves representing agentic topologies as computational graphs or code and optimizing interaction structures to solve complex problems [11,39]; (3) solution refinement, in which the system engages in recursive feedback loops to critique, debug, and iteratively improve its own outputs [17,19,20]; and (4) data augmentation/generation, where the models distill knowledge into datasets by synthesizing high-quality reasoning trajectories for self-training [15,25,29,37], or employ adversarial strategies to generate challenging training instances [12,40]. Distinct from these prior approaches, IFCodeEvolve implements a meta-level evolution of the data generation mechanism itself. By iteratively evolving the instruction schema and actor, we sustain an adaptive curriculum for coding data generation. 3 Method In this section, we introduce IFCodeEvolve, a co-evolving instruction-following coding data generation framework. We first formulate the problem and objective in Section 3.1, and elaborate the core components in Section 3.2. The implementation of the proposed co-evolving data synthesis framework is presented in Section 3.3 and Section 3.4. The pesudocode can be found in Algo.1. 3.1 Problem Formulation Given a set of coding problems, solutions, and functional unit testsD=(p k ,s k ,f k test (·)) N k=1 , where each coding problemp k includes a question description, function signature, and input/output example, the goal of our proposed framework is to augment each instance by synthesizing a set of instruction constraints: p ′ k ,s ′ k ,ι k = f IFCoEvolve (p k ,s k )(1) whereι k =ι k,j M k j=1 is the set of instructions fork-th problem. We focus on verifiable instructions, i.e., the instructions can be automatically validated via checking functions, such as the variable name convention, data structure usage, or interface design. The instructions impose strict constraints without altering the core algorithmic semantics, enforcing the model to reason within a narrowed solution space. Challenges Constructing high-quality IF coding datasets presents two primary challenges: (1) Constraint compatibility and solvability: Unlike general chat instructions, coding constraints must be logically consistent with both the problem context and each other. For instance, a restriction on variable naming conventions can directly conflict with a directive like “must not use intermediate variables”. Ensuring that the aggregated instructions remain soluble without creating logical conflict is non-trivial. (2) Balancing difficulty with validity: The generated instructions must be sufficiently challenging to push the model’s reasoning boundary, targeting problems where the model currently fails. However, generating fundamentally unsolvable problems would easily satisfy such a criterion while hack the objective. The core challenge lies in generating samples that are adversarially hard yet strictly valid. 3.2 Instruction Representation and Sampling We introduce parametric instruction schema for steerable data synthesis, and an MTCS sampler used to explore instruction combinations from the vast schematic space. Parametric Instruction Previous methods [5,31] prompt LLMs to brainstorm instruction candidates using static text representations. Such representations treat parametric variations of the same logic (e.g., variable length > 5 vs. < 5) as distinct entities, leading to a redundant search space. 3 Question Description: Write a Python function to check if a string is a palindrome. Your code should_use utilize construct_type. Your code must not utilize slicing. Instantiation Proof-by-Construction Instruction: 1.Your code must not utilize slicing. def is_palindrome(s): return s == s[::-1] : Your code should_use utilize construct_type. Success or Retry ... Instruction: 1.Your code must not utilize slicing. 2.Type hints must be provided. 3.Variable must follow camelCase. ... ... ...... ...... ...... Execution-Based Routing Finalized Dataset ... def is_palindrome(s): l, r = 0, len(s) - 1 while l < r: if s[l] != s[r]: return False l += 1; r -= 1 return True Figure 3 Illustration of multi-round IF coding data augmentation. In light of this, we apply parametric instruction [32], representing each instruction as an instantiable function schema, with configurable constraint attributes serving as parameters. Formally, a concrete instructionιis derived by instantiating a schema τ which consists of: •A textual skeleton with variable slots, e.g., “The length of variable names comparison exceed length characters.”. •An argument space defining valid types and scope for the slots, e.g.,comparison ∈[“must”, “must not”] and length ∈ int. • f IF-test (·): An AST 1 -based verification function that maps a given solution to0,1, indicating whether it strictly adheres to the instantiated instruction. The instantiation process is driven by an LLM conditioned on the problem context, as detailed in Section 3.3. We construct an initial libraryTof 27 diverse schema (see Appendix C), which serves as the foundation for the subsequent sampling procedure. Monte Carlo Tree Search-based Sampler Building uponT, we formulate the IF data synthesis process as a sequential tree search structure, where each node expansion corresponds to instantiating a schema and a root-to-leaf trajectory constitutes the final aggregated instruction set. To navigate this vast combinatorial space, we employ the sampler based on Monte Carlo Tree Search (MCTS) [10], which manages the sampling by quantifying the effectiveness of schema, i.e., estimating the potential of a branch to yield a challenging instruction set. Formally, letτ <t =τ 1 ,· ,τ t denote the state at stept, representing the sequence of schema sampled so far. Each schema corresponds to an instantiated instruction, forming the setι <t =ι 1 ,· ,ι t . At each step, the MCTS selects a small batch of the schema from the remaining libraryT \τ <t based on the estimated value: τ C ∼ softmax τ∈T\τ <t Q(τ <t ,τ )(2) whereCis the candidate size. The value is calculated by aggregating the exploitation and exploration terms: Q(τ <t ,τ ) = W (τ <t ,τ ) + α· P (τ ) N (τ <t ,τ ) + α + U (τ <t ,τ )(3) whereW(τ <t ,τ) andN(τ <t ,τ) are accumulated success count and visit count of schemaτat stateτ <t ,P(τ) represents the global prior success rate ofτ, andαis a smoothing factor (an integer>1) to mitigate high variance in low-visit nodes [38]. The exploration term U (τ <t ,τ ) is calculated as: U (τ <t ,τ ) =    ηif N (τ <t ,τ ) = 0, η q lnN(τ <t ) N(τ <t ,τ) if N (τ <t ,τ ) > 0. (4) whereN(τ <t ) is the state visit count andηis the exploration weight. For a schemaτthat has never been sampled at the current state, i.e.,N(τ <t ,τ) =W(τ <t ,τ) = 0, the valueQ(τ <t ,τ) simplifies toP(τ) +η, which prioritize schema with high global performance. 1 https://docs.python.org/3/library/ast.html 4 Discussion The utilization of parametric instructions enables us to cover a vast instruction space with a manageable set of schema. As such, it allows the application of MCTS, which is grounded in statistical priors to significantly enhance data synthesis efficiency. 3.3 Multi-Round Augmentation As illustrated in Figure 3, given a coding data and schema library, our framework iteratively adding one sampled instruction from MCTS to the problem, validating solvability via a proof-by-construction instantiation and filtering instances through execution-based routing. Proof-by-Construction Instantiation The dataset at steptis denoted asD t , where each instance comprises the problem, the current solution, and the accumulated instructions, represented as (p t ,s t ,ι <t ). For the subsequent step, MCTS samples a batch of candidate schema via Equ. 2. The transition to the next state is performed by an LLM: p t+1 ,s t+1 ,ι <t+1 = π gen (p t ,s t ,ι <t ,τ C )(5) where we consider the LLM serves as data generatorπ gen . Specifically, the LLM is prompted to rank the candidate schema based on the compatibility, subsequently instantiating the selected schema to maximize difficulty. To guarantee validity, the LLM is required to synthesize a witness solution by modifying the current solutions t conditioned on the problem, the historical instruction set, and the newly added instruction. The instantiation is deemed successful only if the witness solution passes the cumulative set of IF verification functions as well as the functional unit tests, thereby demonstrating the solvability of the augmented problem. Overall, instead of validating from scratch, we frame the augmentation as a progressive code adaptation task, injecting deterministic external feedback via execution to rigorously prove constraint feasibility. This step-wise verification enforces inductive solvability: by ensuring that every increment is valid, the final problem is guaranteed to be soluble. Execution-Based Feedback Routing We implement a feedback-driven curriculum strategy by deploying an LLM as an actor model to dynamically probe the solvability of the generated data. The actor’s execution pass rate serves as a real-time indicator of difficulty, which is calculated as: E s f test (s) < λ ∧   E s " Y ι f ι IF-test (s) # < λ   (6) whereλdenotes the difficulty threshold and solutions are drawn from the actor, i.e.,s∼ π actor (·|p t ,ι <t ). We estimate these expectations via Best-of-N sampling [26]. This equation indicates that a problem is deemed difficult if both the average functional pass rate and the average IF all-pass rate fall below a specific threshold. The calculated indicator drives a conditional routing policy: after augmenting the dataset with new instructions at stept, instances that the actor fails to solve are early-stopped for finalization, whereas successfully solved instances are propagated to the next iteration for continued augmentation. This indicator also functions as the win condition for MCTS. That is, if the current instruction can induce failure, it is counted as a win (i.e., W (τ <t ,τ )← W (τ <t ,τ ) + 1)). 3.4 Co-Evolving Paradigm Our proposed augmentation paradigm establishes a robust pipeline for IF code data generation. However, adhering to a static schema library and a fixed actor limits problem diversity and difficulty at the actor’s performance. Motivated by this, we extend the static pipeline into a self-evolving framework, which facilitates the co-evolution of both the actor and the schema library, ensuring that the data generation process dynamically scales with the model’s capabilities. 5 Actor Evolution In our framework, the actor serves as a dynamic filter, routing instances that it fails to solve into the finalized problem set. To ensure the generation of progressively challenging problems, we implement an iterative evolution paradigm: upon completing each augmentation round, the actor is post-trained on the newly synthesized data. This optimized actor is used in the subsequent generation phase, raising the difficulty bar for future samples. The evolving procedure of actor build a curriculum generation process: the problem patterns that the actor initially fails can potentially overcome post-training, prompting the paradigm to pivot towards generating strictly harder instances that remain unsolved by the improved actor. Instruction Schema Evolution While the initial library T provides a foundation, a fixed schema set can let actor easily adapt to fixed patterns. In light of this, we propose to refineTvia two strategies: composition and mutation. As for the composition, we leverage MCTS trajectory statistics to identify synergistic schema compositions. Specifically, the global joint success rate for a transitionτ i → τ j is aggregated over all visited statesSending in τ i : ̄ R(τ i ,τ j ) = P τ <t ∈S τ i W (τ <t ,τ j ) P τ <t ∈S τ i N (τ <t ,τ j ) (7) whereS τ i =τ <t | last(τ <t ) =τ i denotes the subset of states where the most recently sampled schema isτ i . A pair (τ i ,τ j ) is selected for composition if its occurrence frequency demonstrates a synergistic difficulty gain compared to the individual priors P (·): ̄ R(τ i ,τ j ) > max P (τ i ),P (τ j ) + δ(8) whereδdenotes the minimum margin of difficulty improvement. Selected pairs are merged to form a new composite schema τ i⊕j , which unifies the logic of both constituents into a single parametric function. As for the mutation, it targets the under-performing schema based on their global prior success rateP(τ), i.e., T weak ⊂Twith the lowestP(τ) values. For eachτ ∈T weak , we retrieve a set of question instances including the instruction instantiated fromτ, where the actor successfully passes this question in both functionality and IF. These instances with actor’s solutions are fed into an LLM for adversarial analysis. The LLM is prompted to: • Diagnose: Analyze why the current constraintτwas easily satisfied by the actor (e.g., identifying loopholes, loose parameter boundaries, or lack of corner-case coverage). •Mutate: Propose an evolved schemaτ ′ that strictly tightens the constraint or closes the identified loophole, such that the actor’s original solution would fail under τ ′ . • Validate: Generate two positive witness examples forτ ′ to ensure the mutated schema remains logically sound. Only mutated schemas that pass the validation check are incorporated intoT. This adversarial process effectively forces the actor to generalize rather than exploit specific problem patterns in the instruction design. Discussion Fundamentally, the generated problem distribution is determined by the interplay between the actor model and the schema library. Our co-evolving paradigm optimizes this distribution by advancing these components in a mutually complementary manner, where the actor’s capability growth necessitates higher complexity while the evolving library supplies the necessary structural diversity. 4 Experiment In this section, we present the experimental results of proposed IFCodeEvolve and a curated benchmark dataset based on the generated data, i.e., IFCodeBench. 6 Table 1 Comparison on public IF-Coding benchmarks. IFEvalCodeCodeIF Inst. PromptInst. Prompt Proprietary Models GPT-5.20.9391 0.85710.9460 0.6667 Gemini-3 Pro0.9271 0.82860.8757 0.4023 Claude-4.5-Sonnet0.9273 0.80480.8724 0.3879 Claude-4.5-Opus 0.9249 0.79520.9281 0.5575 Seed-1.6-thinking0.9023 0.72380.7949 0.2040 Seed-1.8-thinking0.9152 0.78570.6764 0.3621 Open-Source Models GLM-4.70.8989 0.70000.7163 0.3563 Kimi-K20.8606 0.57140.8290 0.2931 GPT-OSS-20B0.8894 0.71900.7090 0.3649 GPT-OSS-120B 0.9144 0.75710.5910 0.3362 Seed-OSS-36B 0.7363 0.63810.8231 0.3362 Qwen3-Coder-480B0.8466 0.57620.8144 0.2358 DeepSeek-V3.20.8923 0.67410.8747 0.3506 DeepSeek-Coder-V20.7681 0.41900.7274 0.1494 IFCodeEvolve Qwen2.5-Coder-7B0.6837 0.28000.5986 0.0718 + 1st-Generation0.8298 0.58570.8281 0.2557 + 2nd-Generation 0.8458 0.59040.8357 0.2844 + 3rd-Generation 0.8701 0.61430.8435 0.3007 Seed-Coder-8B0.8051 0.46670.7337 0.1523 + 1st-Generation0.8382 0.55710.8430 0.2902 + 2nd-Generation0.8552 0.59520.8509 0.3017 + 3rd-Generation0.8519 0.60950.8520 0.3333 Table 2 Accuracy analysis on IFEvalCode. ActorOriginal + 1st + 2nd + 3rd Qwen2.5-Coder-7B0.1523 0.2230 0.2523 0.2571 Seed-Coder-8B0.2904 0.2952 0.3285 0.3142 Table 3 Performance after training on syn- thetic data. IFEvalCodeCodeIF Inst. PromptInst. Prompt DeepSeek-Coder-1.3B0.6446 0.23810.5725 0.0805 w/ RL Training0.7822 0.43330.7985 0.2011 Qwen2.5-Coder-14B0.7549 0.41900.7571 0.1351 w/ RL Training0.8706 0.65710.8762 0.3563 Qwen2.5-Coder-32B0.8492 0.57620.8161 0.2500 w/ RL Training0.8962 0.70950.8883 0.4147 Table 4 Contamination analysis. IFEvalCodeCodeIF n = 5 n = 13n = 5 n = 13 Qwen2.5-Coder-7B2.14%0%6.13%0% Seed-Coder-8B5.06%0%3.02%0% Table 5 Ablation Study on Base Model of Generator. IFEvalCodeCodeIF Inst. PromptInst. Prompt Qwen2.5-Coder-7B0.6837 0.28000.5986 0.0718 w/ Seed-1.60.8701 0.61430.8435 0.3007 w/ GPT-OSS-120B0.8703 0.62380.8408 0.3161 w/ Qwen2.5-72B0.7925 0.48570.7753 0.1667 Seed-Coder-8B0.8051 0.46670.7337 0.1523 w/ Seed-1.60.8519 0.60950.8520 0.3333 w/ GPT-OSS-120B0.8619 0.63330.8400 0.3100 w/ Qwen2.5-72B0.8438 0.58570.8024 0.2098 4.1 Results of IFCodeEvolve Source Dataset We clean up and construct a coding dataset by integrating three diverse public datasets, including MBPP [2], LeetCode (Easy Level) [28], ClassEval [6]. These datasets span distinct programming domains, covering basic logic, competitive algorithm, and class-level design. We filter out problems with ambiguous descriptions, specifically instances where the test cases test cases are under-specified. Furthermore, we augment the dataset by generating missing function signatures and input/output examples. The final dataset comprises 1,565 coding problems, with 958, 519, and 88 samples derived from each respective source. Benchmark We evaluate our method using two recent benchmarks dedicated to IF coding tasks: IFEval- Code [34] and CodeIF [32]. As our study focuses on Python, we exclusively conduct evaluations on the Python subset of these benchmarks. Crucially, these benchmarks are curated from sources distinct from our seed dataset, ensuring no data overlap. Using an LLM to verify instruction compliance, we report two granularities as defined in [41]: • Inst.: The average proportion of satisfied instructions per problem (Instruction-level pass rate). • Prompt: The proportion of problems where all instructions are satisfied (Prompt-level pass rate). Base Model We evaluate the performance of 13 representative LLMs, comprising 5 proprietary models and 8 open-source reasoning models. Specifically, we employSeed-1.6as the generator and utilize two coder models, i.e.,Qwen2.5-Coder-7BandSeed-Coder-8B, as actor models, reporting their performance throughout 7 Table 6 Comparison on IFCodeBench. 1∼ 3 Instructions4∼ 6 Instructions≥ 7 Instructions Func. Unit TestLLM Func. Unit TestLLM Func. Unit TestLLM Inst. PromptInst. PromptInst. PromptInst. PromptInst. PromptInst. Prompt Proprietary Models GPT-5.20.95830.9768 0.96520.9826 0.95130.96930.9800 0.90800.9720 0.87730.87200.9453 0.76800.9542 0.8160 Gemini-3 Pro0.91660.9803 0.96520.9953 0.98610.84290.9775 0.91570.9767 0.93860.76800.9602 0.79200.9361 0.7840 Claude-4.5-Sonnet0.92360.9534 0.90970.9710 0.93750.85070.9604 0.83520.9583 0.83900.79200.9618 0.73600.9650 0.7600 Claude-4.5-Opus0.93750.9806 0.94440.9791 0.93750.90030.9653 0.87350.9650 0.90030.81600.9541 0.79200.9254 0.7760 Seed-1.6-thinking0.93050.9324 0.88190.9675 0.93050.81990.9448 0.78920.9480 0.76240.65600.8840 0.65600.9337 0.6000 Seed-1.8-thinking0.90270.9571 0.92360.9745 0.93050.80840.9495 0.81990.9515 0.78920.68000.9287 0.71200.9516 0.7280 400B+ Open-Source Models Kimi-K20.81250.8331 0.72910.8877 0.73610.72410.8684 0.53250.8637 0.51720.68800.8268 0.20800.8288 0.2240 DeepSeek-V3.20.86800.8412 0.75690.9050 0.77080.80840.9310 0.69730.9141 0.63600.71200.9186 0.48800.9057 0.4720 Qwen3-Coder-480B0.86110.8033 0.65270.8761 0.69440.81990.8867 0.60150.8805 0.55930.76000.8830 0.36800.8622 0.3440 100B+ Open-Source Models GLM-4.70.88190.9569 0.93050.9363 0.90970.71640.8834 0.77390.8371 0.72790.64000.9133 0.73600.8267 0.6960 DeepSeek-Coder-V20.81940.6806 0.50000.7766 0.48610.68580.8109 0.39840.7763 0.33710.61600.7776 0.12800.7211 0.0960 GPT-OSS-120B0.92360.9623 0.93750.9745 0.94440.86590.9399 0.85050.9295 0.83140.74400.8960 0.70400.8477 0.7040 20B+ Open-Source Models Seed-OSS-36B0.88880.9108 0.85410.9594 0.89580.78160.9246 0.77390.9551 0.83140.66400.8839 0.64000.9660 0.7920 Qwen2.5-Coder-32B0.68050.8038 0.68050.8750 0.70130.59770.8658 0.54400.8819 0.57470.56000.8320 0.28800.8537 0.3040 GPT-OSS-20B0.90270.9187 0.85410.9375 0.88190.76240.8884 0.74320.8302 0.69730.62400.8301 0.56000.7210 0.5200 7B+ Open-Source Models DeepSeek-Coder-V2-Lite0.77770.4490 0.20130.6145 0.20830.72790.5722 0.07660.5504 0.04980.64000.5996 0.01600.5520 0.0160 Qwen2.5-Coder-14B0.70830.6127 0.41660.7407 0.42360.66280.7505 0.26430.7272 0.25280.58400.6874 0.08800.6611 0.0800 Seed-Coder-8B0.72910.6664 0.46520.7581 0.43750.62830.7789 0.36780.7657 0.32950.59200.7360 0.11200.7370 0.1200 Qwen2.5-Coder-7B 0.68750.5025 0.26380.6469 0.26380.67430.6579 0.14170.6127 0.11110.58400.6425 0.02400.5956 0.0320 the evolution. By default, we use theInstructvariants for all models. All the prompts can be found in Appendix F. Hyperparameter & Implementation We setC= 3,α= 10,η= 0.5,λ= 1, andδ= 0.1. We pick 3 schema with the lowest values for mutation. GRPO [24] is applied to post-train actor models across evolving process. As for the reward function, we define two indicator terms:r func =I(f test (s)) for functional correctness and r IF-dense = 1 |ι| P ι I(f ι IF-test (s)) for instruction adherence rate. The final reward is computed as: r = r func + r IF-dense + 2· (r func · r IF-dense )(9) where we incentivize satisfying both constraints simultaneously. Additional experiments of reward functions can be found in Appendix D. Benchmarking Results As detailed in Table 1, proprietary models generally outperform open-source baselines, while theGPT-OSSseries exhibits competitive capability. The metrics Prompt on CodeIF are significantly lower than those on IFEvalCode across all evaluated models. We attribute this gap to the higher number of instructions per problem in CodeIF, which imposes a substantially more rigorous challenge. Crucially, IFCodeEvolve consistently enhances both actor models, ultimately achieving performance comparable to SOTA reasoning models. This validates the efficacy of IFCodeEvolve’s co-evolving paradigm. In addition, Table 2 reports the percentage of problems satisfying both unit tests and all instruction constraints. While both models show clear gains driven by enhanced instruction adherence,Seed-Coder-8Bexhibits more moderate improvement, as its stronger baseline leaves limited room for further growth. We evaluate the contamination of the training dataset generated across all iterations on IFEvalCode and CodeIF, presenting the n-gram overlap results in Table 4. It can be found that the contamination rate remains at 0% under the strictn= 13 standard, confirming that our generated data is free from leakage and that the observed performance gains stem from genuine capability enhancement. Weak-to-Strong Scalability To investigate the cross-model generalizability of our synthesized data, we employ the corpus generated by the smaller actor (Seed-Coder-8B) to supervise larger models. Specifically, we 8 Figure 4 Percentage of finalized problems across rounds. Figure 5 Schema with top 10 global success rate. def largestLocal(grid: list[list[int]]): n, size = len(grid), n - 2 result = [[0] * size for _ in range(size)] def fill_result(k: int) -> None: if k >= size * size: return i, j = k // size, k % size current_max = max(grid[x][y] for x in \ range(i, i+3) for y in range(j, j+3)) result[i][j] = current_max fill_result(k + 1) fill_result(0) return result Question: Generate an (n-2)*(n-2) matrix where each element represents the maximum value of the corresponding 3*3 sub-grid in input. Instruction: 1. The solution must be built with a recursive function. 2. The function named fill_result takes exactly one parameter. Question: W rite a function to calculate the sum of the cubes of the first n natural numbers, where the sum is [(n(n+1))/2]^2. Instruction: 1. Ensure the code contains exactly one if-else statement and exactly one while loop. 2. Add type annotations for function and variable. def sum_series(calcNumber: int) -> int: if calcNumber < 1: return 0 else: calcTotal: int = 0 calcI: int = 1 while calcI <= calcNumber: calcTotal += calcI ** 3 calcI += 1 return calcTotal Question: Determine the count of unsorted columns in a matrix formed b y the input strings. Instruction: 1. All variable names in the code must adhere to camelCase naming convention. 2. Ensure your code contains exactly one if-else statement and exactly two while loops. def minDeletionSize(strs): deleteCount = colIndex = rowIndex = 0 while colIndex < len(strs[0]): while rowIndex < len(strs) - 1: if strs[rowIndex][colIndex] \ > strs[rowIndex + 1][colIndex]: deleteCount += 1 break rowIndex += 1 colIndex += 1 rowIndex = 0 return deleteCount (a)(b)(c) variable_naming_convention Al l variable names in your code must follow the convention naming convention. Instruction Evolving variable_naming_with_digit Al l variable names in your code must follow the convention naming convention and contain at least one digit. Figure 6 Representative examples of the three schema categories with adapted solutions. (a) A constraint from the initial library, i.e.,function_parameter_count(2nd instruction). (b) A composed schema combining control flow constraints, i.e.,conditional_count + loop_count(1st instruction). (c) A mutated schema illustrating the evolution from a standard convention (variable_naming_convention) to a stricter variant requiring digits (variable_naming_with_digit). aggregate the synthesized data across all three iterations to post-train three models of varying sizes, utilizing Equ. 9 as the reward signal. The finalized dataset includes 4,241 coding problems, and more details can be found in Appendix C. As presented in Table 3, all models exhibit significant improvements in instruction following. Most notably,Qwen2.5-Coder-32Bachieves performance parity with proprietary SOTA models, validating the effectiveness of our weak-to-strong supervision. 4.2 IFCodeBench We curated a high-quality benchmark from our generated data using a strict human-in-the-loop verification process. We designed a rubric consisting of four binary sanity checks and two scalar quality metrics. An initial LLM-based filter first discarded invalid samples and identified high-potential candidates (Value = 3). These candidates then underwent manual review, where human annotators applied the same criteria. Only instances confirmed by humans to possess high value were included in the final test set with 530 problems. Full details are provided in Appendix E. As shown in Table 6, we divide IFCodeBench into three distinct subsets based on the number of instructions, and report five key metrics to capture both functional correctness and instruction-following capability: • Func.(Functional Correctness): The pass rate on the original functional unit tests. • Inst. and Prompt: We apply the same metrics as mentioned in Section 4.1 with either rigorous execution- based checkers or LLM-based judge. The evaluation results derived from the LLM judge align closely with those from the execution-based Unit Tests, demonstrating the reliability and accuracy of our execution-based verification. Besides, a clear inverse correlation between the number of instructions and performance can be observed. A substantial performance gap exists between proprietary models and smaller open-source models, where smaller models struggle significantly to satisfy complex, concurrent constraints. 9 4.3 Further Analysis Sampling Efficiency To analyze sampling efficiency, we track the ratio of finalized problems to total sampled candidates per round. Specifically, the actor’s pass rate serves a routing signal, where a problem is deemed finalized if the actor fails to solve the augmented problem (Section 3.3). An efficient sampler is expected to learn from interaction, progressively shifting the sampling distribution towards hard-to-solve cases. As visualized in Figure 4, the finalization rate initially climbs and converges as instructions accumulate, demonstrating search efficiency. Besides, the two actor models exhibit distinct learning dynamics. For the strongerSeed-Coder-8B, the sampling process becomes increasingly challenging after evolution. Case Study Figure 5 illustrates the top-10 instruction schema ranked by global success rate, i.e.,P(π), for the 3rd-generationSeed-Coder-8B. It can be found that top-ranked schema comprise initial constraints (e.g., function_parameter_count), composite constraints (e.g.,conditional_count+loop_count), and mutated variants (e.g., variable_naming_with_digit). The detailed definition can be found in Appendix C. Furthermore, we showcase representative examples for each of these three categories in Figure 6. The case study highlights three distinct difficulty mechanisms. Case (a) shows that even simple constraints like function_parameter_countbecome challenging when coupled with contextual requirements such as recursion. Case (b) showcases the complexity arising from the composition of multiple control-flow constraints. Case (c) exemplifies the adversarial evolution process: the sampler detects patterns in the current solution and synthesizes a mutated constraint, i.e., enforcing digits in variable names, that renders the previous solution invalid. Ablation Study onπ gen We analyze the effect of the generator backboneπ gen . In addition to our default Seed-1.6, we test two open-source alternatives, i.e.,GPT-OSS-120B(reasoning model) andQwen2.5-72B (non-reasoning IF model). Table 5 demonstrates that our method drives consistent improvements across all generator configurations. Importantly, the open-sourceGPT-OSS-120Bachieves parity with theSeed-1.6 model, underscoring the feasibility and robustness of our approach. Ablation study on the core modules can be found in Appendix D. 5 Conclusion We presented IFCodeEvolve, a co-evolutionary data synthesis framework that unifies parametric instruction design with MCTS-guided validation. By structurally evolving the schema library alongside the actor model, our approach guarantees strict solvability while continuously pushing the frontier of problem complexity. Empirical results demonstrate that IFCodeEvolve effectively overcomes the limitations of static data generation, establishing a robust foundation for training next-generation instruction-following code LLMs. 10 References [1]Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report.arXivpreprint arXiv:2303.08774, 2023. [2]Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models.arXivpreprint arXiv:2108.07732, 2021. [3]Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Yunxuan Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models.JournalofMachine LearningResearch, 25(70):1–53, 2024. [4]Kaustubh Deshpande, Ved Sirdeshmukh, Johannes Baptist Mols, Lifeng Jin, Ed-Yeremai Hernandez-Cardona, Dean Lee, Jeremy Kritz, Willow E Primack, Summer Yue, and Chen Xing. Multichallenge: A realistic multi-turn conversation evaluation benchmark challenging to frontier llms. InFindingsoftheAssociationforComputational Linguistics:ACL2025, pages 18632–18702, 2025. [5]Guanting Dong, Keming Lu, Chengpeng Li, Tingyu Xia, Bowen Yu, Chang Zhou, and Jingren Zhou. Self-play with execution feedback: Improving instruction-following capabilities of large language models.arXivpreprint arXiv:2406.13542, 2024. [6] Xueying Du, Mingwei Liu, Kaixin Wang, Hanlin Wang, Junwei Liu, Yixuan Chen, Jiayi Feng, Chaofeng Sha, Xin Peng, and Yiling Lou. Classeval: A manually-crafted benchmark for evaluating llms on class-level code generation. arXivpreprintarXiv:2308.01861, 2023. [7]Yuanqi Du, Botao Yu, Tianyu Liu, Tony Shen, Junwu Chen, Jan G Rittig, Kunyang Sun, Yikun Zhang, Zhangde Song, Bo Zhou, et al. Accelerating scientific discovery with autonomous goal-evolving agents.arXivpreprint arXiv:2512.21782, 2025. [8]Huan-ang Gao, Jiayi Geng, Wenyue Hua, Mengkang Hu, Xinzhe Juan, Hongzhang Liu, Shilong Liu, Jiahao Qiu, Xuan Qi, Yiran Wu, et al. A survey of self-evolving agents: On path to artificial super intelligence.arXivpreprint arXiv:2507.21046, 2025. [9] Daya Guo, Qihao Zhu, Dejian Yang, Zhenda Xie, Kai Dong, Wentao Zhang, Guanting Chen, Xiao Bi, Yu Wu, YK Li, et al. Deepseek-coder: When the large language model meets programming–the rise of code intelligence. arXivpreprintarXiv:2401.14196, 2024. [10] Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, et al. Deepseek-r1: Incentivizing reasoning capability in llms via reinforcement learning.arXiv preprintarXiv:2501.12948, 2025. [11]Shengran Hu, Cong Lu, and Jeff Clune. Automated design of agentic systems.arXivpreprintarXiv:2408.08435, 2024. [12] Chengsong Huang, Wenhao Yu, Xiaoyang Wang, Hongming Zhang, Zongxia Li, Ruosen Li, Jiaxin Huang, Haitao Mi, and Dong Yu. R-zero: Self-evolving reasoning llm from zero data.arXivpreprintarXiv:2508.05004, 2025. [13] Yuxin Jiang, Yufei Wang, Xingshan Zeng, Wanjun Zhong, Liangyou Li, Fei Mi, Lifeng Shang, Xin Jiang, Qun Liu, and Wei Wang. Followbench: A multi-level fine-grained constraints following benchmark for large language models. InProceedingsofthe62ndAnnualMeetingoftheAssociationforComputationalLinguistics(Volume 1:LongPapers), pages 4667–4688, 2024. [14]Carlos E Jimenez, John Yang, Alexander Wettig, Shunyu Yao, Kexin Pei, Ofir Press, and Karthik Narasimhan. Swe-bench: Can language models resolve real-world github issues?arXivpreprintarXiv:2310.06770, 2023. [15]John Jumper, Richard Evans, Alexander Pritzel, Tim Green, Michael Figurnov, Olaf Ronneberger, Kathryn Tunyasuvunakool, Russ Bates, Augustin Žídek, Anna Potapenko, et al. Highly accurate protein structure prediction with alphafold.nature, 596(7873):583–589, 2021. [16] Yujia Li, David Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, et al. Competition-level code generation with alphacode.Science, 378 (6624):1092–1097, 2022. 11 [17]Jiaye Lin, Yifu Guo, Yuzhen Han, Sen Hu, Ziyi Ni, Licheng Wang, Mingguang Chen, Hongzhang Liu, Ronghao Chen, Yangfan He, et al. Se-agent: Self-evolution trajectory optimization in multi-step reasoning with llm-based agents.arXivpreprintarXiv:2508.02085, 2025. [18]Renze Lou, Kai Zhang, and Wenpeng Yin. A comprehensive survey on instruction following.arXivpreprint arXiv:2303.10475, 1, 2023. [19] Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, et al. Self-refine: Iterative refinement with self-feedback.Advancesin NeuralInformationProcessingSystems, 36:46534–46594, 2023. [20]Alexander Novikov, Ngân V ̃u, Marvin Eisenberger, Emilien Dupont, Po-Sen Huang, Adam Zsolt Wagner, Sergey Shirobokov, Borislav Kozlovskii, Francisco JR Ruiz, Abbas Mehrabian, et al. Alphaevolve: A coding agent for scientific and algorithmic discovery.arXivpreprintarXiv:2506.13131, 2025. [21]Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. Training language models to follow instructions with human feedback. Advancesinneuralinformationprocessingsystems, 35:27730–27744, 2022. [22] Valentina Pyatkin, Saumya Malik, Victoria Graf, Hamish Ivison, Shengyi Huang, Pradeep Dasigi, Nathan Lambert, and Hannaneh Hajishirzi. Generalizing verifiable instruction following.arXivpreprintarXiv:2507.02833, 2025. [23]ByteDance Seed, Yuyu Zhang, Jing Su, Yifan Sun, Chenguang Xi, Xia Xiao, Shen Zheng, Anxiang Zhang, Kaibo Liu, Daoguang Zan, et al. Seed-coder: Let the code model curate data for itself.arXivpreprintarXiv:2506.03524, 2025. [24]Zhihong Shao, Peiyi Wang, Qihao Zhu, Runxin Xu, Junxiao Song, Xiao Bi, Haowei Zhang, Mingchuan Zhang, YK Li, Yang Wu, et al. Deepseekmath: Pushing the limits of mathematical reasoning in open language models. arXivpreprintarXiv:2402.03300, 2024. [25]David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George Van Den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. Mastering the game of go with deep neural networks and tree search.nature, 529(7587):484–489, 2016. [26]Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models.arXivpreprintarXiv:2203.11171, 2022. [27]Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. InProceedingsofthe61st annualmeetingoftheassociationforcomputationallinguistics(volume1:longpapers), pages 13484–13508, 2023. [28]Yunhui Xia, Wei Shen, Yan Wang, Jason Klein Liu, Huifeng Sun, Siyue Wu, Jian Hu, and Xiaolong Xu. Leetcodedataset: A temporal dataset for robust evaluation and efficient training of code llms.arXivpreprint arXiv:2504.14655, 2025. [29] Huajian Xin, Daya Guo, Zhihong Shao, Zhizhou Ren, Qihao Zhu, Bo Liu, Chong Ruan, Wenda Li, and Xiaodan Liang. Deepseek-prover: Advancing theorem proving in llms through large-scale synthetic data.arXivpreprint arXiv:2405.14333, 2024. [30]Ran Xin, Chenguang Xi, Jie Yang, Feng Chen, Hang Wu, Xia Xiao, Yifan Sun, Shen Zheng, and Ming Ding. Bfs-prover: Scalable best-first tree search for llm-based automatic theorem proving. InProceedingsofthe63rd AnnualMeetingoftheAssociationforComputationalLinguistics(Volume1:LongPapers), pages 32588–32599, 2025. [31]Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, Qingwei Lin, and Daxin Jiang. Wizardlm: Empowering large pre-trained language models to follow complex instructions. InTheTwelfth InternationalConferenceonLearningRepresentations, 2024. [32] Kaiwen Yan, Hongcheng Guo, Xuanqing Shi, Jingyi Xu, Yaonan Gu, and Zhoujun Li. Codeif: Benchmarking the instruction-following capabilities of large language models for code generation.arXivpreprintarXiv:2502.19166, 2025. [33]Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. Large language models as optimizers. InTheTwelfthInternationalConferenceonLearningRepresentations, 2023. 12 [34]Jian Yang, Wei Zhang, Shukai Liu, Linzheng Chai, Yingshui Tan, Jiaheng Liu, Ge Zhang, Wangchunshu Zhou, Guanglin Niu, Zhoujun Li, et al. Ifevalcode: Controlled code generation.arXivpreprintarXiv:2507.22462, 2025. [35]Mert Yuksekgonul, Federico Bianchi, Joseph Boen, Sheng Liu, Pan Lu, Zhi Huang, Carlos Guestrin, and James Zou. Optimizing generative ai by backpropagating language model feedback.Nature, 639(8055):609–616, 2025. [36]Daoguang Zan, Zhirong Huang, Wei Liu, Hanwu Chen, Linhao Zhang, Shulin Xin, Lu Chen, Qi Liu, Xiaojian Zhong, Aoyan Li, et al. Multi-swe-bench: A multilingual benchmark for issue resolving.arXivpreprintarXiv:2504.02605, 2025. [37] Eric Zelikman, Yuhuai Wu, Jesse Mu, and Noah D Goodman. Star: Self-taught reasoner bootstrapping reasoning with reasoning. InProc.the36thInternationalConferenceonNeuralInformationProcessingSystems, volume 1126, 2024. [38]Chengxiang Zhai and John Lafferty. A study of smoothing methods for language models applied to ad hoc information retrieval. InAcmsigirforum, volume 51, pages 268–276. ACM New York, NY, USA, 2017. [39] Jiayi Zhang, Jinyu Xiang, Zhaoyang Yu, Fengwei Teng, Xionghui Chen, Jiaqi Chen, Mingchen Zhuge, Xin Cheng, Sirui Hong, Jinlin Wang, et al. Aflow: Automating agentic workflow generation.arXivpreprintarXiv:2410.10762, 2024. [40] Andrew Zhao, Yiran Wu, Yang Yue, Tong Wu, Quentin Xu, Matthieu Lin, Shenzhi Wang, Qingyun Wu, Zilong Zheng, and Gao Huang. Absolute zero: Reinforced self-play reasoning with zero data.arXivpreprint arXiv:2505.03335, 2025. [41]Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. Instruction-following evaluation for large language models.arXivpreprintarXiv:2311.07911, 2023. 13 Appendix A Limitation A limitation of our current study is its focus on Python, chosen for its dominance in algorithmic reasoning benchmarks and competitive programming. While the proposed co-evolutionary framework and MCTS sampler are inherently language-agnostic, extending IFCodeEvolve to a multi-lingual setting (e.g., C++, Java, or Rust) requires adapting the underlying AST verifiers and defining language-specific instruction schemas. Future work will focus on generalizing the schema library to support diverse programming syntaxes and broader software engineering paradigms beyond algorithmic tasks. B Implementation Pseudocode Here we provide the pesudocode of our proposed IFCodeEvolve in Algo.1. Besides, we apply a dynamic expansion rule: if the candidate pool size drops below 500 (due to routing), each problem is augmented twice in the next iteration. This redundant augmentation ensures a consistent data supply. For each finalized question, we will apply an LLM to rephrase the instruction without modifying the logic to improve the diversity. To enhance diversity, we employ a LLM to perform semantics-preserving paraphrasing for instructions of finalized questions. The prompt can be found in Appendix F. Algorithm 1 IFCodeEvolve Require: Seed Dataset D seed , Instruction Schema Library T , Generator π gen , Actor π actor 1: Initialize sampler π MCTS using T 2: D final ←∅ 3: for iteration k = 1 to K do 4: // Phase 1: Multi-Round Augmentation 5: D new ← AugmentData(D seed ,T ,π gen ,π actor ,π MCTS ) 6: D final ←D final ∪D new 7: // Phase 2: Actor Evolving 8: π actor ← PostTrain(π actor ,D new ) 9: // Phase 3: Sampler Evolving 10: π MCTS ← Reset(D new ,π MCTS ,π actor ) 11: // Phase 4: Schema Evolving 12: τ i⊕j ← Composition(π MCTS ,π actor ) Eq. 7,8 13: T ←T ∪τ i⊕j 14: T weak ← SelectWeakSchema(T ,π MCTS ) 15: T ′ weak ← Mutation(D final ,T weak ,π gen ) 16: T ←T /T weak ∪T ′ weak 17: end for 18: return D final ,π actor Running environment. The experiments are conducted on a single Linux server with Intel(R) Xeon(R) Platinum 8336C CPU, 1.9Ti RAM, and 8 NVIDIA A800-SXM4-80GB. Our method is implemented on PyTorch 2.8.0 and Python 3.11.2. Training All the RL training are based on an open-source framework verl 2 . We set the group size toG= 8, with a global training batch size of 256 and a mini-batch size of 64. We employ dynamic batching with a maximum of 16,384 tokens per GPU. The learning rate is fixed at 1×10 −6 with 10 warmup steps. We employ an asymmetric PPO clipping range of [0.2,0.28] and disable the KL penalty. All training is conducted with FSDP2 and vLLM integration, supporting a maximum sequence length of 2048 tokens. 2 https://github.com/verl-project/verl 14 C Dataset C.1 Schema The library of parametric schema is detailed in Table 7. For improved presentation, certain schema identifiers have been abbreviated (e.g., variable_existence is denoted as var_existence). Each schema is paired with an AST-based verification function, which dynamically adapts to the instantiated parameters. Taking the variable_existence (Code 1) and conditional_count (Code 2) as examples, the verifier traverses the Abstract Syntax Tree to identify all variable assignments and control flow structures. Invariable_existence, the parametermodegoverns the validation logic: if set to ‘should’, the function confirms the presence of the specified variable; if ’should not’, it ensures its absence. As forconditional_- count, the parametercomparison(e.g., ‘at least’, ‘exactly’) combined with a targetcountdefines the numerical constraint. The verifier counts the occurrences ofast.Ifandast.Comparenodes, subsequently evaluating whether the total satisfies the specified relational condition. Code 1 AST-based verification logic for variable_existence. import ast def check_variable_existence(code, variable_name, mode): tree = ast.parse(code) assert mode in ["should", "should not"] defined_vars = set() for node in ast.walk(tree): if isinstance(node, ast.Assign): for target in node.targets: if isinstance(target, ast.Name): defined_vars.add(target.id) # Also catch annotated assignment: x: int = 1 elif isinstance(node, ast.AnnAssign): if isinstance(node.target, ast.Name): defined_vars.add(node.target.id) exists = variable_name in defined_vars if str(mode).lower() == "should not": return not exists return exists Code 2 AST-based verification logic for conditional_count. import ast def check_conditional_count(code, count, comparison): tree = ast.parse(code) assert comparison in ["at least", "at most", "exactly"] if_nodes = [n for n in ast.walk(tree) if isinstance(n, ast.If)] actual = len(if_nodes) target = int(count) if "at least" in comparison: return actual >= target if "at most" in comparison: return actual <= target if "exactly" in comparison: return actual == target return False 15 Table 7 Library of parametric instruction schema. SchemaInstruction TemplateParameter Category: Variable & Structures var_existenceYour code mode define a variable named name. mode: [should, should not]; name: str no_inter_var Your code must not use intermediate variables.- naming_conv Variable names must follow style conven- tion. style: [camelCase, PascalCase, snake_case] global_varMust define a global variable named name. name: str init_valueDefine variable name initialized with val. name: str; val: str name_lenVariable name length mode exceed n char- acters. mode: [should, should not]; n: int var_prefix Variable names must start with prefix ‘pre’.pre: str var_suffixVariable names must end with suffix ‘suf’. suf: str data_structThe data structure ds mode be used. ds: [dict, set, list, tuple, ...];mode: [must, must not] Category: Logic & Control Flow loop_countInclude comp n type loop(s). comp: [at least, at most, exactly];type: [while, for]; n: int forbid_loopYour code must not use any type loops. type: [while, for] cond_countInclude comp n if-else statement(s). comp: [exactly, at least, at most]; n: int switch_stmt Must include a switch (or match/case) state- ment. - recursionYou must implement the solution using recur- sion. - Category: Interface & Type interfaceYou should define a type named name. type: [class, interface]; name: str type_hintType annotations must be used for functions and vars. - func_defYour code must define a function named name. name: str param_count Function name must accept exactly n params. name: str; n: int Category: Library & Tools import_libYour code mode import the library lib. mode: [must, must not]; lib: str no_importYour code must not import any library.- forbid_func You are not allowed to use built-in(s): funcs.funcs: list[str] require_funcYou must use the built-in function(s): funcs.funcs: list[str] sugar_usageYour code mode utilize type. mode: [must, must not];type: [list comp, lambda, ternary operator, generator exp, ...] sugar_countUtilize comp n type. type: [list comp, lambda, ternary operator, gen- erator exp, ...];n: int;comp: [at least, at most, exactly] Category: Style & Formatting comment_cntInclude comp n line(s) of comments. comp: [exactly, more than, less than]; n: int comment_lang Comments must be written in lang. lang: [en, zh] code_linesThe solution must contain comp n lines of code. comp: [exactly, at least, at most]; n: int 16 C.2 Training Dataset Figure 7 The distributions of schema category and the number of instructions. In Section 4.1, we evaluate the efficacy of our approach by post-training a series of LLMs across different parameter scales using the synthesized dataset. The dataset was curated via IFCodeEvolve, employingSeed-Coder-8Bas the base actor model. Our pipeline yielded a total of 4,241 high-quality IF coding instances, with an average of approximately 1,400 problems generated per iteration. The taxonomic distribution of schema categories and the corresponding instruction counts are detailed in Figure 7. It can be found that our dataset exhibits a robust hierarchy of complexity. The inner circle denotes the number of concurrent constraints per problem, ranging from 1 to over 10. Notably, a significant portion of the dataset comprises problems with 4 to 9 instructions, indicating a high level of task difficulty. The outer ring reveals a balanced distribution across functional domains, such as Logic & Control Flow and Interfaces & Types, ensuring that the model undergoes comprehensive training across diverse coding dimensions. Examples We present representative examples of the gener- ated IF coding data across varying complexity levels, specifically featuring 2, 4, 6, and 8 instructions. Each instance comprises a formal problem definition, a list of constraints, example input-output pairs, a function signature, and a reference solution. Note that the foundational instruction, i.e., ’Implement the solution in Python’, is treated as a global prerequisite and is excluded from the total instruction count. Example with 2 instructions # Problem Description Write a function that takes a list of tuples where each tuple contains a name and a mark. The function should compute the total marks for each name by summing their marks, then return the tuple consisting of the name with the highest total marks and their total. # Instruction 1. Implement the solution using Python. 2. It is forbidden to use the built-in ’max’ function in your code. 3. The code must make use of list comprehension. # Example Input and Output Example Input 1: [(’Juan Whelan’,90),(’Sabah Colley’,88),(’Peter Nichols’,7),(’Juan Whelan’,122),(’Sabah Colley’,84)] Example Output 1: (’Juan Whelan’, 212) Example Input 2: [(’Juan Whelan’,10),(’Sabah Colley’,20),(’Peter Nichols’,30),(’Juan Whelan’,40),(’Sabah Colley’,50)] Example Output 2: (’Sabah Colley’, 70) # Function Signature def max_aggregate(stdata): pass # Solution def max_aggregate(stdata): 17 aggregated = [ (name, sum(mark for n, mark in stdata if n == name)) \ for name in n for n, _ in stdata ] max_name = None max_total = 0 for name, total in aggregated: if max_name is None or total > max_total: max_name = name max_total = total return (max_name, max_total) Example with 4 instructions # Problem Description Write a Python function to check whether the given number can be represented as the sum of two squares of positive integers. # Instruction 1. Develop the solution using Python. 2. Define a variable called found and set its initial value to False in your code. 3. Utilize the set data structure in your implementation. 4. Include a switch (or match/case) statement within the code. 5. Make sure to import the math library in your code. # Example Input and Output Example 1: Input: 25 Output: True (since 3² + 4² = 25) Example 2: Input: 24 Output: False # Function Signature def sum_Square(n): pass # Solution import math def sum_Square(n): found = False squares = set() i = 1 while i <= math.isqrt(n): square = i * i squares.add(square) remainder = n - square match remainder: case x if x in squares and x != 0: found = True break i += 1 18 return found Example with 6 instructions # Problem Description Write a function to determine if a given number is abundant. An abundant number is a number for which the sum of its proper divisors (excluding the number itself) is greater than the number. # Instruction 1. Implement the solution using Python. 2. Ensure every variable name in your code adheres to the snake_case naming convention. 3. Do not import the math library in your code. 4. Make use of list comprehension in your implementation. 5. Avoid using any for loops in your code. 6. Incorporate the set data structure in your solution. 7. Include exactly one while loop in your code. # Example Input and Output Example Input 1: 12 Example Output 1: True Example Input 2: 15 Example Output 2: False # Function Signature def check_abundant(n): pass # Solution def check_abundant(n): divisors = set() i = 1 while i * i <= n: if n % i == 0: divisors.add(i) if i != n // i: divisors.add(n // i) i += 1 proper_sum = sum([d for d in divisors if d != n]) return proper_sum > n Example with 8 instructions # Problem Description Given a list of toy types, return true if a permutation of the list could form a palindrome sequence (reads the same forwards and backwards) and false otherwise. # Instruction 1. Implement the solution using Python. 2. The code must include exactly one list comprehension and one while loop. 3. Ensure your code does not define a variable with the name "index". 4. It is mandatory to incorporate a switch (or match/case) statement in the code. 19 5. The solution must make use of a generator expression. 6. You must employ the built-in ‘collections.Counter‘ function. 7. A global variable named ‘palindrome_check_enabled‘ must be defined in the code. 8. Full type annotations (type hints) are required for every function and variable. 9. The built-in ‘sum‘ function is strictly prohibited from use in the solution. # Example Input and Output Example 1: Input: s = "code" Output: false Example 2: Input: s = "aab" Output: true Example 3: Input: s = "carerac" Output: true # Function Signature def canPermutePalindrome(s: str) -> bool: pass # Solution from collections import Counter from typing import Generator, Iterable, Iterator palindrome_check_enabled: bool = True def canPermutePalindrome(s: str) -> bool: global palindrome_check_enabled if not palindrome_check_enabled: return False chars: list[str] = [c for c in s] char_count: Counter[str] = Counter(chars) odd_counts: Generator[int, None, None] = (v % 2 for v in char_count.values()) odd_count_total: int = 0 odd_iter: Iterator[int] = iter(odd_counts) while True: try: val: int = next(odd_iter) match val: case 1: odd_count_total += 1 case _: pass except StopIteration: break return odd_count_total < 2 20 D Additional Experiments Reward Function We investigate the impact of reward shaping by ablating the reward function design on the same training corpus, as shown in Figure 8(a). Compared to the dense reward (Eq. 9), we examine two distinct variants: • Sparse Reward: We remove the dense shaping term, providing instruction feedback only upon perfect compliance. r = r func + I(r IF-dense = 1) + 2· (r func · r IF-dense )(10) •Conditional Reward: We employ a gated formulation where instruction adherence is rewarded only if the code is functionally correct. r = r func · 1 + r dense + 2.0· I(r dense = 1) (11) As shown in Figure 8, we present the learning curves on entropy and reward across the training on Seed-Coder-8B(a) andQwen2.5-Coder-7B(b). The Conditional Reward exhibits the poorest performance due to its strict gating mechanism, which masks instruction adherence signals whenever functional correctness fails. This creates reward sparsity during early training. In contrast, the Dense/Sparse Rewards provides continuous feedback independent of functional success, facilitating a smoother optimization process. (a)(b) Figure 8 Learning curves of different RL reward functions usingQwen2.5-Coder-7B(a) andSeed-Coder-8B(b) as base models Table 8 Ablation study on core modules. IFEvalCodeCodeIF Inst. PromptInst. Prompt Qwen2.5-Coder-7B 0.8701 0.61430.8271 0.2672 + IFCodeEvolve w/o MCTS0.8542 0.59520.8388 0.2989 w/o Actor Evolving0.8701 0.61430.8435 0.3007 w/o Instruction Evolving0.8700 0.60010.8421 0.3132 Seed-Coder-8B 0.8519 0.60950.8520 0.3333 + IFCodeEvolve w/o MCTS0.8514 0.59050.8382 0.2902 w/o Actor Evolving0.8433 0.58700.8421 0.2903 w/o Instruction Evolving 0.8521 0.61430.8376 0.2874 Ablation Study We conduct an ablation study to assess the effectiveness of the core components of IFCodeEvolve. Specifically, we consider three variants: (1) w/o MCTS, where the MCTS-guided search is replaced by uniform random sampling; (2) w/o Actor Evolving, which freezes the actor model and generates the full volume of data in a single-round synthesis; and (3) w/o Instruction Evolving, where the schema library remains the same. The results presented in Table 8 indicate that removing either core module leads to a decline in actor model’s performance mostly. E Rubric We detail the evaluation rubric employed to curate IFCodeBench, comprising six distinct criteria. These are divided into two categories: Sanity Metrics and Quality Metrics. The first four, i.e., Consistency, Redundancy, 21 Validity, and Alignment, serve as binary filters; any candidate failing a single criterion (scoring 0) is discarded. The remaining two, including Transformation and Value, assess the substantive difference between the vanilla and constrained solutions. Our curation pipeline begins with an automated phase whereSeed-1.6filters candidates, retaining only those meeting the sanity requirements alongside thresholds ofTransformation≥2 andValue≥2. These candidates then undergo rigorous manual inspection using the same rubric. Ultimately, IFCodeBench consists of 530 gold-standard instances that satisfy all four sanity checks and achieve a maximum score ofValue= 3. The scoring strategies and representative examples for each criterion are provided below. • Consistency: Each instruction must not conflict with the problem description or other instructions. – Scoring: 0 / 1 –Negative Example 1: "Ensure every variable name in your code adheres to the camelCase naming convention." AND "The code must avoid using any intermediate or temporary variables.". –Negative Example 2: "Import the ‘math‘ library in your code." AND "Do not use the ‘math.pow‘ built-in function". • Redundancy: The instructions must be unique and necessary. – Scoring: 0 / 1 – Negative Example 1: "All variable names must start with the prefix ‘num’ and adhere to the camelCase naming convention." AND "All variable names must follow camelCase and include at least one digit.". –Negative Example 2: "The code must include type annotations for every function and variable, and you are required to use the built-in ‘collections.Counter’ function." AND "Ensure the code has exactly two lines of comments and provides type hints for all functions and variables.". • Validity: Instructions must be verifiable. – Scoring: 0 / 1 – Negative Example 1: "The code must be elegant.". – Negative Example 2: "The code must be highly readable and use meaningful variable names.". • Alignment: The provided solution must satisfy all instructions. – Scoring: 0 / 1 – Negative Example 1: "Implement a function to calculate the factorial ofnusing recursion" AND solution: def factorial(n): result = 1 for i in range(1, n + 1): result *= i return result – Negative Example 2: "The code cannot use any built-in sorting functions" AND solution: def sort_array(arr): return sorted(arr) • Transformation: Evaluate the extent of structural modification between the vanilla solution (generated without specific constraints) and the constrained solution. – Scoring: 1 / 2 / 3 (1: Surface, 2: Medium, 3: High) – Surface-level: Cosmetic changes only. The vanilla solution: 22 import heapq def merge_sorted_list(num1,num2,num3): num1 = sorted(num1) num2 = sorted(num2) num3 = sorted(num3) result = heapq.merge(num1,num2,num3) return list(result) The constrained solution: import heapq def merge_sorted_list(num1, num2, num3): sorted_num1 = sorted(num1) sorted_num2 = sorted(num2) sorted_num3 = sorted(num3) sorted_result = heapq.merge(sorted_num1, sorted_num2, sorted_num3) return list(sorted_result) – Medium-level: Apply different data structure, control flow, but still the same logical flow. The vanilla solution: def text_match_three(text): for i in range(len(text) - 3): if text[i] =='a' and text[i+1] =='b' and text[i+2] =='b' and text[i+3] =='b': return'Found a match!' return'Not matched!' The constrained solution: import re def text_match_three(text): patterns ='ab3?' if re.search(patterns, text): return'Found a match!' else: return('Not matched!') – High-level: Paradigm Shift / Algorithmic Change. The vanilla solution: def permutation_coefficient(n, k): P = [[0 for i in range(k + 1)] for j in range(n + 1)] for i in range(n + 1): for j in range(min(i, k) + 1): if (j == 0): P[i][j] = 1 else: P[i][j] = P[i - 1][j] + (j * P[i - 1][j - 1]) if (j < k): P[i][j + 1] = 0 return P[n][k] The constrained solution: from functools import reduce def permutation_coefficient(N, K): if K == 0: return 1 if K > N: return 0 return reduce(lambda X, Y: X * Y, range(N, N - K, -1)) 23 •Value: The goal is to assess whether the instruction leads to meaningful logical changes, algorithmic optimizations, or paradigm shifts in the code, rather than simply making changes for the sake of change (or making things worse). – Scoring: 1 / 2 / 3 (1: Negative, 2: Neutral, 3: Positive) –Negative value: Instruction forces the model to generate code that is more difficult to read, less secure, and technically unsound. The vanilla solution: def extract_string(str, l): result = [e for e in str if len(e) == l] return result The constrained solution: def data_Recursive_Len(data_String): if not data_String: return 0 return 1 + data_Recursive_Len(data_String[1:]) data_Filter_Check = lambda data_String, data_Len: data_Recursive_Len(data_String) == data_Len def extract_string(data_InputList, data_TargetLength): data_FilteredResult = list( e for e in data_InputList if data_Filter_Check(e, data_TargetLength) ) return data_FilteredResult –Neutral value: The instructions led to changes in the code’s structure, but the core logic, complexity, and engineering quality remained essentially unchanged. The vanilla solution: def extract_index_list(l1, l2, l3): result = [] for m, n, o in zip(l1, l2, l3): if (m == n == o): result.append(m) return result The constrained solution: def extract_index_list(l1, l2, l3): result = [] index = 0 min_len = min(len(l1), len(l2), len(l3)) while True: if index >= min_len: break m, n, o = l1[index], l2[index], l3[index] if m == n == o: result.append(m) index += 1 return result –Positive value: The instruction led to a solution that is educationally valuable, algorithmically alternative, or engineering-robust. Although the new solution may not necessarily be better than the original one, it demonstrates a different way of thinking. The vanilla solution: def permutation_coefficient(n, k): P = [[0 for i in range(k + 1)] for j in range(n + 1)] 24 for i in range(n + 1): for j in range(min(i, k) + 1): if (j == 0): P[i][j] = 1 else: P[i][j] = P[i - 1][j] + (j * P[i - 1][j - 1]) if (j < k): P[i][j + 1] = 0 return P[n][k] The constrained solution: from functools import reduce def permutation_coefficient(N, K): if K == 0: return 1 if K > N: return 0 return reduce(lambda X, Y: X * Y, range(N, N - K, -1)) F Prompt Actor prompt As a programming assistant, your task is to generate code snippets based on the user question and instructions given below: ## Requirements - Make sure you follow the user instructions. If the instruction says to use a specific language or a specific method, use exactly that. - Your output should be a valid code snippet in the programming language indicated in the question or the instructions. - Remember to import any necessary libraries or modules if needed. - Remember to import Typing if the signature contains type hints. ## Output Format The output should only be a valid code snippet without any explanations, comments, or text outside the code. ## Problem prompt LLM evaluator for instruction following # Role You are an expert Code Compliance Auditor. Your task is to verify whether a model-generated code solution strictly follows a specific set of user instructions. # Workflow You must follow these two steps strictly: ## Step 1: Step-by-Step Analysis - Iterate through each instruction in the provided list. - For each instruction, examine the "Model-generated Code" to see if the requirement is met. 25 - Briefly explain your reasoning for each instruction. - Conclude each analysis with a "Yes" or "No". ## Step 2: Final Output Generation - Collect your "Yes" or "No" verdicts into a Python- style list. - Ensure the list length is exactly the same as the number of instructions. - Wrap this list inside ‘<answer>’ and ‘</answer>’ tags. - The content inside ‘<answer>’ must be **only** the list (e.g., ‘[‘Yes’, ‘No’]’), without any reasoning text. # Evaluation Criteria - **Strict Adherence:** If a specific language, library, algorithm complexity, or naming convention is requested, it must be present. - **Syntax:** The code must be syntactically valid for the requested language. - **Order:** The i-th element in your result list must correspond to the i-th instruction. # Input Data ## Coding Question question ## Instructions instruction ## Model-generated Code code # Execution Please begin your Step 1 Analysis now, followed immediately by the Step 2 Final Output. Proof-by-construction by generator You are an expert Python coding challenge designer and data synthesizer. Your task is to "mutate" a given programming problem (Seed Question) by applying a specific **instruction**, thereby generating a new, more challenging version of the problem. # Input Format You will receive an XML snippet containing: 1. ‘<question>’: The original problem description, existing instructions, the reference solution (Code), programming language, and language. 2. ‘<Mutations>’: A list of ‘<Mutation>’ tags. Each mutation is a template for a new instruction and a list of ‘<params>’ required to instantiate that template. # Workflow (Step-by-Step) Before generating the final XML output, you must perform the following reasoning steps inside a ‘<thought>’ tag: 1. Analyze the Original Code: - Understand the algorithm, complexity, and existing instructions of the seed code. 2. Strategic Parameter Selection and Applicability Check: - Iterate through **each** provided ‘<Mutation>’ in the list. - For each candidate, evaluate two criteria: - **Compatibility**: Does this mutation make sense for this problem and the current instructions? - **Challenge Level**: How much does this force a refactor? 26 - **Selection Strategy**: - Discard Incompatible mutations. - If multiple have the same challenge level, prioritize the one that aligns best with Pythonic best practices or specific algorithmic concepts. 3. Parameter Instantiation: - For the selected mutation, look at its ‘<params>’. - If ‘<Mutation>/<params>’ is empty, skip parameter selection and proceed directly to conflict detection. The instruction text is fixed. - If ‘<Mutation>/<params>’ is not empty: - **Maximize Challenge**: Choose parameter options that contradict the *current* implementation (e.g., if code uses a ‘for‘ loop, and options are ‘[’while’, ’recursion’]‘, choose ‘recursion‘ if valid). - **Feasibility**: Ensure the chosen parameters allow for a valid solution. 4. Conflict Detection: - Ensure the selected mutation and its parameters do not contradict the original ‘<instruction>’. - Such as ’Your code must utilize exactly 1 list comprehension.’ and ’Your code must not use any for loops.’, which are incompatible with each other. 5. Refactor Code (Only if Successful): - If successful, rewrite the reference code to **strictly adhere** to the new instantiated constraint. - The modified code must produce the exact same output for the same inputs as the original code. 6. Synthesize Output: - Generate the ‘<success>’ tag first. - If ‘<success>false</success>’, **STOP** after closing the tag. Do not generate params or question. - If ‘<success>true</success>’, generate ‘<instantiated_params>’ and the modified ‘<question>’. - If the input ‘<Mutation>/<params>’ was empty, the tag <instantiated_params> must also be empty (i.e., ‘<instantiated_params></instantiated_params>’). # Output Format Return the result strictly in XML format. Do not include markdown code block markers (like “‘xml). <output > <thought ><![CDATA [1. Analysis: [Analysis of original code] 2. Evaluation: - Mutation ID 1: [Compatibility: Yes/No] | [Challenge: Low/Med/High] | [Reasoning] - Mutation ID 2: ... - Decision: Selected Mutation ID [X] because ... 3. Parameter Selection: [Reasoning for chosen params] 4. Refactoring Strategy: [How the code will change ]]]></thought > <success >[true/false]</success > <instantiated_params > <param > <name>[Parameter Name , e.g., variable_name]</name> <value>[Selected Value , e.g., total_score]</value> </param > </instantiated_params > <question > <question_desc >[Original description. Modify only if the mutation fundamentally changes the output format , otherwise keep as is.]</question_desc > <instruction > [Existing instructions] [New instantiated instruction] </instruction > <code><![CDATA[ [The full , modified Python code] 27 ]]></code> <function_signature ><![CDATA[ function signature that should be aligned with the code ]]></function_signature > <lg>[en/zh]</lg> <programming_language >python </programming_language > </question > </output > # Constraints - Do **not** alter the core algorithmic intent (e.g., if the problem is "Sum of list", do not change it to "Average of list"). - The ‘<instruction>’ field must contain **all** previous instructions plus the new one, separated by newlines and numbering. - The code must be wrapped in ‘<![CDATA[ ... ]]>’. Instruction mutation You are an expert in program analysis and synthetic coding data generation. Your task is to EVOLVE an existing instruction into a STRICTLY STRONGER, AST-CHECKABLE instruction that INVALIDATES ALL GIVEN EXAMPLES, while still SUBSUMING the original instruction. The goal is to generate a higher-difficulty, verifier-backed instruction that exposes genuine un- constrained syntactic degrees of freedom. ## Input You will receive an XML describing an existing instruction, including: - type - instruction template - checking function Multiple XML examples of solved coding problems, containing: - question_desc - instruction(s) - example_input_and_output - code - instantiated_params All provided examples are VALID solutions to the ORIGINAL instruction. ## Goals You MUST design an EVOLVED instruction such that: 1. Subsumption - Any program that satisfies the EVOLVED instruction MUST also satisfy the ORIGINAL instruction. - You must explicitly justify this subsumption. - The evolved instruction MUST introduce at most ONE new AST-level constraint beyond those already enforced by the original instruction. - The evolved instruction shouldn’t be overly complicated 2. Examples-as-Negative - ALL provided example programs MUST FAIL the EVOLVED instruction. 28 - These failures must arise from a GENERAL, STRUCTURAL, AST-level constraint. - You MUST NOT reference specific identifiers, literals, or fingerprints unique to the examples. 3. AST-Checkability - The evolved instruction MUST be checkable using static AST analysis. - No runtime execution, no I/O, no performance, no semantic reasoning. 4. Generalization - The evolved instruction MUST generalize beyond the given examples. - You are FORBIDDEN from writing constraints that merely exclude the examples without structural meaning. You MUST NOT: - Mention or encode specific variable names, constants, or literals that appear only in the examples. - Refer to line counts, whitespace, formatting, or comments. - Use vague or semantic constraints such as: "readable", "clean", "efficient", "reasonable", "well- structured". - Encode example-specific signatures or hashes. - Introduce constraints that can be satisfied or violated trivially. ## Output format (MUST be valid XML, no extra text): The output should only be a valid XML document without any extra text: ```xml <output > <thought ><![CDATA[ A detailed explanation covering: - Interpretation of the original instruction - How the examples satisfy it - What syntactic freedom is being restricted - Why the evolved instruction is strictly stronger - Why it subsumes the original - Why all provided examples fail it ]]></thought > <instruction ><![CDATA[ The evolved instruction template (may contain placeholders) ]]></instruction > <function > <name>FUNCTION_NAME </name> <params > <name>param_name </name> <type>string | integer | boolean | enum</type> <options >...</options >  </params > <params > <name>param_name </name> <type>string | integer | boolean | enum</type> <options >...</options >  </params > ... <impl><![CDATA[ def FUNCTION_NAME(tree , code_str , param1 , param2 , ** kwargs): # AST -based deterministic checking logic return True or False ]]> </impl> </function > 29 <positive_cases > <case> <instantiated_params >  </instantiated_params > <code> <![CDATA[ # Python code that SATISFIES the evolved instruction under the given parameter instantiation ]]> </code> </case> <case> <instantiated_params >  </instantiated_params > <code> <![CDATA[ # Python code that SATISFIES the evolved instruction under the given parameter instantiation ]]> </code> </case> </positive_cases > </output > ``` Problem rephrase You are an expert Data Augmentation Specialist for code generation tasks. Your goal is to rewrite coding instructions to increase linguistic diversity while STRICTLY preserving the original semantic meaning and logic. # Task You will receive a list of coding instructions (formatted as a numbered list). You must generate a rephrased version of this list. The new instructions should sound natural, diverse, and human-written. # Transformation Rules 1. **Rephrase**: This is your main tool. Use synonyms, change sentence structures, and vary the tone to express the exact same requirement. 2. **Combine**: Merge two or multiple instructions into a single instruction. 3. **Preserve Logic**: - Do NOT change any values (e.g., if it says "length < 5", do not output "length <= 5"). - Do NOT change the requirement type (e.g., "must use" cannot become "can use"). - Do NOT omit any constraints. 4. **Format**: The output must be a numbered list separated by newlines. 30

Instant research discovery