Paper deep dive
SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion
Yu He, Weikai Yang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%
Last extracted: 8/18/2026, 5:49:57 AM
Summary
The paper introduces SkillCommit, an online framework for evolving LLM agent skills by converting historical experience into a hierarchical library of reusable procedural knowledge. Unlike existing methods that rely on semantic similarity, SkillCommit uses behaviorally validated scope expansion, involving feedback-guided patch induction, behavioral compatibility grouping via cross-instance replay, and source-preserving consolidation. Experiments on RuleArena, OpenExempt, and KOR-Bench demonstrate that SkillCommit consistently improves agent performance and enables cross-model experience transfer.
Entities (13)
Relation Signals (12)
SkillCommit → evaluatedon → RuleArena
confidence 95% · Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance
SkillCommit → evaluatedon → OpenExempt
confidence 95% · Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance
SkillCommit → evaluatedon → KOR-Bench
confidence 95% · Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance
SkillCommit → usesprocess → Feedback-Guided Patch Induction
confidence 95% · SkillCommit organizes skill evolution into three stages. First... Feedback-Guided Patch Induction derives an instance-specific patch
SkillCommit → usesprocess → Behavioral Compatibility Grouping
confidence 95% · As patches accumulate, Behavioral Compatibility Grouping evaluates whether multiple patches exhibit transferable behavior
SkillCommit → usesprocess → Source-Preserving Consolidation
confidence 95% · Finally, Source-Preserving Consolidation abstracts behaviorally compatible patches into a higher-level skill
SkillCommit → outperforms → ExpeL
confidence 90% · SkillCommit consistently outperforms no-skill settings and strong baselines... ExpeL... ReasoningBank... ACE... Trace2Skill
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language model (LLM) agents can continually improve without parameter updates by converting historical experience into reusable procedural knowledge. However, existing methods often consolidate experience based on semantic similarity or LLM judgments, which may merge superficially related but behaviorally incompatible strategies and thereby degrade performance. To address the issue, we propose SkillCommit, an online skill evolution framework that continuously transforms experience into a hierarchical library of reusable skills. Each new experience is initially preserved as an instance-specific patch, retaining the behavior validated in its local context. As related skills accumulate, SkillCommit abstracts those sharing a common behavioral mechanism into higher-level skills. Specifically, for each incoming skill, embedding-based retrieval first identifies candidate related skills. Cross-instance replay and an LLM-based mechanism check determine whether these skills transfer across cases and share a common underlying mechanism. Candidates that pass both checks are abstracted into a higher-level skill and committed only if it preserves the validated behavior of all constituent skills. Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance across diverse domains. Moreover, the learned skills transfer across model scales and families, enabling cross-model experience transfer.
Tags
Links
- Source: https://arxiv.org/abs/2608.15165v1
- Canonical: https://arxiv.org/abs/2608.15165v1
Trouble viewing inline? Open PDF directly →
Full Text
44,145 characters extracted from source content.
Expand or collapse full text
SkillCommit: Evolving Agent Skills through Behaviorally Validated Scope Expansion Abstract Large language model (LLM) agents can continually improve without parameter updates by converting historical experience into reusable procedural knowledge. However, existing methods often consolidate experience based on semantic similarity or LLM judgments, which may merge superficially related but behaviorally incompatible strategies and thereby degrade performance. To address the issue, we propose SkillCommit, an online skill evolution framework that continuously transforms experience into a hierarchical library of reusable skills. Each new experience is initially preserved as an instance-specific patch, retaining the behavior validated in its local context. As related skills accumulate, SkillCommit abstracts those sharing a common behavioral mechanism into higher-level skills. Specifically, for each incoming skill, embedding-based retrieval first identifies candidate related skills. Cross-instance replay and an LLM-based mechanism check determine whether these skills transfer across cases and share a common underlying mechanism. Candidates that pass both checks are abstracted into a higher-level skill and committed only if it preserves the validated behavior of all constituent skills. Experiments on RuleArena, OpenExempt and KOR-Bench demonstrate that SkillCommit consistently improves agent performance across diverse domains. Moreover, the learned skills transfer across model scales and families, enabling cross-model experience transfer. Introduction Large language model (LLM) agents can use external tools and interact with their environments with increasing autonomy (22; 14; 21; 3; 29). Yet reliable deployment in specialized domains often requires domain knowledge, procedural conventions, and operational constraints that base models do not acquire during pretraining. Reusable skill provide a modular way to externalize such procedures and guide agents without modifying model parameters (19; 12; 10; 16; 26; 13). In practice, however, such knowledge is rarely available as a complete instruction document and instead remains scattered across practitioners’ experience, execution histories, and task feedback. Manually eliciting and formalizing such knowledge is costly and may still omit rare but critical boundary cases. This gap motivates skill evolution, in which agents automatically extract, consolidate, and refine reusable skills from accumulated trajectories and feedback (13; 23; 25; 1; 17). Existing methods typically use LLM-based abstraction to distill individual trajectories and feedback into instance-specific patches that agents can retrieve and reuse when solving similar tasks (16; 26; 13). However, since these patches are narrow in scope, retaining them individually leads to growing memory and retrieval costs as experience accumulates (1; 12; 25). To build a more compact and general skill library, subsequent methods consolidate related instance-specific patches into higher-level skills based on semantic similarity (20; 10) or model judgments (20; 19; 12; 10; 1). However, they do not guarantee behavioral compatibility, because semantically similar patches may encode conflicting strategies, whereas dissimilar patches may share a common underlying mechanism. Figure 1 illustrates this mismatch between semantic similarity and behavioral compatibility. Figure 1: Semantic similarity may retrieve behaviorally incompatible patches while overlooking those that share a common behavioral mechanism. Reliable skill evolution therefore requires criteria that go beyond surface similarity. Research in cognitive science suggests that concrete experience must first be transformed into procedural knowledge that can guide future action (7). Such knowledge transfers beyond its original context only when learners identify the relational structure shared across experiences rather than relying on surface resemblance (4; 5; 2). Moreover, newly formed abstractions must be integrated without interfering with previously validated competence (11). Together, these principles suggest that skill evolution should establish the local validity, justify its broader reuse through evidence of a shared underlying mechanism, and preserve validated behavior as that knowledge is consolidated at higher levels of abstraction. To meet these requirements, we propose SkillCommit, an online skill evolution framework that organizes skill evolution into three stages. First, when the agent fail to complete a task, Feedback-Guided Patch Induction derives an instance-specific patch based on the failed trajectory and feedback and locally validates it by replaying this task instance. As patches accumulate, Behavioral Compatibility Grouping evaluates whether multiple patches exhibit transferable behavior and share a common underlying mechanism, thereby determining the appropriate scope of a candidate higher-level skill. Finally, Source-Preserving Consolidation abstracts behaviorally compatible patches into a higher-level skill and commits it when it preserves the validated behavior of every constituent skill. This process allows the skill library to expand, reorganize, and compress over time while retaining empirically validated behavior. We evaluated SkillCommit across RuleArena (28), OpenExempt (15) and KOR-Bench (9), where it consistently outperforms no-skill settings and strong baselines across all evaluated subsets. Further experiments show that the induced skills transfer effectively across model scales and retain utility even when transferred across model families. Our contributions are: • We formulate reliable skill evolution as behaviorally validated scope expansion across skill admission, compatibility grouping, and higher-level abstraction, requiring replay evidence for every broader behavioral claim. • We propose SkillCommit, an online framework to build compact skill libraries while retaining previously validated behavior. • We evaluate SkillCommit across RuleArena, OpenExempt, and KOR-Bench, showing consistent gains over no-skill and strong baselines, and examine the transferability of the induced skills across model scales and families. Related Work LLM Agents and Skill Systems. Agent skills have emerged as reusable procedural artifacts that extend LLM agents with task instructions, operational knowledge, and executable workflows beyond one-off prompting and atomic tool use (18; 19; 8; 12; 10). Prior work has studied how such external knowledge is organized and associated, with A-MEM (20) dynamically linking accumulated memory notes and CA3Mem (25) recombining related experience through a structured memory graph. Agent Workflow Memory (19) instead induces recurring workflows from agent trajectories and selectively retrieves them for subsequent tasks. Meanwhile, SkillsBench (8) evaluates whether curated skills improve agent performance, whereas SkillLearnBench (27) examines whether agents can continually acquire reusable skills from task experience. These studies establish how skills can be organized, retrieved, and evaluated, but leave open what behavioral evidence should justify the broader applicability claimed by an evolving skill. Skill Acquisition and Self-Evolution. Early experience-based (24; 6) transform execution feedback into reusable textual knowledge, including verbal reflections in Reflexion (16), transferable insights in ExpeL (26), and reasoning strategies distilled from successful and failed experience in ReasoningBank (13). More recent systems move beyond retaining individual memories by continually refining an evolving playbook in ACE (23) or hierarchically consolidating trajectory-local lessons in Trace2Skill (12). SkillGen (10) evaluates the net intervention effect of a synthesized skill, while SkillCAT (1) replays candidate patches on their source tasks before hierarchical merging. Although these methods introduce validation at particular stages, they do not uniformly govern the successive scope expansions from a textual candidate to a validated patch, from individual patches to a behaviorally compatible group, and from a group abstraction to an active pattern. In contrast, SkillCommit separately validates these decisions through teacher-referenced source replay, directed cross-instance replay, and source-retention replay. SkillCommit Reliable skill evolution requires not only extracting procedural knowledge from individual experiences, but also determining how broadly that knowledge can be generalized without compromising previously validated behavior. SkillCommit addresses this challenge by treating skill-library construction as a sequence of behaviorally validated scope expansions. As illustrated in Figure 2, Feedback-Guided Patch Induction derives an instance-specific patch and verifies that the frozen executor can use it to solve the source instance. Behavioral Compatibility Grouping uses cross-instance replay and mechanism assessment to identify patches that can support a shared higher-level skill. Source-Preserving Consolidation commits the induced skill only when source replay confirms its validity across all instances within its claimed scope. Figure 2: SkillCommit overview. Feedback-Guided Patch Induction derives and source-validates instance patches; Behavioral Compatibility Grouping uses cross-instance replay to expand an existing skill or form a new one; and Source-Preserving Consolidation commits a candidate only after full-scope source replay. Feedback-Guided Patch Induction This stage aims to transform feedback from a failed task execution into a procedural patch that the frozen executor can use to improve its behavior. It follows an iterative two-step process: generating a candidate patch from teacher feedback and validating the patch through source replay. Patch Generation. The core idea is to contrast the executor’s failed behavior with a teacher-generated reference for the same task. Specifically, when the executor fails on a source instance xix_i and produces a trajectory τi− _i^-, a teacher model is provided with xix_i, the verified ground-truth answer, and τi− _i^-. Conditioned on this information, the teacher produces a reference trajectory τi+ _i^+ that reaches the correct answer and diagnoses the discrepancy between the failed and successful behaviors. Based on this contrast, the teacher generates a candidate patch cic_i containing concise procedural guidance, such as behavioral constraints, decision rules, or checkable intermediate steps intended to address the failure observed in τi− _i^-. Note that the teacher may use the same underlying model as the executor, but operates under a separate teacher prompt with access to privileged feedback that is never exposed during execution. To prevent solution leakage, candidates containing the final answer or an instance-specific solution to xix_i are discarded before validation. Source-replay Validation. Each candidate patch cic_i is validated by replaying the source instance with the same frozen executor. During replay, the executor receives only the original task context xix_i and the candidate patch cic_i. The resulting answer is evaluated by the task verifier, which retains access to the ground truth. If the executor successfully solves xix_i, the candidate is admitted as a source-validated instance-specific patch. Otherwise, the rejected patch, the new replay trajectory, and the verifier feedback are returned to the skill author to guide the next generation attempt. This generate-and-replay cycle is repeated for at most K=3K=3 attempts, terminating as soon as a patch enables the frozen executor to solve the source instance. In practice, 88.0% of failed instances yield a validated patch within 3 attempts, with 91.8% successful patches obtained within the first 2 trials. Behavioral Compatibility Grouping This stage determines how a locally-validated patch can be reused beyond its original instance. Given an incoming patch qtq_t and its source instance xtx_t, SkillCommit first examines whether qtq_t is behaviorally compatible with an existing higher-level skill. If a compatible skill is found, the framework attempts to expand that skill to cover the new patch. Otherwise, it searches for a group of compatible residual patches and uses them to induce a new higher-level skill. In both cases, semantic similarity is used only to retrieve a bounded set of plausible candidates. The actual decision is based on behavioral replay and mechanism assessment, since semantically similar patches may implement incompatible strategies, while patches expressed differently may follow the same underlying procedure. Note that the detailed prompt templates used for behavioral compatibility assessment and higher-level skill induction are provided in Appendix A. Expanding an Existing Skill. SkillCommit first retrieves the active higher-level skills whose descriptions are most similar to the incoming patch qtq_t. We embed the repair text of qtq_t and the structured description of each active skill using the same encoder, rank the active skill versions by cosine similarity, and retain the top R candidates, with R=4R=4 in our experiments. For each retrieved skill p, the framework collects behavioral evidence by applying the two memories beyond their currently validated scopes. Specifically, the existing skill p is replayed on the incoming source instance xtx_t, while qtq_t is replayed on a set of task instances (p)W(p) cover by p. Let ρ(m,x)∈0,1ρ(m,x)∈0,1 indicate whether the frozen executor solves instance x when guided only by memory m. We compute the behavioral compatibility between p and qtq_t as: C(p,qt)=ρ(p,xt)+∑xj∈(p)ρ(qt,xj)1+|(p)|.C(p,q_t)= ρ(p,x_t)+ _x_j (p)ρ(q_t,x_j)1+|W(p)|. (1) The pair passes the behavioral compatibility screen when C(p,qt)≥γC(p,q_t)≥γ, where γ∈[0,1]γ∈[0,1] is a predefined threshold. This cross-replay provides graded evidence that the two memories operate over related behavioral contexts. Because partial behavioral transfer may still arise from different strategies, a pattern author subsequently assesses whether p and qtq_t can be explained by a shared procedural mechanism. If the compatibility score and mechanism assessment both support consolidation, the author proposes an expanded version of p whose scope includes the incoming source. The proposal remains a candidate until full source replay in the next stage confirms that the expanded skill preserves the behavior validated across its claimed scope. Forming a New Skill. If no existing skill passes the previous checks, SkillCommit proceeds to form a new skill from residual patches, i.e, those source-validated patches that have not yet been incorporated into any higher-level skill. Similarly, SkillCommit retrieves a set of semantically related residual patches tC_t and evaluates their behavioral compatibility through cross-instance replay. Based on these replay outcomes, SkillCommit searches for an anchor patch whose procedure transfers to the source instances of all other patches in a candidate group. An eligible group must include the incoming patch qtq_t and contain at least m members. By default, the anchor is required to transfer to every member, while reverse transfer may be imposed as a stricter optional criterion. Among eligible groups, SkillCommit selects the one with the broadest behavioral coverage, using semantic similarity and a deterministic ordering only to resolve ties. If no eligible group is found, qtq_t remains residual and no new skill is created. Source-Preserving Consolidation After identifying either a behaviorally coherent group of patches or a compatible existing skill, SkillCommit induces a candidate skill over the proposed scope. Because abstraction may omit constraints essential to individual source instances, consolidation is governed by full source replay. A candidate is committed over its proposed scope only if it preserves all previously validated behavior within that scope; otherwise, its scope may be narrowed to a replay-supported subset or the candidate is rejected. Candidate Skill Induction. A candidate skill can arise in two ways. If the previous stage identifies a compatible group of residual patches, the pattern author attempts to induce a new higher-level skill from that group. If the incoming patch is compatible with an existing skill, the pattern author produces an expanded version of that skill. In both cases, the induced skill specifies an applicability condition, executable procedural guidance, and an exclusion condition, represented by when, repair, and avoid_when, respectively. To encourage mechanism-level abstraction rather than instance memorization, the pattern author is not given source questions, answers, teacher trajectories, or provenance information. For forming a new skill, it receives only the anonymized patch texts in the proposed group. For an expansion, it additionally receives the current skill description. Full-Scope Source Replay. Let S denote the proposed source set of a candidate. To verify that the abstraction preserves previously validated behavior, SkillCommit replays the candidate on every source instance in S using the frozen executor. If replay succeeds on every source, the candidate is regarded as fully supported. Otherwise, the pattern author incrementally refines the current candidate using its previous version together with the replay trajectories and verifier feedback, while ground-truth answers and instance identities remain hidden. Each refinement produces a new candidate version, which is replayed on the same source set before further refinement. This process continues for at most L attempts, and all candidate versions together with their replay outcomes are retained for auditing. Source-Preserving Commit and Split. To ensure that every committed skill preserves the validated behavior of every source instance within its declared scope, SkillCommit uses source replay as the sole criterion for consolidation. Depending on the replay outcome, each candidate skill follows one of three transitions. Commit. If a candidate successfully replays on every source instance within its proposed scope, it is committed as a higher-level skill. When the candidate expands an existing skill, the new version replaces the active version while preserving previous versions for traceability. When the candidate is induced from a group of residual patches, it becomes a new active skill. Split. If replay succeeds on only a subset of the proposed sources, SkillCommit selects the candidate version with the strongest replay support. If the supported source set of this version contains at least m instances, it will be split by restricting its scope to the supported sources and replayed once more on this reduced scope. A successfully revalidated split is committed as an independent higher-level skill. Typically, we set m to either 22 or 33 depending on the dataset, as detailed in the Appendix B. Reject. If no candidate the candidate is discarded. The existing skill hierarchy remains unchanged, and the corresponding patches continue to be retained as instance-specific skills. If no candidate version yields a replay-supported subset of at least m source instances, or if the narrowed candidate fails revalidation, the candidate is rejected. The existing skill hierarchy remains unchanged, while the corresponding patches remain available as instance-specific skills. Together, these transitions ensure that higher-level skills are expanded only when their broader applicability is supported by replay, while unsupported abstractions are conservatively narrowed or rejected. Experiments We evaluate whether SkillCommit improves frozen executors across heterogeneous reasoning domains, yields consistent task-level positive transfer, and produces skills that remain effective across model scales and families. We further examine how its three stages contribute to the performance. Experimental Setup Datasets and Metrics. We evaluate SkillCommit as an online skill evolution framework on three complementary reasoning benchmarks. To examine skill evolution in complex rule-based reasoning, we use the Airline and NBA subsets from RuleArena (28). Specifically, we split 200 Airline cases into 100 induction cases and 100 held-out test cases, while assigning 215 NBA cases into 170 induction cases and 45 testing cases. To evaluate generalization in the legal domain, we select Exemption Classification (EC) and Exemption Valuation (EV) from the Advanced Competency suite of OpenExempt (15), with 210 induction cases and 105 held-out test cases for each task. Finally, we include the Logic and Cipher tasks from KOR-Bench (9) to assess generalization in knowledge-orthogonal abstract reasoning, using 175 cases for induction and 75 cases for held-out testing in each task. The evaluation metrics reflect the output structure of each benchmark. RuleArena and KOR-Bench require verifiable final answers, so we report strict accuracy, counting an instance as correct only when its parsed answer passes the corresponding benchmark verifier. OpenExempt instead requires structured predictions of exemption citations or valuation claims; we therefore follow the official evaluator and report macro-F1, which reflects both coverage of applicable items and avoidance of unsupported predictions. Baselines. We compare SkillCommit with No Skill and four representative experience-learning methods. ExpeL (26) extracts reusable insights from accumulated experiences, while ReasoningBank (13) stores and retrieves reasoning memories derived from previous executions. ACE (23) iteratively evolves an external playbook through reflection and curation. Trace2Skill (12) hierarchically distills trajectory-level lessons into a reusable skill document. We adapt each method to the same benchmark interface and induction data while preserving its original skill-construction mechanism. Within each model configuration, all methods use the same frozen executor, base task prompt, output parser, verifier, and per-instance generation limit. Model Configurations and Implementation Details. Our model configurations separate matched-model effectiveness from transfer across model scales and families. We first consider two matched-model settings, using DeepSeek-V4-Flash and Qwen3.6-35B-A3B, respectively, for both skill construction and task execution. These settings establish the effectiveness of SkillCommit when the skill author and user are identical, across models with different capacity levels. We then use the smaller Qwen3.5-9B model as the skill user, reflecting a practical deployment scenario in which skills are constructed by a more capable model but repeatedly executed by a less costly model. Skills authored by Qwen3.6-35B-A3B evaluate in-family, cross-scale transfer, and skills authored by GPT-5.4 evaluate transfer across both model family and scale. All model parameters remain frozen throughout skill construction and evaluation, with generation settings and method-specific budgets provided in the Appendix B. Main Results Skill Author Skill User Method RuleArena OpenExempt KOR-Bench Avg.↑ Air.↑ NBA↑ EV↑ EC↑ Logic↑ Cipher↑ DeepSeek V4-Flash DeepSeek V4-Flash No Skill 72.00 28.89 76.86 86.33 72.00 73.33 68.24 ExpeL 88.00 53.33 50.17 88.36 80.00 78.67 73.09 ReasoningBank 66.00 31.11 65.54 74.25 68.00 72.00 62.82 ACE 51.00 6.67 87.32 95.65 72.00 70.67 63.89 Trace2Skill 50.00 28.89 84.30 75.50 70.67 70.67 63.34 SkillCommit (Ours) 96.00 71.11 94.28 96.01 82.67 85.33 87.57 Qwen3.6 35B-A3B Qwen3.6 35B-A3B No Skill 51.00 26.67 29.02 47.46 53.33 76.00 47.25 ExpeL 54.00 31.11 7.76 21.53 65.33 78.67 43.07 ReasoningBank 44.00 26.67 22.37 27.64 58.67 74.67 42.34 ACE 32.00 11.11 38.42 45.71 73.33 77.33 46.32 Trace2Skill 55.00 31.11 35.68 29.34 61.33 82.67 49.19 SkillCommit (Ours) 85.00 66.67 72.46 79.46 74.67 85.33 77.27 Qwen3.6 35B-A3B Qwen3.5 9B No Skill 55.00 15.56 9.97 15.61 60.00 65.33 36.91 ExpeL 54.00 26.67 4.18 10.73 62.67 70.67 38.15 ReasoningBank 53.00 22.22 16.42 22.68 52.00 65.33 38.61 ACE 14.00 15.56 26.31 34.57 41.33 74.67 34.41 Trace2Skill 35.00 31.11 24.76 25.43 60.00 73.33 41.61 SkillCommit (Ours) 81.00 60.00 54.28 62.41 76.00 84.00 69.62 Table 1: Main results across three skill-author/skill-user configurations. RuleArena and KOR-Bench report strict accuracy, while OpenExempt reports macro-F1. Avg. is the unweighted mean of the six task scores. The best result in each configuration is shown in bold, and the second-best result is underlined. Arrow subscripts show changes over matched baselines. SkillCommit consistently improves every evaluated setting over no-skill execution and other baseline methods. Table 1 summarizes evaluation results of the experiments on the three complementary reasoning benchmarks. Across all 18 task configuration pairs in Table 1, SkillCommit improves over No Skill and achieves the best result on every task, raising the unweighted mean of the reported task metrics from 50.80% to 78.15%. When DeepSeek-V4-Flash and Qwen3.6-35B-A3B are both skill author and skill user, the six-task average increases by 19.33 and 30.02 points, respectively. By contrast, 31 of the 72 task-level results from the four competing skill methods fall below No Skill and another six merely match it. SkillCommit exceeds the strongest competing method in average score by 14.48, 28.08, and 28.01 points in the three configurations, respectively. Figure 3: Instance-level effects relative to No Skill on matched DeepSeek-V4-Flash RuleArena runs. Help counts cases that No Skill misses but the corresponding method solves, while Hurt counts cases that No Skill solves but the method breaks. Experience-derived memories are not reliably beneficial without behavioral validation and may introduce more errors than they correct. The instance-level decomposition in Figure 3 shows that modest or even negative aggregate changes do not imply that these methods fail to learn from experience; rather, their gains are frequently offset by newly introduced errors. ExpeL repairs 24 Airline and 14 NBA failures—matching SkillCommit’s 24 repairs on Airline—but also overturns eight and three baseline-correct predictions, respectively, reducing its net gains to 16 and 11 cases. This pattern is consistent with extracting useful insights without execution-grounded admission and replay-validated scope control, allowing a valid lesson to be reused outside the context in which it applies. ReasoningBank helps/hurts 3/9 Airline cases and 2/1 NBA cases, suggesting that retrieving a relevant reasoning memory without behavioral compatibility evidence can produce negative transfer despite a slight gain on NBA. ACE helps/hurts 3/24 and 3/13 cases on Airline and NBA, respectively, indicating that globally evolving a playbook without regression-testing its updates can overwrite substantially more correct behavior than it corrects. Trace2Skill helps/hurts 2/24 Airline cases and 7/7 NBA cases, consistent with trajectory-level lessons being generalized beyond their behaviorally supported scope. By comparison, SkillCommit repairs 24 Airline and 21 NBA failures while regressing on only zero and two cases, yielding net gains of 24 and 19. The central distinction is therefore not merely whether a method can extract a useful memory, but whether each expansion of that memory’s scope is supported by replay evidence. SkillCommit provides these safeguards through source-replay patch admission, directed cross-instance compatibility checks, and full-scope source replay before commit; the component ablation below examines their respective contributions. Analysis of Skill Transferability Figure 4: Matched-model and cross-model skill reuse. Legend arrows denote skill author → frozen skill user. No Skill bars show the matched baseline for the Qwen-9B user; Qwen-35B → Qwen-35B is the matched-model refference. Bars report strict accuracy. In-family cross-scale transfer. To test whether skills authored by a larger model remain usable by a smaller executor from the same family, we deploy the library authored by Qwen3.6-35B-A3B with a frozen Qwen3.5-9B skill user. Across the four tasks in Figure 4, the transferred skills raise the Qwen3.5-9B average from 48.97 without skills to 75.25 and improve performance on every task. Transfer is substantial but not lossless: matched Qwen3.6-35B-A3B execution remains higher on Airline (85.00 vs. 81.00), NBA (66.67 vs. 60.00), and Cipher (85.33 vs. 84.00), while the smaller user performs slightly better on Logic (76.00 vs. 74.67). This pattern indicates that the learned skills preserve a reusable procedural core across model scales, while part of their effectiveness still depends on executor-specific interpretation and prompting conventions. At the same time, consistent gains over No Skill and the reversal on Logic argue against the library merely encoding instructions tailored to the author model. These results are consistent with feedback-guided induction and source-preserving consolidation converting local experience into explicit, replay-supported procedures. Cross-family and cross-scale transfer. To examine whether skills remain reusable when the author and user differ in both model family and scale, we further use GPT-5.4 as the skill author and Qwen3.5-9B as the skill user. Across the four tasks in Figure 4, GPT-5.4-authored skills raise the Qwen3.5-9B average from 48.97 without skills to 69.75 and improve performance on every task, demonstrating transfer across both model-family and scale boundaries. The transfer remains weaker than in-family cross-scale reuse on Airline (67.00 vs. 81.00), Logic (70.67 vs. 76.00), and Cipher (81.33 vs. 84.00), resulting in a 5.50-point average gap. This task-dependent degradation is consistent with residual mismatch between how different model families express and execute procedural guidance, even when the underlying mechanism is reusable. Nevertheless, the two author models reach the same accuracy on NBA (60.00), and GPT-5.4-authored skills remain within four points of matched Qwen3.6-35B-A3B performance on both Logic and Cipher. The NBA tie does not establish equivalent skill quality, since both libraries are ultimately executed by the same smaller user and may encounter a common user-side ceiling; however, it rules out a uniform cross-family penalty and suggests that sufficiently explicit decision procedures can survive changes in author family. Overall, the results indicate that behavioral induction and consolidation provide a portable interface between heterogeneous models, while not completely eliminating family- and executor-specific conventions. Component Ablations Condition Success Δ (p) SkillCommit (Full) 71.11 0.000.00 No Skill 28.89 ↓42.22 42.22 w/o Feedback-Guided Patch Induction 53.33 ↓17.78 17.78 w/o Behavioral Compatibility Grouping 66.67 ↓4.44 4.44 w/o Source-Preserving Consolidation 66.67 ↓4.44 4.44 Feedback-Guided Patch Induction only 64.44 ↓6.67 6.67 Behavioral Compatibility Grouping only 60.00 ↓11.11 11.11 Source-Preserving Consolidation only 57.78 ↓13.33 13.33 Table 2: Ablation results on RuleArena NBA. Δ denotes the absolute percentage-point drop from no ablation. To distinguish whether each stage is necessary within the complete pipeline from whether it is sufficient on its own, Table 2 pairs every w/o condition with a corresponding component-isolation condition. The w/o variants measure the marginal contribution of a stage given the remaining pipeline, whereas the only variants execute the named stage using the upstream inputs required by its actual interface. Source-replayed patch induction supplies the dominant performance gain. Removing Feedback-Guided Patch Induction reduces performance from 71.11% to 53.33%, the largest drop among the three leave-one-stage-out conditions. By contrast, deploying source-validated instance patches directly reaches 64.44%. Since the former condition still receives a parseable first-pass teacher patch, this contrast attributes the gain to validating and refining the proposed patch rather than merely generating additional memory text. The high Patch Induction-only score therefore shows that most immediate accuracy improvement comes from converting a plausible proposal into guidance that the frozen executor has demonstrated it can enact. It does not, however, measure the broader scope control provided by the evolving skill library. Compatibility grouping improves transfer by screening where a patch can be reused. Removing Behavioral Compatibility Grouping lowers performance from 71.11% to 66.67%, showing that semantic retrieval alone does not recover all of the benefit obtained from cross-instance behavioral evidence. Compatibility Grouping in isolation reaches 60.00% when operating on first-pass patches without source validation or subsequent consolidation. The gap between these conditions indicates that compatibility screening is useful when supplied with reliable local repairs, but cannot compensate for unsupported upstream patches or independently preserve the resulting abstraction. Its role is therefore not to create the original correction, but to determine the behavioral contexts in which that correction can be safely reused. Consolidation acts as a preservation gate rather than a standalone repair mechanism. Removing Source-Preserving Consolidation similarly reduces performance to 66.67%, showing that full-scope replay helps prevent an authored abstraction from discarding validated behavior. The smaller drop relative to Patch Induction reflects a distinct role: compatibility evidence can support a group without guaranteeing that one abstraction preserves every source. Full-scope replay tests that abstraction and narrows or rejects candidates with incomplete support. Consolidation alone reaches 57.78%, the weakest component-isolation result, because applying a replay gate to semantic groups of unvalidated patches cannot replace source-validated repairs or behavioral evidence for group formation. Conclusion We presented SkillCommit, an online skill evolution framework that transforms accumulated agent experience into a compact, hierarchical skill library without sacrificing previously validated behavior. Rather than relying solely on semantic similarity or model judgment, SkillCommit grounds consolidation in behavioral evidence, admitting every broader behavioral claim only after it survives replay. Experiments on RuleArena, OpenExempt, and KOR-Bench demonstrate consistent improvements over no-skill settings and strong skill-evolution baselines. The resulting skills also transfer across model scales and families, suggesting that procedural knowledge can be authored by more capable models and reused by smaller, more economical agents. We view this as a practical step toward agents that keep improving from experience while their accumulated knowledge remains compact, reusable, and behaviorally reliable. References Chen et al. (2026) K. Chen, Q. Zhong, J. Liu, and B. Du SkillCAT: contrastive assessment and topology-aware skill self-evolution for LLM agents. External Links: 2606.13317, Link Cited by: Introduction, Introduction, Skill Acquisition and Self-Evolution.. Chi et al. (1981) M. T. H. Chi, P. J. Feltovich, and R. Glaser Categorization and representation of physics problems by experts and novices. Cognitive Science 5 (2), p. 121–152. External Links: Document Cited by: Introduction. Deng et al. (2023) X. Deng, Y. Gu, B. Zheng, S. Chen, S. Stevens, B. Wang, H. Sun, and Y. Su Mind2Web: towards a generalist agent for the web. In Advances in Neural Information Processing Systems, Vol. 36, p. 28091–28114. External Links: Link Cited by: Introduction. Gentner (1983) D. Gentner Structure-mapping: a theoretical framework for analogy. Cognitive Science 7 (2), p. 155–170. External Links: Document Cited by: Introduction. Gick and Holyoak (1983) M. L. Gick and K. J. Holyoak Schema induction and analogical transfer. Cognitive Psychology 15 (1), p. 1–38. External Links: Document Cited by: Introduction. Gupta et al. (2024) P. Gupta, S. Kirtania, A. Singha, S. Gulwani, A. Radhakrishna, G. Soares, and S. Shi MetaReflection: learning instructions for language agents using past reflections. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, p. 8369–8385. External Links: Document, Link Cited by: Skill Acquisition and Self-Evolution.. Kolb (2014) D. A. Kolb Experiential learning: experience as the source of learning and development. FT press. Cited by: Introduction. Li et al. (2026) X. Li, Y. Liu, W. Chen, B. You, Z. Di, Y. He, S. Zheng, et al. SkillsBench: benchmarking how well agent skills work across diverse tasks. External Links: 2602.12670, Link Cited by: LLM Agents and Skill Systems.. Ma et al. (2025) K. Ma, X. Du, Y. Wang, H. Zhang, Z. Wen, X. Qu, J. Yang, J. Liu, M. Liu, X. Yue, W. Huang, and G. Zhang KOR-Bench: benchmarking language models on knowledge-orthogonal reasoning tasks. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: Introduction, Datasets and Metrics.. Ma et al. (2026) Y. Ma, Y. Huang, H. Bao, H. Zhuang, S. Shukla, M. Galley, X. Zhang, and S. Feuerriegel SkillGen: verified inference-time agent skill synthesis. External Links: 2605.10999, Link Cited by: Introduction, Introduction, LLM Agents and Skill Systems., Skill Acquisition and Self-Evolution.. McClelland et al. (1995) J. L. McClelland, B. L. McNaughton, and R. C. O’Reilly Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological Review 102 (3), p. 419–457. External Links: Document Cited by: Introduction. Ni et al. (2026) J. Ni, Y. Liu, X. Liu, Y. Sun, M. Zhou, P. Cheng, D. Wang, E. Zhao, X. Jiang, and G. Jiang Trace2Skill: distill trajectory-local lessons into transferable agent skills. External Links: 2603.25158, Link Cited by: Introduction, Introduction, LLM Agents and Skill Systems., Skill Acquisition and Self-Evolution., Baselines.. Ouyang et al. (2026) S. Ouyang, J. Yan, I. Hsu, Y. Chen, K. Jiang, Z. Wang, R. Han, L. T. Le, S. Daruki, X. Tang, V. Tirumalashetty, G. Lee, M. Rofouei, H. Lin, J. Han, C. Lee, and T. Pfister ReasoningBank: scaling agent self-evolving with reasoning memory. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: Introduction, Introduction, Skill Acquisition and Self-Evolution., Baselines.. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessi, R. Raileanu, M. Lomeli, E. Hambro, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems, Vol. 36, p. 68539–68551. External Links: Link Cited by: Introduction. Servantez et al. (2026) S. Servantez, S. B. Lawsky, R. Jain, D. W. Linna, and K. Hammond OpenExempt: a diagnostic benchmark for legal reasoning and a framework for creating custom benchmarks on demand. External Links: 2601.13183, Link Cited by: Introduction, Datasets and Metrics.. Shinn et al. (2023) N. Shinn, F. Cassano, A. Gopinath, K. Narasimhan, and S. Yao Reflexion: language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, Vol. 36. External Links: Link Cited by: Introduction, Introduction, Skill Acquisition and Self-Evolution.. Suzgun et al. (2026) M. Suzgun, M. Yuksekgonul, F. Bianchi, D. Jurafsky, and J. Zou Dynamic cheatsheet: test-time learning with adaptive memory. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p. 7080–7106. External Links: Document, Link Cited by: Introduction. Wang et al. (2023) G. Wang, Y. Xie, Y. Jiang, A. Mandlekar, C. Xiao, Y. Zhu, L. Fan, and A. Anandkumar Voyager: an open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291. External Links: Document, Link Cited by: LLM Agents and Skill Systems.. Wang et al. (2025) Z. Z. Wang, J. Mao, D. Fried, and G. Neubig Agent workflow memory. In Proceedings of the 42nd International Conference on Machine Learning, Proceedings of Machine Learning Research, Vol. 267, p. 63897–63911. External Links: Link Cited by: Introduction, Introduction, LLM Agents and Skill Systems.. Xu et al. (2025) W. Xu, Z. Liang, K. Mei, H. Gao, J. Tan, and Y. Zhang A-MEM: agentic memory for LLM agents. In Advances in Neural Information Processing Systems, Vol. 38. External Links: Link Cited by: Introduction, LLM Agents and Skill Systems.. Yang et al. (2023) R. Yang, L. Song, Y. Li, S. Zhao, Y. Ge, X. Li, and Y. Shan GPT4Tools: teaching large language model to use tools via self-instruction. In Advances in Neural Information Processing Systems, Vol. 36, p. 71995–72007. External Links: Link Cited by: Introduction. Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. R. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In The Eleventh International Conference on Learning Representations, External Links: Link Cited by: Introduction. Zhang et al. (2026a) Q. Zhang, C. Hu, S. Upasani, B. Ma, F. Hong, V. Kamanuru, J. Rainton, C. Wu, M. Ji, H. Li, U. Thakker, J. Zou, and K. Olukotun Agentic context engineering: evolving contexts for self-improving language models. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: Introduction, Skill Acquisition and Self-Evolution., Baselines.. Zhang et al. (2024) W. Zhang, K. Tang, H. Wu, M. Wang, Y. Shen, G. Hou, Z. Tan, P. Li, Y. Zhuang, and W. Lu Agent-pro: learning to evolve via policy-level reflection and optimization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 5348–5375. External Links: Document, Link Cited by: Skill Acquisition and Self-Evolution.. Zhang et al. (2026b) Z. Zhang, W. Bu, K. Pan, B. Miao, W. Zhang, G. Wang, W. Ji, R. Tang, J. Li, and S. Tang Evolving generalist virtual agents with generative and associative memory. Proceedings of the AAAI Conference on Artificial Intelligence 40 (15), p. 13006–13014. External Links: Document, Link Cited by: Introduction, Introduction, LLM Agents and Skill Systems.. Zhao et al. (2024) A. Zhao, D. Huang, Q. Xu, M. Lin, Y. Liu, and G. Huang ExpeL: LLM agents are experiential learners. Proceedings of the AAAI Conference on Artificial Intelligence 38 (17), p. 19632–19642. External Links: Document, Link Cited by: Introduction, Introduction, Skill Acquisition and Self-Evolution., Baselines.. Zhong et al. (2026) S. Zhong, Y. Lu, J. Ning, Y. Wan, L. Feng, Y. Ao, L. F. R. Ribeiro, M. Dreyer, S. Ammirati, and C. Xiong SkillLearnBench: benchmarking continual learning methods for agent skill generation on real-world tasks. External Links: 2604.20087, Link Cited by: LLM Agents and Skill Systems.. Zhou et al. (2025) R. Zhou, W. Hua, L. Pan, S. Cheng, X. Wu, E. Yu, and W. Y. Wang RuleArena: a benchmark for rule-guided reasoning with LLMs in real-world scenarios. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 550–572. External Links: Document, Link Cited by: Introduction, Datasets and Metrics.. Zhou et al. (2024) S. Zhou, F. F. Xu, H. Zhu, X. Zhou, R. Lo, A. Sridhar, X. Cheng, T. Ou, Y. Bisk, D. Fried, U. Alon, and G. Neubig WebArena: a realistic web environment for building autonomous agents. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: Introduction.