Paper deep dive
Revision or Re-Solving? Decomposing Second-Pass Gains in Multi-LLM Pipelines
Jingjie Ning, Xueqi Li, Chengyu Yu
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 4/2/2026, 3:32:41 AM
Summary
This paper introduces a four-condition decomposition framework to analyze the performance gains in multi-LLM revision pipelines. By isolating re-solving, scaffold, and content effects, the authors demonstrate that gains in MCQ tasks are primarily driven by the stronger model's independent re-solving capabilities, whereas code generation gains are heavily influenced by structural scaffolding, with weak-model content often acting as a harmful anchor.
Entities (7)
Relation Signals (3)
Multi-LLM revision pipelines → exhibitsgainsdrivenby → Re-solving effect
confidence 95% · On MCQ tasks... most gains are consistent with stronger-model re-solving
Multi-LLM revision pipelines → exhibitsgainsdrivenby → Scaffold effect
confidence 95% · On code generation tasks... two-stage prompting remains useful because even semantically null drafts can provide substantial structural scaffolding
Weak-model content → isharmfulin → Code generation tasks
confidence 92% · On code generation tasks, however... weak draft content can be harmful.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-LLM revision pipelines, in which a second model reviews and improves a draft produced by a first, are widely assumed to derive their gains from genuine error correction. We question this assumption with a controlled decomposition experiment that uses four matched conditions to separate second-pass gains into three additive components: re-solving, scaffold, and content. We evaluate this design across two model pairs on three benchmarks spanning knowledge-intensive MCQ and competitive programming. Our results show that the gains of multi-LLM revision are not monolithic, but depend on task structure, draft quality, and the type of draft information. On MCQ tasks, where the answer space is constrained and drafts provide little structural guidance, most gains are consistent with stronger-model re-solving, and directly routing queries to the stronger model can be more effective than revising a weak draft. On code generation tasks, however, two-stage prompting remains useful because even semantically null drafts can provide substantial structural scaffolding, while weak draft content can be harmful. Finally, role-reversed experiments show that strong drafts clearly benefit weak reviewers. Ultimately, our findings demonstrate that the utility of multi-LLM revision is dynamically bottlenecked by task structure and draft quality, necessitating more targeted pipeline designs rather than blanket revision strategies.
Tags
Links
- Source: https://arxiv.org/abs/2604.01029v1
- Canonical: https://arxiv.org/abs/2604.01029v1
Trouble viewing inline? Open PDF directly →
Full Text
49,701 characters extracted from source content.
Expand or collapse full text
Published as a conference paper at COLM 2026 Revision or Re-Solving? Decomposing Second-Pass Gains in Multi-LLM Pipelines Jingjie Ning ∗ Xueqi Li ∗ Chengyu Yu ∗ School of Computer Science, Carnegie Mellon University jening, xueqil, cy4@andrew.cmu.edu Abstract Multi-LLM revision pipelines, in which a second model reviews and im- proves a draft produced by a first, are widely assumed to derive their gains from genuine error correction. We question this assumption with a controlled decomposition experiment that uses four matched conditions to separate second-pass gains into three additive components: re-solving, scaffold, and content. We evaluate this design across two model pairs on three benchmarks spanning knowledge-intensive MCQ and competitive programming. Our results show that the gains of multi-LLM revision are not monolithic, but depend on task structure, draft quality, and the type of draft information. On MCQ tasks, where the answer space is constrained and drafts provide little structural guidance, most gains are consistent with stronger-model re-solving, and directly routing queries to the stronger model can be more effective than revising a weak draft. On code generation tasks, however, two-stage prompting remains useful because even semanti- cally null drafts can provide substantial structural scaffolding, while weak draft content can be harmful. Finally, role-reversed experiments show that strong drafts clearly benefit weak reviewers. Ultimately, our findings demonstrate that the utility of multi-LLM revision is dynamically bottle- necked by task structure and draft quality, necessitating more targeted pipeline designs rather than blanket revision strategies. 1 Introduction Figure 1: Illustration of the decomposition. Using one language model to critique and refine another model’s output has become a common way to improve re- sponse quality beyond single-pass genera- tion (Madaan et al., 2023; Shinn et al., 2023; Chen et al., 2024). The usual intuition is that a stronger reviewer can identify er- rors in a weaker model’s draft and correct them, and empirical results often appear consistent with this view. We argue that this interpretation is in- complete. When the reviewer is also the stronger model, an apparent revision gain may arise simply because the reviewer can independently re-solve the problem, even without using the draft. Distinguish- ing draft utility from re-solving matters directly for system design. If gains mostly come from re-solving, routing the query to the stronger model may be enough; if they come from scaffold or content, multi-stage ∗ These authors contributed equally to this work. 1 arXiv:2604.01029v1 [cs.SE] 1 Apr 2026 Published as a conference paper at COLM 2026 prompting may still be useful. These implications may differ across task types, especially between knowledge-intensive MCQ and code generation. Prior work has largely measured overall revision gains without separating these mecha- nisms (Madaan et al., 2023; Paul et al., 2024; Olausson et al., 2024). The few studies that include re-solving controls are narrower in scope and do not test role-reversed settings or separate the structural and semantic contributions of the draft in a unified controlled design (Stechly et al., 2023; Huang et al., 2023). We introduce a four-condition design that additively decomposes second-pass gains into three effects: re-solving, scaffold, and content, as shown in Figure 1. We evaluate this design on two model pairs across GPQA Diamond (Rein et al., 2023), HLE (Phan et al., 2025), and LiveCodeBench (Jain et al., 2024), with an additional role-reversed supplementary setting. Our main findings are: 1. On knowledge-intensive MCQ tasks, the content effect is non-significant across both model pairs and datasets, and most observed gains are consistent with stronger- model re-solving. 2.On code generation, weak-draft content is significantly worse than a structure-only scaffold. Gains are driven mainly by scaffold, and the content disadvantage grows with problem difficulty. 3. In the role-reversed setting, strong-model drafts significantly help weaker reviewers on both MCQ and code tasks, suggesting that draft utility depends on draft quality and on the kind of information the draft provides. 2 Related work Revision-style inference and self-correction are widely used to improve large language model (LLM) outputs beyond single-shot generation. Our work sits at the intersection of (i) iterative refinement and critique pipelines, (i) analyses questioning when self-correction truly works, and (i) multi-agent and capability-asymmetric collaboration. Iterative refinement with language feedback. Self-Refine (Madaan et al., 2023) popular- izes a feedback→refine loop in which an LLM critiques and iteratively revises its own output without additional training. Reflexion (Shinn et al., 2023) extends this to agentic settings by storing verbal reflections in memory for future attempts. REFINER (Paul et al., 2024) trains a critic to provide structured feedback over intermediate reasoning representations, while DeCRIM (Ferraz et al., 2024) applies a decompose–critique–refine pattern to multi-constraint instruction following. In programming, Self-Debugging (Chen et al., 2024) uses execution results to enable iterative code repair, and CRITIC (Gou et al., 2024) interleaves generation with tool-based validation. These tool-grounded approaches highlight that revision is most reliable when the feedback channel is dependable. Limits of intrinsic self-correction. A growing line of work argues that intrinsic self- correction, revision without reliable external feedback, is often ineffective and may even degrade performance (Huang et al., 2023). Stechly et al. (2023) show that apparent gains from iterative prompting can be explained by sampling and selection effects rather than by critique content. Surveys synthesize these mixed findings and conclude that self-correction works best with credible external feedback or explicit correction training (Kamoi et al., 2024; Pan et al., 2024). Related analyses suggest that latent factors such as confidence influence whether revisions help, supporting the view that “revision” is not a single mechanism but a composition of distinct effects (Li et al., 2024). Draft quality, capability asymmetry, and multi-agent decomposition.In code generation, self-repair gains are often modest, underscoring that editing low-quality drafts is not uniformly beneficial (Olausson et al., 2024). Multi-agent debate frameworks improve reasoning and factuality (Du et al., 2024; Liang et al., 2024), but recent decompositions reveal simple ensembling accounts for much of the observed gain (Choi et al., 2025). Weak-to- strong generalization studies find stronger models benefit from weak supervision but may 2 Published as a conference paper at COLM 2026 still underperform strong ceilings (Burns et al., 2024), highlighting the need to understand when weak drafts provide usable signal. Our framework complements these lines by isolating re-solving, scaffold, and content effects within a single controlled design. Re-solving and test-time scaling.Recent test-time scaling frameworks such as Re 2 (Wang et al., 2026) and Socratic Self-Refine (Shi et al., 2025) find that abandoning a flawed reasoning trajectory and re-solving from scratch can outperform attempts to patch it. Our isolation of re-solving as the dominant driver on MCQ tasks aligns with these findings, while our scaffold and content effects provide a finer-grained view of what the second pass contributes beyond independent re-solving. 3 Method As discussed in Section 1, the total second-pass gainx 2 − x 1 confounds model capability, review framing, and draft semantics. We introduce a four-condition design that decomposes this gain into three interpretable components. 3.1 Four Conditions We define four conditions,x 1 –x 4 , and use these symbols to refer to both the conditions and their corresponding accuracies. We evaluate two model pairs, Pair 1 (Gemini Flash Lite→GPT-5-mini) and Pair 2 (GPT-4o-mini→Gemini Flash), each in a Primary setting (weak→ strong) and a Supplementary setting (roles swapped). x 1 : Generator baseline. The generator answers the question directly. The result is cached and reused by all downstream conditions. x 2 : Standard cross-model revision. The reviewer receives the question together with the generator ’s output and is asked to review and improve the answer. The prompt template is identical to x 4 ; only the draft argument differs. x 3 : Re-solving control. The reviewer receives only the question and answers it directly, using the same prompt asx 1 with no review framing. Any framing benefit from the review task therefore accrues to x 2 , making x 2 > x 3 a conservative test of draft utility. x 4 : Scaffold control. The reviewer receives the question together with a semantically null placeholder draft (described in Section 3.3), using the same review prompt as x 2 . No prompt in any condition reveals the identity of the model that produced the draft. For MCQ, all conditions use the format “Reason 1 / Reason 2 / Answer: X”, placing the answer letter last to prevent it from anchoring the stated reasons. 3.2 Effect Decomposition Using the four conditions, the total second-pass gain x 2 − x 1 decomposes additively as: The content effect, which is our primary quantity of interest, measures the marginal value of the weak model’s actual output relative to a format-matched null draft. A negative content effect indicates that the weak model’s draft actively misleads the strong reviewer. The scaffold effect captures any benefit from the review framing and structural template. The re-solving effect isolates the capability gap between the two models. 3.3 Null Draft Design The null draft inx 4 preserves the format of a genuinex 1 response while removing task- specific content. We do not use another question’s real output as a decoy, because it may carry latent transfer signal. A semantically null but well-formed draft avoids such transfer 3 Published as a conference paper at COLM 2026 Figure 2: Signed decomposition of second-pass gains in the (weak→strong) setting into re-solving, scaffold, and content. MCQ gains are mostly re-solving, whereas LiveCodeBench is scaffold-dominated with negative content. Error bars show paired 95% CI. while still eliciting normal reviewer behavior. For MCQ tasks, we use a generic two-reason template followed byAnswer: X, whereX∈A, B, C, D. For code tasks, the null draft is a syntactically valid but semantically empty stub. The full null draft is shown in Appendix A. 3.4 Datasets and Evaluation We evaluate on three benchmarks. GPQA Diamond (Rein et al., 2023) (198 questions) con- sists of graduate-level multiple-choice science questions in physics, chemistry, and biology. HLE (Humanity’s Last Exam) (Phan et al., 2025) (451 questions) contains expert-level ques- tions across many disciplines and serves as a higher-difficulty MCQ complement to GPQA. LiveCodeBench (Jain et al., 2024) (1,054 problems) provides competitive-programming problems released after model training cutoffs, reducing contamination concerns. Problems are partitioned into Easy (322), Medium (382), and Hard (350) using the benchmark’s own difficulty labels, and solutions are evaluated against all public test cases. All statistical tests use two-tailed McNemar’s test with Yates continuity correction atα = 0.05. 4 Results and Analysis Figure 2 gives the global picture for the primary setting, our main setting of interest. Second- pass gains are not monolithic. On MCQ, they are dominated by re-solving, whereas on LiveCodeBench they are dominated by scaffold, with weak-draft content becoming harmful. We therefore discuss the two task types separately and use the supplementary strong→weak setting only as an asymmetry check on draft quality. 4.1 MCQ tasks Across both datasets, MCQ behavior is best understood as re-solving-dominated in the primary direction and draft-quality-dependent in the supplementary direction. 4 Published as a conference paper at COLM 2026 PrimarySupplementary Accuracy (%)Effects (p)Effects (p) Dataset Pair x 1 x 2 x 3 x 4 Re-solv. Scaff. ContentRe-solv.Scaff. Content GPQA P161.6 79.3 76.3 74.7 +14.6 ∗ −1.5 ns +4.5 ns −14.6 ∗ −5.1 ns +14.6 ∗ P233.3 87.4 89.9 90.4 +56.6 ∗ +0.5 ns −3.0 ns −55.6 ∗ −3.5 ns +26.8 ∗ HLE P112.6 20.2 17.5 18.4+4.9 ∗ +0.9 ns +1.8 ns −3.8 ns −0.7 ns +2.9 ns P211.3 37.5 41.7 39.9 +30.4 ∗ −1.8 ns −2.4 ns −30.6 ∗ −1.1 ns +12.0 ∗ Table 1: Accuracy and effect decomposition on MCQ tasks. The primary setting (weak→ strong) reports accuracy and all three effects; the supplementary setting (strong→weak) reports effects only (full accuracy in Appendix Table 3). ∗ p< .001; ∗ p< .05; ns: not significant (two-tailed McNemar’s test with Yates correction). 4.1.1 Primary Setting Table 1 shows a consistent pattern across GPQA and HLE. In the weak→strong direction, most of the second-pass gain is already captured by re-solving, while scaffold remains negligible and content stays near zero. The clearest examples are GPQA Pair 2 and HLE Pair 2, where re-solving contributes+56.6 p and+30.4 p, respectively, while content remains small and non-significant. Even when revision substantially improves over the weak baseline, the gain is therefore largely consistent with the stronger reviewer solving the question independently rather than extracting reliable semantic value from the weak draft. Mechanistically, this pattern is plausible for knowledge-intensive MCQ tasks because the answer space is constrained and the draft provides little task-specific structural guidance. A stronger reviewer can often discard a weak rationale and reconstruct the answer from its own parametric knowledge, so the draft functions less as a useful intermediate object than as a noisy prefix. This also explains why content remains near zero rather than consistently negative. The weak draft is not always inert, but its helpful and harmful influences mostly wash out, leaving re-solving as the dominant source of gain. 4.1.2 Supplementary Setting The supplementary strong→weak results reverse this picture. Once the incoming draft is strong and the reviewer is weak, content becomes clearly useful. It is significantly positive in both GPQA pairs and in HLE Pair 2, while HLE Pair 1 remains weak and non-significant. The most natural explanation is draft quality. In GPQA and HLE Pair 2, the stronger generator produces drafts that are good enough for the weaker reviewer to reuse; in HLE Pair 1, the supplementary generator reaches only 18.0% accuracy, leaving little reliable signal to transfer. The MCQ results therefore support a threshold-like view of draft utility that weak drafts add little beyond what a stronger reviewer can do on its own, but sufficiently strong drafts can materially help weaker reviewers. 4.1.3 Benefit–Harm View of the Content Effect Figure 3 provides a complementary view of the content effect by separating draft benefits from draft harms. On MCQ, the primary-setting points lie near the diagonal, consistent with near-cancellation. On LiveCodeBench, they shift clearly toward net harm, while the supplementary setting moves toward net benefit. We next show that this asymmetry is sharper on code generation. 4.2 Code Generation Code generation exhibits a different regime from MCQ. In the primary direction, the dominant gain comes from scaffold rather than from re-solving alone, and weak-draft content becomes actively harmful. 5 Published as a conference paper at COLM 2026 Figure 3: Benefit–harm view of the real-draft content effect (x 2 − x 4 ). Points below the diagonal indicate net benefit and points above it net harm. MCQ lies near cancellation, whereas LiveCodeBench shifts from net harm to net benefit across settings. PrimarySupplementary Accuracy (%)Effects (p)Effects (p) Pair x 1 x 2 x 3 x 4 Total Re-solv. Scaffold ContentRe-solv. Scaffold Content Pair 1 50.6 83.9 61.1 87.0 +33.3 +10.5 ∗ +25.9 ∗ −3.1 ∗ −7.3 ∗ +9.2 ∗ +3.2 ∗ Pair 2 28.5 78.1 43.1 86.0 +49.6 +14.6 ∗ +42.9 ∗ −7.9 ∗ −15.6 ∗ −1.7 ns +25.1 ∗ Table 2: Accuracy and effect decomposition on LiveCodeBench. The primary setting (weak →strong) reports accuracy and all three decomposed effects; the supplementary setting (strong→ weak) reports effects only (full accuracy in Appendix Table 3). 4.2.1 Primary Setting Table 2 shows that the core pattern on LiveCodeBench is not re-solving-dominated but scaffold-dominated. In both model pairs, the null scaffold outperforms the standard revision pipeline, and the content effect is significantly negative. This means that once the reviewer is given a weak code draft, the problem is no longer just whether the stronger model can solve the task, but whether it can avoid being pulled into a bad intermediate artifact. Figure 4: LiveCodeBench difficulty split for Pair 1. The content effect becomes increasingly negative from easy to hard. A plausible mechanism is artifact-level anchoring. Unlike MCQ, where the draft mainly provides a reasoning pre- fix, code drafts expose a partially instan- tiated executable object including func- tion signatures, parsing logic, wrapper structure, and implementation choices. Even when the weak model’s high-level algorithm is not entirely wrong, these concrete implementation decisions can still trap the reviewer in a brittle local trajectory. The strong reviewer therefore behaves less like an independent solver and more like an editor working within the weak draft’s structural constraints so that weak-draft content can actively hurt performance even when the second pass remains overall beneficial. 6 Published as a conference paper at COLM 2026 Figure 5: Mechanism-level decomposition of second-pass outcomes in the primary setting. On MCQ, diagnostic cases are dominated by the re-solving family; on LiveCodeBench, they are dominated by scaffold-positive cases. Figure 4 reinforces this interpretation. For Pair 1 in the primary setting, the content effect becomes increasingly neg- ative as difficulty rises from easy to medium and to hard.As problems become harder, weak drafts are more likely to contain deeply flawed implementation structure, increasing the anchoring cost. The scaffold effect is therefore not just a statistical residual but part of the mechanism. A null code draft provides no algorithmic solution, yet it still offers a syntactically valid, code-shaped object that waits for completion and repair. In the review framing, this appears to shift the second pass away from free-form solving and toward submission repair that fills in wrappers, normalizes I/O structure, and produces a complete executable artifact. Unlike MCQ, code tasks can benefit from this kind of structural intermediate even when its semantic content is empty. An appendix ablation comparing the Null Scaffold and the True-Null Scaffold shows only negligible differences (Appendix Table 9), suggesting that these scaffold gains are not primarily driven by trivial identifier retention. 4.2.2 Supplementary Setting The supplementary code results again show that draft utility is quality-dependent. When the draft comes from the stronger model, the content effect becomes positive in both pairs, indicating that code drafts are useful when their quality is high enough. Together with the harmful weak-draft content in the primary setting, this asymmetry suggests that draft usefulness in code depends on whether the incoming artifact provides usable structure and semantics rather than brittle implementation baggage. 4.3 Mechanistic Case Studies of Second-Pass Behavior The aggregate decompositions above identify which component dominates on average; this subsection asks how those effects appear in individual examples. 7 Published as a conference paper at COLM 2026 Per-question diagnostic taxonomy. Each question induces an outcome tuple(x 1 x 2 x 3 x 4 ), where✓denotes correct and✗denotes incorrect. We exclude the two non-diagnostic extremes,(✗)and(✓), and assign the remaining 14 patterns to exactly one family using a priority rule. We first check Content, becausex 2 vs.x 4 is the cleanest contrast where the prompt is identical and only the draft content differs. Among the remaining cases, we then check Scaffold, asking whether the null scaffold changes the outcome relative to unassisted re-solving. Residual cases are assigned to Re-solving, where the reviewer behaves the same regardless of the draft and only differs from the generator baseline. Full 16-way counts appear in Appendix Table 10. • Content (x 2 ̸= x 4 ): the real draft changes the outcome relative to the matched null scaffold. Positive meansx 2 is correct whilex 4 is incorrect, e.g.,(✗✓✗). Negative means x 2 is incorrect while x 4 is correct, e.g., (✓✗✓). •Scaffold (x 2 = x 4 butx 3 ̸= x 4 ): draft content is inert, but review framing with the null scaffold changes the outcome relative to pure re-solving. Positive meansx 3 is incorrect whilex 4 is correct, e.g.,(✗✓✗✓). Negative meansx 3 is correct whilex 4 is incorrect, e.g., (✗✓✗). •Re-solving (x 2 = x 4 = x 3 butx 1 ̸= x 2 ): neither draft content nor review framing matters; only the reviewer-generator capability gap remains. Positive means the reviewer succeeds where the generator fails, e.g.,(✗✓). Negative / drift means the generator is correct but the reviewer independently errs, e.g., (✓✗). Figure 5 shows that these families distribute very differently across task types. On MCQ, diagnostic cases are dominated by re-solving, whereas on LiveCodeBench they shift toward scaffold-positive cases and content-negative cases. We next examine representative examples of these contrasting behaviors. 4.3.1 MCQ: Re-solving and second-pass drift Positive Re-solving. In the left panel of Figure 6, the weak generator (x 1 ) starts from an incorrect premise and answers incorrectly, while the reviewer-side conditions reach the correct answer through a different conceptual route. The key point is that the reviewer does not refine the weak draft. Instead, it abandons the draft and reconstructs the answer from scratch, so the gain comes from the stronger model’s independent reasoning rather than from reliable semantic value in the weak draft. Figure 6: Representative MCQ cases. Left: positive re-solving, where the reviewer abandons the weak premise and reaches the correct answer independently. Right: negative re-solving, where the second pass overrides a correct weak answer and drifts to an incorrect one. Negative Re-solving. The right panel shows the complementary failure mode. Here the weak generator is already correct, but all reviewer-side conditions replace that correct answer with new, incorrect trajectories. Rather than preserving a correct baseline, the second pass re-engages the question and drifts away from it. On MCQ, then, the same replacement behavior that can rescue a weak answer can also overwrite a correct one. 8 Published as a conference paper at COLM 2026 4.3.2 Code: Artifact-level anchoring and scaffold repair On code-generation tasks, the second pass behaves less like reasoning revision and more like artifact repair. Weak drafts can anchor the reviewer to brittle implementations, while scaffolds can improve performance by inviting structured completion. Negative Content. In the left panel of Figure 7, the harm does not come from the weak draft’s high-level idea alone, but from its concrete implementation form. The weak draft exposes brittle parsing and wrapper decisions, and the reviewer remains anchored to this local structure instead of rebuilding a more robust solution. The result is a submission that inherits implementation fragility even when the stronger reviewer could, in principle, solve the problem correctly on its own. Positive Scaffold. The right panel clarifies why the null scaffold can help. The scaffold carries no algorithmic solution, but it does provide a syntactically valid, code-shaped object that invites completion. In the review framing, this shifts the task away from free-form solving and toward submission repair including finishing the wrapper, normalizing I/O structure, and producing a complete executable artifact. This kind of structural benefit has no close analogue on MCQ. Figure 7: Representative code cases. Left: negative content, where a draft anchors revision to a brittle artifact. Right: positive scaffold, where scaffold supports a robust solution. Why the mechanisms diverge across tasks. We hypothesize that this divergence reflects differences in output-space structure. MCQ has a constrained answer space, so a strong reviewer can often override a bad prefix and reconstruct the answer independently. How- ever, code generation is far more open-ended. As later tokens depend heavily on earlier structural choices, the reviewer is more vulnerable to anchoring by a weak draft and more likely to benefit from a scaffold that frames the task as repair rather than fresh generation. 5 Conclusion Our decomposition framework challenges the prevailing assumption in multi-LLM collab- oration that improved performance naturally stems from genuine revision. Instead, our results show that second-pass gains are highly task-dependent and can arise from different underlying mechanisms. On knowledge-intensive MCQ tasks, apparent improvements are overwhelmingly driven by the stronger model independently re-solving the problem, with content effects remaining near zero across settings, suggesting that weak-to-strong revision often adds little beyond directly querying the stronger model. On code generation tasks, however, multi-stage prompting remains useful for a different reason. Weak-draft semantics can actively degrade performance, while even semantically empty scaffolds provide substantial value by supporting structured repair. Our supple- mentary experiments further show that draft utility is highly asymmetric, with weak drafts often anchoring stronger reviewers to brittle trajectories, whereas high-quality drafts can genuinely help weaker reviewers. Taken together, these results show that the efficacy of revision is not a uniform mechanism, but depends on task structure, draft quality, and the type of information the draft contributes. 9 Published as a conference paper at COLM 2026 These findings have direct implications for system design. For constrained reasoning tasks, dynamically routing queries to a stronger model may often be preferable to multi-stage refinement. Yet for structured generation tasks like coding, multi-stage pipelines remain valuable because of their scaffolding properties. Ultimately, our work establishes that weak- to-strong revision should no longer be deployed as a universal template, paving the way for more adaptive, task-aware collaborative LLM architectures. 10 Published as a conference paper at COLM 2026 References Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold As- chenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeffrey Wu. Weak-to-strong generalization: Eliciting strong capabilities with weak supervision. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. doi: 10.48550/arXiv.2312.09390. URL https://arxiv.org/abs/2312.09390. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. Teaching large language models to self-debug. In The Twelfth International Conference on Learning Representations (ICLR), 2024. Hyeong Kyu Choi, Xiaojin Zhu, and Yixuan Li. Debate or vote: Which yields better decisions in multi-agent large language models? arXiv preprint arXiv:2508.17536, 2025. doi: 10.48550/arXiv.2508.17536. URL https://arxiv.org/abs/2508.17536. Yilun Du, Shuang Li, Antonio Torralba, Joshua B. Tenenbaum, and Igor Mordatch. Im- proving factuality and reasoning in language models through multiagent debate. In Proceedings of the 41st International Conference on Machine Learning (ICML), 2024. doi: 10.48550/arXiv.2305.14325. URL https://arxiv.org/abs/2305.14325. Thomas Palmeira Ferraz, Kartik Mehta, Yu-Hsiang Lin, Haw-Shiuan Chang, Shereen Oraby, Sijia Liu, Vivek Subramanian, Tagyoung Chung, Mohit Bansal, and Nanyun Peng. LLM self-correction with DeCRIM: Decompose, critique, and refine for enhanced following of instructions with multiple constraints. In Findings of the Association for Computational Linguistics: EMNLP, 2024. URLhttps://aclanthology.org/2024.findings-emnlp.458/. arXiv:2410.06458. Zhibin Gou, Zhihong Shao, Yeyun Gong, Yelong Shen, Yujiu Yang, Nan Duan, and Weizhu Chen. CRITIC: Large language models can self-correct with tool-interactive critiquing. In International Conference on Learning Representations (ICLR), 2024. URLhttps://arxiv.org/ abs/2305.11738. arXiv:2305.11738. Jie Huang, Xinyun Chen, Swaroop Mishra, Huaixiu Steven Zheng, Adams Wei Yu, Xinying Song, and Denny Zhou. Large language models cannot self-correct reasoning yet. arXiv preprint arXiv:2310.01798, 2023. Naman Jain, King Han, Alex Gu, Wen-Ding Li, Fanjia Yan, Tianjun Zhang, Sida Wang, Ar- mando Solar-Lezama, Koushik Sen, and Ion Stoica. LiveCodeBench: Holistic and contam- ination free evaluation of large language models for code. arXiv preprint arXiv:2403.07974, 2024. Ryo Kamoi, Yusen Zhang, Nan Zhang, Jiawei Han, and Rui Zhang. When can LLMs actually correct their own mistakes? a critical survey of self-correction of LLMs. Transactions of the Association for Computational Linguistics, 12:1417–1440, 2024. doi: 10.1162/tacl_a_00713. URL https://aclanthology.org/2024.tacl-1.78/. Loka Li, Zhenhao Chen, Guangyi Chen, Yixuan Zhang, Yusheng Su, Eric Xing, and Kun Zhang. Confidence matters: Revisiting intrinsic self-correction capabilities of large language models. arXiv preprint arXiv:2402.12563, 2024. doi: 10.48550/arXiv.2402.12563. URL https://arxiv.org/abs/2402.12563. Tian Liang, Zhiwei He, Wenxiang Jiao, Xing Wang, Yan Wang, Rui Wang, Yujiu Yang, Shuming Shi, and Zhaopeng Tu. Encouraging divergent thinking in large language models through multi-agent debate. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2024. URLhttps://aclanthology.org/ 2024.emnlp-main.992/. arXiv:2305.19118. Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Sean Welleck, Peter Clark, Antoine Bosselut, Eric Smith, Luke Zettlemoyer, and Hannaneh Hajishirzi. Self-refine: Iterative refinement with self-feedback. In Advances in Neural Information Processing Systems, volume 36, 2023. 11 Published as a conference paper at COLM 2026 Theo X. Olausson, Jeevana Priya Inala, Chenglong Wang, Jianfeng Gao, and Armando Solar-Lezama. Is self-repair a silver bullet for code generation? In The Twelfth International Conference on Learning Representations (ICLR), 2024. Liangming Pan, Michael Saxon, Wenda Xu, Deepak Nathani, Xinyi Wang, and William Yang Wang. Automatically correcting large language models: Surveying the landscape of diverse automated correction strategies. Transactions of the Association for Computational Linguistics, 12:484–506, 2024. doi: 10.1162/tacl_a_00660. URLhttps://transacl.org/ index.php/tacl/article/view/5825. Debjit Paul, Mete Ismayilzada, Maxime Peyrard, Beatriz Borges, Antoine Bosselut, Robert West, and Boi Faltings. REFINER: Reasoning feedback on intermediate representations. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2024. Long Phan, Alice Gatti, Ziwen Han, Nathaniel Li, Josephina Hu, Hugh Zhang, Arman Habibi, Aditya Chatterjee, Lianghao Zhao, Michael Chen, et al. Humanity’s last exam. arXiv preprint arXiv:2501.14249, 2025. David Rein, Betty Li Hou, Asa Cooper Stickland, Jackson Petty, Richard Yuanzhe Pang, Julien Dirani, Julian Michael, and Samuel R. Bowman. GPQA: A graduate-level google- proof Q&A benchmark. arXiv preprint arXiv:2311.12022, 2023. Haizhou Shi, Ye Liu, Bo Pang, Zeyu Leo Liu, Hao Wang, Silvio Savarese, Caiming Xiong, Yingbo Zhou, and Semih Yavuz. Ssr: Socratic self-refine for large language model reasoning. arXiv preprint arXiv:2511.10621, 2025. doi: 10.48550/arXiv.2511.10621. URL https://arxiv.org/abs/2511.10621. Noah Shinn, Federico Cassano, Edward Berman, Ashwin Gopalan, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, 2023. Kaya Stechly, Matthew Marber, and Subbarao Kambhampati. GPT-4 doesn’t know it’s wrong: An analysis of iterative prompting for reasoning problems. In NeurIPS 2023 Workshop on Foundation Models for Decision Making, 2023. Pinzheng Wang, Shuli Xu, Juntao Li, Yu Luo, Dong Li, Jianye Hao, and Min Zhang. Re 2 : Unlocking LLM reasoning via reinforcement learning with re-solving. In The 14th Interna- tional Conference on Learning Representations (ICLR), 2026. 12 Published as a conference paper at COLM 2026 Contents of Appendix A Experimental Setup and Prompt Templates13 B Complete Experimental Results15 C Disclosure of LLM Usage18 A Experimental Setup and Prompt Templates Method Details Our experiments instantiate the same revision design with two model pairs and two role directions. In the primary setting, Pair 1 uses Gemini Flash Lite as the generator and GPT-5-mini as the reviewer, while Pair 2 uses GPT-4o-mini as the generator and Gemini Flash as the reviewer. In the supplementary setting, the generator and reviewer roles are swapped within each pair. Concretely, the code defines four model assignments: primary pair 1 (Gemini Flash Lite→GPT-5-mini), supplementary pair 1 (GPT-5-mini→ Gemini Flash Lite), primary pair 2 (GPT-4o-mini→Gemini Flash), and supplementary pair 2 (Gemini Flash→GPT-4o-mini). The prompt logic is held fixed across all four settings and only the model assignment changes. This makes the two series byte-for-byte identical in prompt structure and condition logic, isolating model-pair effects from implementation artifacts. Within each setting, we evaluate four main conditions.x 1 is the generator-only baseline, where the generator answers the question directly.x 2 is standard cross-model revision, where the reviewer receives the question together with the real draft produced under x 1 .x 3 is a pure re-solving control, where the reviewer receives the question only and answers directly with no review framing.x 4 is a scaffold control, where the reviewer receives the same review prompt as inx 2 but the draft is replaced with a semantically null placeholder. As implemented in the experimental script,x 2 andx 4 share the exact same critique-prompt template; only thedraftargument differs. Likewise,x 3 uses the same direct-answer template asx 1 , ensuring that anyx 2 > x 3 gap cannot be attributed to privileged prompt framing. No critique prompt includes source attribution indicating which model produced the draft. For the coding task, we additionally define an ablation conditionx 5 , used only to test possible identifier leakage in the null scaffold. Compared withx 4 , the only change is that the placeholder no longer attempts to preserve any task-specific function name or format signal. This produces a maximally neutral scaffold and allows us to distinguish the effect of format-matching from the effect of weak-draft content itself. The semantic-null draft inx 4 is constructed to preserve output format while carrying no task-specific semantics. For MCQ, the placeholder always follows the same three-line structure (Reason 1,Reason 2,Answer) as a real model output. The answer letter is assigned deterministically byMD5(question)[0] % 4, which makes the placeholder reproducible across runs and machines while distributing letters uniformly overA–D. For code, the null draft is a syntactically valid Python stub. Function-level problems receive a placeholder function whose name is extracted from the problem statement via regex so that the signature matches the required identifier exactly; stdin/stdout problems receive amain()scaffold with apassbody. This design avoids contaminating the null baseline with accidental content signals such as a wrong function name. By contrast,x 5 uses a fixed generic stub (e.g.,def solution(*args, **kwargs): ...) with no task-specific lexical cues. The implementation also uses a selective cache to ensure clean comparisons. Generator steps (x 1 and x r 1 ) are cached and reused by downstream conditions so that all revision and scaffold variants for a given question operate on the same generator draft. Critique and re-solving steps (x 2 ,x 3 ,x 4 , andx 5 , together with their supplementary counterparts) always bypass the cache and are issued as fresh API calls. The cache key is the tuple(model_key, 13 Published as a conference paper at COLM 2026 MCQ direct-answer prompt (x 1 , x 3 ) Answer the following multiple-choice question. Reply in this EXACT format (no other text): Reason 1: <one short reason supporting your answer> Reason 2: <one short reason why the most plausible alternative is wrong> Answer: <letter> QUESTION Code direct-answer prompt (x 1 , x 3 ) Function-level tasks Write a Python function that solves the following problem. Return only the function code, no explanation. Problem: QUESTION stdin/stdout tasks Solve the following competitive programming problem in Python. Your solution MUST read input from stdin and write output to stdout. Return only the complete Python solution, no explanation. Problem: QUESTION MCQ review prompt (x 2 , x 4 ) Here is an answer to the following multiple-choice question: Question: QUESTION Answer attempt: DRAFT Review the answer and reasons above. Identify any errors in the reasoning or the chosen letter, then provide your own best answer. Reply in this EXACT format (no other text): Reason 1: <one short reason supporting your answer> Reason 2: <one short reason why the most plausible alternative is wrong> Answer: <letter> Code review prompt (x 2 , x 4 , x 5 ) Here is a Python solution to the following problem: Problem: QUESTION Solution: DRAFT Review this solution carefully, identify any errors or suboptimal logic, and provide a corrected and improved version. For function-level tasks: Return only the improved function code, no explanation. For stdin/stdout tasks: Return only the complete improved Python solution (reads from stdin, writes to stdout), no explanation. tag, prompt), which prevents collisions between different model roles or condition types even when prompts share overlapping content. Prompt Templates We separate the prompt templates by function rather than placing all prompts in a single omnibus box. Direct-answer prompts are used forx 1 andx 3 , review prompts are used forx 2 ,x 4 , andx 5 , and null-draft templates are listed separately because they define the scaffold and true-null controls. Hyperparameters The LLM wrapper defines four model-role keys and maps them to concrete API model names:gpt_strong → gpt-5-mini,gpt_weak → gpt-4o-mini,gemini_strong → gemini-3-flash-preview,andgemini_weak → gemini-3.1-flash-lite-preview.OpenAIcallsareissuedthrough chat.completions.createwith a single user message, and Gemini calls are issued through generate_content with the prompt as the contents field. Importantly, the wrapper does not explicitly set decoding controls such as temperature, top-p, max tokens, or random seed. As a result, all runs use the providers’ default genera- tion settings. The only explicit execution-time control exposed in the wrapper isuse_cache. Whenuse_cache=True, the system reads from a persistent cache if a matching entry already exists and otherwise writes the result on a cache miss. Whenuse_cache=False, the system always makes a fresh API call and does not read from or write to the cache. In our experi- ments, generator steps always use caching so that all downstream conditions share the exact same draft for a given question, whereas critique and re-solving steps always bypass the cache to avoid reusing reviewer outputs. The persistent cache file is stored ascache.pkl, and the wrapper reads API credentials from the environment variablesOPENAI_API_KEYand GEMINI_API_KEY. The cache is guarded by a thread lock, so reads and writes are synchronized even if API calls are issued concurrently. 14 Published as a conference paper at COLM 2026 MCQ null draft for x 4 Reason 1: This answer considers several possibilities but does not address the current question. Reason 2: The reasoning pattern is generic and may not be relevant here. Answer: X X is assigned deterministically by MD5(question)[0] % 4. Code null scaffold for x 4 Function-level def FUNC_NAME(*args, **kwargs): # placeholder implementation # structure only; not related to the current task pass stdin/stdout import sys def main(): # placeholder implementation # structure only; not related to the current task pass main() Code true-null draft for x 5 def solution(*args, **kwargs): """Placeholder draft with no task-specific information.""" raise NotImplementedError B Complete Experimental Results B.1 Full Accuracy Table Table 3 reports accuracy for all 48 experimental conditions across both experiment pairs. DatasetSettingPairx 1 x 2 x 3 x 4 GPQA (198) Primary Pair 1122 (61.6%)157 (79.3%)151 (76.3%)148 (74.7%) Pair 266 (33.3%)173 (87.4%)178 (89.9%)179 (90.4%) Supplem. Pair 1153 (77.3%)143 (72.2%)124 (62.6%)114 (57.6%) Pair 2178 (89.9%)114 (57.6%)68 (34.3%)61 (30.8%) HLE (451) Primary Pair 157 (12.6%)91 (20.2%)79 (17.5%)83 (18.4%) Pair 251 (11.3%)169 (37.5%)188 (41.7%)180 (39.9%) Supplem. Pair 181 (18.0%)74 (16.4%)64 (14.2%)61 (13.5%) Pair 2188 (41.7%)99 (22.0%)50 (11.1%)45 (10.0%) LCB (1054) Primary Pair 1533 (50.6%)884 (83.9%)644 (61.1%)917 (87.0%) Pair 2300 (28.5%)823 (78.1%)454 (43.1%)906 (86.0%) Supplem. Pair 1627 (59.5%)681 (64.6%)550 (52.2%)647 (61.4%) Pair 2461 (43.7%)544 (51.6%)297 (28.2%)279 (26.5%) Table 3: Accuracy for all conditions. Format: correct count (percentage). Pair 1 uses Gemini Flash Lite as generator and GPT-5-mini as reviewer. Pair 2 uses GPT-4o-mini as generator and Gemini Flash as reviewer. 15 Published as a conference paper at COLM 2026 B.2 Complete Effect Decomposition Table 4 reports all effect sizes in percentage points. DatasetSettingPairTotalRe-solv.ScaffoldContent GPQA Primary Pair 1 +17.7+14.6 −1.5+4.5 Pair 2 +54.0+56.6+0.5 −3.0 Supplem. Pair 1 −5.1 −14.6 −5.1+14.6 Pair 2 −32.3 −55.6 −3.5+26.8 HLE Primary Pair 1 +7.5+4.9+0.9+1.8 Pair 2 +26.2+30.4 −1.8 −2.4 Supplem. Pair 1 −1.6 −3.8 −0.7+2.9 Pair 2 −19.7 −30.6 −1.1+12.0 LCB Primary Pair 1 +33.3+10.5+25.9 −3.1 Pair 2 +49.6+14.6+42.9 −7.9 Supplem. Pair 1 +5.1 −7.3+9.2+3.2 Pair 2 +7.9 −15.6 −1.7+25.1 Table 4: Effect decomposition for all datasets, settings, and model pairs. All values are in percentage points. Re-solving= x 3 − x 1 ; Scaffold= x 4 − x 3 ; Content= x 2 − x 4 ; Total = x 2 − x 1 . B.3 McNemar’s Test Results Tables 5–7 report full McNemar’s test statistics for all key comparisons. Significance: ∗ p<.05, ∗ p<.01, ∗ p<.001.n ✓✗ : condition A correct, B wrong.n ✗✓ : condition A wrong, B correct. SettingPairComparisonAcc AAcc B∆n ✓✗ n ✗✓ χ 2 p Primary Pair 1 Content (x 2 − x 4 )79.374.7 −4.51672.78.095 ns Scaffold (x 4 − x 3 )74.776.3 +1.59120.19.663 ns Re-solving (x 3 − x 1 )76.361.6 +14.6451612.85.0003*** Pair 2 Content (x 2 − x 4 )87.490.4 +3.04101.79.181 ns Scaffold (x 4 − x 3 )90.489.9 −0.5650.001.00 ns Re-solving (x 3 − x 1 )89.933.3 +56.61142106.2<.001*** Supplem. Pair 1 Content (x 2r − x 4r )72.257.6 −14.6421314.26.0002*** Scaffold (x 4r − x 3r )57.662.6 +5.122321.50.221 ns Re-solving (x 3r − x 1r )62.677.3 −14.6164512.85.0003*** Pair 2 Content (x 2r − x 4r )57.630.8 −26.8641136.05<.001*** Scaffold (x 4r − x 3r )30.834.3 +3.524310.65.418 ns Re-solving (x 3r − x 1r )34.389.9 −55.62112104.2<.001*** Table 5: McNemar’s tests on GPQA (two-tailed, Yates correction). B.4 LiveCodeBench Difficulty Breakdown (Pair 1, primary) Table 8 shows a breakdown of the accuracy of the four conditions and content effect from different difficulty levels on LiveCodeBench. B.5 Scaffold Ablation on Identifier Retention To test whether the gains of the scaffold control are partly driven by task-specific identifier cues, we compare the Null Scaffold against a more neutral True-Null Scaffold. The two 16 Published as a conference paper at COLM 2026 SettingPairComparisonAcc AAcc B∆n ✓✗ n ✗✓ χ 2 p Primary Pair 1 Content (x 2 − x 4 )20.218.4 +1.836440.61.434 ns Scaffold (x 4 − x 3 )18.417.5 +0.928320.15.699 ns Re-solving (x 3 − x 1 )17.512.6 +4.964424.16.041* Pair 2 Content (x 2 − x 4 )37.539.9 −2.444331.30.254 ns Scaffold (x 4 − x 3 )39.941.7 −1.841330.66.416 ns Re-solving (x 3 − x 1 )41.711.3 +30.41662994.85<.001*** Supplem. Pair 1 Content (x 2r − x 4r )16.413.5 +2.934471.78.182 ns Scaffold (x 4r − x 3r )13.514.2 −0.732350.06.807 ns Re-solving (x 3r − x 1r )14.218.0 −3.847642.31.129 ns Pair 2 Content (x 2r − x 4r )22.010.0 +12.0247827.54<.001*** Scaffold (x 4r − x 3r )10.011.1 −1.131360.24.625 ns Re-solving (x 3r − x 1r )11.141.7 −30.62816696.75<.001*** Table 6: McNemar’s tests on HLE (two-tailed, Yates correction). SettingPairComparisonAcc AAcc B∆n ✓✗ n ✗✓ χ 2 p Primary Pair 1 Content (x 2 − x 4 )83.987.0 −3.176438.61.003** Scaffold (x 4 − x 3 )87.061.1 +25.933306218.2<.001*** Re-solving (x 3 − x 1 )61.150.6 +10.532521422.45<.001*** Pair 2 Content (x 2 − x 4 )78.186.0 −7.91274439.32<.001*** Scaffold (x 4 − x 3 )86.043.1 +42.916468420.3<.001*** Re-solving (x 3 − x 1 )43.128.5 +14.625810464.67<.001*** Supplem. Pair 1 Content (x 2r − x 4r )64.661.4 +3.2951294.86.027* Scaffold (x 4r − x 3r )61.452.2 +9.26215941.70<.001*** Re-solving (x 3r − x 1r )52.259.5 −7.322930610.80.001** Pair 2 Content (x 2r − x 4r )51.626.5 +25.128293217.1<.001*** Scaffold (x 4r − x 3r )26.528.2 −1.739573.01.083 ns Re-solving (x 3r − x 1r )28.243.7 −15.611628067.09<.001*** Table 7: McNemar’s tests on LiveCodeBench (two-tailed, Yates correction). Tiernx 1 x 2 x 3 x 4 Content Easy32269.3%94.1%74.8%94.7% −0.6 p Medium38247.1%86.1%66.8%89.5% −3.4 p Hard35037.1%72.0%42.3%77.1% −5.1 p Table 8: Accuracy by difficulty tier on LiveCodeBench, Pair 1 primary setting. Content effect = x 2 − x 4 . conditions use the same review prompt and differ only in the placeholder shown to the reviewer. The Null Scaffold preserves a task-shaped placeholder, including the extracted function name when available, whereas the True-Null Scaffold replaces this with a fixed generic stub. Quantitatively, the difference is negligible in both model pairs. For Pair 1, accuracy changes only from 87.0% under the Null Scaffold to 86.4% under the True-Null Scaffold (p =0.617); for Pair 2, it changes from 86.0% to 85.1% (p =0.481). We therefore do not find evidence that the strong performance of the scaffold control is primarily driven by the retained function name or by simple task-specific lexical cues. 17 Published as a conference paper at COLM 2026 PairNull ScaffoldTrue-Null Scaffoldp Pair 187.086.40.617 Pair 286.085.10.481 Table 9: Scaffold ablation on LiveCodeBench. Left: comparison between the Null Scaffold and the True-Null Scaffold. Right: forward-setting accuracies and significance tests for their difference. At the same time, this ablation should be interpreted narrowly. It does not imply that all structural signals are irrelevant, nor that the two scaffold variants are behaviorally identical in every respect. Rather, it shows that the gains of the scaffold control are not reducible to trivial identifier retention alone. The main benefit appears to come from presenting the reviewer with a syntactically valid, code-shaped intermediate object under review framing, rather than from the specific lexical content of the extracted function name. B.6 Pattern-Family Counts Dataset Pair n✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓✗✓ GPQA P119817134853351113122399 P219810336231104200100162 HLE P1451281228121981430341116158 P2451195182423138121071702357314 LCB P1105477211045158316254552571975307 P21054971045213426110222111101020194 Table 10: Pattern counts across all 2 4 per-question outcome patterns for the two primary model pairs. Patterns are written in literalx 1 x 2 x 3 x 4 order, where✓denotes correct and✗ denotes incorrect. Table 10 shows the pattern counts of all the 16 patterns for GPQA, HLE, and LiveCodeBench. C Disclosure of LLM Usage All research contributions in this paper, including the experimental design, the four- condition decomposition framework, the choice of controls, the statistical analysis method- ology, the interpretation of results, and the iterative refinement of the study, were conceived and carried out entirely by the authors. We used LLMs in the following assistive capacities during the preparation of this work: •Code implementation and debugging. LLMs assisted with implementing portions of the experimental pipeline (e.g., API wrapper logic, evaluation scripts, and result aggregation) and with debugging runtime errors. All code was reviewed and validated by the authors. •Visual assets. Certain illustrative icons in Figure 1 were generated with the aid of an LLM-based image tool. All data-driven figures (tables, charts, and plots) were produced by author-written code from raw experimental outputs. 18 Published as a conference paper at COLM 2026 •Writing assistance. LLMs were used to polish prose, improve clarity, and sug- gest structural edits to the manuscript text. All substantive content, claims, and arguments are the authors’ own. 19