Paper deep dive
PatchWrite: One Line, Not One Section -- Compile-Gated, Validity-Preserving Editing for AI-Drafted Manuscripts
Weiwei Yang
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Automated manuscript pipelines often regenerate an entire section to repair a local defect, allowing unrelated metrics and citations to change even when the resulting PDF still builds. PatchWrite instead constrains how candidate edits become committed manuscript states: it reuses bounded EDIT N M editing and rollback, but tightens compilation acceptance with fatal-log checks and adds evidence locks that require every cited key and experimental numeric token to be attested by a reference registry or experimental log. Candidates that fail either check are rejected and the previous HEAD is retained. On a 24-manuscript x 8-fault oracle stress test (768 jobs, evenly split between compile-breaking and content-only faults), whole-slot rewriting mutated an unrelated "12-layer" line in every case (0/192 preserved; numeric Jaccard 0.6667), whereas PatchWrite preserved it in 192/192 cases. Removing the compile gate reduced acceptance to 0, while removing the evidence gate allowed a hallucinated citation to pass. The same pattern held across all eight faults. To test the protocol with generation rather than oracle edits, we reran the 192 jobs with the writer model proposing the edits. The model's candidates were accepted in 75% of cases; nearly all rejections came from one reproducible failure mode in which the model attempted to delete a line using an empty replacement unsupported by the current grammar. Every accepted candidate passed both gates, and 93.75% fixed the injected fault; the remaining cases involved a technically valid but sentence-inappropriate citation and one markup-changing near-miss. In a blind evaluation of sixteen PDF pairs, both raters preferred PatchWrite for preserving lab-grounded facts (C1 Likert 5.0 vs. 2.0), while rating prose quality nearly identically. Logs from 193 in-product drafting tasks show the same classes of failures occurring in practice.
Tags
Links
- Source: https://arxiv.org/abs/2608.23001v1
- Canonical: https://arxiv.org/abs/2608.23001v1
Trouble viewing inline? Open PDF directly →
Full Text
53,390 characters extracted from source content.
Expand or collapse full text
PatchWrite: One Line, Not One Section — Compile-Gated, Validity-Preserving Editing for AI-Drafted Manuscripts Weiwei Yang Affiliation: Solus (https://solus.xin) Email: wxs123x@gmail.com Abstract Automated manuscript pipelines often regenerate a whole section to fix a local defect, and the PDF still builds even though unrelated metrics and citations have quietly moved. PatchWrite instead applies one line-interval EDIT N M (span ≤40≤ 40), committing only when pdflatex is clean and every key and numeric token is attested in a registry or experimental log, and rolling back to the last compiling HEAD otherwise. On a 24-manuscript × 8-fault oracle stress test (768 jobs, split evenly between compile-breaking and content-only faults), whole-slot rewrite mutates an unrelated “12-layer” line in every single case (0/1920/192 preserved, numeric Jaccard 0.66670.6667), while PatchWrite preserves it 192/192192/192 times. Disabling the compile gate drives the accept rate to 00; disabling the evidence gate lets a hallucinated citation through. The pattern holds uniformly across all eight faults. Because oracle repairs test the mechanism rather than generation quality, we reran the same 192192 jobs with the writer model itself proposing the EDIT. It is accepted 75%75\% of the time before a one-line prompt fix, and the shortfall traces almost entirely to one reproducible failure mode: the model tries to delete a line using an empty replacement, which the grammar rejects. Telling the model explicitly to comment out rather than empty the line raises acceptance to 100%100\% and eliminates the fallback, though only 84.9%84.9\% (down from 93.75%93.75\%) of the now-accepted patches pass our strict fault-fixed check, since one fault class is resolved by commenting out the defect marker rather than removing it. Every accepted patch still clears both gates: no hallucinated citation and no unattested number gets through. The author and a second rater, both blind to condition, preferred PatchWrite on all sixteen PDF pairs sampled from the oracle corpus when the criterion is keeping lab-grounded facts intact (C1 Likert 5.05.0 vs. 2.02.0), and scored the prose nearly identically. Logs from 193193 in-product drafting tasks show the same failure modes occurring in the wild. Keywords automated scientific writing ⋅· LaTeX revision ⋅· compile gates ⋅· evidence locks ⋅· multi-agent systems 1 Introduction Large language models can draft conference-shaped LaTeX (9; 7; 13). A remaining failure is non-monotonic revision: a reviewer asks to fix one undefined citation; the agent rewrites Results; the PDF still builds; a lab-grounded number has moved. Dashboards that only plot compile rate will celebrate the new PDF. The unit of mutation was the section, not the line. This is one instance of a broader question the agent community is only now making a first-class research topic: not whether an agent’s output looks fluent, but whether it can be verified to satisfy a domain-specific correctness predicate before being trusted, the focus of an emerging line of work on agent reliability and verification (for instance the NeurIPS 2026 workshop “Who Verifies the Agents? Toward Reliable Agent Development,” https://verify-agents-workshop.github.io/, non-archival at the time of writing). This paper grounds that question in one narrow, high-stakes domain, LaTeX manuscript revision, where the correctness predicate is a compile gate plus an evidence lock rather than a hand-wavable quality score. PaperOrchestra’s refinement loop historically emits a whole-slot fence. A slot is a named placeholder region of the venue template, for instance an entire Method or Results section, that the writer model regenerates in full each round rather than editing in place: compile failure is a continue, success replaces the whole slot (9). Agent Laboratory’s PaperSolver already uses EDIT N M with immediate pdflatex (7), but on English ICLR-shaped shells (the outer document-class/template scaffold a paper is typeset into, as distinct from its content) and with LLM self-scores as quality. EasyPaper generates structured LaTeX from metadata (2) (software, not a peer-reviewed writing operator). We combine bounded interval editing, a compile gate that does not trust a nonstopmode PDF, cite/number locks, and PaperOrchestra’s venue shells into one invariant: the file on disk is always the last compiling, evidence-clean manuscript. PatchWrite sits in the refinement loop (Figure 1). The proposer sees numbered source and emits one EDIT N M. The runtime commits a compiling, evidence-clean T′T or returns T. Illegal or over-long EDITs fall back to the legacy slot rewrite, so the operator is only as strong as the proposer’s interval. PatchWrite is deployed as the revision operator inside Solus (https://solus.xin), a research-writing workspace that formats drafts to a chosen venue’s template. We ask five questions: 1. Does surgical EDIT keep unrelated tokens that slot rewrite mutates? 2. Does the compile gate reject a bad patch rather than a nonstopmode PDF? 3. Does the evidence gate catch a compiling patch that invents a cite key? 4. When the writer model itself proposes the EDIT instead of an oracle, how often is it legal, does it pass the gates, and does the accepted patch actually fix the fault? 5. When two readers compare the resulting PDFs, do they prefer the surgical draft if lab-grounded facts must not move? Questions 1–3 are answered with scripted oracle repairs (24 mini-articles, eight faults spanning both compile-breaking and content-only defects, four conditions) so generation quality is not a confound. Question 4 is answered by rerunning the same 24×824× 8 corpus with the writer model (qwen3.7-plus) proposing the patch itself, single call per job, no oracle (Section 4.2); this is exactly the measurement Questions 1–3 deliberately deferred by isolating the mechanism. Question 5 is answered with a dual-pass rater annotation of sixteen held-out PDF pairs from the oracle corpus (Section 4.3). Product logs from 193193 drafting tasks motivate the gates but are not a substitute for either pass. The contribution here is not a new edit grammar but a stricter, empirically validated acceptance predicate applied to an existing bounded-edit mechanism. Contributions: (i) a compile gate that scans the pdflatex log for fatals rather than trusting a nonstopmode PDF, applied to PaperSolver’s EDIT N M and rollback-on-failure mechanism (tex_patch.py, ContentRefinementAgent); (i) cite and numeric locks reused from PaperOrchestra, combined with (i) for the first time as a single acceptance criterion; (i) an oracle stress protocol at 24×824× 8 scale plus two ablations isolating each gate’s contribution, with table cells copied from evals/results/summary.json; (iv) a live-model companion at the same 24×824× 8 scale across two independent writer models (qwen3.7-plus, Kimi K2.6) measuring parse, compile, gate, and fault-fixed rates for each model’s own EDIT proposals, including a one-line prompt fix that closes one model’s fallback gap while surfacing a new fault-fixed trade-off, cells copied from evals/results/llm_edit_summary.json and evals/results_kimi/llm_edit_summary.json; (v) a sixteen-pair rater inspection with cells copied from raw_materials/human_eval.json. 2 Related Work Tool use. PatchWrite’s EDIT N M is a narrow instance of the broader reasoning-plus-acting paradigm: a model interleaves generation with one constrained, well-typed action rather than free-form output (15), and can be taught to invoke that action only when it improves the outcome (6). Neither line of work ties the action’s acceptance to a compile gate or an evidence lock; PatchWrite adds both as the acceptance criterion for the single tool it exposes. Paper agents. PaperOrchestra maps idea plus experimental log onto venue templates (9): IEEE/arXiv shells, Chinese journal and thesis templates, OpenAlex citation policy, claim–evidence checks. Refinement historically rewrites a slot, not a line. The host workbench already runs those templates in production; PatchWrite replaces only the mutation operator inside ContentRefinementAgent. AI Scientist-v2 searches experiments then writes (13); we compare only the writing/revision module, not discovery or workshop acceptance. EasyPaper is a metadata-to-LaTeX stack with optional typesetting (2). PaperSolver vs. PatchWrite. Agent Laboratory’s PaperSolver supplies the edit grammar we reuse (EDIT N M, compile, rollback) (7). The differences that matter for this paper are locks and templates, not the fence syntax. PatchWrite reuses bounded interval editing and rollback-on-failure from PaperSolver, but tightens the compile predicate — a fatal-log scan rather than treating any nonstopmode PDF as success — and adds an external evidence constraint (cite-key and numeric-token grounding against a registry and experimental log) to the acceptance criterion. Existing bounded edit + stricter compile predicate + evidence locks ⇒ validity-preserving acceptance. PaperSolver scores drafts with LLM reviewers and iterates on English ICLR-shaped shells. PatchWrite does not use an overall LLM score as a halt condition; it binds new keys and empirical numbers to a registry and experimental log, scans the pdflatex log for fatals instead of treating a nonstopmode PDF as success, and runs on PaperOrchestra slot files (with Chinese-venue surrogates in the stress corpus). Table 1 summarizes writing/revision modules only. Code editing. PatchWrite’s constrained patch generation is structurally the same problem SWE-bench poses for source code: given a defect description, produce a patch that resolves it against a fixed correctness signal (3). SWE-agent’s finding, that a narrow, purpose-built edit interface outperforms raw file access for LLM patch generation (14), is the same design bet EDIT N M makes for LaTeX manuscripts instead of a Git repository. The correctness signal differs: SWE-bench checks a held-out test suite, PatchWrite checks pdflatex plus a citation/number registry, so a head-to-head comparison across the two settings would need matched infrastructure we do not build here. Concurrent work: PaperJury. PaperJury argues that “load-bearing safety and completion logic should reside in deterministic orchestration rather than model discretion” (12), the same HEAD-invariant instinct this paper builds around, reached independently and posted earlier in 2026. The target differs: PaperJury pre-submission-hardens human-authored CS papers through a due-process review–verdict–revise–verify loop with anchor-bounded edits and terminal outcomes (invalid-drop, valid-fixable, author-required), evaluated by expert review on held-out vision/NLP/ML papers against four baselines. PatchWrite sits earlier in the pipeline, inside LLM-drafted revision (PaperOrchestra’s refinement loop), with a narrower, more mechanical gate: one line-interval EDIT N M, a pdflatex log scan, and cite/number set-membership against a registry. There is no due-process trial and no author-required terminal state; in their place is an oracle-plus-live-model stress test (Section 4) rather than expert review. The two systems converge on a thesis and diverge on where in the authoring pipeline to enforce it and how heavy the gate should be. We have read only PaperJury’s public abstract, not run its benchmark, and not had it run on ours, so Table 1 deliberately leaves out a cell-by-cell row for it: a comparison built from an abstract alone would overstate what either side has verified about the other. Long-form synthesis. STORM (8) and AutoSurvey (11) outline-then-write encyclopedic or survey text. They are not compile-gated LaTeX editors and do not bind numeric tokens to a lab log. Their quality metrics (citation coverage, outline coherence) answer a different question than “did this revision move a number the injector never touched.” Judges. Agent Laboratory showed automated reviewers over-score generated papers versus humans (7). We therefore report human preference on PDF pairs as a separate table (Section 4.3), and we do not use LLM overall-score as a halt condition. Table 1: Writing/revision modules only. “Compile gate”: failed pdflatex or a fatal log rolls back HEAD. “ZH shells”: does the system ship a Chinese-venue shell (defined in Section 2)? EasyPaper is listed as software. System Edit unit Compile gate Evidence lock ZH shells PaperOrchestra slot fence compile, no slot rollback cite/claim yes PaperSolver EDIT N M yes no (LLM score) no AI Scientist-v2 (writeup) paper draft typesetting experiment search no EasyPaper (software) section / paper optional citation tools n/a PatchWrite EDIT N M ≤ 40 yes + log scan cite + numbers surrogates 3 Method HEAD invariant. Let T be a compiling .tex file. PatchWrite replaces T only if every enabled gate returns true. Failed candidates leave the bytes on disk unchanged. Figure 1 is the control flow inside ContentRefinementAgent when use_tex_patch = True (disable with PATCHWRITE = 0). Figure 1: PatchWrite in the PaperOrchestra refinement loop. An illegal or over-long EDIT falls back to legacy slot rewrite. Compile or evidence failure rolls back to HEAD; only both gates passing commits T′T . Numbered view. The proposer sees 1-based N | text (number_lines) and may emit one fence‘EDIT N M replacement lines ‘ with 1≤N≤M1≤ N≤ M and M−N+1≤40M-N+1≤ 40. The interval is replaced by the body (apply_edit). Over-long, empty, non-integer, or past-EOF ranges are rejected before compile. Bare EDIT N M is also parsed, matching PaperSolver (7). Runtime (try_patch). The runtime takes two inputs: the current compiling document T, and cmd, a candidate edit string supplied by a model or an oracle. It proceeds as follows. 1. Parse one EDIT block (the function parse_edit). On failure, return T unchanged. 2. Apply the closed-interval replacement (apply_edit) to obtain a candidate document T′T . 3. If the compile gate is on: write a unique jobname, delete stale .aux/.pdf/.log files, and run PaperOrchestra’s compile_latex. Accept only a PDF larger than 64 bytes whose log contains none of emergency stop, fatal error, ! latex error, or a forced-fail marker. pdflatex -interaction=nonstopmode often still writes a PDF after a syntax error, so “a PDF exists” is not by itself a passing compile gate. 4. If the evidence gate is on: a cite key is allowed if it appears in the bibliography, the citation map, or HEAD itself; any other key is flagged unknown_cite:*. Every empirical number is checked against the experimental log. A violation of either check rolls the candidate back. 5. On success, commit T′T as the new HEAD; on any failure, HEAD stays T. Fallback. If no legal EDIT is parsed or a gate fails, the loop falls back to slot-fence rewrite so jobs still finish. That fallback re-introduces whole-slot mutation; production logs should count it. Whole-slot REPLACE remains a human-explicit escape hatch. Line numbers are regenerated each round after a committed insert; otherwise N,MN,M drift. Worked example. 9 |The encoder uses 12 layers and a frozen tokenizer~ smith2020. 10 | Results 11 |Accuracy was 91.4 compared with the baseline 68.2~ lee2019. A fake_cite injector rewrites line 11 to fake2024. The oracle emits EDIT 11 11 with the original sentence. Slot rewrite also restores a valid Results line, but it also changes line 9 to “16 layers.” Both PDFs build; only one is monotonic (Figure 4). 4 Experiments 4.1 Oracle revision stress Twenty-four mini-articles make up the corpus (16 English, 8 Chinese-venue pdflatex surrogates with ASCII titles). Each HEAD contains “The encoder uses 12 layers …” and Results numbers drawn from a per-document log. Eight scripted faults split into two classes. Content-class faults leave the broken HEAD compiling: an undefined fake2024, a 99.9 absent from the log, a TODO line, and swapped accuracy/baseline numbers. Compile-class faults break pdflatex outright: an unclosed $, an unmatched brace next to a , an unclosed quote environment, and an undefined control sequence. Four conditions times 192192 copies gives 768768 jobs. • po_slot: scripted whole-block rewrite that does invert the fault and also rewrites “12 layers”→“16 layers”. This is a PaperOrchestra-style slot mutation, not a live LLM rewrite of a conference paper. • patchwrite: one-line EDIT with both gates on. • patchwrite_nocompile: replacement patchwrite-forced-fail. • patchwrite_nogate: surgical fix plus hallucinated2024, evidence gate off. Metrics: accept, compile_ok, gate_ok, cite/numeric Jaccard vs pre-fault HEAD, layers_preserved. Table 2 cells match evals/results/summary.json after wiping evals/results/work. Table 2: Oracle revision stress (24×824× 8 faults =192=192 per condition). Jaccard vs pre-fault HEAD. The 0.000.00 vs. 1.001.00 layer split is by construction of the oracles: it tests whether gates preserve HEAD, not whether an LLM finds the patch. Condition n Acc. Comp. Gate Cite J. Num. J. Layers PO-slot rewrite 192 1.00 1.00 1.00 1.00 0.6667 0.00 PatchWrite 192 1.00 1.00 1.00 1.00 1.00 1.00 PW, no compile gate 192 0.00 0.00 — 1.00 1.00 1.00 PW, no evidence gate 192 1.00 1.00 0.00 0.6667 1.00 1.00 Figure 2: Oracle stress (n=192n=192 per bar). Left: accept rate. Middle: numeric Jaccard vs pre-fault HEAD. Right: whether the unrelated “12 layers” sentence survives. Compile-off never replaces HEAD; evidence-off keeps layers but drops cite Jaccard (not plotted here; Table 2). What Table 2 shows. Slot rewrite compiles at a 1.001.00 rate while layers_preserved sits at 0.000.00. The 0.66670.6667 numeric Jaccard is exactly the 12→1612→ 16 token swap. Compile-off never replaces HEAD, so accept stays at 00. Evidence-off keeps the layers line but drops cite Jaccard to 2/32/3, once smith2020 and lee2019 are joined by hallucinated2024. The pattern is identical on every one of the eight faults (n=24n=24 per fault-condition cell, Figure 3), on the content-class/compile-class split (n=96n=96 per condition each), and on the EN/ZH subsets (n=128n=128/6464 per condition); ZH here means a shared pdflatex engine, not ctex camera copies. Compile-class faults are mechanically the harder case for the oracle, since the broken HEAD does not build at all, so patchwrite has to restore compilation and stay surgical at the same time. It still lands at 1.001.00 on every preservation metric there. Mean latexmk wall time is ≈0.22≈ 0.22 s in every condition (Appendix C); latency is not the claim. Figure 3: layers_preserved broken out by fault (n=24n=24 per bar). PO-slot is 0.000.00 and PatchWrite is 1.001.00 on all eight faults, including the four compile-class faults added to widen the corpus beyond the original fake_cite/bad_number/broken_math/todo set. Case: en_vision_01 / bad_number. Figure 4 shows the two compiling PDFs after the injector wrote “Accuracy was 99.9”. Both oracles restore 91.491.4. Only the slot rewrite also rewrites the Method sentence. A compile-rate dashboard cannot tell them apart. Figure 4: Side-by-side source after repairing bad_number on en_vision_01. Both PDFs build. PO-slot changes encoder depth 12→1612→ 16; PatchWrite EDIT 11 11 does not. 4.2 Live-model EDIT proposals Table 2 measures the operator with the correct patch already known; it says nothing about how often a real writer model finds a legal patch on its own. We reran the same 24×824× 8 corpus with the oracle replaced by one live call to the project’s own writer model (Alibaba Cloud DashScope, model id qwen3.7-plus). Given the numbered broken source, the experimental log, and a one-line reviewer or compiler note describing the defect (not the fix), the model has to emit its own EDIT N M, single-shot. The prompt follows the same numbered-source-plus-single-EDIT-fence contract as ContentRefinementAgent._revise_via_tex_patch’s production prompt, though it uses a standalone system prompt and a synthetic one-line defect note rather than a live reviewer pass and the full production system instruction. So this measures the EDIT-fence contract under realistic constraints, not a byte-identical replay of production. The resulting patch runs through the same compile and evidence gates as Table 2. Cells: evals/results/llm_edit_summary.json / llm_edit_slices.json (n=192n=192; code llm_edit_stress.py, https://github.com/Baiang/editnm). One thing shapes how to read every number below: the OpenAI-compatible DashScope call path strips any temperature parameter, so each row is a single stochastic draw at the server default rather than a repeated or averaged measurement (unlike Table 2’s oracle cells, which are bit-identical on rerun). The clean fault-level split (six faults at 1.001.00/1.001.00, two at 0.000.00/0.000.00) reads as systematic per-fault behavior rather than a coin flip, but that is a reading, not a confirmed replication. Scope and caveats for this section are collected once, in full, in Section 6. Table 3: Live-model EDIT proposals, n=192n=192 (same corpus as Table 2, oracle replaced by qwen3.7-plus). fault_fixed and layers_preserved are conditioned on accepted. Metric Value Parse ok (legal EDIT emitted) 0.750.75 Compile ok, of parsed 1.001.00 Gate ok, of compiled 1.001.00 Accept rate 0.750.75 Fallback rate 0.250.25 Fault fixed, of accepted 0.93750.9375 Layers preserved, of accepted 0.94440.9444 Figure 5: Left: the six headline rates from Table 3. Right: parse and accept rate by fault (n=24n=24 each). todo and undefined_cmd are 0.000.00 on both; the other six faults are 1.001.00 on both. What Table 3 shows. Six of the eight faults, fake_cite, bad_number, broken_math, unmatched_brace, unclosed_env, and swapped_acc, parse and get accepted every single time. Every parsed patch that reaches the gates also compiles and clears the evidence gate 100%100\% of the time (gate_ok =1.00=1.00 of compiled), matching zero hallucinated cites or unattested numbers. The 25%25\% overall fallback rate is not spread across the eight faults at all; it belongs entirely to the other two. todo and undefined_cmd fall back 24/2424/24 times each, for exactly the same reason, and none of the remaining six faults contributes a single fallback job: 4848 of the 4848 fallback jobs are todo or undefined_cmd, and 00 of the remaining 144144 jobs fall back. The reason is specific enough to name. The model’s instinct is to delete the injected line, so it submits EDIT N N with an empty body (‘EDIT 10 10 ‘). parse_edit rejects an empty replacement (empty_replacement), because the grammar has no delete verb, only replace-with-at-least-one-line, and a one-line reviewer note never tells the model the workaround the oracle uses: replace the line with a % comment. That single gap accounts for the entire observed fallback mass, all 4848 of the 48/19248/192 fallback jobs, not merely a large share of it. It reads as a fixable prompt-contract issue rather than evidence that gate architecture explains fallback in general. (broken_math is accepted every time; its shortfall shows up in fault_fixed instead, discussed next.) fault_fixed of accepted is 0.93750.9375, below 1.001.00 for two distinct reasons visible in the per-fault cells. broken_math accepts 24/2424/24 but only fixes 16/2416/24: the miss rewrites The encoder uses 12 layers to The encoder uses $12$ layers instead of removing the stray $. That still compiles and still passes the evidence gate, since the numeric token 12 is unchanged, but it does not byte-match the original prose. It is a compiling, gate-clean patch that is nonetheless not the identity repair. fake_cite fixes 23/2423/24; the miss replaces fake2024 with smith2020, a real key defined in this file’s own bibliography, so the evidence gate accepts it. But smith2020 is the Method-encoder citation, not the correct key for this Results-accuracy sentence. That is the evidence gate’s real boundary, stated with an example rather than an assertion: it checks that a cite key is attested, not that it is the correct attested key for this specific claim. Mean latency rises from ≈0.22≈ 0.22 s (Table 2, compile only) to ≈1.38≈ 1.38 s (LLM call plus compile). The model call now dominates wall time, not the gate. Section 5 returns to what this result does and does not license us to claim. Table 3b: Live-model EDIT proposals, second model (Kimi K2.6, n=192n=192; same corpus, prompt-fixed harness). Metric Qwen3.7-plus (prompt-fixed) Kimi K2.6 Parse ok 1.001.00 0.8750.875 Compile ok, of parsed 1.001.00 1.001.00 Gate ok, of compiled 1.001.00 1.001.00 Accept rate 1.001.00 0.8750.875 Fallback rate 0.000.00 0.1250.125 Fault fixed, of accepted 0.8490.849 0.91070.9107 Layers preserved, of accepted 0.97920.9792 0.91670.9167 Mean latency (s) 1.56511.5651 26.8726.87 4.3 Rater inspection of sixteen PDF pairs Oracle metrics do not say whether a reader would treat the two compiling PDFs as equally submission-ready. We therefore annotated sixteen pairs drawn from the stress work directory: en_vision_01 and zh_med_09 × all eight faults, up from four in an earlier four-fault pass (protocol and cells: raw_materials/human_eval.json). The author and a second rater scored each pair with condition names hidden (sheets labeled A/B). Rubric: • C1 factual (1–5). Consistency with HEAD: encoder depth 12 and log-attested accuracy. • C2 defect repaired (1–5). The injected fault is gone. • C3 stub prose (1–5). Readability of the mini-article (both drafts are stubs). • Preference. Which PDF is more submission-ready if lab-grounded facts must not move. This is a designed rater dual-pass on real oracle PDFs, not a multi-PhD panel and not LLM-as-judge, and it is disclosed at that scale rather than dressed up as a validated study: standard guidance for human evaluation of generated text treats a two-rater, non-independent pool as exploratory rather than confirmatory (10), and raw preference agreement (here 16/1616/16) is not a substitute for a chance-corrected agreement statistic computed over a larger, independent rater pool (1). Likert means for C1/C2 follow the visible 12→1612→ 16 rewrite (every PO-slot file) versus its absence (every PatchWrite file). C3 is near-tied by construction of the stub corpus; one annotator marked a single PatchWrite stub one point lower on prose, the same single disagreement as the earlier eight-pair pass, now diluted over twice as many pairs (3.94→3.973.94→ 3.97). Table 4: Rater dual-pass on sixteen oracle PDF pairs. Preference agreement 16/1616/16. Likert is 11–55. Cells match human_eval.json. Per-pair rows: Appendix E. Exploratory two-rater pool (author plus one unpaid rater; Section 4.3), not a validated human-evaluation study. PO-slot PatchWrite Preferred as submission-ready (pairs) 0/160/16 16/1616/16 Mean factual errors listed per PDF 1.001.00 0.000.00 C1 factual (mean Likert) 2.002.00 5.005.00 C2 defect repaired (mean Likert) 5.005.00 5.005.00 C3 stub prose (mean Likert) 4.004.00 3.973.97 Figure 6: Left: pairwise preference (n=16n=16). Right: mean Likert. Both oracles repair the injected defect (C2); readers split on facts (C1) and therefore on preference. Prose of these stubs is a near-tie. What Table 4 shows. C2 being tied at 5.05.0 is the interesting negative: slot rewrite does fix the local bug, which is why compile rate alone is a bad halt condition. C1 and preference isolate the silent 12→1612→ 16 change. These sixteen pairs measure whether a reader notices a monotonicity failure that automatic Jaccard already flags; scope and caveats for this annotation are collected in Section 6. 4.4 In-the-wild workbench logs These are not operator-level blind SxS. They are product artifacts from the host writing workbench (Postgres task store plus per-task JSON), exported 2026-08-23 (Table 5; Figure 7; raw_materials/in_the_wild.json). The author is affiliated with Solus (https://solus.xin), the platform these logs are drawn from and where PatchWrite is deployed as its revision operator; see Section 7 for the full conflict-of-interest statement. 193193 paper tasks and 240240 revisions were in the store. Among 157157 evidence-gate snapshots, 4545 fail (28.7%28.7\%); the 118118 recorded issues are all table_number_not_in_log. Among 156156 LaTeX sanity snapshots, 4040 files have unmatched braces and 77 orphan cite tokens appear. Nine coach reviews (Solus’s structured, per-round reviewer-feedback pass) flag placeholders, compile problems, truncated quotes, or claim–evidence gaps. Fifteen share-link comments (freeform comments left by viewers of a paper’s shareable preview link, a separate product feature from the coach) exist on 1616 links; they are UI-test strings and are not coded as pairwise preference. The logs show that unattested numbers and broken TeX already occur in real drafting jobs, the same classes the oracle injects, without any claim that users preferred PatchWrite PDFs in the product UI. Table 5: In-product drafting logs (not a 6–8 pair blind SxS). Counts from task artifacts and the paper-task database. Signal Denominator Count Paper tasks / revisions — 193 / 240 Evidence-gate fail (unsupported table number) 157 snapshots 45 fail (118 issues) Unmatched TeX braces 156 sanity files 40 Orphan cite tokens 156 sanity files 7 Share-link notes (uncoded) 16 links 15 items Figure 7: Rates from in-product snapshots. Evidence-gate failures and unmatched braces are common; orphan cites are rarer. Coach foci (7/97/9 reviews) are a small, separately labeled set. 5 Analysis False fluency. Compile rate =1.00=1.00 is what a dashboard would celebrate. The regression is in a line the injector never touched. PatchWrite’s result is the same compile rate with layers preserved. The sixteen-pair inspection is the reader-facing restatement of that split: C2 tied, C1 and preference not tied. Widening the fault set does not change the story. We doubled the corpus (12 to 24 manuscripts) and doubled the fault set (4 to 8) in two directions rather than one. Three of the four additions, unmatched_brace, unclosed_env, and undefined_cmd, are compile-class: the broken HEAD fails pdflatex, so the oracle repair has to do more work than a content-only fix, restoring compilation rather than merely restoring a token. The fourth, swapped_acc, is a second content-class fault that exercises the numeric-Jaccard metric differently from bad_number, since two tokens swap instead of one being replaced. None of the four new faults moves any rate in Table 2 or Figure 3: layers_preserved stays 0.000.00/1.001.00 and accept_rate follows the same per-condition pattern on every one of the eight faults taken separately (n=24n=24 each). The compile-class faults are mechanically harder, since the pre-repair file will not build at all, yet they show exactly the same split as the original four content-class faults. That is evidence the result comes from the gate architecture, not from which particular injector happened to be scripted first. We read this as ruling out one specific threat to validity, that four hand-picked faults happened to favor PatchWrite, not as a claim that fault diversity is now exhausted. Section 6 still flags that all faults are scripted rather than adversarially searched. The nonstopmode trap. If the predicate were “PDF exists,” patchwrite_nocompile would accept: pdflatex still writes a file after . Scanning the log is the difference between a compile gate and a file-existence gate. Production wrappers that call latexmk in nonstopmode and then glob for *.pdf will silently promote broken math into HEAD. Fallback is the real production risk. Oracle jobs always emit a legal short EDIT, so they never take the slot-fallback branch. A model that dumps Results into EDIT 1 80 is rejected on span and gets slot-rewritten instead, which is the regression documented in Table 2. Table 3 now reports a live-LLM fallback rate rather than deferring it: 25%25\% of 192192 jobs, concentrated rather than diffuse. todo and undefined_cmd fall back every single time (24/24 each) because the model tries to delete the injected line with an empty EDIT body, and the grammar has no delete verb. That single, named gap is the entire observed fallback mass, all 4848 of the 4848 fallback jobs; the other six faults contribute zero. Two mitigations belong in the runtime rather than the prompt. First, count fallbacks per job and surface them in the UI, broken out by cause (parse vs. compile vs. gate), the way Table 3 does. Second, regenerate numbered source after every committed insert so subsequent N,MN,M stay aligned. A third, cheaper fix follows directly from Section 4.2: tell the proposer explicitly that deletion means replacing with a comment, not replacing with nothing. We reran Section 4.2’s live-LLM suite with one added sentence in the todo and undefined_cmd defect notes, telling the proposer that deletion means replacing the line with a % comment rather than leaving it empty. The overall accept rate moved from 0.750.75 to 1.001.00 and fallback dropped from 25%25\% to 0%0\%, closing the gap the abstract originally reported; todo alone went from 0/240/24 to a clean 24/2424/24 both accepted and fixed. But fault_fixed_rate_of_accepted moved the other way overall, from 0.93750.9375 to 0.8490.849: all 2424 accepted undefined_cmd patches compile clean and pass the evidence gate, yet none satisfies our string-presence check for “fixed,” because the model deleted by commenting out % rather than replacing it with the oracle’s clean % removed; the same defect signature survives in the text, inert but present. This is the same false-fluency pattern flagged earlier in this section for compile rate: closing the accept-rate gap surfaced a second gap our own fault_fixed metric was designed to catch. A second model, a different seam. A second model surfaces the same shape of failure, at a different fault. We reran the prompt-fixed harness (same corpus, gates, and defect notes as the fix above, unchanged) against a second writer model, Kimi K2.6 (kimi-k2.6, via Moonshot’s OpenAI-compatible API; Table 3b). It accepts 87.5%87.5\% of proposals and, of those, fixes the fault 91.1%91.1\% of the time, preserving layers 91.7%91.7\% of the time; mean latency is 26.926.9 s per call (reasoning-heavy generation), against Qwen3.7-plus’s much lower per-call latency. Fallback is again concentrated rather than diffuse, but at a different fault than before: 2323 of the 2424 fallback jobs are unclosed_env, a fault the prompt fix’s two added sentences never mentioned, and on which Qwen3.7-plus never fell back even before that fix. The one-line prompt fix that closed Qwen’s gap is therefore a fix for the specific fault types one model happened to fail on, not a general solution to the grammar’s missing delete verb; a different model exposes the same underlying gap at a different seam. A second, independent near-miss replicates the earlier fault_fixed boundary: broken_math accepts 2323 of 2424 but fixes only 99, because Kimi sometimes writes “$12$ layers” instead of “12 layers”, compiling and gate-clean, but not a byte match, the same class of miss as Qwen’s undefined_cmd comment-out. Two models independently hitting the same metric boundary is stronger evidence that the boundary is in fault_fixed’s strict string check, not particular to one model’s phrasing habits. Unlike DashScope’s silent parameter-stripping, Moonshot’s API explicitly rejects any temperature other than 1.01.0 for kimi-k2.6, so variance here can only be estimated from repeated draws at that fixed setting, not from a pinned low temperature. Stale aux. An early eval run failed until evals/results/work was wiped; latexmk reused .aux across colliding jobnames. Unique jobnames plus deleting stale auxiliaries are part of the compile gate, not an implementation footnote. Evidence-gate conservatism. In-product snapshots fail most often on table_number_not_in_log (118118 issues across 4545 failing files). That is a true positive if the number is invented, and a false positive if the log is incomplete. PatchWrite will refuse a compiling patch in the second case. Authors who want the number in the PDF must first put it in the log; that is the intended workflow, not a compile workaround. Attested is not correct. The fake_cite miss in Section 4.2 is a real example of the evidence gate’s boundary, not a hypothetical one: run_evidence_gates checks set membership, whether a key is defined anywhere in the document, which a citation log can attest cheaply. It cannot check that a key is the semantically correct citation for a specific sentence, which would need a much richer claim-to-source index than tex_patch.py builds today. It is also a different, cheaper problem than external citation-hallucination detection, which grounds a cite against real scholarly metadata such as CrossRef, Semantic Scholar, and OpenAlex, rather than a document’s own bibliography (4), or checks that a whole .bib entry’s fields (author, venue, year) match the real record rather than just its key (5). PatchWrite’s gate would accept a citation to a real paper’s key even if that paper does not say what the sentence claims, and it would accept a fabricated-but-locally-defined .bib entry; both are out of scope for a set-membership check and would need exactly that kind of external grounding. What the human table is not. Given the protocol in Section 4.3, sixteen oracle pairs will always prefer the draft that keeps “12 layers” if that is the rubric. Table 4 checks that the Jaccard split is visible to readers; it does not measure venue-level prose. A live-LLM SxS judging the model’s own accepted-but-imperfect patches from Section 4.2 (the broken_math and fake_cite misses would be a pointed test case) remains future work; scope for this section is collected in Section 6. 6 Limitations • Table 2 is oracles, not writers, by design. It assumes the correct patch is known so the gates can be isolated from generation quality. Table 3/3b are the writer-model companions: two models (qwen3.7-plus, Kimi K2.6), each single-shot on this same synthetic corpus, no repeats for either model. The two models concentrate their fallback on different fault types (todo/undefined_cmd for Qwen before its prompt fix, unclosed_env for Kimi), suggesting the prompt-level fix is reactive rather than general. Read this as two data points, not a general LLM-proposed-EDIT success rate. • Short files. Mini articles, not 8–10 page venue PDFs. Full IEEE/ctex templates exist in the host tree and were not this table’s compile target. • Chinese surrogates share pdflatex with English; they are not camera ctex theses. • Scripted faults, not adversarial ones. Eight faults across two classes (content-class, compile-class) rule out a four-fault coincidence, but all eight are hand-written injectors, not a search over the space of possible defects or faults sampled from a live LLM’s actual error distribution. • Annotation scale. Two raters × sixteen pairs (Section 4.3), judging oracle PDFs, not the live model’s own (sometimes imperfect) patches from Section 4.2. This is what the end of Section 4.3 and “What the human table is not” (Section 5) both point to. Product share-link comments were UI tests and were not recoded as preference. • Fallback measured once, not tracked in production. Section 4.2 gives a live-LLM fallback rate for the first time (25%25\% before the one-line prompt fix in Section 5 closes it to 0%0\%, one model, this corpus), but the runtime does not yet log per-cause fallback rate (parse vs. compile vs. gate) on real drafting jobs the way Table 3 does on the oracle corpus. 7 Ethics PatchWrite does not create experimental facts. Generating a domain paper without a lab log, or inventing citations, is misconduct. A compile gate can still ship a fluent falsehood if the evidence gate is off or the log is fabricated. Authors remain responsible for every claim. The system design and experimental protocol are the author’s own; LLM assistance was used, under the author’s direction, to help write and debug the implementation (tex_patch.py, revision_stress.py, llm_edit_stress.py, corpus.py) and to draft parts of this manuscript. Table cells are copied from summary.json, llm_edit_summary.json, in_the_wild.json, and human_eval.json. Table 3 additionally uses an LLM as the object of measurement, not just an assistant: every cell there is the recorded behavior of one live call to qwen3.7-plus per row, not a hand-edited or cherry-picked transcript. The human table is rater inspection of artifacts we generated, disclosed as such; it is not a hired-annotator study. The second rater (A2) is a friend of the author, not a co-author and not compensated; both raters scored blind to which system produced each PDF (Section 4.3). Conflict of interest: the author is affiliated with Solus (https://solus.xin), a research-writing platform; PatchWrite is deployed there as its revision operator, and the in-product logs in Section 4.4 are drawn from that platform. 8 Conclusion Writing agents should not treat “the section compiled” as “the paper did not change underfoot.” PatchWrite makes the mutation a bounded interval and compile-plus-evidence the only path from HEAD to HEAD. On a 24×824× 8 oracle stress test spanning content-class and compile-class faults, that is the difference between layers_preserved =0.00=0.00 and 1.001.00, reproduced identically at 4×4× the scale of an earlier four-fault pass. Replacing the oracle with the writer model itself on the same 192192 jobs shows the gates hold under real generation too: every parsed, compiling patch passes the evidence gate; the 25%25\% fallback rate traced to one named prompt gap and closed to 0%0\% once the proposer is told explicitly to comment out rather than empty a deleted line, though that same fix leaves 84.9%84.9\% (down from 93.75%93.75\%) of now-accepted patches meeting our strict fault-fixed check rather than open-ended unreliability. On sixteen PDF pairs, readers prefer the surgical draft 16/1616/16 when facts must not move, while both drafts repair the local defect. In-product logs show unattested numbers and broken TeX already appearing in real jobs. The next measurement is LLM-proposed EDITs on full venue-length templates with multiple models, not this mini-article corpus with one model, together with a live-LLM comparison judging the model’s own accepted-but-imperfect patches rather than just the oracle’s. Beyond LaTeX manuscripts. Nothing in the argument above is specific to LaTeX. Strip away pdflatex and the citation registry and what is left is a three-part pattern: a bounded edit (a small, syntactically checkable change rather than a free-form rewrite), a domain-specific validity predicate (a mechanical pass/fail check, not a fluency score), and rollback to the last-known-good state on any failure. The same shape describes a code-review agent that edits one function and runs the test suite before keeping the change, a contract- or compliance-document assistant that edits one clause and runs a compliance checker before keeping the change, or a database-schema-migration agent that edits one migration step and runs a schema check before keeping the change. We have not built or evaluated any of these; the claim here is only that the pattern transfers in principle, not that we have shown it works. Whether bounded-edit-plus-predicate-plus-rollback holds up once pdflatex is replaced by a test suite, a compliance checker, or a migration linter, each with its own failure modes and cost of a false accept, is an open question this paper does not answer. Reproducibility. The mechanism (tex_patch.py) and both eval harnesses (revision_stress.py, llm_edit_stress.py), plus the 24-manuscript corpus (corpus.py), are public under the MIT license: https://github.com/Baiang/editnm. This is a standalone extraction with no dependency on the private codebase PatchWrite ships inside; see that repository’s README for the two components (the compile step and the numeric-evidence check) that were reimplemented clean-room rather than copied over. Tests: pytest tests/ (network-free). Oracle eval (Table 2): python revision_stress.py --out results. Live-model eval (Table 3; calls qwen3.7-plus via DashScope, needs DASHSCOPE_API_KEY, real billed calls): python llm_edit_stress.py --out results. Second-model eval (Table 3b; calls Kimi K2.6 via Moonshot’s OpenAI-compatible API, real billed calls): OPENAI_API_KEY=<key> OPENAI_BASE_URL=https://api.moonshot.cn/v1 python llm_edit_stress.py --model kimi-k2.6 --max-tokens 3000 --out results_kimi (the script’s client is OpenAI-compatible-endpoint-generic, not Moonshot-specific; --max-tokens is raised because kimi-k2.6 spends most of a small budget on hidden reasoning tokens before any visible completion, unlike qwen3.7-plus). Figure generation, the human-annotation sheet, and the in-the-wild export (Table 5) use internal tooling not included in that repository. PatchWrite runs in production as part of Solus (https://solus.xin). Appendix A Corpus titles Generated by corpus.py (https://github.com/Baiang/editnm). English (16): Patch-Level Robustness in Vision Encoders; Instruction Tuning Without Answer Leakage; Streaming ASR with Delayed Token Loss; Two-Tower Retrieval with Hard Negatives; Message Passing with Residual Gates; Adaptive Clipping for Noisy Gradients; Offline RL with Conservative Q Backup; Variant Effect Prediction on Sparse Panels; Dense Captioning with Region-Level Losses; Non-Autoregressive Translation with Latent Alignment; Self-Supervised Speech Tokens for Speaker ID; Graph Networks for Reaction Yield Prediction; Long-Context Retrieval for Case Law; Satellite Time Series for Crop Mapping; Gaze-Aware Interface Adaptation; Membership Inference on Fine-Tuned LLMs. Chinese-venue surrogates (8, ASCII titles): Variant Effect Prediction (Chinese venue surrogate); Retrieval Augmented Classroom QA (Chinese venue surrogate); Contrastive Pretraining for Industrial Time Series (CN surrogate); Compile-Gated Scholarly Revision (Chinese thesis surrogate); Legal Judgment Prediction (Chinese venue surrogate); Crop Mapping from Satellite Series (CN surrogate); Contrastive Learning for Credit Risk (CN surrogate); Compile-Gated Editing for CJK Theses (CN thesis surrogate). The last eight EN and last four ZH titles were added to widen the corpus from the original 8 EN / 4 ZH. Cite keys smith2020, lee2019 are fixtures. Appendix B Fault recipes Implemented in revision_stress.py (no LLM), eight total. Content-class (broken HEAD still compiles): fake_cite: lee2019→ fake2024. bad_number: Accuracy token → 99.9. todo: a TODO line before Results. swapped_acc: the accuracy and baseline numbers trade places in the Results sentence. Compile-class (broken HEAD fails pdflatex): broken_math: unclosed $. unmatched_brace: an extra inserted next to a . unclosed_env: an unterminated quote inserted before Results. undefined_cmd: an undefined control sequence ( ) inserted before Results. Oracle PatchWrite inverts only the span in every case. Oracle po_slot also rewrites “12 layers” to “16 layers” regardless of which fault it is repairing. Appendix C Latency Mean wall time including latexmk (same 192-job cells as Table 2): PO-slot 0.22360.2236 s; PatchWrite 0.22090.2209 s; no compile gate 0.21770.2177 s; no evidence gate 0.21850.2185 s. All conditions invoke latexmk; latency is not the claim. These are wall-clock numbers from one machine under ordinary load, not a controlled benchmark. A repeat run of this same suite measured 0.21000.2100/0.20950.2095/0.20920.2092/0.21160.2116 s, and the original 48-job four-fault pass measured ≈0.23≈ 0.23 s; all three runs agree to within ±0.01± 0.01–0.020.02 s and none changes the ordering or the qualitative claim (compile dominates; the gates add negligible overhead). Appendix D Per-fault uniformity Each fault × condition cell has n=24n=24 (up from n=12n=12 in the original four-fault pass). For every one of the eight faults (fake_cite, bad_number, todo, swapped_acc, broken_math, unmatched_brace, unclosed_env, undefined_cmd), po_slot has accept =1.00=1.00, layers =0.00=0.00, numeric Jaccard =0.6667=0.6667; patchwrite has all preservation metrics =1.00=1.00; patchwrite_nocompile has accept =0.00=0.00; patchwrite_nogate has cite Jaccard =0.6667=0.6667 (Figure 3). Content-class and compile-class faults each aggregate to n=96n=96 per condition with the identical pattern. EN 128128 jobs, ZH 6464 jobs per condition; same pattern on both subsets. Source: evals/results/slices.json. Appendix E Per-pair rater annotation Sixteen pairs (up from eight); two raters (A1, A2); preference always PatchWrite. Every PO-slot PDF lists one factual error (encoder depth 12→1612→ 16). C3 is stub-prose Likert; A2 scored zh_med_09/todo PatchWrite prose as 33 rather than 44 (the sole disagreement, unchanged from the original eight-pair sheet). Table 6: Per-pair sheet. Pref. == PatchWrite. Factual errs. counted vs HEAD. Paper Fault Errs PO / PW C1 PO / PW C2 both Pref. A1/A2 en_vision_01 fake_cite 1 / 0 2 / 5 5 PW / PW en_vision_01 bad_number 1 / 0 2 / 5 5 PW / PW en_vision_01 broken_math 1 / 0 2 / 5 5 PW / PW en_vision_01 todo 1 / 0 2 / 5 5 PW / PW en_vision_01 unmatched_brace 1 / 0 2 / 5 5 PW / PW en_vision_01 unclosed_env 1 / 0 2 / 5 5 PW / PW en_vision_01 undefined_cmd 1 / 0 2 / 5 5 PW / PW en_vision_01 swapped_acc 1 / 0 2 / 5 5 PW / PW zh_med_09 fake_cite 1 / 0 2 / 5 5 PW / PW zh_med_09 bad_number 1 / 0 2 / 5 5 PW / PW zh_med_09 broken_math 1 / 0 2 / 5 5 PW / PW zh_med_09 todo 1 / 0 2 / 5 5 PW / PW zh_med_09 unmatched_brace 1 / 0 2 / 5 5 PW / PW zh_med_09 unclosed_env 1 / 0 2 / 5 5 PW / PW zh_med_09 undefined_cmd 1 / 0 2 / 5 5 PW / PW zh_med_09 swapped_acc 1 / 0 2 / 5 5 PW / PW References Artstein and Poesio (2008) R. Artstein and M. Poesio Inter-coder agreement for computational linguistics. Computational Linguistics 34 (4), p. 555–596. Cited by: §4.3. EasyPaper (2026) EasyPaper EasyPaper: a multi-agent academic paper generation system. Note: https://github.com/PinkGranite/EasyPaperSoftware repository Cited by: §1, §2. Jimenez et al. (2024) C. E. Jimenez, J. Yang, A. Wettig, S. Yao, K. Pei, O. Press, and K. Narasimhan SWE-bench: can language models resolve real-world GitHub issues?. In International Conference on Learning Representations, Cited by: §2. Khajavi et al. (2026) K. Khajavi, S. Sadeghi, R. Adhikari, and A. Tessier CiteCheck: retrieval-grounded detection of LLM citation hallucinations in scientific text. arXiv preprint arXiv:2605.27700. Cited by: §5. Rao and Callison-Burch (2026) D. Rao and C. Callison-Burch BibTeX citation hallucinations in scientific publishing agents: evaluation and mitigation. arXiv preprint arXiv:2604.03159. Cited by: §5. Schick et al. (2023) T. Schick, J. Dwivedi-Yu, R. Dessì, R. Raileanu, M. Lomeli, L. Zettlemoyer, N. Cancedda, and T. Scialom Toolformer: language models can teach themselves to use tools. In Advances in Neural Information Processing Systems, Cited by: §2. Schmidgall et al. (2025) S. Schmidgall, Y. Su, Z. Wang, X. Sun, J. Wu, X. Yu, J. Liu, M. Moor, Z. Liu, and E. Barsoum Agent laboratory: using LLM agents as research assistants. arXiv preprint arXiv:2501.04227. Cited by: §1, §1, §2, §2, §3. Shao et al. (2024) Y. Shao, Y. Jiang, T. A. Kanell, P. Xu, O. Khattab, and M. S. Lam Assisting in writing Wikipedia-like articles from scratch with large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Cited by: §2. Song et al. (2026) Y. Song, Y. Song, T. Pfister, and J. Yoon PaperOrchestra: a multi-agent framework for automated AI research paper writing. arXiv preprint arXiv:2604.05018. Cited by: §1, §1, §2. van der Lee et al. (2021) C. van der Lee, A. Gatt, E. van Miltenburg, and E. Krahmer Human evaluation of automatically generated text: current trends and best practice guidelines. Computer Speech & Language 67, p. 101151. Cited by: §4.3. Wang et al. (2024) Y. Wang, Q. Guo, W. Yao, H. Zhang, X. Zhang, Z. Wu, M. Zhang, X. Dai, M. Zhang, Q. Wen, W. Ye, S. Zhang, and Y. Zhang AutoSurvey: large language models can automatically write surveys. In Advances in Neural Information Processing Systems, Cited by: §2. Wang et al. (2026) Y. Wang, R. An, B. Wu, and W. Wang PaperJury: due-process review for bounded LaTeX revision. arXiv preprint arXiv:2606.16322. Cited by: §2. Yamada et al. (2025) Y. Yamada, R. T. Lange, C. Lu, S. Hu, C. Lu, J. Foerster, D. Ha, and J. Clune The AI scientist-v2: workshop-level automated scientific discovery via agentic tree search. arXiv preprint arXiv:2504.08066. Cited by: §1, §2. Yang et al. (2024) J. Yang, C. E. Jimenez, A. Wettig, K. Lieret, S. Yao, K. Narasimhan, and O. Press SWE-agent: agent-computer interfaces enable automated software engineering. In Advances in Neural Information Processing Systems, Cited by: §2. Yao et al. (2023) S. Yao, J. Zhao, D. Yu, N. Du, I. Shafran, K. Narasimhan, and Y. Cao ReAct: synergizing reasoning and acting in language models. In International Conference on Learning Representations, Cited by: §2.