Paper deep dive
Correct Choices, Failed Sequences: Choice2Seq Benchmarks Correctness Retention Across Output Interfaces
Tianyou Lai, Wentao Yue, Zhiying Deng, Qilei Li
Intelligence
Status: not_run | Model: - | Prompt: - | Confidence: 0%
Entities (0)
Relation Signals (0)
No relation signals yet.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:A model may select the correct answer yet fail to construct it through a sequence of local decisions. We introduce Choice2Seq, a diagnostic benchmark that measures correctness retention across complete-answer selection and sequential construction in independent runs on the same problems and candidate sets. Sequence Composition Rate (SCR) measures the fraction of correctly selected items that are also answered correctly through local construction. Jev achieves 97.1% and 99.5% selection accuracy on arithmetic and lexical retrieval, yet retains only 33.0% and 2.1% of those correct choices under serial construction, respectively. Substantial gaps also emerge for StartLux and Qwen, including when Qwen emits characters directly. Restricting local options to candidate-supported characters improves construction, while explicit position indexing improves copying fidelity, even for incorrect selections. These interventions distinguish task correctness, plan fidelity, and output compliance. Matched dialogue-state evaluations reveal that local field construction largely preserves Jev's correct states but reduces Qwen's mean-turn accuracy, highlighting task- and interface-dependent differences. Sequence-level confidence and request-cost analyses further characterize completed outputs.
Tags
Links
- Source: https://arxiv.org/abs/2610.16191v1
- Canonical: https://arxiv.org/abs/2610.16191v1
Trouble viewing inline? Open PDF directly →
Full Text
127,718 characters extracted from source content.
Expand or collapse full text
Correct Choices, Failed Sequences: Choice2Seq Benchmarks Correctness Retention Across Output Interfaces Tianyou Lai 1 * Wentao Yue 2 * Zhiying Deng 3 Qilei Li 3† 1 Carnegie Mellon University 2 Shenzhen University 3 Central China Normal University Abstract A model may select the correct answer yet fail to construct it through a sequence of lo- cal decisions. We introduce Choice2Seq, a diagnostic benchmark that measures correct- ness retention across complete-answer selec- tion and sequential construction in indepen- dent runs on the same problems and candi- date sets. Sequence Composition Rate (SCR) measures the fraction of correctly selected items that are also answered correctly through local construction. Jev achieves 97.1% and 99.5% selection accuracy on arithmetic and lexical retrieval, yet retains only 33.0% and 2.1% of those correct choices under serial con- struction, respectively. Substantial gaps also emerge for StartLux and Qwen, including when Qwen emits characters directly. Restricting lo- cal options to candidate-supported characters improves construction, while explicit position indexing improves copying fidelity, even for incorrect selections. These interventions distin- guish task correctness, plan fidelity, and out- put compliance. Matched dialogue-state eval- uations reveal that local field construction largely preserves Jev’s correct states but re- duces Qwen’s mean-turn accuracy, highlight- ing task- and interface-dependent differences. Sequence-level confidence and request-cost analyses further characterize completed out- puts. Code and data will be released soon. 1 Introduction “The reductionist hypothesis does not by any means imply a ‘constructionist’ one.” — Philip W. Anderson, More Is Different (1972) Correctness is observed through an output inter- face, not in isolation. Choosing among complete candidate answers tests a single decision, whereas constructing an answer character by character re- quires successive local decisions at each output * These authors contributed equally to this work. † Corresponding author: qilei.li@ccnu.edu.cn. position, conditioned on the model’s own prefix. This distinction is especially relevant to Jev, which exposes finite, typed decisions (Almeida, 2026) and has documented difficulties with repeated charac- ter choices (TypeSafe AI, 2026). Strong accuracy on individual decisions alone therefore does not establish correctness of the completed sequence. Research Question How reliably does correctness in complete- answer selection carry over to sequential con- struction on the same problems, and how does it vary across interface conditions? Previous Jev studies examine decision accuracy, selective judgment, option-name sensitivity, and probability coherence (Deußer et al., 2026; Li et al., 2026b; Sun et al., 2026; Li et al., 2026a). Yet ag- gregate accuracy across interfaces obscures the key quantity: which correctly selected items remain correct when constructed sequentially. Interpret- ing the gap is also difficult because the interfaces change decision granularity, option sets, and con- text together. Furthermore, task correctness must be distinguished from copying fidelity and output compliance: faithfully reproducing a selected an- swer can still preserve an error. These challenges require item-level pairing and targeted controlled comparisons with distinct evaluation criteria. To address these challenges, we introduce Choice2Seq (Figure 1), which separates measur- ing correctness retention from diagnosing its lim- its. Native selects a candidate, whereas Serial con- structs an answer through local decisions condi- tioned on its own prefix. Both run independently on the same problems and candidate sets. Sequence Composition Rate (SCR) measures the fraction of Native-correct items also solved by Serial, using arithmetic and lexical targets with matched can- didate ambiguity along correct prefixes. Rather than attributing an accuracy gap to a mechanism, 1 arXiv:2610.16191v1 [cs.AI] 12 Oct 2026 Matched Problems Arithmetic (384 items) Lexical retrieval (384 items) 39 × 15 = ? 585589 385389 485489 685689 8 candidates (equal length) Length 3-8 appleangle chair crown riverratio tabletoken 8 candidates (equal length) Length 3-8 Matched prefix ambiguity Problems designed so multiple candidates share prefixes. Independent Output Interfaces Shared problem and candidates Native: select one complete answer Which candidate is correct? 585 589 385 . . . Selected answer: 585 Selects one answer from 8 candidates (same length). Serial: choose one character at each position Position 1? (1-L) 5 (empty) Position 2? (1-L) 8 5 Position 3? (1-L) 9 58 Constructed answer: 589 Arithmetic: 0-9 0123456789 Lexical: a-z abcz. . . Uses the original problem and candidates plus its own evolving prefix. Correctness Retention SCR = Both correct / Native correct Serial Native CorrectIncorrect Correct Incorrect Both correct (numerator) Native correct only Serial correct only Both incorrect Results for Jev (primary evaluation) Jev / Arithmetic (384 items) Native 97.1% SCR 33.0% Jev / Lexical (384 items) Native 99.5% SCR 2.1% SCR uses Native correct as the denominator (not Serial EM). Controlled Diagnostics A. Candidate-supported options Position-wise union over all 8 candidates. 585 489 385 689 . . . Position 1 3456 Position 2 8 Position 3 59 Improved construction B. Indexed evidence Use a position table to copy supplied answers. "589" (supplied plan) Position123 Index012 Character589 Reproduced output: 589 Fidelity (matches plan) Correctness (wrong answer) Improved copying fidelity C. Code vs. character Both valid when requested. A5 Code: A Character: 5 Output compliance Evaluation Extensions Dialogue fields Intent (what user wants) Slots (key information) Task-dependent Additional evaluation of completed outputs ConfidenceRequest costStopping Figure 1: Choice2Seq distinguishes correctness retention, copying fidelity, and output compliance. Upper: inde- pendent interfaces on matched problems: correct complete-answer selection, incorrect Serial construction. Lower: controls test support, evidence access, rendering; extensions assess dialogue-state and completed-output reliability. we vary local choice support and context, and test reproduction of supplied answers, including incor- rect selections, under different evidence and output formats. These diagnostics distinguish task correct- ness, copying fidelity, and format compliance. Our paired results reveal a large but nonuniform retention gap. Jev’s Native accuracy reaches 97.1% on arithmetic and 99.5% on lexical retrieval, yet SCR is only 33.0% and 2.1% (Table 1). Sim- ilar losses appear for StartLux and persist for Qwen under direct character output, showing that typed response codes alone cannot explain the gap. Candidate-supported options improve construction, whereas indexed evidence improves copying fi- delity even for incorrect selected answers (Fig- ure 11; Table 4). Matched symbolic tasks and dialogue fields show that poor retention is not uni- versal: Jev largely preserves correct dialogue states, while Qwen loses mean-turn accuracy (Figure 9). Our contributions are: • We introduce Choice2Seq and SCR to quan- tify item-level correctness retention across in- dependently executed output interfaces, be- yond aggregate accuracy comparisons. •We show that task correctness, copying fi- delity, and output compliance require separate evaluation, using targeted controls over local choices, input evidence, and output formats. •We establish task- and model-dependent boundaries of retention through symbolic and dialogue evaluations, and assess confidence and request costs for completed outputs. Key Findings •High selection accuracy does not ensure correctness retention. Large Native–Serial gaps persist across models, including when Qwen emits characters directly (Table 1). •Faithful construction need not yield cor- rect answers. Candidate-supported choices improve task accuracy, while indexed evi- dence improves copying fidelity, even for in- correct selections (Figure 11 and Table 4). •Sequential construction is not inherently unreliable. Matched symbolic tasks demon- strate high retention; in dialogue states, Jev largely preserves correct predictions while Qwen loses mean-turn accuracy (Figure 9). 2 Related Work Structured decisions and output interfaces. Jev produces finite, typed choices with associated prob- abilities (Almeida, 2026). Decision-model stud- ies examine general capabilities, security special- ization, confidence-based escalation, legal judg- ment, and downstream scientific decisions (Deußer et al., 2026; Chen et al., 2026; Li et al., 2026b; Zhang et al., 2026; Deng et al., 2026). Output con- straints also affect generative-model accuracy (Tam et al., 2024), while JSONSchemaBench evaluates constraint coverage, output quality, and efficiency (Geng et al., 2025). Choice2Seq pairs complete- 2 answer and local outcomes on identical problems and candidates, identifying retained, lost and newly correct items. Matched fields in Schema-Guided Dialogues extend this comparison beyond enumer- ated complete answers (Rastogi et al., 2020). Construction and readout. Multiple-choice se- lection is sensitive to labels and positions (Zheng et al., 2024), while token-based scoring can dis- agree with generated answers (Wang et al., 2024). Jev’s decisions vary with option names and answer- preserving context (Sun et al., 2026; Xu et al., 2026), paralleling prompt-format sensitivity (Sclar et al., 2024). Spelling research distinguishes char- acter information encoded in model representations from reliable use during generation (Itzhak and Levy, 2022; Edman et al., 2024; Hiraoka and Inui, 2025). Jev’s indirection sensitivity raises a concern (TypeSafe AI, 2026). Exposure-bias studies exam- ine error accumulation and recovery under gener- ated prefixes (He et al., 2021; Arora et al., 2022). Unlike continuous decoding, Choice2Seq probes prefix feedback through fresh position queries with external state, alongside controls for choice support, evidence access, and output contracts. Supplied-answer controls separate copying fidelity from task correctness, and matched digit–letter op- erations test representation dependence. Confidence and trajectory outcomes. Calibra- tion depends on the correctness event being scored (Guo et al., 2017; Jagannatha and Yu, 2020; Jiang et al., 2021). Uncertainty research distinguishes token-, sequence-, and semantic-level assessments (Malinin and Gales, 2021; Kuhn et al., 2023), while Jev coherence tests examine consistency across related requests (Li et al., 2026a). Metric effec- tiveness varies across output contexts (Deviyani and Diaz, 2025). Choice2Seq tests whether scores aggregated from successive local choices predict completed-string exact match, comparing ranking with length-only references and calibration with prevalence baselines. Trajectory-aware evaluation captures intermediate outcomes beyond terminal success (Diaz, 2026). Our dialogue analyses there- fore report mean-turn, final-state, and trajectory accuracy alongside replay-based state recovery. 3 Evaluation Protocol Choice2Seq compares correctness across whole- answer selection and local construction in character tasks and semantic fields. Paired outcomes reveal item-level correctness turnover; interventions iden- tify interface conditions that preserve correctness. 3.1 Interfaces and outcomes Each item contains a problemx i , targety i of known lengthL i , and eight distinct, equal-length candi- datesA i . Native selects a complete candidate. In- dependent queries each position without a prefix. Serial starts empty and appends one character per position. Each step freshly requests the original an- swer’s character at an absolute position, receiving the problem, candidates, length and current pre- fix as external state. The prefix is a fallible draft. Local options are ten digits for arithmetic or low- ercase letters for lexical retrieval. Algorithm 1 in Appendix A gives the complete procedure. LetN i indicate a correct Native choice andS i an entirely correct Serial string. Sequence Compo- sition Rate (SCR) measures the conditional correct- ness of Serial on the Native-correct items: SCR = P i 1[N i = S i = 1] P i 1[N i = 1] .(1) The paired table records both correct, Native only, Serial only and both wrong. Marginal EM com- bines losses and gains, which can cancel or mask replacement of successful items. SCR summarizes the overlap within Native successes; Serial success on Native-wrong items records gains outside that population. This distinction reveals turnover even when Serial EM improves (Table 2). The runs are independent: neither receives the other’s answer. For Native EMaand Serial EMb,|SCR− b| ≤ 1− a, so near-perfect Native accuracy makes the two rates numerically close. We report the joint counts, Native coverage, SCR numerator and de- nominator, and gain rate alongside Serial content EM on all items. Loss mass is the Native-only count divided by all items (Appendix A.1). Fidelity instead compares construction with an answer supplied to it. TeacherForced supplies gold prefixes; gold Copy supplies the complete target; SelfPlanSpell and fresh-plan copying supply the model’s selected answer, including incorrect se- lections. Gold EM scores the original task, while fidelity scores copying. Program return emits the selected string without another request, preserving Native accuracy. A common-parent comparison appears in Appendix B.2. Independent and Serial executeL i positions in the content track; a separate common alphabet adds punctuation, whitespace and END to evaluate termination (Appendix A). 3 3.2 Tasks and candidate geometry The formal test contains 384 items per domain, balanced over lengths three to eight, with disjoint development data. Multiplication crosses operand- length balance and carry complexity, the latter com- puted by a column-arithmetic simulator. Lexical items retrieve English nouns from WordNet defini- tions with wordfreq metadata (Miller, 1995; Prince- ton University, 2006; Speer, 2022). Screening and human review establish sense-specific acceptability within the displayed candidates, excluding target cues, same-synset distractors and semantic con- flicts. Strict EM requires an exact match to the tar- get; an output audit distinguishes candidate mem- bership, WordNet indexing and blinded semantic judgments on reviewed outputs (Appendix D.17). Frozen frequency and repetition quotas govern sam- pling; the independent confidence cohort uses sep- arate targets and length quotas determined by vo- cabulary availability (Appendix A). Candidate geometry determines how much in- formation a prefix supplies. For a correct prefix of lengthk,K i (k)counts candidates consistent with that prefix. Each arithmetic item is paired with a lexical item sharing the full vector of correct-prefix counts, includingK i (0) = 8andK i (1) = 2. We also record prefix uniqueness and entropy under uniform candidate probabilities. This matches am- biguity along the reference path, while off-path branching and position-wise character support re- main task-specific. This matching does not equal- ize computation or lexical retrieval difficulty. A Na- tive control receives only the candidates to measure selection from candidate-set cues. An isomorphic COPY/REVERSE panel uses bijective digit/letter encodings to match source structure, operation, ten- symbol inventory, and candidate geometry. A se- mantic bridge constructs named Boolean fields or binary outputs from the same source facts. 3.3 Controlled interventions The interventions vary feasible outputs, evidence access and rendering. Full Serial allows the full do- main alphabet at every position. Union restricts lo- cal choices to characters appearing at that position among the candidates, permitting cross-candidate recombinations. Trie restricts each choice to con- tinuations compatible with the generated prefix and candidate set, so every valid completed string be- longs to the original eight candidates. Support fil- tering preserves option identities and relative order. A matched panel crosses these supports with controller filling. On valid trajectories, Call vari- ants query every position, including singletons; Fill variants supply singleton values without a re- quest. FullFill supplies only position-union single- tons and keeps the full alphabet elsewhere. These contrasts distinguish output-space restriction from saved requests; ambiguous decision counts accom- pany both. Contemporary Native and Full repeats use all original items for Jev and a frozen, stratified 192-item Qwen3.5-27B subset. Original Union re- sults retain their separate panel (Appendix D.6). Raw supplies a string; Indexed adds its position table; Direct also supplies the queried character. Fixed-criterion copying holds questions, options and order constant across evidence and prefix con- ditions. Padding and Batch controls test length and bundling; selected and incorrect targets distinguish fidelity from correctness. Rebinding and matched code/character contracts test binding and render- ing. Paired samples, prompt variants and inference families appear in Appendices A and D. Schema-Guided Dialogues (Rastogi et al., 2020) extend the comparison to known-service intents and categorical slots. With identical history and schemas, Joint requests all fields in one call; Inde- pendent requests each field separately; Serial also receives its own earlier predictions for the current turn. Local arms receive neither complete-state candidates nor Joint’s answer. A separate replay compares transcript recomputation with predicted or gold prior state. Endpoints include dialogue- averaged mean-turn EM, final-state and trajectory accuracy, and request costs (Appendix C). 3.4 Models and statistical analysis Jev’s formal construction uses choices returned by the service. Readout and plan-copying diag- nostics instead use a predefined argmax over re- ported probabilities. Both selections and reported probabilities are retained. StartLux uses pinned BF16 weights and released decision code with supplied names and descriptions (StartLuxLabs, 2026). Qwen3-8B and Qwen3.5-27B use full BF16 weights with thinking disabled (Qwen Team, 2025, 2026). Queries for legal codes verify allowable code tokens, while direct character and complete- string responses use greedy generation. Serial mea- sures external sequential construction: each new request reconstructs state and asks an absolute- position question. WholeGenerate solves the same Qwen subset in one uninterrupted response. Pre- 4 fixContinue appends the current prefix to the as- sistant prompt and consumes the first character of each generated suffix, without an absolute-position query. Different output contracts retain separate Na- tive denominators. Invalid responses fail strict scor- ing. Complete prompts, decoding budgets and run- time identities appear in Appendices A, C and E. An independence reference multiplies observed position accuracies within model, domain and length. Comparing Independent/TeacherForced string EM with this reference captures deviations potentially arising from item heterogeneity or de- pendent errors; Serial additionally varies prefix con- text. Whole-item bootstrap preserves heterogeneity and correlated errors (Appendix D.10). Intervals resample whole items or source clus- ters within frozen strata, redrawing conditional de- nominators and keeping repeated measurements to- gether (Dror et al., 2018). Primary exact McNemar tests use Holm correction (McNemar, 1947; Holm, 1979); supplementary panels specify their own fam- ilies. Dialogue bootstrap retains whole dialogues and checks groups linked by shared responses. Ex- ploratory GEE clusters by item and uses only con- verged fits with finite estimates and no separation (Liang and Zeger, 1986). Zero-success SCR cells use exact two-sided binomial intervals; other main- table cells use the item bootstrap. Appendices give seeds and prompts. Prompt robustness reports indi- vidual settings, mean and worst performance across variants, and common-parent ranking stability. 4 Correctness Retention Across Interfaces We first quantify the paired interface gap across tasks and model families, then examine how con- trolled interventions affect correctness retention. High whole-answer accuracy coexists with low local construction accuracy. Jev excels in Na- tive but retains fewer correct answers under Serial, especially in lexical retrieval (Table 1). Every origi- nal paired decrease survives Holm correction. Can- didates alone yield near-chance selection, support- ing a role for problem evidence. Qwen’s code and character contracts yield different Native-correct populations; shared-correct comparisons preserve the Jev/Qwen domain reversal (Appendix D.1). Na- tive ranks direct Qwen above Jev on arithmetic, while Serial reverses that ordering. Marginal improvement can replace previ- ously correct answers. High Native accuracy makes SCR close to Serial EM in the natural tasks, with the same model ordering. Named semantic fields reveal correctness turnover (Table 2): Qwen3- 8B gains nine successes overall but retains six of 28 Native successes, losing 22 and gaining 31. SCR distinguishes preserved successes from their re- placement; the joint table records both. Jev’s sim- ilar marginal accuracies accompany high overlap. Complete transitions appear in Appendix D.16. Continuous generation can succeed where po- sition queries fail. On the same Qwen3.5-27B checkpoint and 96 parents per domain, Whole- Generate solves almost all items in one response, whereas Serial position queries lose many suc- cesses despite valid character output (Table 3). Pre- fixContinue reconstructs the assistant prefix on ev- ery request: arithmetic remains accurate, while lexical trajectories develop both content and format failures. This controller is distinct from uninter- rupted decoding. All 1,056 Serial position requests finish within two output tokens, below their eight- token cap. The contrast demonstrates failure un- der the evaluated external construction interface; prompt-state representation, assistant prefill and rendering also vary (Appendices D.8 and D.18). The gap persists across tested prompts, with interface-dependent magnitude. Jev arithmetic retention decreases with length; lexical retention is low even for short targets (Figure 2). Prompt, order and repeat perturbations show the same broad Jev pattern, though paired intervals are wide. Across three templates, direct Qwen Serial EM ranges from 7.3–28.1% in arithmetic and 39.6–56.2% in lexical retrieval; Native accuracy remains higher in both domains throughout. The v2 improvements in both domains survive Holm correction, and all re- sponses remain format-valid (Table D3). Mean and worst-template SCR and common-parent ranking stability appear in Table D4 and Appendix D.4. Takeaway Compare which items are correct as well as how many. Paired outcomes distinguish pre- served successes from losses and gains that marginal accuracy can otherwise obscure. 5 Diagnosing Correctness and Fidelity Controls distinguish solving the original task, faith- fully copying supplied answers, and correctly ren- dering outputs under different response contracts. 5 Table 1: Paired correctness on 384 items per domain. Joint counts give both correct / Native only / Serial only / neither; EM and SCR are percentages. SCR counts are both correct / Native correct. Intervals are length-stratified item bootstrap 95% CIs, except zero-success cells (†), which show exact two-sided binomial intervals. Model / interfaceDomain Native EMJoint counts Serial EM SCR countSCR [95% CI] Jevmath97.1 123/250/0/1132.0123/373 33.0 [29.0, 37.0] Jevlexical99.58/374/0/22.18/3822.1 [0.8, 3.7] StartLuxmath61.28/227/1/1482.38/2353.4 [1.3, 5.9] StartLuxlexical93.81/359/0/240.31/3600.3 [0.0, 0.8] Qwen3-8B (code)math15.40/59/0/3250.00/590.0 [0.0, 6.1]† Qwen3-8B (code)lexical94.50/363/0/210.00/3630.0 [0.0, 1.0]† Qwen3.5-27B (code)math25.31/96/1/2860.51/971.0 [0.0, 3.4] Qwen3.5-27B (code)lexical99.75/378/0/11.35/3831.3 [0.3, 2.6] Qwen3.5-27B (direct) math98.439/339/0/610.239/37810.3 [7.4, 13.3] Qwen3.5-27B (direct) lexical99.0132/248/0/434.4132/380 34.7 [30.0, 39.6] Table 2: A marginal gain can replace most Native suc- cesses. Named semantic fields use 128 records per model. Joint counts are both correct / Native only / Se- rial only / neither; EM and SCR are percentages. ModelNative/Serial EM Joint counts SCR Jev89.8/90.6111/4/5/8 96.5 Qwen3-8B21.9/28.96/22/31/69 21.4 StartLux51.6/71.1 51/15/40/22 77.3 Table 3: One-response solving largely preserves accu- racy; repeated position queries do not. Qwen3.5-27B uses the same 96 items per domain, with no supplied an- swer. Correct counts are content EM; valid trajectories are arithmetic/lexical percentages; calls are per item. InterfaceArithmetic LexicalValid Calls Native94/9696/96 100/100 1.00 WholeGenerate94/9691/96 100/100 1.00 Position query14/9627/96 100/100 5.50 PrefixContinue94/960/96 100/9.4 5.50 5.1 Candidate support and prefix context Candidate-consistent support substantially im- proves lexical construction. Union permits un- listed character recombinations; TrieCall preserves Native’s eight feasible complete outputs while still querying every singleton choice. On matched Jev lexical items, EM rises from 1.8% under Full to 24.0% under UnionCall and 82.0% under TrieCall (Figure 12). Only 26.3% of UnionCall outputs belong to the candidates; every valid Trie output does. Trie improves over Full in both models and domains, and over lexical Union after correction. Jev’s domain contrast shrinks to 2.6 points [−2.9, 7.8], although Native remains more accurate. The observed lexical recovery is consistent with differences in candidate geometry. Match- ing correct-prefix counts leaves arithmetic unions with eight feasible strings, but lexical unions admit many cross-candidate recombinations (Table D7). TrieCall prevents these recombinations while re- taining every position query. A wrong Trie branch excludes gold, so recovery still requires selecting the right candidate through local decisions. Restricted support and saved requests have different effects. FullFill improves arithmetic ac- curacy while keeping full choices at ambiguous positions. Union and Trie restrict those choices; Fill additionally skips singletons. TrieCall still queries all positions; TrieFill reduces calls from 5.5 to about two per item while keeping the same eight feasible outputs. In this panel, lexical Union has no singleton positions and queries every posi- tion. On twelve-character reversals, queried depth and branch placement affect Jev more than alphabet size. Complete comparisons retain decision counts and feasible outputs (Appendices D.6 and D.7). Local improvement need not recover com- plete strings. Generated prefixes help Jev arith- metic over Independent queries; both lexical out- comes remain low. Gold prefixes improve item- averaged character accuracy, especially lexically, without comparable gains in complete-string suc- cess. The paired arithmetic TeacherForced string- success difference has an interval spanning zero (Appendix B). The product of Independent posi- tion accuracies yields a low lexical EM reference. Arithmetic Serial exceeds this reference at most lengths, consistent with useful prefix feedback (Fig- ure 14). Deviations from this reference may reflect dependent errors and item heterogeneity; the Se- rial contrast also changes prefix context. Together, these controls distinguish local error accumula- tion from additional interface effects. Imperfect position decisions already predict low string EM; prefix feedback can change those decisions; and 6 345678 Target length 0 25 50 75 100 Native accuracy (%) (a) Arithmetic: Native accuracy 345678 Target length 0 25 50 75 100 SCR (%) (b) Arithmetic: SCR 358 0 10 20 Low SCR (with intervals) 345678 Target length 70 80 90 100 Native accuracy (%) (c) Lexical: Native accuracy 345678 Target length 0 25 50 75 100 SCR (%) (d) Lexical: SCR 358 0 10 20 Low SCR (with intervals) Jev StartLux Qwen3-8B (code) Qwen3.5-27B (code) Qwen3.5-27B (direct) Figure 2: High Native accuracy does not guarantee Serial retention. Across 64 paired items per domain and length, (a,c) show near-ceiling Native accuracy for Jev and direct-output Qwen3.5-27B. In (b,d), Jev’s arithmetic retention declines with length while lexical retention remains near zero; direct Qwen instead retains more lexical than arithmetic answers. SCR conditions on Native-correct items; bands indicate 95% item-bootstrap intervals. candidate consistency determines which complete strings they can form. Trie recovery and accurate WholeGenerate outputs demonstrate that lexical losses depend on how the external controller ex- poses and combines choices, rather than represent- ing failures shared by all sequence interfaces. 5.2 Evidence access and plan fidelity An available answer still needs a usable repre- sentation. Raw gold Copy and SelfPlanSpell fail on many strings, especially lexical targets. A po- sition table derived from the supplied answer im- proves both position readout and complete-string Serial fidelity for Jev and StartLux in every fixed- criterion model/domain comparison. Questions, options and order remain constant while evidence conditions are crossed with Independent and Se- rial querying (Figure 15). Indexing simultaneously changes addressability, salience and prompt length. Better copying can preserve an incorrect an- swer. Fresh-plan indexing uses each model’s se- lected string, including incorrect selections. It im- proves fidelity for both models and domains under paired correction (Table 4; Figure 4), but faithfully copied incorrect plans fail gold EM. On supplied targets enriched with errors, Jev faithfully copies Table 4: Serial plan-copying fidelity and indexed task correctness. Raw and Indexed measure fidelity to freshly selected model plans, including incorrect selections; Gold EM measures indexed outputs against the task target. Each model–domain pair contains 30 plans. ModelDomainRaw Indexed Gold EM Jevmath17/3030/3028/30 Jevlexical4/3029/3029/30 StartLux math8/3025/3016/30 StartLux lexical1/3029/3028/30 every indexed incorrect target without solving the original problem (Appendix D). Fidelity and task correctness therefore require separate scores. Structured evidence helps beyond increasing its length. Indexed copying outperforms nonin- dexed padding, including Qwen controls matched on rendered input tokens (Table D9). Information and salience still differ. Controller and Batch con- trasts appear in Appendix D.11; program return provides the zero-additional-request reference. 5.3 Option binding and output contracts Option names and output contracts affect read- out. Remapping controls motivated by option- 7 name conflicts (Sun et al., 2026) show corrected ex- tra losses for StartLux arithmetic and Qwen lexical readout, with none observed for Jev (Appendix B). Matched Qwen generation favors character out- put with indexing but codes with direct lexical evidence. In the full prompt/order grid, Qwen3- 8B has corrected advantages in both output direc- tions, whereas Qwen3.5’s corrected differences fa- vor codes (Figure 17). Many strict Qwen3.5 charac- ter failures instead return the uppercase code bound to the correct criterion. Natural Serial retains con- tent errors despite valid character output. Complete contracts, paired results and request examples dis- tinguish these outcomes (Appendices D and E). 5.4 Task and representation effects Task structure changes the retention pattern. Matched COPY/REVERSE operations yield a small letter-minus-digit Serial contrast whose inter- val spans both directions, rather than reproducing the natural lexical disadvantage. Jev also retains most correct named and binary semantic fields (Ta- ble B2). The natural arithmetic–lexical contrast therefore combines solving with candidate geom- etry and evidence access. It narrows under Trie, while direct Qwen reverses the domain ordering. The twelve-character factor panel further identifies queried depth and branch placement as contributors to construction accuracy. Together, these controls associate retention with task structure and the out- put interface, beyond digit-versus-letter labels. Takeaway Choose the evaluation endpoint before judg- ing any intervention. Evaluate solving with task accuracy, copying with plan fidelity, and rendering with output-contract compliance. 6 Reliability of Completed Outputs 6.1 Confidence, cost, and stopping Confidence must be assessed against the com- pleted object. We score chosen-path product, ge- ometric mean and minimum top-choice probabil- ity using Brier, ECE, AUROC and average preci- sion (AP) (Brier, 1950; Guo et al., 2017; Davis and Goadrich, 2006). Product improves arithmetic ranking over the length baseline. Calibration er- ror persists. Independent lexical evaluation yields 13/2,347 successes (0.55%; exact 95% interval 0.30–0.95%). With so few successes, AUROC/AP gains over length and the Brier difference from a prevalence-only baseline have intervals spanning zero. At 10% coverage, Product precision is only 3.8% [1.7, 6.4]. Successes concentrate at short lengths; common-length weighting changes the population, not the success count (Appendix D). Cost and termination require separate end- points. Serial adds requests, tokens and recorded request time relative to Native plus program re- turn. The common-alphabet track also separately scores premature/missing END and cap exhaustion alongside strict content EM (Appendix B). 6.2 Field construction and downstream state Field construction tests granularity without complete-state candidates. Under Serial, Jev re- tains 420 of 429 Joint-correct turns; its mean-turn difference interval spans zero. Qwen shows a cor- rected Serial mean-turn loss, while intervals for Serial–Joint final-state differences span zero for both models (Figure 9). Local construction assem- bles fields without Joint’s selected state and incurs higher per-turn model request costs (Appendix C). Trajectory EM can hide differences in later states. Oracle/Rollout tests gold versus predicted prior state: Rollout has lower mean-turn and final- state accuracy despite identical trajectory EM, which stays zero after the first error (Appendix C). Trajectory-aware evaluation similarly preserves progress beyond terminal success (Diaz, 2026). Takeaway Score states after errors, not just error-free trajectories. Report later-state accuracy and re- covery alongside trajectory EM, and track con- struction costs separately for each interface. 7 Conclusion We asked whether correctness under whole-answer selection carries over to independently executed local construction. We developed Choice2Seq to compare interfaces on paired tasks and examine conditions preserving correctness. High selection accuracy cannot guarantee retention: candidate support, evidence representation and generation protocols shape construction outcomes. Copying fidelity can improve without improving task cor- rectness, while later-state accuracy reveals differ- ences hidden by trajectory EM. These findings call for interface-aware evaluation that tracks preserved successes and treats task correctness, copying fi- delity and output compliance as distinct endpoints. 8 Limitations Choice2Seq evaluates correctness retention under explicitly defined output interfaces on the same un- derlying problem instances. The primary tasks use known lengths of three to eight and eight displayed candidates, covering arithmetic and English noun retrieval. A twelve-character symbolic panel ex- tends the length range under matched operations. These settings characterize short, constrained con- struction rather than unrestricted long-form or mul- tilingual generation. Full and Union can admit out- puts beyond the candidate set, while Trie preserves complete candidate membership through a differ- ent local decision structure from Native. Matching correct-prefix geometry controls candidate ambi- guity along the reference path; off-path branching, computation and vocabulary remain task-specific. The natural-generation comparison evaluates continuous responses, position queries and recon- structed character prefills as distinct interface con- figurations. It does not attribute their differences to a single internal decoding mechanism. Like- wise, indexing controls compare evidence represen- tations, with addressability and salience potentially changing together. The lexical audit classifies all outputs in the matched panel against candidates and WordNet, complemented by blinded seman- tic review of all indexed noncandidate pairs and a stratified unindexed sample. Human judgments therefore support the reviewed population, while resource-level conclusions cover the full panel. Dialogue experiments extend the framework to known-service intents and categorical fields, us- ing state-specific endpoints rather than character SCR. Fresh-plan fidelity comparisons use 30 plans per model/domain, with exact intervals quantifying finite-sample precision. Confidence and cost find- ings apply to the evaluated length mixtures, success prevalence, prompts and runtime configurations. Ethical Considerations We use generated arithmetic and synthetic facts, WordNet and wordfreq with source notices, and public Schema-Guided Dialogues under C BY- SA 4.0. Lexical resources can contain dated senses or terminology. The release preserves study prove- nance and scientific response records while exclud- ing credentials, account identifiers and identifying paths. Human review and automated lexical screen- ing are documented separately. References Diogo Almeida. 2026. Introducing System One models & Jev. TypeSafe AI official release. Accessed 10 October 2026. Kushal Arora, Layla El Asri, Hareesh Bahuleyan, and Jackie Chi Kit Cheung. 2022. Why exposure bias matters: An imitation learning perspective of error accumulation in language generation. In Findings of the Association for Computational Linguistics: ACL 2022, pages 700–710, Dublin, Ireland. Association for Computational Linguistics. Glenn W. Brier. 1950. Verification of forecasts ex- pressed in terms of probability. Monthly Weather Review, 78(1):1–3. Zheng Chen, Fei Yu, Haohao Huang, Yang Li, Anlong Chen, and Lei Chen. 2026. SecJev: Bringing security expertise to System One decision models. arXiv preprint arXiv:2610.03073. Version 1, 2 October 2026. Jesse Davis and Mark Goadrich. 2006. The relation- ship between Precision-Recall and ROC curves. In Proceedings of the 23rd International Conference on Machine Learning, pages 233–240. Association for Computing Machinery. Boyuan Deng, Shuyi Fan, Hongyang Zhang, and Xin- hong Xie. 2026. Jev for scientific decisions: Evaluat- ing semantic choices and their consequences. arXiv preprint arXiv:2609.24965. Version 2, 23 September 2026. Tobias Deußer, Lorenz Sparrenberg, and Rafet Sifa. 2026. Evaluating and benchmarking the System One model Jev. arXiv preprint arXiv:2609.37647. Ver- sion 1, 29 September 2026. Athiya Deviyani and Fernando Diaz. 2025. Contextual metric meta-evaluation by measuring local metric accuracy. In Findings of the Association for Compu- tational Linguistics: NAACL 2025, pages 4921–4940. Association for Computational Linguistics. Fernando Diaz. 2026. Offline preference-based trajec- tory evaluation. arXiv preprint arXiv:2606.17541. Rotem Dror, Gili Baumer, Segev Shlomov, and Roi Reichart. 2018. The hitchhiker’s guide to testing statistical significance in natural language processing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1383–1392. Association for Computational Linguistics. Lukas Edman, Helmut Schmid, and Alexander Fraser. 2024. CUTE: Measuring LLMs’ understanding of their tokens. In Proceedings of the 2024 Confer- ence on Empirical Methods in Natural Language Processing, pages 3017–3026. Association for Com- putational Linguistics. 9 Saibo Geng, Hudson Cooper, Michał Moskal, Samuel Jenkins, Julian Berman, Nathan Ranchin, Robert West, Eric Horvitz, and Harsha Nori. 2025. JSON- SchemaBench: A rigorous benchmark of struc- tured outputs for language models. arXiv preprint arXiv:2501.10868. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Wein- berger. 2017. On calibration of modern neural net- works. In Proceedings of the 34th International Con- ference on Machine Learning, volume 70 of Pro- ceedings of Machine Learning Research, pages 1321– 1330. PMLR. Tianxing He, Jingzhao Zhang, Zhiming Zhou, and James Glass. 2021.Exposure bias versus self- recovery: Are distortions really incremental for au- toregressive text generation? In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 5087–5102, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. Tatsuya Hiraoka and Kentaro Inui. 2025. Spelling-out is not straightforward: LLMs’ capability of tokeniza- tion from token to characters. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 13340–13353. Association for Computa- tional Linguistics. Sture Holm. 1979. A simple sequentially rejective mul- tiple test procedure. Scandinavian Journal of Statis- tics, 6(2):65–70. Itay Itzhak and Omer Levy. 2022. Models in a spelling bee: Language models implicitly learn the character composition of tokens. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 5061–5068. Associa- tion for Computational Linguistics. Abhyuday Jagannatha and Hong Yu. 2020. Calibrat- ing structured output predictors for natural language processing. In Proceedings of the 58th Annual Meet- ing of the Association for Computational Linguistics, pages 2078–2092, Online. Association for Computa- tional Linguistics. Zhengbao Jiang, Jun Araki, Haibo Ding, and Graham Neubig. 2021. How can we know when language models know? on the calibration of language models for question answering. Transactions of the Associa- tion for Computational Linguistics, 9:962–977. Lorenz Kuhn, Yarin Gal, and Sebastian Farquhar. 2023. Semantic uncertainty: Linguistic invariances for un- certainty estimation in natural language generation. In The Eleventh International Conference on Learn- ing Representations. Keyi Li, Yihao He, and Quanyi Li. 2026a. Beyond calibration: Do a typed-decision model’s probabil- ities obey the probability axioms? arXiv preprint arXiv:2609.33209. Version 1, 27 September 2026. Yubo Li, Yidi Miao, Ramayya Krishnan, and Rema Padman. 2026b.JEV-as-a-judge: Accept when confident, escalate when unsure. arXiv preprint arXiv:2609.26550. Version 4, 6 October 2026. Kung-Yee Liang and Scott L. Zeger. 1986. Longitu- dinal data analysis using generalized linear models. Biometrika, 73(1):13–22. Andrey Malinin and Mark Gales. 2021. Uncertainty estimation in autoregressive structured prediction. In International Conference on Learning Representa- tions. Quinn McNemar. 1947. Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2):153–157. George A. Miller. 1995. WordNet: A lexical database for English. Communications of the ACM, 38(11):39– 41. Princeton University. 2006. WordNet 3.0. Lexical database and documentation. Corpus copyright 2006; documentation accessed 6 October 2026; source li- cense retained. Qwen Team. 2025. Qwen3-8B. Official model card. Accessed 6 October 2026; evaluated checkpoint iden- tity recorded separately. Qwen Team. 2026.Qwen3.5-27B.Offi- cialmodelrelease.Evaluatedrevision fc05daec18b0a78c049392ed2e771dde82bdf654. Abhinav Rastogi, Xiaoxue Zang, Srinivas Sunkara, Raghav Gupta, and Pranav Khaitan. 2020.To- wards scalable multi-domain conversational agents: The Schema-Guided Dialogue dataset. Proceedings of the AAAI Conference on Artificial Intelligence, 34(05):8689–8696. Melanie Sclar, Yejin Choi, Yulia Tsvetkov, and Alane Suhr. 2024. Quantifying language models’ sensitiv- ity to spurious features in prompt design or: How I learned to start worrying about prompt formatting. In The Twelfth International Conference on Learning Representations. Robyn Speer. 2022. rspeer/wordfreq: v3.0. Zenodo software release. Archived release v3.0.2; this evalu- ation uses package version 3.1.1. StartLuxLabs. 2026. StartLux-Decision: Typed deci- sion models. Official software and model release. Pinned model and code revisions are recorded in Ap- pendix A. Yu Sun, Junhao Xu, Jiajia Shi, and Zijin Yang. 2026. Type-safe is not error-free: Typed decision models follow the option name, not the definition bound to it. arXiv preprint arXiv:2609.26758. Version 3, 8 October 2026. 10 Zhi Rui Tam, Cheng-Kuang Wu, Yi-Lin Tsai, Chieh- Yen Lin, Hung-yi Lee, and Yun-Nung Chen. 2024. Let me speak freely? a study on the impact of format restrictions on large language model performance. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Track, pages 1218–1236. Association for Computa- tional Linguistics. TypeSafe AI. 2026. Jev 1.13 jaggedness. Official model documentation. Accessed 6 October 2026. Xinpeng Wang, Chengzhi Hu, Bolei Ma, Paul Röttger, and Barbara Plank. 2024.Look at the text: Instruction-tuned language models are more robust multiple choice selectors than you think. In First Conference on Language Modeling. Zixiang Xu, Zirui Song, Chiyu Zhang, Xiuying Chen, Xi Liu, Xiyang Hu, and Yue Zhao. 2026. JevOut: Natural context can flip decision models. arXiv preprint arXiv:2609.30243. Version 2, 8 October 2026. Fan Zhang, Yankai Chen, Zhuohan Xie, Yixi Zhou, Sijia Peng, Lei Fan, Xinhua Ji, Cunyuan Zheng, Huangy- ong Shan, Philip S. Yu, Xue Liu, Yu Chen, Preslav Nakov, and Songwei He. 2026. Same scores, differ- ent decisions: Evaluating JEV and language models for legal document understanding. arXiv preprint arXiv:2609.27678. Version 1, 23 September 2026. Chujie Zheng, Hao Zhou, Fandong Meng, Jie Zhou, and Minlie Huang. 2024. Large language models are not robust multiple choice selectors. In The Twelfth Inter- national Conference on Learning Representations. 11 A Detailed Protocol and Reproducibility A.1 Design and independent units The independent unit is a natural parent item, shared symbolic source, semantic fact record or complete dialogue. Positions, conditions, repeti- tions and option permutations remain attached to that unit. Cross domain contrasts resample the designated arithmetic and lexical partner pair to- gether. Character construction probes the inter- face. The matched field task extends evaluation to states without complete-answer candidates. Algorithm 1 gives the fixed length procedure for Full, Union and prefix-aware Trie support. Native’s selection is retained for paired scoring. Each local call uses the independently constructed prefix. Paired outcomes and SCR. Letp ab denote the proportion with Native correctnessaand Serial correctnessb. Native EM isp 11 + p 10 , Serial EM isp 11 +p 01 , and SCR isp 11 /(p 11 +p 10 ). Loss and gain mass arep 10 andp 01 ; their difference is the marginal accuracy gap. For Native EMa > 0and Serial EM b, the overlap is bounded by max 0, a + b− 1 a ≤ SCR≤ min 1, b a . Thus marginals bound the overlap, and paired ob- servations identify it. Also|SCR− b| ≤ 1− a: SCR equals Serial EM when Native is perfect and is undefined when Native has no successes. For a natural-task illustration, Qwen arithmetic FullFill and UnionFill both solve 36/96 items but retain 35/94 and 36/94 Native successes. This one- item difference illustrates distinct information, not a controller advantage. A larger turnover appears in the named semantic Qwen cell: 28 Native and 37 Serial successes overlap on six items, so 22 losses and 31 gains produce a net gain of nine (Ta- ble B2). Independent requests can have correlated correctness through shared item difficulty. Copy- ing an existing selection is evaluated separately by plan fidelity. Table D1 compares models on a common Native-correct population. A.2 Arithmetic and candidate construction Positive operands have products of exactly L dig- its. Balanced operand lengths differ by at most one digit; unbalanced lengths differ by at least two. The grade school simulator records positive out- going carries within each multiplication row and the final partial product summation. Carry chain Algorithm 1 Paired Native and Serial evaluation. Require:Problemx, candidatesA, lengthL, al- phabet Σ Require:Supports ∈ FULL, UNION, TRIE, fill flag f 1: n← Native(x,A) 2: ˆy ← ε 3: for t = 1,...,L do 4: V ← Σ 5:if s = UNION then 6:V ←a t : a∈ A 7:else if s = TRIE then 8:V ←a t : a∈ A, a <t = ˆy 9:end if 10:if V =∅ then 11:return n, INVALID 12:end if 13:if f ∧|V| = 1 then 14:z ← sole(V ) 15:else 16:z ← Local(x,A,L,t, ˆy,V ) 17:end if 18:if z /∈ V then 19:z ← INVALIDMARKER 20:end if 21:ˆy ← ˆy∥ z 22: end for 23: return n, ˆy length counts consecutive carry events within a row or summation, resetting between them. Ele- mentary multiplications, operand lengths, separate multiplication/addition carries, full carry traces, and zero/repeated digit counts are retained. Digit counts use concatenated operands. For each length and structure, a frozen bounded pool defines low carry at or below its median and high carry above it. Final selection takes 16 items from each structure/carry cell per length. Products are unique in test. Pilot/development products and unordered operand pairs are excluded. Simulator output is checked against integer multiplication. Primary arithmetic candidates use a four by two construction matching an accepted lexical set’s vec- tor of candidate counts compatible with the gold prefix. All have equal length and no leading zero. For prefix length k, K(k) counts compatible com- plete candidates. Stored metadata includes K(0) through K(L), uniform candidate entropy log2 K(k), first unique prefix, mean entropy over k=0 through L−1, and next character branch entropy. Every 12 Problem + eight complete candidates One choice of a complete answer Program returns selection Task accuracy = Native EM (a) Native Same problem + same candidates Local choice at each position Feedback from earlier choices Complete string EM SCR: retention given Native correct (b) Serial Selected answer (correct or incorrect) Raw or indexed evidence Independent or Serial Plan fidelity vs. gold EM Preserve a plan vs. solve the task (c) Copying a selected answer Figure 3: Correctness retention and plan fidelity require different references. (a) Program return preserves a selected answer without another model request. (b) Serial solves the same problem separately, measuring overlap with Native correctness. (c) Copying scores fidelity to the supplied plan separately from gold-answer correctness, distinguishing faithful copying of a wrong plan from successful solving. natural pair has the same K vector along the gold path, with K(0)=8 and K(1)=2. Branching out- side the gold path and support at each position can differ. Entropy quantifies uncertainty under a uniform distribution over candidates. A.3 Lexical sources and review WordNet 3.0 supplies noun synsets and unchanged definitions. The wordfreq 3.1.1 supplies Zipf fre- quency. Targets are ASCII lowercase single words of length three to eight with Zipf at least 2.5. Each source sense has one eligible lemma. Screening excludes proper name senses, multiword forms, tar- get/form cues, shared synset candidates, adjacent taxonomic senses and documented semantic con- flicts. Records retain raw lemmas, same-length aliases, candidate senses and possible outside an- swers. Acceptance applies to the specified sense within the closed candidate set. Construction begins by sampling four groups by initial letter, with two members each. All eight can- didates require acceptable noun definition sources and a frequency range at most 1.5 Zipf units. Gold is sampled uniformly from the constructed set be- fore duplicate rejection, review and quota selection. Frequency strata split at Zipf 4.0. The presence of repeated letters defines the second stratification factor. Frozen length quotas select the first eligible bank ranks in each cell. Those acceptance steps condition the final distribution. Controls using only the candidate set audit remaining set cues. Devel- opment and test share no gold targets or unordered operand pairs; distractors may recur. Before formal experiments, two reviewers in- dependently assessed all 576 bank items with- out prior answer records or model outputs. Both selected the intended answer and judged it uniquely acceptable among the eight candidates for all items. The planned resolution procedure was joint source checking and discussion. No answer/uniqueness adjudication was needed. Inclusion separately applies source/ambiguity rules: 543 items were eligible, 33 held (26 for gloss clarity, seven for candidate overlap), and 384 selected by frozen quotas. The remaining 159 eligible items were unused. For example, mug has an underspecified gloss fragment, while granola competes with listed oatmeal. Records include full definitions and candidates. The independent calibration bank contains 2,400 previously unused targets, all receiving LLM as- sisted lexical screening. Two humans indepen- dently review 225 subjects: an audit of 192 pre- sampled items, 32 per length, plus 33 deduplicated targeted cases covering screening holds. In the random audit, answer and uniqueness judgments agree throughout, including one jointly nonunique item. Definition sufficiency differs on 15 items. Across both groups, 44 subjects receive a negative or uncertain judgment from either reviewer. Their union with unresolved screening holds excludes 53 subjects, leaving 2,347. Review records retain dis- agreements and identify all human-reviewed items. Short eligible nouns are scarce after frequency, sense, target isolation and candidate set require- ments. Quotas based on available sources yield 61, 315, 442, 511, 548 and 470 retained subjects at lengths three through eight. Quotas precede 13 model outcomes, and every retained item is run regardless of success. The main paired population remains balanced. Common length standardization accounts for the cohorts’ different length mixtures. A.4 Output conditions and stopping Native makes one request with eight choices. Its control uses only the candidates and withholds the problem. Independent and Serial query all L po- sitions with the same problem and candidates. In- dependent provides an absolute position without a prefix. Serial supplies its actual generated prefix. TeacherForced supplies the gold prefix. Gold Copy supplies the target, and original SelfPlanSpell sup- plies that item’s main Native selection. Fidelity compares output with the selected plan. Gold accu- racy compares with the task answer. The main content track uses ten digits or 26 low- ercase letters and makes no END request. The separate track with 51 choices uses 26 letters, ten digits, eleven centrally configured punctuation sym- bols, space/newline/tab and END on 72 preselected natural items. Punctuation is defined by this project because the original demonstration did not enumer- ate it. The cap is L+3 requests. Exact completion requires gold immediately followed by END. Con- tent EM and content in the first L positions are separately scored. Exclusive stopping categories follow this priority: exact completion, premature END, correct content without END, step cap ex- haustion, wrong content. Auxiliary indicators pre- serve overlapping exhaustion/content errors. The prefix intervention preselects one position per item and crosses correct/error drafts with read- out of the whole answer or a local character. An error draft flips one preceding character. It need not remain compatible with a complete candidate. Schedules retain the supplied states and are inde- pendent of observed outcomes. A.5 Representation and readout controls The isomorphic panel uses 20 bases per length, stratified by repetition and early/late uniqueness. COPY/REVERSE and digit/letter encodings create four presentations per base, with shared geome- try and matched order. The main bijection maps digits 0 through 9 to letters a through j. A sec- ond mapping to etaoinshrd uses twelve fixed bases. The semantic panel has 128 source facts, 16 per Boolean pattern, rendered as named fields or binary encodings. Three two option decisions construct each output. Field and binary representations use different wording and labels. The single position panel has 120 natural parents and twelve symbolic COPY bases with two encod- ings: 144 records, 132 independent clusters. It supplies a gold string and preselected absolute po- sition. Raw provides string/position. Indexed adds the entire position table. Direct additionally pro- vides the extracted character. A true/false identity check is separate. These representations change addressability, salience and prompt length together. Opaque, opaque remapped, readable aligned and readable conflicting names retain the character descriptions.A permutation with no fixed points rebinds names.Gold fol- lows descriptions.Extra conflict loss is (aligned−conflicting)−(opaque−remapped). Jev has three repeats.Cross model estimates use the first two shared repetition labels. This supplementary readout uses reported probability argmax with displayed order tie breaking and retains returned selections. StartLux’s ordered JSON bypasses automatic renaming. Qwen retains name/description mappings and records full vocabulary mass on legal codes. Matched ordinary Qwen generation crosses three evidence sources and code/character outputs on all 144 records, giving 864 requests. Both contracts use disabled thinking, greedy decoding, a budget of eight tokens, identical evidence and matched or- der. Malformed outputs fail. Subsequent marginal prompt/order checks and the full crossed grid re- tain the same parents. Appendices D and E give inference families and verbatim contracts. The fresh own plan panel selects five natural items per domain/length without outcomes. Each decision model makes 60 fresh Native choices, then copies its selected string under raw/indexed evidence and Independent/Serial control. The in- dexed table is computed from that selection, in- cluding errors. Each model uses 1,380 requests. Native is shared across its four copying arms. Fi- delity across all items, gold accuracy, SCR condi- tioned on fresh Native correctness, errors, survival and costs use these plans, separately from original SelfPlanSpell. Four indexing contrasts and four controller–representation interactions form sepa- rate families for Holm correction. A.6 Confidence and statistical inference Chosen path product multiplies reported probabili- ties of emitted characters. Geometric mean expo- nentiates the mean log probability. Minimum top 14 choice probability takes the smallest per step maxi- mum and can differ from chosen path scores when returned choices disagree with argmax. Teacher- Forced also records gold path product and mean. Brier, ten fixed equal width reliability bins/ECE, AUROC and average precision evaluate completed string correctness. One class discrimination is un- defined. Evaluation preserves original scores and exact floating-point ties across cohorts. Native to position projection sums complete can- didate probabilities by character and compares them with independent queries. Raw mass and separately normalized TV/JS are labelled. Unsup- ported mass covers characters absent from every candidate at that position. Its uniform reference is one minus support size divided by alphabet size. Bootstrap uses item-averaged position scores. Original inference uses 99,999 bootstrap draws, seed 20261005. Natural strata are domain/length. Symbolic bases use length, repetition and profile. Semantic bases use gold pattern. Conditional de- nominators are recomputed, with empty ones un- defined. Boundary counts receive supplementary conditional binomial intervals. For the zero success Qwen3-8B cells in Table 1, the two sided exact up- per bounds are 6.1% for 0/59 and 1.0% for 0/363. Empirical bootstrap intervals with zero width de- scribe the observed sample boundary. The pri- mary six Native/Serial exact McNemar tests share a Holm correction at 0.05. Later panels state their own seeds, draw counts and correction families. Character observations remain grouped by item. Exploratory GEE fits cluster by item (Liang and Zeger, 1986). Failed, quasi separated or nonfinite fits are not used for inference. A.7 Models and reproducibility Jev is pinned to jev-1.13.0 with SDK 0.7.0. Formal construction, independent calibration and candi- date support studies score the choice returned by the service. The readout/own plan diagnostics pre- define probability argmax. Both are retained with reported rounding and disagreements. Separate repeat records capture hosted response variation. StartLux-Decision-4B uses BF16 revision93 02aedb7f7bd889994336f2ae919dffbf5dbee1 and codec61bfab3b4fa3ae9d155d457247a7 fd0a299a24 , Torch 2.8/CUDA 12.8 and Trans- formers 5.8.1. Released temperature, wide choice grouping and the slow Torch fallback are pre- served. Qwen3-8B uses full BF16 (checkpoint digestd45786379a66d7e29f82e4f694b5eab4 7fe7c15ce85777b3b8b0fa30fb73e7e2), dis- abled thinking and Transformers 4.57.1 with the same Torch/CUDA version. Legal code tokens are checked at the actual prompt boundary. Qwen3.5- 27B uses BF16 revisionfc05daec18b0a78c0493 92ed2e771dde82bdf654 , Torch 2.8/CUDA 12.8 and Transformers 5.17.0. Direct queries allow eight new tokens per character, 32 per string and 256 per dialogue JSON. Thinking is disabled. Only outer whitespace is stripped; extra text, wrong case, il- legal values and truncation fail. Checkpoint, tok- enizer and source hashes accompany the responses. Frozen schedules interleave domains and con- ditions within collection windows. Logs retain exact prompts, candidate/option orders, prefixes, references, selections, available probabilities, ser- vice confidence, model/SDK identity, timing, er- rors and retries. Generated outputs without proba- bility maps are explicitly marked. Costs report logical requests, input tokens, avail- able physical forwards and summed recorded re- quest times within each recorded runtime. Timing covers requests and excludes loading and orchestra- tion overhead. Hosted and local runs use different hardware. Jev costs are calculated from recorded usage at the frozen rate of USD 0.042 per million in- put tokens. The reported values represent estimated charges under this fixed rate. Own-plan costs com- prise one Native and L copy calls; experimental spend shares Native across arms. The reproducibility materials contain frozen items, schedules, complete requests/responses, lex- ical review records and offline reconstruction. The evidence index maps exhibits to data and analy- ses; verbatim prompts and contracts appear in Ap- pendix E. B Character Controls B.1 Prefix feedback and supplied targets Independent omits the draft; Serial supplies its generated prefix; TeacherForced supplies the gold prefix on 60 preselected items per domain. Char- acter accuracy is averaged within item. Teacher- Forced string success requires every gold-prefix- conditioned decision to be correct. Fresh-plan copying uses 30 Native selections per model/domain, shared across Raw/Indexed and In- dependent/Serial. The position table is computed from the selected string, including errors. Fidelity scores that string; gold EM scores the original task (Figure 4). Holm correction treats four indexing 15 020406080100 Plan fidelity (%) StartLux: Serial StartLux: Independent Jev: Serial Jev: Independent 13/30 30/30 17/30 30/30 2/30 23/30 8/30 25/30 (a) Arithmetic 020406080100 Plan fidelity (%) 3/30 24/30 4/30 29/30 0/30 29/30 1/30 29/30 (b) Lexical Indexed gold EMRawIndexed Figure 4: Indexing improves fidelity to selected plans without guaranteeing task correctness. Each panel uses 30 fresh plans per domain/model. (a) Arithmetic fidelity can exceed gold EM because incorrect plans are copied faithfully; (b) lexical fidelity also improves for both models. Intervals are 95% item bootstrap. contrasts and four controller–representation inter- actions as separate families. B.2 Three workflows on the same selected-plan parents On the original 60 selected-plan parents per do- main/model, program return emits the recorded Na- tive choice with perfect fidelity and no additional call. Independent Serial solves without receiving that choice; SelfPlanSpell copies it throughLchar- acter queries. Jev arithmetic copying is faithful on 40/60 but gold-correct on 38/60: two incorrect selections are faithfully preserved. The workflows share parents, with Native and Serial independently solving each primary item. B.3 Natural whole-answer generation Qwen3-8B receives each of the 384 original prob- lems and candidate sets and emits the complete answer in one greedy response, with thinking dis- abled and 64 new tokens. No selected or gold answer is supplied. The same checkpoint and items are used in Native and legal-code Serial; the re- sponse contract changes to ordinary text. Joint, lost and gained counts below retain this pairing. B.4 Digit and letter representations and semantic controls COPY/REVERSE uses 120 shared source bases. Mapping digits 0–9 to letters a–j preserves strings, candidate geometry and ten local choices; both operations are averaged within base for inference. The semantic panel uses 128 shared fact records, 16 per Boolean pattern, rendered as named fields or Table B1: Whole-answer generation improves Qwen3- 8B accuracy while changing which items succeed (384 per domain). Serial legal-code EM is zero in both do- mains. Joint, lost and gained counts compare indepen- dent Native and whole-generation outcomes. DomainNative Whole Joint Lost Gained Arithmetic591814118140 Lexical363372358514 binary strings. Each local output requires three bi- nary decisions. SCR conditions on representation- specific Native successes; all-item EM scores the full population of 128 fact records. B.5 Readout at a single position and option binding Sixty parents per domain and twelve symbolic bases each supply a target and one preselected position. Raw provides string and position; In- dexed adds the position table; Direct also sup- plies the extracted character. Cross-model esti- mates average two shared repeats within parent; Jev’s three-repeat estimate is separate.Qwen scores legal codes by conditional softmax. Opaque remapping, readable alignment and readable con- flict preserve character criteria. Extra conflict loss is(aligned − conflicting) − (opaque − remapped). Figure contrasts and cross-model con- trasts receive separate Holm corrections. B.6 Matching generation contracts Code and character requests use identical evidence and order, disabled thinking, greedy decoding and eight new tokens. Malformed responses fail. The 16 −7.5−5.0−2.50.02.55.0 Letter − digit Serial EM (p) Qwen3-8B StartLux Jev (a) Matched digits and letters 0204060 Extra name conflict loss (p) Qwen3-8B StartLux Jev * * (b) Binding difference in differences ArithmeticLexical Figure 5: Matched symbol operations yield small letter–digit contrasts, while option-name conflicts remain interface dependent. (a) On 120 shared COPY/REVERSE bases, contrast intervals span both directions. (b) Extra conflict losses survive correction for StartLux arithmetic and Qwen3-8B lexical readout. Paired intervals are 95%; asterisks mark Holm p < 0.05 after correction within each comparison family. Table B2: Jev preserves most correct semantic decisions under either encoding. SCR reports retained/correct Native counts and 95% intervals separately for named and binary outputs. ModelRepresentation Native EM Serial EM SCR countSCR [95% CI] Jevnamed89.890.6111/115 96.5 [93.0, 99.2] Jevbinary68.085.280/87 92.0 [86.0, 96.8] StartLuxnamed51.671.151/66 77.3 [67.6, 86.6] StartLuxbinary39.129.725/50 50.0 [40.4, 60.5] Qwen3-8B named21.928.96/2821.4 [8.0, 36.0] Qwen3-8B binary14.815.61/195.3 [0.0, 16.7] 020406080100 Accuracy at queried position (%) Qwen3-8B StartLux Jev (a) Arithmetic 020406080100 Accuracy at queried position (%) (b) Lexical RawIndexedDirect Figure 6: On 60 supplied-target parents per domain, (a,b) Jev and StartLux approach perfect indexed readout from lower raw accuracy; Qwen3-8B legal-code accuracy stays lower. Intervals are 95% item bootstrap. 144-record panel crosses three evidence forms with two contracts; all repetitions stay within parent. The full prompt/order grid is in Appendix D.13; exact contracts appear in Appendix E. B.7 Distribution, stopping and cost Candidate probabilities are projected onto each po- sition by summing complete-answer mass for each character. TV/JS use separately normalized distri- butions; raw mass is retained. Unsupported mass is compared with1−|support|/|Σ|, and positions 17 0255075100 Accuracy (%) Direct Indexed Raw 11.7 / 100.0 21.7 / 100.0 76.7 / 100.0 30.0 / 98.3 78.3 / 98.3 93.3 / 95.0 (a) Arithmetic: accuracy / validity 0255075100 Accuracy (%) Direct Indexed Raw 6.7 / 100.0 41.7 / 100.0 98.3 / 100.0 20.0 / 98.3 80.0 / 96.7 85.0 / 96.7 (b) Lexical: accuracy / validity −200204060 Character − code accuracy (p) Direct Indexed Raw (c) Arithmetic: paired difference −200204060 Character − code accuracy (p) Direct Indexed Raw (d) Lexical: paired difference CodeCharacter Figure 7: The better output contract depends on evidence presentation. Under matched greedy Qwen3-8B requests, (a,b) format validity stays high despite accuracy differences. (c,d) Indexing favors characters in both domains, whereas direct lexical evidence favors codes. Malformed responses fail; paired intervals are 95%. are averaged within item. The separate 51-choice track uses 72 preselected parents and a cap ofL + 3requests. It scores cor- rect END, premature END, correct content without END, cap exhaustion and wrong content under the priority in Appendix A.4. Request times exclude loading and orchestration; costs count logical re- quests and recorded usage. Native followed by program return needs one request. C Dialogue Evaluation C.1 Prior-state access Schema-Guided Dialogue (Rastogi et al., 2020) supplies simulated English dialogues with crowd- sourced paraphrases (C BY-SA 4.0). The 96 test prefixes contain four, six or eight user turns (32 each), cover ten services (at most twelve dialogues per service), and have one to three categorical slots plus active intent. A service persists throughout each prefix; selection requires state changes and a nonempty categorical value. Twelve development prefixes are disjoint by split, identifier and content. HistoryRecompute receives the transcript and schema. Rollout receives its previous predicted state and the latest SYSTEM/USER exchange; Or- acle substitutes the gold prior state. Initial intent is NONE, absent slots UNSET and explicit in- difference DONTCARE. Current gold state is re- served for scoring. Jev returns multiple typed fields, StartLux retains semantic names/descriptions, and Qwen emits strict JSON with precisely the re- quested keys and legal values. Mean-turn EM weights dialogues equally. Tra- jectory and final EM score complete prefixes and terminal states. Each model has 1,924 logical re- quests including perturbations and 1,800 physical responses; identical wire inputs share 124 rows. Six rows connect separate dialogues into 94 response- sharing components. Bootstrap either resamples complete dialogues or these components, retaining the scores for receiving dialogues. C.2 Matched current-state construction Joint, Independent and Serial use the same visi- ble history and schema on 96 dialogues, 576 user turns and 1,716 field turns. Joint requests all fields 18 Table C1: Equal trajectory accuracy coexists with different later-state accuracy (96 dialogues per arm). Mean turn EM weights dialogues equally. Trajectory and final EM evaluate complete trajectories and terminal states. ModelModeTrajectory Mean turn Final Valid format JevHistoryRecompute27.175.3 53.1100.0 JevOracle33.383.2 68.8100.0 JevRollout33.377.0 61.5100.0 StartLuxHistoryRecompute25.069.7 59.4100.0 StartLuxOracle30.280.6 71.9100.0 StartLuxRollout30.271.9 59.4100.0 Qwen3.5-27B HistoryRecompute30.276.3 58.3100.0 Qwen3.5-27B Oracle26.080.0 61.5100.0 Qwen3.5-27B Rollout26.073.5 53.1100.0 Table C2: Gold prior states improve mean-turn and final-state accuracy without changing trajectory EM. Differences are Oracle minus Rollout (percentage points), with paired-dialogue and response-component 95% intervals. ModelEndpointDifference [95% CI] Component 95% CI Jevtrajectory0.0 [-2.1, 2.1][-3.1, 3.1] Jevmean turn6.2 [3.7, 8.6][3.1, 9.5] Jevfinal7.3 [2.1, 12.5][1.0, 14.3] Jevmean field2.4 [1.5, 3.3][1.3, 3.6] StartLuxtrajectory0.0 [0.0, 0.0][0.0, 0.0] StartLuxmean turn8.7 [6.7, 10.7][5.6, 12.0] StartLuxfinal12.5 [7.3, 17.7][6.2, 19.6] StartLuxmean field3.7 [2.9, 4.6][2.4, 5.2] Qwen3.5-27B trajectory0.0 [0.0, 0.0][0.0, 0.0] Qwen3.5-27B mean turn6.5 [4.3, 8.9][3.9, 9.5] Qwen3.5-27B final8.3 [2.1, 14.6][1.0, 15.8] Qwen3.5-27B mean field3.0 [2.0, 4.1][1.7, 4.5] Table C3: Failed trajectories can recover correct later or final states. Error populations vary by model and arm. ModelModeFailed Any recovery Final recovery JevHistoryRecompute703625 JevOracle644734 JevRollout643627 Qwen3.5-27B HistoryRecompute673727 Qwen3.5-27B Oracle714734 Qwen3.5-27B Rollout713326 StartLuxHistoryRecompute724733 StartLuxOracle675640 StartLuxRollout673828 together; Independent requests each separately; Se- rial also receives earlier field predictions for that turn. Neither local arm receives Joint’s output. Field order uses seed 20261009. All arms allow 256 new tokens, disabled think- ing and greedy decoding; invalid JSON, extra keys, illegal values and truncation fail. The primary end- point averages joint state turn EM within dialogue, then across dialogues. Two Serial-minus-Joint sign tests share Holm correction (99,999 draws, plus-one correction). State retention is dialogue- weighted local/Joint jointly-correct mass divided by Joint-correct mass, distinct from character SCR. Conditional denominators are redrawn. Final-state and trajectory intervals are descriptive. Each re- quest receives a separate response. Perturbation schedule. Twelve test dialogues receive one task-preserving paraphrase; six of these also receive one permutation of displayed semantic values. Comparisons preserve dialogue clusters and shared responses across arms. D Supporting Results and Robustness D.1 Shared Native-correct population The primary population contains 384 items per do- main and 64 per length for each model. The shared- correct comparison below conditions on items both Jev and direct-output Qwen select correctly. Inter- vals use 99,999 length-stratified paired item boot- 19 Table C4: Local field construction reduces Qwen mean-turn accuracy; the Jev contrast spans zero. Accuracy (%) has 95% dialogue bootstrap intervals. Dialogues receive equal mean-turn weight. Final-state EM scores the terminal dialogue state; trajectory EM requires correctness at every turn. ModelArmMean turnFinal stateTrajectory JevJoint76.0 [71.0, 80.7] 55.2 [44.8, 64.6] 33.3 [24.0, 42.7] JevIndependent 76.0 [71.0, 80.8] 56.2 [45.8, 65.6] 32.3 [22.9, 41.7] JevSerial75.5 [70.8, 80.0] 52.1 [41.7, 62.5] 27.1 [18.8, 36.5] Qwen3.5-27B Joint75.4 [70.8, 79.8] 57.3 [46.9, 66.7] 28.1 [19.8, 37.5] Qwen3.5-27B Independent 70.6 [66.1, 75.0] 58.3 [47.9, 67.7] 20.8 [12.5, 29.2] Qwen3.5-27B Serial72.3 [67.9, 76.6] 59.4 [49.0, 68.8] 24.0 [15.6, 32.3] Table C5: High state retention still requires more local requests. Format validity is in percent. Retention intervals weight dialogues equally and include raw retained turn counts. ModelArmRetained turns Retention [95% CI] Valid Calls Input tokens JevIndependent423/42998.5 [97.2, 99.6] 100.0 17161651282 JevJointreferencereference 100.0576822825 JevSerial420/42997.9 [96.6, 99.2] 100.0 17161686253 Qwen3.5-27B Independent388/42691.9 [88.9, 94.7]99.2 17161059954 Qwen3.5-27B Jointreferencereference 100.0576591013 Qwen3.5-27B Serial395/42693.3 [90.6, 95.8]99.4 17161084404 05101520 Oracle − Rollout (p) Qwen3.5-27B StartLux Jev TrajectoryMean turnFinal state Figure 8: Identical trajectory accuracy conceals differ- ences in later states. On 96 dialogues per model, gold prior states improve mean-turn and final-state accuracy over predicted prior states, while trajectory EM is un- changed. Exploratory 95% intervals resample dialogue groups linked by shared responses. strap draws with seed 20261008. D.2 Repeated Native observations Seventy-two parents (six per domain/length) re- ceive three identical Native requests and three Se- rial trajectories under pinned Jev. Repeats remain grouped by parent. Choice stability and probability stability have distinct endpoints. D.3 Natural task robustness under complete trajectories The 72-parent natural panel varies two paraphrases, two option orders and two additional trajectories −10−505 Serial − Joint (p) Trajectory Final state Mean turn JevQwen3.5-27B Figure 9: The accuracy effect of local field construction differs across models. On 96 dialogues with identical visible history and schemas, Qwen3.5-27B shows a cor- rected mean-turn loss, while Jev’s interval spans zero. Final-state intervals span zero for both models. Intervals are 95% dialogue bootstrap. separately from the reference, yielding seven set- tings per model. Each setting has its own Native conditioning; fixed-reference conditioning is addi- tionally retained. Complete trajectories use actual prefixes and whole-item paired intervals. These natural solving requests receive no target answer, unlike the supplied-target readout panel. D.4 Direct output natural prompt robustness Qwen3.5-27B uses 96 parents per domain, 16 per length selected before outcomes, under three in- terleaved templates. Each has its own Native re- 20 Table D1: The arithmetic–lexical contrast persists on a common Native-correct population. Coverage, retained counts and 95% item bootstrap intervals stratified by length are shown. ModelDomain Coverage RetainedSCR [95% CI] Jevmath367/384 118/367 32.2 [28.1, 36.2] Qwen3.5-27B math367/38438/36710.4 [7.4, 13.4] Jevlexical379/3848/3792.1 [0.8, 3.7] Qwen3.5-27B lexical379/384 131/379 34.6 [29.8, 39.3] Table D2: Across three repeats, Native choices agree but Serial successes vary. Counts are retained/Native correct. Domain Native correct Serial 0 Serial 1 Serial 2 math35/3612/3513/3512/35 lexical36/360/361/361/36 050100 SCR (%) Repeat 2 Repeat 1 Prompt v3 Prompt v2 Order 2 Order 1 Reference 12/35 11/35 10/35 13/34 12/35 13/35 12/35 (a) Jev Arithmetic −20020 Serial EM change (p) (b) Jev Arithmetic 050100 SCR (%) 0/36 2/36 1/36 2/36 0/36 1/36 1/36 (c) Jev Lexical −20020 Serial EM change (p) (d) Jev Lexical 050100 SCR (%) Repeat 2 Repeat 1 Prompt v3 Prompt v2 Order 2 Order 1 Reference 1/20 1/22 3/22 2/23 2/20 1/20 1/20 (e) StartLux Arithmetic −20020 Serial EM change (p) (f) StartLux Arithmetic 050100 SCR (%) 0/36 1/34 0/35 0/36 0/36 0/36 0/36 (g) StartLux Lexical −20020 Serial EM change (p) (h) StartLux Lexical 050100 SCR (%) Repeat 2 Repeat 1 Prompt v3 Prompt v2 Order 2 Order 1 Reference 0/5 0/10 0/7 0/8 0/5 0/5 0/5 (i) Qwen3-8B Arithmetic −20020 Serial EM change (p) (j) Qwen3-8B Arithmetic 050100 SCR (%) 0/34 0/33 0/34 0/34 0/34 0/34 0/34 (k) Qwen3-8B Lexical −20020 Serial EM change (p) (l) Qwen3-8B Lexical Figure 10: The Jev domain gap survives natural-task perturbations. Across 72 items, (a,c) Jev retains more arithmetic than lexical answers; (e,g) StartLux retention stays low; (i,k) Qwen3-8B legal-code construction has no successes. Paired changes are imprecise. SCR uses setting-specific Native successes, with 95% item bootstrap intervals. 21 quest and Full Serial trajectory. Bootstrap retains templates within parent/length; four nonreference Serial contrasts share one exact McNemar/Holm family. All responses satisfy the character format. The verbatim instructions appear in Appendix E. Means average the seven tested marginal settings in the common 72-parent panel or the three tem- plates in the separate Qwen panel. They are not expectations over an unspecified prompt distribu- tion. Worst-setting SCR and conditional denomina- tors are recomputed in each parent bootstrap draw. Within each original setting, SCR and Serial EM agree on model order: Jev leads all seven arith- metic settings and five lexical settings; two lexical settings have no successes. Table D4: Worst-template retention remains well below Native accuracy. Direct Qwen3.5-27B SCR (%) has 95% whole-item bootstrap intervals. Each domain re- samples 96 parents and recomputes its worst setting. DomainMean [95% CI]Worst [95% CI] Arithmetic 16.2 [12.1, 20.6] 7.4 [3.1, 11.7] Lexical48.6 [40.3, 56.9] 40.0 [30.5, 49.5] D.5 Candidate supported local construction All 768 natural parents receive fresh Native, Full Serial and Union Serial requests. Union filters the seeded alphabet to characters that occur at the cur- rent position in any displayed candidate, preserving choice identities and order. Singleton unions are filled with no call or assigned probability. Both lo- cal arms use their own actual prefix; neither queries END. Domains and conditions are interleaved. Jev scores returned choices, StartLux preserves seman- tic options and Qwen emits characters directly. Six Full/Union McNemar tests share Holm cor- rection. Intervals use 99,999 length-stratified boot- strap draws with seed 20261009. The tables distinguish paired recoveries/losses, actual requests, candidate membership and Carte- sian support. A position union can leave the com- plete candidate set; its constraint differs from Trie. Only two arithmetic positions have non-singleton unions; every lexical position does. D.6 Matching complete feasible outputs and controller calls Jev evaluates all 768 parents; Qwen uses 192 se- lected by item hash before outcomes (16 per do- main/length), with every arm of an item on one instance. Seed 20261010 interleaves conditions within position rounds. Full uses the domain alpha- bet. UnionCall uses position unions; TrieCall re- stricts choices to continuations of candidates com- patible with the generated prefix, preserving ex- actly eight feasible outputs. Call queries singletons. UnionFill/TrieFill instead supply them; FullFill supplies position-union singletons and otherwise uses Full. A wrong Trie branch can exclude gold. Malformed Trie prefixes abort, and later positions are logged with zero calls. Membership is checked for valid completions. Confidence is recorded for model calls, not deterministic fills. Instructions and filtered option order are fixed. Eight TrieCall versus Full/Union contrasts share one Holm family; twelve filling contrasts form a separate family. Bootstrap uses 9,999 whole-item draws within length. The exploratory domain con- trast pairs the 384 gold-prefix- geometry partners. Jev’s lexical-minus-arithmetic EM is−26.3points [−30.2,−22.4] under Full and 2.6 [−2.9,7.8] un- der TrieCall. The TrieCall interval includes both a negative and a positive domain contrast. D.7 Crossed controller factors on longer strings Thirty-two previously unused twelve-character sources over a–j are reversed. Each source has two eight-candidate presentations with the same four-by-two branching and Hamming geometry: branches occur at positions 2/3 or 10/11. Distrac- tor symbols are sampled from a–j; all remaining positions agree across candidates. The local al- phabet (a–j or a–z) and number of queried posi- tions (two or six) are crossed with branch place- ment. Both branch positions are always queried; six-query controllers also query positions 1,4,8,12. Other positions are filled only from singleton candi- date unions. These fills use no gold answer. Query count varies exposure at unambiguous positions while keeping ambiguous branch count fixed. Native is shared across the four local arms of each presentation. All requests retain the original problem, candidates, Serial instruction and actual prefix. Seed 20261011 freezes items and inter- leaves factors within position rounds. Each model makes 1,088 requests. All eight configurations re- main grouped by source in 9,999 bootstrap draws. Three average factor contrasts per model share six- test Holm correction; exact sign-permutation dis- tributions use source-level differences. This fixed- vocabulary reversal task isolates controller factors. 22 Table D3: Prompt wording changes Qwen content accuracy under valid character output. SCR uses template-specific Native successes. Changes from v1 have paired 95% intervals and four-test Holm correction. Domain Prompt Serial EM RetainedSCR [95% CI] EM change [95% CI] Holm p lexicalv139.638/95 40.0 [30.5, 49.5]referenceN/A lexicalv256.254/96 56.2 [46.9, 65.6]16.7 [7.3, 26.0]0.005 lexicalv350.047/95 49.5 [39.6, 59.4]10.4 [1.0, 19.8]0.105 mathv17.37/957.4 [3.1, 12.5]referenceN/A mathv228.127/94 28.7 [21.5, 35.8]20.8 [13.5, 28.1] <0.001 mathv313.512/9512.6 [7.3, 18.8]6.2 [-1.0, 13.5]0.180 020406080100 Complete string EM (%) Qwen3.5-27B StartLux Jev (a) Arithmetic 020406080100 Complete string EM (%) (b) Lexical 0102030405060 Union − Full paired EM gain (p) Qwen3.5-27B StartLux Jev * * * (c) Arithmetic: paired difference 0102030405060 Union − Full paired EM gain (p) * * * (d) Lexical: paired difference NativeFull SerialPosition union Figure 11: Candidate support improves Serial success while leaving a gap to Native. On 384 formal items per domain, (a,c) arithmetic gains combine option restriction with deterministic filling; (b,d) lexical gains retain a model request at every position. Intervals are paired 95% item bootstrap; asterisks indicate Holmp < 0.05across six tests. Table D5: Union recovers failed items while losing some Full successes (384 items per domain). Recovered means Full wrong/Union correct; lost means Full correct/Union wrong. Counts expose changes hidden by net accuracy. ModelDomainBoth wrongRecoveredLostBoth correct Jevmath841928100 Jevlexical2859108 StartLuxmath27410334 StartLuxlexical3661710 Qwen3.5-27Bmath2441051421 Qwen3.5-27Blexical1668730101 D.8 Solving through continuous and reconstructed-prefix outputs On the same 192 Qwen parents, WholeGenerate solves from the original problem, candidates and length in one uninterrupted response; no answer is supplied. PrefixContinue removes position/prefix- state fields, appends its constructed prefix to the assistant prompt, generates a suffix and consumes its first character. WholeGenerate uses JSON- text state; PrefixContinue uses structured state. 23 Table D6: Position unions reduce arithmetic calls while permitting noncandidate lexical strings. Membership percentages and recorded model request counts exclude retries. ModelDomain Full calls Union calls Forced Full member Union member Jevlexical2112211202.628.6 Jevmath2112768134433.9100.0 StartLuxlexical2112211201.09.1 StartLuxmath211276813445.2100.0 Qwen3.5-27B lexical21122112040.662.8 Qwen3.5-27B math2112768134420.198.2 Table D7: Matching gold-prefix ambiguity leaves much larger lexical Cartesian output spaces. Mean U averages union sizes within items; Forced counts deterministic positions; Cartesian support multiplies position-wise sizes. Domain Length Items Mean U Forced Min sizeMedianMax math3 to 83841+4/L−2888 lexical3645.220.064134224 lexical4645.430.03368121,568 lexical5645.730.01,8005,18412,544 lexical6645.600.09,60026,04057,624 lexical7645.680.031,104176,400614,656 lexical8645.800.0 201,600 1,128,960 2,370,816 Trie / fill Trie / call Union / call Full Serial 77.9 79.4 76.8 28.1 (a) Jev / arithmetic (384 items) Trie / fill Trie / call Union / call Full Serial 37.5 32.3 32.3 14.6 (b) Qwen3.5-27B / arithmetic (96 items) 0255075100 Complete-string EM (%) Trie / fill Trie / call Union / call Full Serial 81.8 82.0 24.0 1.8 (c) Jev / lexical (384 items) 0255075100 Complete-string EM (%) Trie / fill Trie / call Union / call Full Serial 67.7 65.6 42.7 28.1 (d) Qwen3.5-27B / lexical (96 items) Figure 12: Preserving complete candidate consistency restores much of lexical construction. Trie’s valid outputs belong to Native’s eight candidates; Call still queries singletons, while Fill supplies them. Jev’s domain contrast shrinks, but all local conditions remain below Native (dashed reference). Panels use 384 Jev or 96 Qwen parents per domain; intervals resample whole items within length. Native and Full are fresh, interleaved reruns on the same parents; Table 1 reports the original responses from the primary collection. Both allow 32 new tokens, versus eight for po- sitional queries, with disabled thinking and greedy decoding. Logs preserve every generated suffix and input/output token sequence. PrefixContinue is a repeated reconstruction con- troller, distinct from continuous decoding. Its ini- tial complete suffix is correct on 94/96 arithmetic and 92/96 lexical items. Invalid or empty subse- quent characters fail and enter the next state as an invalid marker. Zero lexical completions have an exact 95% upper limit of 3.8%. Six paired genera- tion contrasts have a separate Holm family. Table 3 in the main text reports the paired con- tent, format and request outcomes. 24 10 choices 2 queries 26 choices 2 queries 10 choices 6 queries 26 choices 6 queries Early Late Candidate branch location 15/3211/321/321/32 0/320/320/320/32 (a) Jev 10 choices 2 queries 26 choices 2 queries 10 choices 6 queries 26 choices 6 queries Early Late Candidate branch location 4/322/321/320/32 1/320/320/320/32 (b) Qwen3.5-27B −30−20−100 EM difference (p) 26 − 10 choices 6 − 2 queries Late − early (c) Paired factor effects Jev Qwen3.5-27B Figure 13: Query count and branch placement affect Jev more than alphabet size in this symbolic panel. On 32 twelve-character reversal sources, (a,b) counts show correct strings under crossed factors. (c) Paired effects have 95% source-bootstrap intervals; Jev’s query-count and branch-location contrasts survive six-test Holm correction. Table D8: Correctness depends on branch location and queried depth despite a fixed eight-candidate set. Each cell contains 32 sources; counts are complete-output correct and joint/Native correct. Query counts exclude deterministic fills. Exact 95% upper SCR limits for zero cells are 13.2% (0/26), 16.1% (0/21) and 20.6% (0/16). BranchChoices / queriesJev correctJev SCR countQwen correctQwen SCR count Early10 / 215/3215/314/323/21 Early26 / 211/3211/312/321/21 Early10 / 61/321/311/321/21 Early26 / 61/321/310/320/21 Late10 / 20/320/261/321/16 Late26 / 20/320/260/320/16 Late10 / 60/320/260/320/16 Late26 / 60/320/260/320/16 D.9 A length control for indexed copying Ninety-six parents (eight per domain/length) re- ceive RawCopy, IndexedCopy and PaddedCopy. Each supplies the gold target and its own prefix overLqueries; question, options and order are fixed. Nonindexed punctuation matches Indexed’s serialized-state character and UTF-8 byte length at the same prefix. Four Indexed-minus-Padded contrasts share Holm correction. A separate Qwen token-length control uses the same parents and three arms. Punctuation, exclud- ing answer symbols and index numbers, matches the pinned tokenizer’s full rendered Indexed re- quest at the same prefix. Every realized match is verified; generated prefixes remain arm-specific. Its two contrasts form a separate Holm family. Whole- item intervals are stratified by length. Both controls change information and salience as well as address- ing, so they do not isolate an internal mechanism. D.10 An empirical reference for composing imperfect decisions For each model/domain/length and matched pop- ulation, letc it indicate position correctness. The empirical independence reference isR = Q t c t , using binary observed outcomes, not reported model probabilities. Independent uses 384 par- ents per domain and TeacherForced 60, with all-items and Native-correct populations. The reference condition’s EM minusRreflects error dependence and item heterogeneity. Serial mi- nus that condition also changes prefix context and instructions. In 9,999 paired draws within length, all position means, products, EM and conditional denominators are recomputed together. The fig- ure shows Jev; all original models are included in the accompanying numerical results. D.11 Copying with fixed position criteria Ninety-six parents (eight arithmetic/lexical pairs per length) receive the same supplied target, absolute-position question and option order. Raw gives the string; Indexed adds its position table. Independent omits the draft; Serial supplies its ac- tual prefix. Batch bundles these questions: native multi-field decisions for Jev/StartLux, strict JSON for Qwen. Single-position generation allows eight tokens; malformed output fails without trimming, 25 Table D9: Indexed copying improves fidelity over nonindexed length controls. Byte matching fixes serialized-state length; token matching uses Qwen’s rendered prompt at a common prefix. Each model/domain has 48 parents. Paired differences have 95% item intervals, with separate Holm correction for byte and token comparisons. ModelDomain Match Indexed Padded Difference [95% CI]Holmp Jevmathbytes43/4817/4854.2 [43.8, 64.6]8.94e-08 Jevlexicalbytes47/488/4881.2 [72.9, 89.6]1.46e-11 Qwen3.5-27B mathbytes23/488/4831.2 [18.8, 43.8]0.00146 Qwen3.5-27B lexicalbytes47/4838/4818.8 [10.4, 27.1]0.00391 Qwen3.5-27B mathtokens23/489/4829.2 [16.7, 41.7]0.00258 Qwen3.5-27B lexicaltokens47/4834/4827.1 [16.7, 37.5] 0.000488 0 25 50 75 100 Complete-string success (%) (a) Math / Independent (384 items) 0 10 20 (b) Lexical / Independent (384 items) 345678 Target length 0 25 50 75 100 Complete-string success (%) (c) Math / TeacherForced (60 items) 345678 Target length 0 10 20 30 (d) Lexical / TeacherForced (60 items) Position-rate productReference string EMSerial string EM Figure 14: Local error rates already imply few correct lexical strings under an independence reference. Jev arithmetic Serial exceeds the Independent reference at most lengths, whereas lexical success remains low in both conditions. TeacherForced comparisons use their smaller matched population. Bands are 95% whole-item bootstrap intervals; the product is an empirical reference, not a calibrated sequence probability. extraction or case repair. The nested binding probe changes literal to opaque names at one preselected position, reusing identical literal requests. Six Serial indexing and six raw Batch-minus- Independent tests share Holm correction. Feedback contrasts and twelve raw/indexed binding contrasts have separate families. Intervals use 99,999 whole- item draws within length, seed 20261008. D.12 Copying supplied incorrect targets Ninety-six parents (eight per domain/length) each receive matched correct/incorrect targets under Raw and Indexed. Incorrect plans comprise 48 observed arithmetic selections, 28 observed lexical selections and 20 constructed lexical candidates; observed errors can come from another model, and two lexical plans are Jev’s own errors. This error-enriched population measures conditional fi- delity, separate from fresh-plan success prevalence. Records retain difference counts and distinguish observed from constructed plans. Six wrong-plan indexing and six indexed wrong- minus-correct tests use separate Holm families. Whole-parent bootstrap preserves target/evidence pairs (99,999 draws, seed 20261008). Exact bi- nomial intervals supplement boundary cells: for example, 48/48 gives [92.6%,100%]. 26 0255075100 Complete copy fidelity (%) Qwen3.5-27B B Qwen3.5-27B S Qwen3.5-27B I StartLux B StartLux S StartLux I Jev B Jev S Jev I 28 → 48 29 → 48 26 → 48 5 → 48 10 → 48 5 → 48 13 → 46 13 → 47 48 → 48 Count / 48 (a) Arithmetic: I / S / B 0255075100 Complete copy fidelity (%) Qwen3.5-27B B Qwen3.5-27B S Qwen3.5-27B I StartLux B StartLux S StartLux I Jev B Jev S Jev I 8 → 48 9 → 48 8 → 48 0 → 47 0 → 43 0 → 47 9 → 48 11 → 48 48 → 48 Count / 48 (b) Lexical: I / S / B 0255075100 Indexed − raw Serial fidelity (p) Qwen3.5-27B StartLux Jev (c) Arithmetic: paired indexing gain 0255075100 Indexed − raw Serial fidelity (p) Qwen3.5-27B StartLux Jev (d) Lexical: paired indexing gain RawIndexed Figure 15: Indexing improves complete copying under fixed position questions. With supplied gold targets and 48 parents per domain, (a,b) indexed fidelity is high across controllers, while raw Batch helps Qwen3.5-27B. (c,d) Serial indexing gains are positive for every model/domain. I/S/B denote Independent/Serial/Batch; paired 95% item bootstrap intervals preserve all conditions within each parent. Table D10: Literal names improve Qwen raw readout; indexed readout is correct under both bindings. Each domain has 48 parents. Differences are percentage points with paired 95% intervals and twelve-test Holm correction. ModelDomain Evidence Literal Opaque Difference [95% CI] Holm p Jevlexicalraw35342.1 [-8.3, 12.5]1 Jevmathraw43430.0 [-6.2, 6.2]1 Qwen3.5-27B lexicalraw371154.2 [41.7, 66.7] 3.3e-07 Qwen3.5-27B mathraw411456.2 [41.7, 68.8] 1.8e-07 StartLuxlexicalraw11110.0 [0.0, 0.0]1 StartLuxmathraw26260.0 [0.0, 0.0]1 D.13 Crossed prompt and option order robustness Each Qwen model receives 3,456 requests on the same 96 Copy parents: Indexed/Direct evidence, code/character output, three templates and three orders at one preselected position. Direct also sup- plies the queried character. Both contracts use opaque names, disabled thinking, greedy decod- ing and eight tokens. Display permutations pre- serve code- character bindings. Malformed re- sponses fail strict accuracy; format- conditioned accuracy is secondary. Seventy-two setting con- trasts share Holm correction; bootstrap uses 99,999 parent draws within length, seed 20261008. Template interactions average character-minus- code differences over orders within parent, then 27 050100 Fidelity to supplied target (%) Qwen3.5-27B StartLux Jev 28/48 48/48 34/48 48/48 12/48 48/48 10/48 47/48 17/48 48/48 15/48 48/48 (a) Arithmetic: correct and wrong targets 050100 Fidelity to supplied target (%) Qwen3.5-27B StartLux Jev 11/48 48/48 10/48 48/48 3/48 45/48 3/48 45/48 11/48 47/48 5/48 46/48 (b) Lexical: correct and wrong targets 050100 Fidelity to incorrect target (%) multiple differences Qwen3.5-27B multiple differences StartLux multiple differences Jev one substitution Qwen3.5-27B one substitution StartLux one substitution Jev 24 → 30 / 30 7 → 30 / 30 10 → 30 / 30 10 → 18 / 18 3 → 17 / 18 5 → 18 / 18 (c) Arithmetic: groups of incorrect targets 050100 Fidelity to incorrect target (%) constructed Qwen3.5-27B constructed StartLux constructed Jev observed error Qwen3.5-27B observed error StartLux observed error Jev 9 → 28 / 28 3 → 27 / 28 4 → 26 / 28 1 → 20 / 20 0 → 18 / 20 1 → 20 / 20 (d) Lexical: groups of incorrect targets Gold EM = 0 for every arm with an incorrect target Correct target / raw Correct target / indexed Wrong target / raw Wrong target / indexed Figure 16: Faithful copying can preserve errors. On 48 parents per domain with paired correct/incorrect targets, (a,b) indexing raises fidelity; Jev copies every indexed incorrect target. Gains cover (c) single/multiple arithmetic character differences and (d) observed/constructed lexical errors. Intervals use 95% item bootstrap. subtract T0 from T1 or T2. Sixteen sign- random- ization tests form a separate Holm family; whole- parent intervals use seed 20261008 and 99,999 draws. Differences of differences can exceed 100 percentage points. The strict failure taxonomy dis- tinguishes valid wrong content, bound-code substi- tutions, other codes, extra text, invalid output and truncation. Appendix E reproduces the complete contracts and output examples. Output scoring. Scoring priority is strict cor- rect, truncated, empty, format-valid wrong, cor- rect bound code, another listed code, extra text, then other invalid output. Codes follow the actual request; whitespace and truncation flags are also retained. All malformed responses fail. Interaction inference is specified in Appendix D.13. D.14 Calibration and discrimination uncertainty The original confidence population has 384 Se- rial items per domain. Reported masses, includ- ing rounding, are not renormalized. Product and geometric mean follow emitted characters; mini- mum top-one probability can differ when service selections disagree with argmax. Ten fixed width bins define ECE. Bootstrap uses 99,999 item draws within length, seed 20261005. Discrimination is undefined in one-class samples; empty reliability bins remain empty. No calibrator is fitted. D.15 Independent lexical calibration study All 2,347 retained independent targets use the original Native/Serial prompts and pinned Jev. Source quotas and review exclusions precede out- comes (Appendix A.3). Every returned choice 28 O0O1O2 T0 T1 T2 Qwen3-8B 72.9*60.4*70.8* 20.82.116.7 -33.3*-22.9-20.8 (a) Arithmetic Indexed O0O1O2 T0 T1 T2 8.339.6*39.6* -41.7*-10.4-8.3 -95.8*-64.6*-68.8* (b) Arithmetic Direct O0O1O2 T0 T1 T2 35.4*33.356.2* 2.1-12.5-8.3 -43.8*-22.9-16.7 (c) Lexical Indexed O0O1O2 T0 T1 T2 -4.2-12.5-12.5 -66.7*-83.3*-85.4* -100.0*-85.4*-89.6* (d) Lexical Direct O0O1O2 T0 T1 T2 Qwen3.5-27B 4.26.24.2 4.28.34.2 -27.1-72.9*-45.8* (e) Arithmetic Indexed O0O1O2 T0 T1 T2 0.02.10.0 0.02.12.1 -41.7*-85.4*-79.2* (f) Arithmetic Direct O0O1O2 T0 T1 T2 0.00.0-2.1 0.00.00.0 -64.6*-87.5*-81.2* (g) Lexical Indexed O0O1O2 T0 T1 T2 0.00.00.0 0.00.00.0 -66.7*-85.4*-81.2* (h) Lexical Direct −100 −75 −50 −25 0 25 50 75 100 Character − code (p) Figure 17: Prompt and option order change which output contract is more accurate. For single-position readout on 96 supplied-target parents, (a–d) Qwen3-8B has corrected code and character advantages in different settings; (e–h) Qwen3.5-27B’s corrected differences favor codes. Malformed outputs fail strict scoring. Asterisks mark Holm p < 0.05 after correction across all 72 paired comparisons. is scored, including probability-argmax disagree- ments. Bootstrap uses 9,999 draws within length, seed 20261008. Conditional exact 95% SCR upper limits are 0.67% (0/546) and 0.79% (0/468) for lengths seven and eight. Thirteen successes overall make fine-grained selection estimates imprecise. Negative target length is a fixed ranking base- line, without Brier/ECE. Scores share outcomes and bootstrap draws; one-class replicates are unde- fined. Source-mixture and equal-length weight- ing are evaluated separately. The independent lexical score-minus-length intervals span zero, so pooled discrimination does not establish incremen- tal screening utility beyond length. Source quotas reflect the limited short-noun pool. Equal-length standardization gives each length weight1/6and each item1/(6n L ); success classes are never rebalanced. Every bootstrap draw recom- putes weighted prevalence, SCR, Brier, ECE, AU- ROC and AP. The prevalence reference uses iden- tical weights; cohorts are not paired item by item. Fixed-coverage precision uses uniform expected inclusion for cutoff ties and reports its uncertainty. Ranking, calibration and selection utility are evalu- ated separately at the observed prevalence. D.16 Paired correctness turnover Table D15 reconstructs the four joint outcomes from paired responses. SCR conditions on Native success; Serial success on Native-wrong items de- scribes gains outside that population. The natural- task model ordering agrees under SCR and Serial EM. Semantic fields have lower Native coverage and larger conditional differences: Qwen named- field accuracy gains coexist with substantial losses of previously correct items. Marginal accuracies alone leave this overlap unidentified; its mathemat- ical bounds appear in Appendix A.1. D.17 Lexical output membership and semantic review We classify every lexical output in the matched Full/UnionCall/TrieCall/ TrieFill panel against its candidates and WordNet 3.0.In order, cate- gories are gold, another candidate, malformed (wrong length or outside lowercase ASCII), an- other gold-synset lemma, another indexed noun, another indexed word, and unindexed string. No off-candidate output is another lemma of the gold synset. Position-wise candidate recombination is recorded separately from semantic classification. Two reviewers independently assessed 50 dis- 29 −125−100−75−50−250 Change in character − code gap (p) Lexical / direct T2 − T0 Lexical / direct T1 − T0 Lexical / indexed T2 − T0 Lexical / indexed T1 − T0 Arithmetic / direct T2 − T0 Arithmetic / direct T1 − T0 Arithmetic / indexed T2 − T0 Arithmetic / indexed T1 − T0 * * * * * * * * (a) Qwen3-8B −125−100−75−50−250 Change in character − code gap (p) * * * * (b) Qwen3.5-27B Figure 18: Wording shifts the relative success of code and character output. Contrasts average option orders within 96 supplied-target parents. (a) Both alternative templates shift Qwen3-8B’s relative accuracy toward codes compared with T0; (b) Qwen3.5-27B changes survive correction for T2 but not T1. Intervals are paired 95% parent bootstrap; asterisks mark Holm p < 0.05 after correction across all sixteen interaction tests. Table D11: Native accuracy stays high while independent lexical Serial successes concentrate at shorter lengths. Item bootstrap 95% intervals stratify by length and redraw the Native denominator. Zero-success cells (†) use exact two-sided binomial intervals in place of degenerate bootstrap bounds. Length Items Native correct Serial correct RetainedSCR (%) 3616177/61 11.48 [4.92, 19.67] 431531033/3100.97 [0.00, 2.26] 544243922/4390.46 [0.00, 1.14] 651150411/5040.20 [0.00, 0.60] 754854600/5460.00 [0.00, 0.67]† 847046800/4680.00 [0.00, 0.79]† All2347232813 13/23280.56 [0.26, 0.86] Table D12: Product ranks arithmetic outcomes better than target length; independent source-mixture lexical gains remain uncertain. Differences compare Product with negative-length ranking, with paired 95% item intervals. Population / mixtureAUROC differenceAP difference Arithmetic0.068 [0.051, 0.086]0.175 [0.135, 0.216] Original lexical0.032 [-0.034, 0.085]0.128 [0.016, 0.425] New lexical / source0.011 [-0.043, 0.064]0.055 [-0.012, 0.249] New lexical / equal0.032 [-0.013, 0.075]0.141 [0.023, 0.444] Table D13: Equal-length weighting narrows the lexical SCR difference between cohorts. Accuracy and SCR (%) have 95% item bootstrap intervals. Equal length weights preserve each cohort’s vocabulary and candidate difficulty. PopulationMixtureNative EMSerial EMSCR Original lexical Equal length 99.5 [98.7, 100.0] 2.1 [0.8, 3.6] 2.1 [0.8, 3.6] New lexicalSource99.2 [98.8, 99.5] 0.6 [0.3, 0.9] 0.6 [0.3, 0.9] New lexicalEqual length99.3 [98.9, 99.6] 2.2 [0.9, 3.6] 2.2 [0.9, 3.6] 30 0.0 0.5 1.0 Arithmetic Observed correctness 0.00.51.0 Composed score 0 1 10 100 Items (a) Product 0.0 0.5 1.0 0.00.51.0 Composed score 0 1 10 100 (b) Geometric mean 0.0 0.5 1.0 0.00.51.0 Composed score 0 1 10 100 (c) Minimum top choice 0.0 0.5 1.0 Lexical Observed correctness 0.00.51.0 Composed score 0 1 10 100 Items (d) Product 0.0 0.5 1.0 0.00.51.0 Composed score 0 1 10 100 (e) Geometric mean 0.0 0.5 1.0 0.00.51.0 Composed score 0 1 10 100 (f) Minimum top choice Gray: all items Color: correct strings Whiskers: 95% intervals Figure 19: High local confidence can coexist with incorrect strings. On 384 Jev items per domain, (a–c) arithmetic scores track correctness but remain miscalibrated; (d–f) lexical successes are sparse, and local-confidence summaries often overstate correctness. Bin counts give each estimate’s support. Whole items are resampled within length. 0.0 0.5 1.0 Independent lexical Observed correctness 0.00.1 0.0 0.1 0.00.51.0 Composed score 0 1 10 100 1000 Items (a) Product 0.0 0.5 1.0 0.00.51.0 Composed score 0 1 10 100 1000 (b) Geometric mean 0.0 0.5 1.0 0.00.51.0 Composed score 0 1 10 100 1000 (c) Minimum top choice Gray: all items Color: correct strings Whiskers: 95% intervals Figure 20: Independent lexical evaluation again separates local confidence from string success. Among 2,347 Jev items with 13 Serial successes, (a) Product concentrates near zero; (b,c) higher geometric mean and minimum top choice scores overstate observed correctness. Bins show sparse successes; intervals resample items within length. tinct item/output pairs: all 26 indexed noncandidate pairs and four unindexed pairs per length, selected by ascending stable subject ID. Materials showed the definition, required length and proposed word, 31 Selected coverage (%) 0 25 50 75 100 Precision (%) (a) Arithmetic Selected coverage (%) 0 10 20 30 Precision (%) (b) Original lexical 102550 Selected coverage (%) 0.0 2.5 5.0 7.5 10.0 Precision (%) (c) Independent lexical 102550 Selected coverage (%) 0 10 20 30 Precision (%) (d) Independent lexical: equal length ProductGeometric meanMinimum top choiceLength Figure 21: Screening precision varies by task and length. (a) Arithmetic precision is high at low coverage; (b) original lexical estimates are imprecise. (c,d) Equal-length weighting raises independent-cohort precision without more successes. Paired 95% item bootstrap intervals use uniform expected inclusion for cutoff ties. Table D14: Lexical calibration and ranking vary with score and length mixture. Intervals are paired 95% item bootstrap. Equal assigns weight 1/6 to each length. Geom. denotes geometric mean. Population MixtureScoreBrierECEAUROCAP OriginalEqualProduct 0.019 [0.009, 0.030] 0.010 [0.005, 0.025] 0.936 [0.877, 0.978] 0.231 [0.081, 0.543] OriginalEqualGeom. 0.109 [0.101, 0.117] 0.286 [0.269, 0.301] 0.849 [0.659, 0.959] 0.164 [0.045, 0.461] OriginalEqualMinimum 0.057 [0.050, 0.064] 0.183 [0.169, 0.198] 0.871 [0.724, 0.966] 0.174 [0.048, 0.477] NewSourceProduct 0.005 [0.003, 0.008] 0.003 [0.001, 0.006] 0.910 [0.830, 0.973] 0.126 [0.041, 0.351] NewSourceGeom. 0.096 [0.093, 0.098] 0.286 [0.281, 0.290] 0.838 [0.706, 0.942] 0.066 [0.020, 0.191] NewSourceMinimum 0.044 [0.042, 0.046] 0.184 [0.180, 0.187] 0.806 [0.680, 0.909] 0.049 [0.012, 0.187] NewEqualProduct 0.019 [0.009, 0.030] 0.008 [0.003, 0.021] 0.926 [0.864, 0.965] 0.248 [0.089, 0.568] NewEqualGeom. 0.107 [0.102, 0.114] 0.283 [0.269, 0.296] 0.857 [0.726, 0.946] 0.175 [0.051, 0.434] NewEqualMinimum 0.056 [0.051, 0.063] 0.181 [0.168, 0.194] 0.859 [0.763, 0.933] 0.161 [0.043, 0.437] withholding gold targets, candidates, models and controllers. Reviewers judged noun validity and definition fit separately using yes, no or uncertain. Their judgments agreed on both dimensions for every pair: 33 outputs were rejected as nouns, 17 were accepted as nouns, and 49 did not fit the defi- nition. The remaining pair, pop for “a quick short straight punch,” received uncertain definition-fit judgments from both reviewers. No reviewed pair received unanimous noun-and- definition acceptance, so adding documented se- mantic alternatives leaves EM unchanged. Accept- ing the uncertain pair would change Jev UnionCall from 92/384 to 93/384 (24.0% to 24.2%); the other controller results remain unchanged. The 50 pairs map to 51 observed outputs because one pair re- curs across controllers. This sensitivity calculation covers reviewed pairs; unindexed outputs outside the sample retain unresolved semantic status. Ta- ble D16 reports resource categories separately from these human judgments. All analyses retain strict closed-set EM as the task score. D.18 Observed generation budgets Raw successful request records retain generated to- ken IDs and the truncation flag. All 1,056 position- query requests stop within two tokens, below the eight-token cap (Table D17). The 32-token whole- 32 Table D15: Marginal improvement can coexist with substantial correctness turnover. Joint counts give both correct / Native only / Serial only / neither. SCR, its difference from Serial EM (∆, percentage points), and Serial success on Native-wrong items (Gain rate) summarize different parts of the same paired table. Model / interfaceTaskItemsJoint countsSCR (%)∆Gain rate (%) Jevlexical3848/374/0/22.1+0.00.0 Jevmath384123/250/0/1133.0+0.90.0 Qwen3-8Blexical3840/363/0/210.0+0.00.0 Qwen3-8Bmath3840/59/0/3250.0+0.00.0 Qwen3.5 / codelexical3845/378/0/11.3+0.00.0 Qwen3.5 / codemath3841/96/1/2861.0+0.50.3 Qwen3.5 / directlexical384132/248/0/434.7+0.40.0 Qwen3.5 / directmath38439/339/0/610.3+0.20.0 StartLuxlexical3841/359/0/240.3+0.00.0 StartLuxmath3848/227/1/1483.4+1.10.7 Jevbinary fields12880/7/29/1292.0+6.870.7 Jevnamed fields128111/4/5/896.5+5.938.5 Qwen3-8Bbinary fields1281/18/19/905.3-10.417.4 Qwen3-8Bnamed fields1286/22/31/6921.4-7.531.0 StartLuxbinary fields12825/25/13/6550.0+20.316.7 StartLuxnamed fields12851/15/40/2277.3+6.264.5 Table D16: Most Full lexical outputs are absent from WordNet, while Trie completions stay within the candidates. Counts list gold / other candidate / malformed / other noun / other word / unindexed. No gold-synset alternatives were found. Recombinations are noncandidate strings compatible with every position-wise union; they can overlap the resource categories. This is an automated resource audit, not semantic adjudication. ModelControllerItemsOutput countsRecombinations JevFull3847/1/0/11/0/36522 JevTrieCall384315/69/0/0/0/00 JevTrieFill384314/70/0/0/0/00 JevUnionCall38492/9/0/11/1/271283 Qwen3.5-27BFull9627/7/0/2/1/592 Qwen3.5-27BTrieCall9663/28/5/0/0/00 Qwen3.5-27BTrieFill9665/31/0/0/0/00 Qwen3.5-27BUnionCall9641/14/22/1/0/1819 Table D17: Position-query failure occurs without hitting the output cap. Budgets and observed maxima are new tokens, including special tokens; truncated counts use recorded request flags. TaskInterfaceRequests Cap Max Truncated lexical Position528820 lexical PrefixContinue52832324 lexical WholeGenerate963240 mathPosition528820 mathPrefixContinue5283290 mathWholeGenerate963290 generation cap also never binds. Four lexical Pre- fixContinue requests reach their cap; its complete- trajectory format failures also include responses below the cap. This audit rules out position-budget truncation in this panel. It leaves prompt-state rep- resentation, assistant prefill tokenization and re- peated requests jointly changed, so the comparison identifies controller outcomes rather than an iso- lated internal autoregressive mechanism. E Complete Prompts and Observable Output Failures E.1Formal content and stopping instructions Boxes reproduce the request text verbatim. Blue braces denote item- specific substitutions; they are not literal model input. The content track queries L absolute positions without END. Native Select the complete correct answer from the provided candidates. 33 Independent Solve the original problem. Select its true answer character at the specified one-based position, independently of other positions. Serial Solve the original problem. Select its true answer character at the specified position. The current prefix is a draft and may contain errors; it does not change the original answer. TeacherForced shares the Serial instruction and substitutes the gold prefix in the state below. Gold Copy Copy copy_target exactly. Select its character at the specified one-based position. Every state contains problem and target_length. Native additionally contains all eight candidates. Independent adds the position, indexed from one. Serial adds position and current_prefix to the same problem/candidate state. TeacherForced uses the same text and state keys but substitutes the gold prefix. Gold Copy contains problem, target_length, position, current_prefix, and copy_target, and with- holds the complete answer candidates. Original SelfPlanSpell uses the Copy instruction and its se- lected plan as copy_target, retaining candidates. State is serialized as canonical JSON. Local cri- teria are all ten digits or all 26 lowercase letters, with deterministic displayed order. Each request reconstructs state. Native instead uses the eight complete strings as criteria. The stopping track permits END among its 51 options and uses the following Serial instruction: Serial with termination Solve the original problem. Select the next character after current_prefix. Select END immediately after the complete answer. It uses the same local state construction with a maximum of L+3 requests. The alphabet is low- ercase a to z, digits 0 to 9, the eleven punctuation symbols period, comma, exclamation mark, ques- tion mark, semicolon, colon, apostrophe, quota- tion mark, hyphen, left parenthesis and right paren- thesis, followed by space, newline, tab and END before deterministic shuffling. These punctuation identities are an explicit project reconstruction. The content track makes L position queries and ex- cludes END from its options. E.2 Complete natural task paraphrases and candidate support The original 72 item robustness panel uses v1 to v3 for Native, Independent and Serial. The Qwen3.5- 27B panel uses the same Native and Serial strings. Natural paraphrases change only the instruction, re- taining the controller’s problem, candidates, length, position and prefix. The T0–T2 readout templates below supply the target answer. v2, Native Which listed full answer correctly solves the problem? v2, Independent Which character occupies the specified one-based posi- tion of the answer to the original problem? v2, Serial Given the possibly incorrect draft prefix, identify the original answer character at the specified position. v3, Native Identify the correct complete solution among the candidate answers. v3, Independent Determine the complete solution and identify the character at the specified position. v3, Serial Use the draft history to determine the true solution char- acter at this position; the draft may be wrong. Position union Serial uses exactly the v1 Serial instruction and full state above. Its displayed local options are the seeded full list filtered to characters occurring at that position in any candidate. Their stable identities and relative order are retained. The controller returns a singleton without a request. Both Serial arms use the problem and full candidate set without receiving the fresh Native selection. For Qwen3.5-27B natural queries, a single user message contains JSON keys state, question, al- lowed_values, and response_format. State is the canonical JSON string used by the other interfaces, and question is the applicable instruction above. The Native response contract is: Native direct-output contract Output only the requested complete string. No explanation or markdown. The local response contract is: 34 Serial / generated-prefix state candidates: candidate_strings current_prefix: generated_prefix position: one_based_position problem: problem target_length: length TeacherForced / gold-prefix state candidates: candidate_strings current_prefix: gold_prefix position: one_based_position problem: problem target_length: length Table E1: Serial and TeacherForced share instructions and canonical JSON state; only prefix sources differ. Local direct-output contract Output only the single requested lowercase letter or digit. Requests use a single user message and greedy decoding with thinking disabled. The budgets are 32 new tokens for Native and eight for a local query. Options constrain the prompt without a token mask. After outer whitespace removal, scoring requires an exact listed value and a complete response. Format failures and truncated responses fail without repair. E.3 Continuous solving and prefix continuation Both generation arms share these instructions: Complete-answer solving Solve the original problem and output only its complete answer, without explanation or markdown. Complete-answer response contract Output only the complete answer. No explanation or markdown. Both receive the original problem, eight candi- dates and target length; neither receives the selected or gold answer. WholeGenerate places canonical JSON text in the user object’sstatefield. Prefix- Continue places the state object there and appends generated_prefixafter the assistant genera- tion boundary. Its state excludes absolute position and current-prefix fields. Both allow 32 new tokens; PrefixContinue consumes only the first suffix char- acter per request. Union/Trie and the controller- factor panel retain the verbatim Serial instruction, with the option/fill rules in Appendix D.6. E.4 Matched field construction contract Every requested field uses the following instruc- tion, followed by the literal suffixField:, the field name, ., and its schema description: Current-state field decision Determine the current service state after the latest USER turn. Copy categorical values exactly from the schema. Preserve earlier user-provided slot values unless the user changes or cancels them. Ignore SYSTEM suggestions unless accepted by the user. UNSET means no value is held; DONTCARE means the user explicitly has no preference. Select the current active intent, or NONE if no intent is active. Return each requested field. Field: field_name. schema_description Joint requests all fields. Independent requests only the current field. Both states contain schema and dialogue_history. Serial requests the current field and additionally includes partial_current_state containing its actual preceding field predictions for that turn. The field order is fixed before outcomes. Criteria use stable value IDs and the corresponding schema value descriptions. The same field order and value order are reused across arms. Jev receives these typed questions directly. Qwen3.5-27B receives one user message JSON ob- ject with state, field_queries, and response_format, with no added system message. The response con- tract is reproduced below: Dialogue JSON response Return only one JSON object mapping precisely the re- quested fields to their exact semantic values, not option IDs. No explanation or markdown. All three arms use a cap of 256 new tokens and greedy decoding with thinking disabled. The re- sponse must parse as one JSON object with pre- cisely the requested keys, each mapped to an exact legal string value. Extra keys, malformed JSON, il- legal values and truncation fail without repair. This field contract specifies JSON keys and legal values. The character contract specifies a single symbol. E.5 Full crossed readout contracts T0, T1 and T2 specify the same supplied target and output contract. Their wording emphasizes the criterion, the choice, and the response boundary differently. The grid measures model sensitivity to these wording differences. Each request contains one system message with the following text and one 35 T0 / Code response Return exactly ONE listed choice code, as a single up- percase letter. Do not output the option name or criterion value. Do not explain, reason aloud, add whitespace, punctuation, or any other text. T0 / Character response Return exactly ONE character described by the correct criterion: a lowercase letter or digit, as appropriate for the listed criteria. Do not output a choice code or option name. Do not explain, reason aloud, add whitespace, punctuation, or any other text. T1 / Code response Your entire response must be exactly one uppercase letter: the code of the selected choice. Use its criterion to decide. Omit the option name and criterion value. Include no explanation, reasoning, whitespace, punctuation, or additional text. T1 / Character response Your entire response must be exactly one character: the lowercase letter or digit specified by the correct choice’s criterion. Omit the code and option name. Include no explanation, reasoning, whitespace, punctuation, or additional text. T2 / Code response Choose the option whose criterion answers the question. Reply with its single uppercase choice code alone. Your response must contain no other characters, explanation, or formatting. T2 / Character response Choose the option whose criterion answers the question. Reply with the single lowercase letter or digit given by that criterion alone. Your response must contain no other characters, explanation, or formatting. Table E2: T0–T2 preserve the target, criterion bindings and user question. Verbatim code/character instructions differ in response representation; rows change emphasis on the criterion, choice and response boundary. user message containing ordered JSON. Thinking is disabled, decoding is ordinary greedy, and the maximum is eight new tokens for both contracts. Model identities are in Appendix A.7. Across T0 to T2, response length, allowable sym- bols and surrounding text remain fixed. T0 names the correct criterion, T1 the correct choice’s crite- rion, and T2 asks for a choice before its criterion. Answer and criterion bindings remain fixed. Shared user question (all six contracts) Select the character of copy_target at the one-based query_position. Auxiliary representations, when present, encode the same target. Each criterion description speci- fies an output character. Select by the criterion descrip- tion; option names identify criteria. Code response_format Select one choice and output only its single-character code. Character response_format Select one choice and output only the single character described by its criterion. User JSON has exactly state, question, choices, and response_format. All arms supply copy_target, query_position, indexing set toone-based, and the complete indexed_characters list. Direct evi- dence additionally supplies query_character. Each ordered choices entry maps a displayed uppercase code to option_name and criterion. Codes bind to characters once per parent. O0 to O2 permute dis- play order while preserving those bindings. Option names, response codes and character criteria are distinct in the opaque condition. E.6 Complete request examples The two requests below use indexed arithmetic evidence, character output and the reference or- der. Each uses its complete T0 or T2 system mes- sage, the shared user question and character re- sponse format above. State and ordered choices are printed in full; no direct query-character field is supplied. Thinking is disabled. Decoded re- sponses are scored without case repair, whitespace trimming or code mapping. 36 Indexed readout / correct character (T0) copy_target: 7470 query_position: 3 indexing: one-based indexed_characters: [1:7, 2:4, 3:7, 4:0] Each pair is a position/character object. Codeoption_namecriterion Ao_266a58860677 Bo_391552c099c11 Co_986b79d4b69d9 Do_a4aab3f1f0803 Eo_98089e6d36f70 Fo_d10a4bc9e0c15 Go_92e9e7e5922d6 Ho_ade1a1c2a61a8 Io_2bf175f9655e4 Jo_c11310c456c2 Complete response: "7". Required character: 7. No truncation. Indexed readout / bound code (T2) copy_target: 20936748 query_position: 3 indexing: one-based indexed_characters: [1:2, 2:0, 3:9, 4:3, 5:6, 6:7, 7:4, 8:8] Each pair is a position/character object. Codeoption_namecriterion Ao_ade1a1c2a61a8 Bo_a4aab3f1f0803 Co_c11310c456c2 Do_266a58860677 Eo_92e9e7e5922d6 Fo_986b79d4b69d9 Go_2bf175f9655e4 Ho_d10a4bc9e0c15 Io_98089e6d36f70 Jo_391552c099c11 Complete response: "F". Required character: 9. No truncation. Table E3: Correct criterion identification does not guarantee output compliance. The left response emits the required character; the right emits its bound code, which fails the same character contract. 37