Paper deep dive
When Stories Evolve: Benchmarking LLM Storytelling Across Agent Architectures in Open-Ended World Simulations
Yuqi Chen, Sixuan Li, Yunfeng Cai, Xueai Li, Ka Man Yan, Ying Li
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 91%
Last extracted: 8/22/2026, 2:44:21 AM
Summary
The paper introduces WSE-bench, a process benchmark for evaluating Large Language Models (LLMs) in open-ended world simulations. It assesses three distinct capabilities: sustained generation (Generation Coverage), canonical coherence (Consistency), and meaningful development (Richness). The study compares 12 models across 12 agent architectures and finds that these capabilities are distinct and sometimes competing, with a non-concave Pareto frontier between Consistency and Richness. Model scale primarily improves sustained generation but does not reliably enhance coherence or development.
Entities (14)
Relation Signals (12)
WSE-bench β evaluates β Generation Coverage
confidence 95% Β· WSE-bench, a process benchmark that separately evaluates sustained generation, canonical coherence, and meaningful development
WSE-bench β evaluates β Consistency
confidence 95% Β· WSE-bench, a process benchmark that separately evaluates sustained generation, canonical coherence, and meaningful development
WSE-bench β evaluates β Richness
confidence 95% Β· WSE-bench, a process benchmark that separately evaluates sustained generation, canonical coherence, and meaningful development
Model scale β improves β sustained generation
confidence 92% Β· Model scale chiefly improves sustained generation
Consistency β competeswith β Richness
confidence 90% Β· Consistency and Richness do not form a smooth trade-off... distinct and sometimes competing capacities.
Model scale β doesnotimprove β canonical coherence
confidence 90% Β· without producing reliable gains in canonical coherence
WMOD β generates β PlotPlan
confidence 90% Β· WMOD optionally organizes proposed actions into a pre-narration PlotPlan
Character Agent β proposesactionsfor β non-player characters
confidence 90% Β· B2 uses one Character Agent to propose for all non-player characters
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models can write fluent stories, but open-ended storytelling requires more than local fluency. In evolving world simulations and AI-native games, models must preserve facts, relationships, causal dependencies, and character states as the world changes. We introduce WSE-bench, a process benchmark that separately evaluates sustained generation, canonical coherence, and meaningful development in dynamic LLM storytelling. Generation Coverage records the proportion of planned narrative steps produced; Consistency tracks when canon breaks; and Richness measures how meaningfully branching, player-shaped trajectories develop. Across frontier models, Consistency and Richness do not form a smooth trade-off: their empirical Pareto frontier is non-concave, with several non-dominated intermediate configurations that no positive linear weighting can select. Added structure can enrich trajectories, but it does not uniformly improve coherence and may shorten them. Model scale chiefly improves sustained generation, without producing reliable gains in canonical coherence or meaningful development. These results show that sustained generation, canonical coherence, and meaningful development are distinct and sometimes competing capacities. WSE-bench makes those dynamics visible by extending narrative evaluation from finished stories to the processes that create them.
Tags
Links
- Source: https://arxiv.org/abs/2608.15654v1
- Canonical: https://arxiv.org/abs/2608.15654v1
Trouble viewing inline? Open PDF directly β
Full Text
90,013 characters extracted from source content.
Expand or collapse full text
When Stories Evolve: Benchmarking LLM Storytelling Across Agent Architectures in Open-Ended World Simulations Yuqi Chen1β π Sixuan Li2β Yunfeng Cai4 Xueai Li1 Ka Man Yan1 Ying Li3,4π 1The University of Hong Kong 2Peking University 3Tsinghua University 4Beijing Institute of Mathematical Sciences and Applications (BIMSA) chenyuqi@hku.hk liying0928@tsinghua.edu.cn β Equal contribution π Corresponding authors Abstract Large language models can write fluent stories, but open-ended storytelling requires more than local fluency. In evolving world simulations and AI-native games, models must preserve facts, relationships, causal dependencies, and character states as the world changes. We introduce WSE-bench, a process benchmark that separately evaluates sustained generation, canonical coherence, and meaningful development in dynamic LLM storytelling. Generation Coverage records the proportion of planned narrative steps produced; Consistency tracks when canon breaks; and Richness measures how meaningfully branching, player-shaped trajectories develop. Across frontier models, Consistency and Richness do not form a smooth trade-off: their empirical Pareto frontier is non-concave, with several non-dominated intermediate configurations that no positive linear weighting can select. Added structure can enrich trajectories, but it does not uniformly improve coherence and may shorten them. Model scale chiefly improves sustained generation, without producing reliable gains in canonical coherence or meaningful development. These results show that sustained generation, canonical coherence, and meaningful development are distinct and sometimes competing capacities. WSE-bench makes those dynamics visible by extending narrative evaluation from finished stories to the processes that create them. 1 Introduction An open-ended story has no final draft. In evolving world simulations and LLM-driven open-narrative games, each action changes the state from which the next scene is generated (17; 8; 26). A promise, injury, or transfer of knowledge can become binding evidence many steps later. Each continuation becomes part of the storyβs canon: the set of facts established so far. Open-ended storytelling can fail in three distinct ways: generation may stop, the story may contradict its accumulated canon, or it may remain coherent only by failing to develop. Evaluation must therefore distinguish sustained generation, canonical coherence, and meaningful development. Most narrative evaluation focuses on completed stories, including book-length quality (28) and consistency bugs (12). Such settings do not reveal when an evolving trajectory first breaks canon, when development stalls, or whether an architecture trades one capability for the other. Interactive systems combine memory, planning, character, and world components (17; 8; 3), but differences in worlds, models, and player policies obscure component effects. Long-horizon benchmarks expose compounding errors hidden by short evaluations (16; 21). Figure 1 illustrates the three requirements at stake: sustained generation, canonical coherence, and meaningful development. Figure 1: Conceptual requirements for open-ended storytelling across agent architectures: meaningful development (Richness), canonical coherence (Consistency), and sustained generation (Generation Coverage). WSE-bench evaluates evolving stories along these three dimensions. To evaluate these requirements, we introduce WSE-bench (When Stories Evolve Benchmark), a process benchmark that operationalizes them as Generation Coverage, Consistency, and Richness. Generation Coverage records the proportion of planned narrative steps successfully produced. Each episode starts from a common initial world and a model-independent opening. A fixed player policy selects attempted actions, the tested architecture determines what follows, and independent evaluators identify contradictions and meaningful developments. Sharing the episode setup and choice-selection schedule across conditions makes architectural differences easier to interpret. We examine how canonical coherence, meaningful development, and sustained generation vary with agent architecture and model choice, and whether dense-model scale produces consistent trends. Within-episode comparisons estimate architecture effects, while cross-model analyses characterize the models evaluated here. Beyond architecture and model differences, we ask whether the origin of a world shapes the evolving canon. We hypothesize that source-derived worlds may be more vulnerable to contradiction when generated trajectories diverge from plots potentially seen during pretraining. Comparing simulation-native settings with worlds adapted from published plays tests this possibility, which we call prior-canon interference. Together, paired architecture comparisons, cross-model analyses, and blinded validation separate story quality from sustained generation. They distinguish configurations that expand the attainable ConsistencyβRichness region from those that move along its boundary. Figure 2: WSE-bench combines standardized episode construction, recurrent generation loop, and systematic comparison of agent architectures. Each episode contains one player-controlled character and four non-player characters. At each step, B1B_1 lets the Story Agent decide consequences directly, B2B_2 uses one Character Agent to propose for all non-player characters, and B3B_3 uses one isolated Character Agent per non-player character; the Story Agent remains the sole narrator. WMOD optionally organizes proposed actions into a pre-narration PlotPlan of at most 12 beats, whereas WMEM optionally records a structured post-narration world-state update. Crossing the three bases with the two binary modules yields 12 architecture conditions. 2 Related Work Planning and long-form story generation Planning-based generators separate storyline construction from surface realization (30). Re3 conditions recursively on a global plan and current story state (29). DOME combines dynamic outlines, temporal memory, and conflict analysis (24), while FACTTRACK tracks temporally qualified world facts at the outline level (15). These systems establish planning and memory as plausible interventions, but generally compare bundled pipelines on completed drafts. Narrative evaluation and LLM judges OpenMEVA tests story metrics for human agreement, discourse-level coherence, and robustness (7). LongStoryEval organizes reader-relevant criteria for book-length narratives (28), and ConStory-Bench targets consistency bugs in long stories (12). G-Eval motivates rubric-based LLM evaluation (14), but creative-writing preference remains difficult even for strong judges (5). WSE-bench therefore uses dimension-specific Judges, blinded human validation, and step-level evidence. This design locates contradictions and substantive developments while keeping generation failures separate from Consistency and Richness. Role-playing and simulated story worlds Generative Agents and SOTOPIA study memory, planning, and multi-agent interaction (17; 32). CharacterBox evaluates role-playing through character trajectories in a narrator-coordinated sandbox (23). Other interactive systems combine authorial and character processes through directorβactor control, narrative planning, or character simulation (8; 26; 3; 31). BOOKWORLD and EvolvingWorld extend these ideas to persistent literary societies and co-evolving characterβworld state (20; 33). These studies usually assess bundled systems. Long-horizon and stateful agent evaluation AgentBoard evaluates multi-turn agents through process-level progress rather than final success alone (16). Long-horizon studies also show how small per-step errors compound with task length (21). Language-model world simulators expose unreliable state transitions (25), while trained text-based world models can learn coherent dynamics when behavioral coverage is adequate (13). Across these lines of work, system comparisons often conflate differences in models, worlds, player behavior, and architecture. WSE-bench holds the world, choice-selection policy, narrator role, output format, and tested model fixed within paired episodes while varying character-decision structure, WMOD, and WMEM independently. It evaluates inference-time behavior through canonical coherence, meaningful development, and sustained generation over long-horizon, open-ended trajectories. 3 Benchmark Framework WSE-bench represents each evolving story as a trajectory grounded in a common initial world. Figure 2 traces the framework from standardized episode construction through recurrent story generation to evaluation. Operationally, B1B_1 assigns character decisions to the Story Agent, B2B_2 adds one Character Agent for all non-player characters, and B3B_3 assigns one isolated Character Agent to each non-player character. WMOD supplies a pre-narration PlotPlan of at most 12 beats, while WMEM applies a structured state update after narration. The resulting trajectories are evaluated with Generation Coverage, Consistency, and Richness. 3.1 Initial Worlds and Episode Construction Each episode is grounded in an InitialWorld, a self-contained canonical starting state that specifies characters, relationships, persistent facts, and time. The benchmark collection used here draws from three source families. Two are simulation-native: Into-the-Painting (ITP), an open-narrative historical game world (2), and Smallville, a simulated society (17). The third consists of literary worlds adapted from the multilingual DraCor drama collection (6). The simulation-native settings do not prescribe a reference continuation. DraCor worlds, by contrast, begin from published plays whose plots and character arcs may be represented in model pretraining. A common InitialWorld format makes architectures comparable across sources. Each EpisodeSeed defines a connected cast, the focal player, their initial relationships and locations, and the instance language. The collection contains 42 five-character seeds, 14 per source family, balanced for focal-player representation and relationship structure. Each seed has a native-language opening created without an evaluated model. All 12 architecture conditions follow the same pre-specified choice schedule. The instances span Chinese, English, Dano-Norwegian, Russian, and Swedish. Appendix A provides further details on episode construction and cross-architecture comparison. 3.2 Evolving Trajectories The opening introduces the player-controlled character and multiple active non-player characters. At step t, a deterministic policy selects an option from the previous narrative. The selected option expresses what the player attempts, and the agent system determines its consequences and continues the story. Each step produces a narrative continuation NtN_t, options for the next action, and indicators of whether the story advances to a new day or ends. To keep the source of canon unambiguous, only the generated narrative can establish new story facts. The history supplied to later steps contains selected player intents, generated narratives, and time. When WMEM is enabled, it records a structured state update after each narrative, but that update cannot rewrite canon. A failed update therefore does not invalidate an already generated step. This trajectory-centered view does not prescribe a correct continuation. Instead, each new step is evaluated against the canon accumulated along its own path and by how meaningfully it develops that path. 3.3 Metrics Generation Coverage measures how much of the planned trajectory is produced; Consistency and Richness measure its canonical coherence and meaningful development. Generation: Trajectory Coverage For trajectory i, let LiL_i be the number of valid narrative steps available for evaluation, and let H be the planned horizon. For a cell containing n trajectories, Generation Coverage is G=100nβHββi=1nLi.G= 100nH _i=1^nL_i. (1) Trajectories with no valid narrative have Li=0L_i=0. Thus, G is the mean percentage of planned narrative steps observed across trajectories. It measures the ability to sustain generation through the planned horizon, not narrative quality. Continuations after generation failures or valid story endings remain unobserved. Consistency: Canonical Coherence A narrative step is inconsistent only when all of the following conditions hold: 1. the starting state or an earlier narrative explicitly establishes a fact, or unavoidably entails it; 2. the current narrative explicitly establishes or unavoidably entails a conflicting claim about the same entity, reference, and applicable time; and 3. the current narrative contains no explicit or clearly implied transition that reconciles the two. These criteria distinguish contradiction from narrated change. Missing information and unfulfilled player intents are not inconsistencies; style, plausibility, repetition, and progress are evaluated separately. The first step meeting these criteria marks the onset of inconsistency; uncertain cases remain separate. A trajectory that ends because generation fails has not demonstrated consistency over the missing future steps, but neither has it demonstrated a contradiction. Such trajectories are therefore right-censored, and a KaplanβMeier curve estimates the proportion remaining contradiction-free at each step (10). Let S^Cβ(t) S_C(t) denote this estimate, and let YtY_t be the number of trajectories still observed at step t with no earlier contradiction. The two Consistency scores are Cobs C_obs =100Hββt=1HS^Cβ(t), = 100H _t=1^H S_C(t), (2) Cadj C_adj =100Hβt=1H[Yt>0]S^C(t). = 100H _t=1^HI[Y_t>0] S_C(t). The primary score, CobsC_obs, carries forward the last estimable value if no trajectory remains under observation. The conservative alternative, CadjC_adj, instead assigns zero after observational support is exhausted. This prevents a low-coverage cell from appearing artificially favorable in the Pareto comparison without treating generation failure as a contradiction or imputing unseen content. Appendix B.2 gives the KaplanβMeier and risk-set definitions. Richness: Meaningful Development Richness Judges identify text-supported units that introduce or advance consequential events, character or relationship changes, story threads, or world elements. Repetition, stylistic elaboration, and text length alone do not count. The Richness score R combines the frequency and breadth of meaningful development. The three breadth dimensions capture event diversity, meaningful character and relationship change, and consequential world or social expansion. Let Qβ[0,1]Qβ[0,1] be the log-scaled frequency of non-repetitive development per observed narrative step, capped at the 95th percentile of the calibration data. Let Bβ[0,1]Bβ[0,1] be the mean normalized rating across these dimensions. Then R=100βQβB.R=100 QB. (3) The geometric mean prevents abundant but narrow activity from fully compensating for broad but sparse development. Appendix B.4 gives the exact calculation of Q and B. Consistency and Richness remain independent during evaluation. A canon-contradicting development still counts toward Richness and is penalized only by Consistency. Conversely, a static trajectory can remain coherent while receiving low Richness. Because these qualities can diverge, the results compare them jointly without collapsing them into a single score. 4 Agent Architectures The evaluated grid combines one of three base architectures with zero, one, or two add-on modules: b,w,p _b,w,p =BbβWMEMwβWMODp, =B_b _w _p, (4) b b β1,2,3,w,pβoff,on. β\1,2,3\, w,pβ\off,on\. Here β denotes architectural composition, and w and p indicate whether WMEM and WMOD are enabled. Each base therefore has four module configurations: neither module, memory only, planning only, or both. For each tested model, the same model and operating mode are used for every enabled component. B1B_1 has no Character Agent. The shared Story Agent decides consequences and narrates. B2B_2 uses one Character Agent to propose actions jointly for all non-player characters, whereas B3B_3 uses a separate Character Agent for each non-player character. These bases reflect three character-action topologies from prior work: single-narrator long-form generation (B1B_1) (29), centralized multi-character coordination (B2B_2) (33), and decentralized character agents (B3B_3) (17; 8; 31). WMOD inserts a stateless World Model before narration. It adjudicates the selected player attempt together with any Character-Agent proposals and returns a PlotPlan of at most 12 dependency-linked beats; the Story Agent receives this plan in place of the raw proposals. WMEM runs after narration: a World Memory Agent compares the realized narrative with prior structured state, emits a MemoryDelta of persistent changes, and a rule-based reducer updates character conditions, relationships, facts, and supporting characters for the next step. The Story Agent remains the sole narrator, and only the initial world and generated narratives establish canon. Paired conditions share the episode setup, choice-selection schedule, Story Agent role, and tested model. Appendix A shows the full information flow, and Appendix C gives results for each base architecture. 5 Evaluation and Validation 5.1 LLM-Judge Evaluation Evaluators receive the InitialWorld, opening, and generated trajectory, but no model, architecture, internal generation data, or source-revealing identifiers. Appendix B details these evaluation inputs. Consistency Judges apply the three criteria above to every narrative step. To mark a contradiction, a Judge must identify the prior fact, the conflicting claim, their mutual exclusion, and the missing transition. Each Judge verdict has three statesβconsistent, inconsistent, or uncertainβand the uncertain state is retained through aggregation. Only a valid majority of consistent or inconsistent votes yields a binary step verdict for the first-contradiction risk set; cases with a missing Judge or no valid majority are excluded from that binary risk set. Richness Judges return the development units and dimension scores defined above , and the trajectory-level median determines R. 5.2 Human Validation of Judges Human annotations provide references on separate held-out samples, using the same rubrics without model, architecture, or Judge outputs. For Consistency, design-weighted comparison of a valid two-of-three Judge majority with human labels on 120 trajectories reaches 85.7% accuracy (95% CI 81.381.3β90.1%90.1\%) and 82.0% balanced accuracy. For Richness, the three-Judge median on a separate 36-trajectory sample achieves MAE 7.637.63 (95% CI 6.266.26β8.958.95) and Pearson r=.861r=.861 against the human reference. We use these validation results to assess the benchmarkβs aggregate Judge measures; Appendix B.5 details sampling and agreement. 5.3 Experimental Design and Analysis The evaluation crosses 12 models, 42 shared episode seeds, and 12 architectures, yielding 6,048 trajectories and 144 cells. One modelβarchitecture cell contains all 42 seeds; its model fills every enabled LLM role. The Qwen panel contains dense Qwen2.5-Instruct 3B, 7B, 14B, 32B, and 72B checkpoints (19), plus Qwen3.5 -397B-A17B MoE. The Frontier panel contains Grok-4.3, GLM-5.2, DeepSeek-V4-Pro, Kimi-K2.6, Gemini-3.1-Pro, and GPT-5.6-sol. Panel summaries refer to and equally weight these twelve tested models. Architecture contrasts pair cells over shared seeds and weight evaluated models equally. Their intervals hierarchically bootstrap source clusters and EpisodeSeed records (Appendix C); source-family contrasts also weight models equally (Appendix E). The heatmap reports CobsC_obs beside Generation Coverage, whereas Pareto analysis uses CadjC_adj to avoid carrying survival beyond observational support. Dense-Qwen, source-family, and horizon analyses respectively describe five-checkpoint association, the studied worlds, and when step-20 events first appear. All analyses retain Generation Coverage, Consistency, and Richness as separate outcomes. 6 Experiments and Results The experiments compare outcome profiles across models and architectures, then estimate paired architectural shifts, dense-Qwen scale associations, source-family patterns, and long-horizon failures. Qwen3.5 MoE is analyzed separately from dense-model scaling. Figure 3: Model-by-architecture point estimates for the Qwen and Frontier panels. Columns index models, and rows index the 12 architecture conditions. The three panels show Generation Coverage G, Consistency CobsC_obs, and Richness R; the dashed rule separates the two analysis groups. Paired architecture contrasts below provide uncertainty intervals. Figure 3 maps how Generation Coverage, Consistency, and Richness vary across the evaluated modelβarchitecture conditions. Generation Coverage shows the clearest between-model separation, including an apparent gradient across the dense Qwen2.5 checkpoints. Consistency remains high for many Frontier conditions but changes heterogeneously across architecture rows, whereas Richness is generally greater in the B2B_2 and B3B_3 rows and produces an ordering distinct from Coverage and Consistency. The cross-panel outcome differences motivate Figure 4βs joint ConsistencyβRichness analysis; the heterogeneous within-model architecture shifts motivate the paired contrasts in Table 1; and the model and scale analysis below quantifies the apparent between-model ordering and checkpoint gradient. 6.1 ConsistencyβRichness Trade-off Figure 4: ConsistencyβRichness point estimates based on CadjC_adj, which sets the survival contribution to zero after a cell loses observational support. Shapes denote B1B_1 (Story Agent only), B2B_2 (one centralized Character Agent), and B3B_3 (distributed Character Agents); white overlays mark WMEM, WMOD, or both. The upper-right boundary identifies non-dominated configurations, and supported configurations lie on its upper concave envelope. At the plotted point estimates, Figure 4 reveals a non-concave empirical ConsistencyβRichness frontier: six modelβarchitecture cells are non-dominated, but only two lie on the upper concave envelope. We use CadjC_adj so the comparison reflects contradiction-free survival while it remains observationally supported. Replacing CadjC_adj with CobsC_obs leaves the same pooled six-cell non-dominated set across all 144 cells (Appendix Figure 7). DeepSeek-V4-Pro and GLM-5.2 occupy the Richness-favoring region, GPT-5.6-sol and Gemini-3.1-Pro occupy the Consistency-favoring end, and Kimi-K2.6 lies between them. DeepSeek-V4-Pro B3+WMOD and Gemini-3.1-Pro B3+WMEM+WMOD are the two supported configurations. The four intermediate cells are unsupported: they are non-dominated among observed configurations but cannot maximize any positive linear weighting of the two outcomes (4). Every Grok-4.3 condition is dominated. The adjacent GPT-5.6-sol cell lies 0.05 Richness points below the envelope; this gap is small relative to sampling uncertainty, leaving that cellβs supportedness uncertain. All pooled-frontier cells come from Frontier models, while Qwen cells exhibit internal trade-offs. The four unsupported cells represent distinct observed compromises that a single weighted score would skip, showing why separate Consistency and Richness results preserve choices hidden by scalar ranking. 6.2 Architectures, Models, and Worlds Beyond the joint ConsistencyβRichness geometry, the individual outcomes vary with architecture, model identity, dense-model scale, and InitialWorld source family. Architecture comparisons are paired within episodes. Agent Architecture Effects Contrast ΞβG G ΞβCoβbβs C_obs ΞβCaβdβj C_adj ΞβR R Qwen series B2 β B1 +0.8 [-1.0, +2.5] -9.2 [-14.5, -3.2] -3.4 [-8.3, +3.3] +4.0 [+2.6, +6.0] B3 β B1 -1.1 [-3.0, +0.9] -12.0 [-17.7, -5.4] -8.4 [-10.5, -1.7] +7.0 [+4.4, +9.0] B3 β B2 -1.9 [-3.5, -0.3] -2.8 [-8.5, +3.6] -5.0 [-7.7, +0.4] +3.0 [+0.2, +4.8] WMEM on β off -18.0 [-19.8, -16.1] -5.6 [-10.2, -0.8] -6.0 [-8.8, -0.6] -1.3 [-2.5, +0.1] WMOD on β off -13.7 [-15.6, -11.7] +4.7 [-1.0, +11.9] -22.9 [-25.7, -16.0] -5.9 [-7.2, -4.2] Frontier models B2 β B1 -4.1 [-6.2, -1.9] -3.2 [-5.2, -1.2] -3.3 [-5.3, -1.4] +9.2 [+7.8, +10.5] B3 β B1 -7.5 [-10.8, -4.1] -2.7 [-5.2, -0.4] -2.7 [-5.4, -0.4] +14.6 [+13.3, +16.0] B3 β B2 -3.4 [-6.1, -0.7] +0.5 [-1.8, +2.8] +0.6 [-1.9, +2.9] +5.5 [+4.5, +6.5] WMEM on β off -14.1 [-16.3, -11.9] +0.3 [-1.3, +2.0] +0.2 [-1.8, +1.7] -2.4 [-3.0, -1.8] WMOD on β off -5.5 [-7.2, -3.8] -5.3 [-7.1, -3.4] -5.2 [-6.9, -3.1] +3.4 [+2.4, +4.2] Table 1: Architecture effects on the four reported outcomes. Each contrast averages over the other architecture factors. Values are mean differences across the evaluated models, weighted equally, with 95% hierarchical cluster-bootstrap intervals. Table 1 shows that paired changes to character decision structure mainly shift the ConsistencyβRichness trade-off. Among Frontier models, B2 and B3 raise Richness by 9.2 and 14.6 points relative to B1 while reducing CobsC_obs by 3.2 and 2.7 points. B3 βs independent proposals may increase both development and coordination burden, echoing evidence that multi-agent collaboration can become counterproductive under excess coordination demands (11; 22). The effect of WMOD depends on the character-decision topology. For Frontier models, it changes Richness by only +0.4β[β1.3,+2.1]+0.4\;[-1.3,+2.1] points at B1. The gains are larger at B2 and B3: +4.3β[+2.6,+6.0]+4.3\;[+2.6,+6.0] and +5.3β[+4.0,+6.6]+5.3\;[+4.0,+6.6], respectively. This pattern is consistent with planning being most useful when the Story Agent must coordinate separately proposed character actions, although WMOD does not produce a corresponding Consistency gain. Appendix C gives the results for each base architecture. The marginal effect of WMOD also differs across model groups. It raises Frontier Richness by 3.4 points but lowers Qwen Richness by 5.9 points, while reducing Generation Coverage in both groups. The same planning scaffold is therefore not uniformly useful; its effect depends on the model used with it. WMEM provides no stable Consistency benefit at any base and reduces Generation Coverage in both groups. Because every condition retains the complete narrative history, this result concerns structured state added to full-history access, not memory as a substitute for history. One possible explanation is that long contexts can obscure distant evidence, while extracted summaries may lose information and introduce an additional failure point (27; 18). Stronger graph-memory systems often add retrieval, temporal provenance, and conflict handling (1; 9). Appendix C gives model-specific results and module interactions. Model and Scale Effects When scores are averaged across architectures, Figure 3 shows that no Frontier model leads all three outcomes. GPT-5.6-sol combines the highest mean Consistency with near-complete Generation Coverage (Cobs=99.0C_obs=99.0, G=97.6G=97.6). DeepSeek-V4-Pro instead has the highest mean Richness (R=68.0R=68.0) but lower Consistency and Generation Coverage. Grok-4.3 nearly matches GPT-5.6-sol in Generation Coverage (G=97.2G=97.2) while producing substantially lower Richness (R=40.8R=40.8). These profiles complement the Pareto analysis in Section 6.1 and show that model differences cannot be reduced to a single quality ordering. Scale is a separate question that can be studied within a comparable model family. Across five dense Qwen2.5 checkpoints, Generation Coverage rises by an estimated 19.32 percentage points per parameter doubling. Appendix D provides the full analysis. Larger parameter count does not yield a reliable monotonic improvement in either story measure. Consistency rises across much of the dense series, but low Generation Coverage among smaller models leaves unstable survival tails. Richness is also non-monotonic. Thus, within the tested Qwen2.5 range, parameter count strongly predicts sustained generation but not canonical coherence or meaningful development. Source-Family Effects Source-family comparisons test the prior-canon interference hypothesis: DraCor worlds may become more inconsistent than the simulation-native ITP and Smallville worlds after branching from potentially familiar plots. Each of the 12 models receives equal weight. Figure 5: Per-model Consistency CobsC_obs across the three world families, with each model pooled over 12 architectures. Markers and whiskers show model-specific means and 95% hierarchical cluster-bootstrap intervals. The plotted profiles show within-model source-family variation; aggregate source-family contrasts give each of the 12 evaluated models equal weight. Contrary to this hypothesis, Figure 5 shows that, across the 12 evaluated models, CobsC_obs averages 82.1 for ITP, 83.5 for DraCor, and 78.1 for Smallville. DraCor exceeds ITP by +1.4 points (95% CI [-4.6, +6.9]) and Smallville by +5.4 points ([-2.2, +9.9]). Both CobsC_obs intervals include zero. Results under CadjC_adj are more favorable to DraCor and likewise do not indicate a DraCor disadvantage. DraCor is not uniquely prone to internal contradiction over the evaluated horizon. Among inconsistent trajectories, the first contradiction appears at approximately step 10 in all three source families (Appendix Table 7). We also repeat the comparison after removing each of the seven DraCor plays in turn. The overall comparison remains stable regardless of which play is removed (Appendix Figure 15). 6.3 Long-Horizon Error Patterns When contradictions appear A 10-step evaluation would miss 36.9% of the contradictions eventually observed by step 20 in the Qwen group and 56.1% in the Frontier group. Among cases with an agreed category, temporal conflicts are the most common in both groups. Identity and relationship conflicts tend to surface early against InitialWorld commitments, whereas character-state and world-fact conflicts more often arise later from the accumulated narrative history. The longer horizon thus exposes contradictions that short continuations systematically miss (Appendix B.6; Table 5). Semantic contradiction versus technical truncation Long-horizon breakdowns comprise semantic contradictions and technical truncations. A failure in a Character Agent, WMOD, or the Story Agent prevents the current narrative from being produced, whereas a WMEM failure follows a valid narrative and can truncate the subsequent trajectory. A failed post-narration update can therefore preserve a coherent prefix while eliminating the future in which its consequences would have unfolded. Some contradictions echo content introduced in Character-Agent proposals or WMOD plans, consistent with a conflict surviving multiple handoffs. Most cases, however, have no strong localized upstream match. The trace identifies possible error pathways without assigning causal origins (Appendix C). 7 Conclusion Evaluating stories as evolving trajectories reveals that sustained generation, canonical coherence, and meaningful development are distinct capacities. A trajectory can remain coherent by changing little, become richly eventful at the expense of accumulated canon, or end before either quality is tested over a long horizon. Keeping these outcomes separate shifts the question from whether a model can write a convincing continuation to whether it can sustain a world through consequential change. More architecture is not necessarily more control. Greater scale chiefly extends how long generation lasts, while additional stages often shift trade-offs among the three outcomes rather than improving all of them. Components should earn their complexity by supplying information, coordination, or enforcement that downstream generation can actually use. The source-family comparison does not support the simple expectation that divergence from a familiar plot necessarily destabilizes canon. More broadly, process evaluation must preserve both time and failure type: contradictions may emerge after substantial canon has accumulated, whereas component failures can end generation without producing contradictory text. The non-concave Frontier-model Pareto set reveals specialized Consistency- and Richness-favoring regimes rather than a smooth route to balance. The open challenge is to build systems whose stories can keep changing without losing what they have already become. References Banerjee et al. (2026) P. Banerjee, M. Moshtaghi, S. Subramanian, A. Misra, and A. Chadha APEX-MEM: agentic semi-structured memory with temporal reasoning for long-term conversational AI. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 16470β16489. External Links: Document Cited by: Β§6.2. Chen (2026) Y. Chen Into the Painting: along the river during the qingming festival. Note: https://fishframe.net/itpAccessed 2026-07-29 External Links: Link Cited by: Β§3.1. Chen et al. (2026) Z. Chen, R. Pan, and H. Li StoryBox: collaborative multi-agent simulation for hybrid bottom-up long-form story generation using large language models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40, p. 30359β30367. External Links: Document Cited by: Β§1, Β§2. Ehrgott (2005) M. Ehrgott Multicriteria optimization. 2 edition, Springer, Berlin, Heidelberg. External Links: Document Cited by: Β§6.1. Fein et al. (2026) D. Fein, S. Russo, V. Xiang, K. Jolly, R. Rafailov, and N. Haber LitBench: a benchmark and dataset for reliable evaluation of creative writing. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), p. 7740β7755. External Links: Document Cited by: Β§2. Fischer et al. (2019) F. Fischer, I. BΓΆrner, M. GΓΆbel, A. Hechtl, C. Kittel, C. Milling, and P. Trilcke Programmable corpora: introducing DraCor, an infrastructure for the research on european drama. In Proceedings of DH2019: Complexities, Utrecht, the Netherlands. External Links: Document Cited by: Β§3.1. Guan et al. (2021) J. Guan, Z. Zhang, Z. Feng, Z. Liu, W. Ding, X. Mao, C. Fan, and M. Huang OpenMEVA: a benchmark for evaluating open-ended story generation metrics. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), p. 6394β6407. External Links: Document Cited by: Β§2. Han et al. (2024) S. Han, L. Chen, L. Lin, Z. Xu, and K. Yu IBSEN: director-actor agent collaboration for controllable and interactive drama script generation. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 1607β1619. External Links: Document Cited by: Β§1, Β§1, Β§2, Β§4. Hu et al. (2026) S. Hu, Y. Wei, J. Ran, X. Han, Z. Yao, H. Wang, R. Chen, and L. Zou Does memory need graphs? a unified framework and empirical analysis for long-term dialog memory. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 26758β26782. External Links: Document Cited by: Β§6.2. Kaplan and Meier (1958) E. L. Kaplan and P. Meier Nonparametric estimation from incomplete observations. Journal of the American Statistical Association 53 (282), p. 457β481. External Links: Document Cited by: Β§3.3. Kim et al. (2026) Y. Kim, K. Gu, C. Park, C. Park, S. Schmidgall, A. A. Heydari, Y. Yan, Z. Zhang, Y. Zhuang, Y. Liu, M. Malhotra, P. P. Liang, H. W. Park, Y. Yang, X. Xu, Y. Du, S. Patel, T. Althoff, D. McDuff, and X. Liu Capable language models can outgrow the benefits of collaboration. Nature Machine Intelligence 8, p. 1157β1172. External Links: Document Cited by: Β§6.2. Li et al. (2026a) J. Li, X. Guo, Y. Wu, R. K. Lee, H. Li, and Y. Xie Lost in stories: consistency bugs in long story generation by LLMs. In Findings of the Association for Computational Linguistics: ACL 2026, p. 8400β8428. External Links: Document Cited by: Β§1, Β§2. Li et al. (2026b) Y. Li, H. Wang, J. Qiu, Z. Yin, D. Zhang, C. Qian, Z. Li, X. Ma, G. Chen, and H. Ji From word to world: can large language models be implicit text-based world models?. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 8084β8111. External Links: Document Cited by: Β§2. Liu et al. (2023) Y. Liu, D. Iter, Y. Xu, S. Wang, R. Xu, and C. Zhu G-Eval: NLG evaluation using GPT-4 with better human alignment. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 2511β2522. External Links: Document Cited by: Β§2. Lyu et al. (2025) Z. Lyu, K. Yang, L. Kong, and D. Klein FACTTRACK: time-aware world state tracking in story outlines. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 2825β2848. External Links: Document Cited by: Β§2. Ma et al. (2024) C. Ma, J. Zhang, Z. Zhu, C. Yang, Y. Yang, Y. Jin, Z. Lan, L. Kong, and J. He AgentBoard: an analytical evaluation board of multi-turn LLM agents. In Advances in Neural Information Processing Systems, Vol. 37. External Links: Document Cited by: Β§1, Β§2. Park et al. (2023) J. S. Park, J. C. OβBrien, C. J. Cai, M. R. Morris, P. Liang, and M. S. Bernstein Generative agents: interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, p. 1β22. External Links: Document Cited by: Β§1, Β§1, Β§2, Β§3.1, Β§4. Pollertlam and Kornsuwannawit (2026) N. Pollertlam and W. Kornsuwannawit Beyond the context window: a cost-performance analysis of fact-based memory vs. long-context LLMs for persistent agents. External Links: 2603.04814, Link Cited by: Β§6.2. Qwen Team (2024) Qwen Team Qwen2.5 Technical Report. arXiv preprint arXiv:2412.15115. External Links: Link Cited by: Β§5.3. Ran et al. (2025) Y. Ran, X. Wang, T. Qiu, J. Liang, Y. Xiao, and D. Yang BOOKWORLD: from novels to interactive agent societies for story creation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 15898β15912. External Links: Document Cited by: Β§2. Sinha et al. (2026) A. Sinha, A. Arun, S. Goel, S. Staab, and J. Geiping The illusion of diminishing returns: measuring long horizon execution in LLMs. In The Fourteenth International Conference on Learning Representations, External Links: Link Cited by: Β§1, Β§2. Sun et al. (2026) Y. Sun, X. Ren, K. Zhang, X. Liu, and J. Guo PerspectiveGap: a benchmark for multi-agent orchestration prompting. External Links: 2606.08878, Link Cited by: Β§6.2. Wang et al. (2025a) L. Wang, J. Lian, Y. Huang, Y. Dai, H. Li, X. Chen, X. Xie, and J. Wen CharacterBox: evaluating the role-playing capabilities of LLMs in text-based virtual worlds. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 6372β6391. External Links: Document Cited by: Β§2. Wang et al. (2025b) Q. Wang, J. Hu, Z. Li, Y. Wang, D. Li, Y. Hu, and M. Tan Generating long-form story using dynamic hierarchical outlining with memory-enhancement. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 1352β1391. External Links: Document Cited by: Β§2. Wang et al. (2024a) R. Wang, G. Todd, Z. Xiao, X. Yuan, M. CΓ΄tΓ©, P. Clark, and P. Jansen Can language models serve as text-based world simulators?. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), p. 1β17. External Links: Document Cited by: Β§2. Wang et al. (2024b) Y. Wang, Q. Zhou, and D. Ledo StoryVerse: towards co-authoring dynamic plot with llm-based character simulation via narrative planning. In Proceedings of the 19th International Conference on the Foundations of Digital Games, External Links: Document Cited by: Β§1, Β§2. Wu et al. (2025) D. Wu, H. Wang, W. Yu, Y. Zhang, K. Chang, and D. Yu LongMemEval: benchmarking chat assistants on long-term interactive memory. In International Conference on Learning Representations, External Links: Link Cited by: Β§6.2. Yang and Jin (2025) D. Yang and Q. Jin What matters in evaluating book-length stories? a systematic study of long story evaluation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 16375β16398. External Links: Document Cited by: Β§1, Β§2. Yang et al. (2022) K. Yang, Y. Tian, N. Peng, and D. Klein Re3: generating longer stories with recursive reprompting and revision. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 4393β4479. External Links: Document Cited by: Β§2, Β§4. Yao et al. (2019) L. Yao, N. Peng, R. Weischedel, K. Knight, D. Zhao, and R. Yan Plan-and-write: towards better automatic storytelling. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33, p. 7378β7385. External Links: Document Cited by: Β§2. Yu et al. (2025) T. Yu, K. Shi, Z. Zhao, and G. Penn Multi-agent based character simulation for story writing. In Proceedings of the Fourth Workshop on Intelligent and Interactive Writing Assistants (In2Writing 2025), p. 87β108. External Links: Document Cited by: Β§2, Β§4. Zhou et al. (2024) X. Zhou, H. Zhu, L. Mathur, R. Zhang, H. Yu, Z. Qi, L. Morency, Y. Bisk, D. Fried, G. Neubig, and M. Sap SOTOPIA: interactive evaluation for social intelligence in language agents. In The Twelfth International Conference on Learning Representations, External Links: Link Cited by: Β§2. Zong et al. (2026) Q. Zong, Y. Guo, M. Yang, Y. Guo, and Y. Song EvolvingWorld: an open-schema framework for co-evolving role-play agents and world model in interactive literary world. External Links: 2607.17250, Link Cited by: Β§2, Β§4. Appendix Appendix A Benchmark Construction and Architecture A.1 Episode Construction and Cross-Architecture Comparison We standardized 17 InitialWorld records and selected the nine worlds in Table 2: two simulation-native settings and seven DraCor openings with self-contained five-character casts. Each world begins from one shared snapshot before player choices create a new timeline. All nine worlds are converted to the same self-contained representation of characters, relationships, persistent facts, and time. Language and source family are not independently crossed, so family contrasts describe the worlds studied here rather than causal language effects. Episode construction enumerates connected, self-contained five-character casts and rotates each character into the player role. A fixed selection procedure balances source family, focal-player representation, and relationship structure. This produces 42 EpisodeSeed records, with 14 per family. The native-language opening and three initial actions are then constructed without using any evaluated model. At each step, a fixed sequence sampled uniformly over the three option positions determines the selected choice. All 12 conditions share this sequence and identical first-step option text, although the meaning of later options can diverge with the trajectory. Architecture may therefore affect the narrative, later options, selected actions, and subsequent continuation. The comparison captures the total effect of an architecture as the story evolves. It is not a direct module effect holding the narrative state fixed. A.2 WMOD as Pre-Narration Adjudication At step t, the stateless World Model receives the selected player attempt PtP_t, any Character-Agent intents ItI_t produced under B2B_2 or B3B_3, the canonical history HtH_t, and structured state MtM_t when WMEM is enabled. It returns a PlotPlan WtW_t rather than reader-facing prose. The plan contains at most 12 PlotBeat records; each specifies beat_id, depends_on, source_intent_ids, and content. At least one beat traces to the player attempt, while background intents may be merged, delayed, prevented, or omitted. With WMOD enabled, the Story Agent receives PtP_t and WtW_t in place of the raw Character-Agent intents. The plan remains operational state until the Story Agent realizes it in Narrative NtN_t, the canon-committing output. A.3 WMEM with Full-History Access After each narrative step, structured world memory records current character conditions, relationships, persistent facts, and supporting characters. A rule-based updater applies explicit model-proposed changes without adding inferred facts. Concretely, the World Memory Agent emits a MemoryDelta of persistent changes, and the rule-based reducer applies those operations atomically to produce Mt+1M_t+1. For the next step, each Character Agent receives a deterministic actor-centric CharacterMemory projection of the global structured state. Unchanged entries persist, and the memory cannot rewrite the generated narrative. Evaluators reconstruct canon independently from the narrative trajectory, so structured memory is never ground truth. Let βH denote the policy of exposing the complete narrative history through the current step, and let M denote the structured current-state projection path. All evaluated conditions keep βH active and vary only the additional memory path. The relevant contrast is therefore ΞM|β=Yβ‘(β,M)βYβ‘(β,β ), _M =Y(H,M)-Y(H, ), (5) averaged over the stated bases and WMOD levels. This contrast measures the contribution of structured current-state information alongside the complete narrative history. The implemented WMEM path consists of extraction and rule-based reduction; retrieval, provenance verification, and conflict enforcement are outside that path. Its effect includes changes to later model inputs and the possible coverage loss caused by another model call. Source Initial world Language Opening snapshot Seeds ITP Bianjing, 1100 CE Chinese Bianjing at the opening state in the year 1100 CE. 14 Smallville 25-agent Smallville English Base simulation at midnight on February 13, 2023. 14 DraCor A Dollβs House DanoβNorwegian Act I, after the nurse brings the children upstairs. 2 DraCor Uncle Vanya Russian Act I, just after Serebryakov, Yelena, and Sonya return from their walk. 2 DraCor Three Sisters Russian Act I, after Andrei joins the guests at Irinaβs name-day gathering. 2 DraCor The Cherry Orchard Russian Act I, immediately after the familyβs arrival in the nursery. 2 DraCor Hamlet English Act I, Scene 2, after the court assembles at Elsinore. 2 DraCor King Lear English Act I, Scene 1, before the division of the kingdom is resolved. 2 DraCor The Father Swedish Opening of Act I, with the Captain, Laura, and Bertha at home. 2 Table 2: Initial worlds used to construct the 42 EpisodeSeed records. The snapshot identifies the shared canonical starting point, not a plot that generated trajectories are expected to follow. (Ht,Mt(1)),β¦,(Ht,Mt(4)) (H_t,M_t^(1)),β¦,(H_t,M_t^(4)) βΆCAt(1),β¦,CAt(4)β₯βΆIt, \CA^(1)_t,β¦,CA^(4)_t\_ I_t, (Pt,It,Ht,Mt) (P_t,I_t,H_t,M_t) βΆWMtβΆWt, _t W_t, (Pt,Wt,Ht,Mt) (P_t,W_t,H_t,M_t) βΆSAtβΆNt, _t N_t, (Mt,Nt) (M_t,N_t) βΆMAtβΆΞβMtβreduceMt+1, _t M_t reduceM_t+1, (Ht,Nt) (H_t,N_t) βΆHt+1,(Ht+1,Mt+1)β»step βt+1. H_t+1, 18.49988pt(H_t+1,M_t+1) t+1. Figure 6: Per-step information flow for B3+WMOD+WMEMB_3+WMOD+WMEM. HtH_t is canonical history, MtM_t structured world memory, and Mt(k)M_t^(k) the actor-specific view. PtP_t, ItI_t, WtW_t, NtN_t, and ΞβMt M_t denote the player intent, Character-Agent intents, World-Model plan, Story-Agent narrative, and memory update. Character Agents do not observe PtP_t, and only NtN_t becomes part of canon. The Story Agent is the sole narrator in all conditions. Character Agents and the World Model are stateless at each step. Only the final narrative establishes canon. Structured memory is updated from that narrative, while unexecuted plans and proposed actions establish no story facts. In the experimental configuration, B2B_2 uses one Character Agent to propose actions for all four non-player characters, whereas B3B_3 uses four isolated calls to propose one action per non-player character. WMOD may reconcile these proposals into a plan of at most 12 ordered beats, but only the final narrative enters canon. Appendix B Evaluation and Validation B.1 Evaluator Blinding Each evaluator receives an independently prepared view containing the InitialWorld, opening, language, and generated trajectory. The view excludes model and architecture identities, source-family labels, internal outputs, memory, failures, costs, and the other evaluatorβs result. DeepSeek-V4-Pro appears both among the evaluated models and in the Judge ensemble, but its identity is hidden during evaluation. Character names and canon remain visible as necessary evidence and may reveal recognizable works. B.2 Consistency Under Unequal Coverage For trajectory i, let LiL_i be its number of narrative steps available for evaluation and set Tinc,i=βT_inc,i=β when no contradiction occurs in that prefix. The exact step-k risk set and first-event count are Yk Y_k =βiβ‘[Liβ₯k,Tinc,iβ₯k], = _iI[L_iβ₯ k,\,T_inc,iβ₯ k], (6) Dk D_k =βiβ‘[Liβ₯k,Tinc,i=k]. = _iI[L_iβ₯ k,\,T_inc,i=k]. A trajectory with its first contradiction at step k belongs to both YkY_k and DkD_k at that step, then leaves later risk sets. A contradiction-free trajectory contributes through LiL_i and is then right-censored. Thus DkD_k counts first events at step k, not all contradictions observed up to that step. For the experiments reported here, H=20H=20. The estimated contradiction-free curve and the two Consistency summaries are S^Cβ(t) S_C(t) =βkβ€t:Yk>0(1βDkYk), = _kβ€ t:\,Y_k>0 (1- D_kY_k ), (7) Cobs C_obs =100Hββt=1HS^Cβ(t), = 100H _t=1^H S_C(t), Cadj C_adj =100Hβt=1H[Yt>0]S^C(t). = 100H _t=1^HI[Y_t>0] S_C(t). Both summaries use the same verdicts, first-contradiction events, and censoring. They differ only after no trajectory remains at risk. CobsC_obs carries forward the last estimable survival value, while CadjC_adj assigns zero after support is exhausted. The heatmap pairs CobsC_obs with a separate generation-coverage axis. The Pareto view uses CadjC_adj. Neither summary recovers unseen continuations or treats generation failure as a contradiction. Figure 7: Comparison of the two Consistency summaries across all 144 modelβarchitecture cells. Panel (a) compares CobsC_obs with CadjC_adj. Panel (b) shows where they differ across architectures, and panel (c) relates that difference to Generation Coverage G. The difference is concentrated in lower-coverage Qwen cells and changes no observed judgment or event location. Figure 7 shows that the two values coincide in 119 of 144 cells. Their differences occur mainly in Qwen conditions with few trajectories remaining at later steps and are nearly absent among Frontier models. Recomputing non-dominance with either definition leaves the same six-cell Pareto set across all 144 cells. The Qwen-only sets differ by one member, so we continue to report both summaries for Qwen comparisons affected by sparse later-step evidence. B.3 Supportedness of the Empirical Pareto Set Let FY_F be the 72 Frontier modelβarchitecture point estimates. A non-dominated point yiy_i is supported if some strictly positive weight vector selects it under a linear scalarization, yiβargβ‘maxyβFββ€βy,ββ>02.y_iβ _y _Fw y, ^2_>0. (8) Only two endpoints are supported: DeepSeek-V4-Pro B3+WMOD and Gemini-3.1-Pro B3+WMEM+WMOD. In increasing Consistency order, the four intermediate non-dominated cells fall 2.84, 8.26, 8.15, and 0.05 Richness points below the chord joining these endpoints. The final gap is negligible relative to sampling uncertainty, so this classification describes the estimated frontier only. B.4 Richness Annotation Let d be the number of non-repetitive development units per observed narrative step, and let d95=3.5d_95=3.5 be the 95th percentile in the calibration data. For each breadth dimension j, sjβ1,β¦,5s_jβ\1,β¦,5\ is its anchored rating. The exact computation is Q Q =clipβ‘(logβ‘(1+d)logβ‘(1+d95),0,1), =clip\! ( (1+d) (1+d_95),0,1 ), (9) B B =13ββjβevent,character,worldsjβ14, = 13 _jβ\event,character,world\ s_j-14, R R =100βQβB. =100 QB. Here clipβ‘(x,0,1)clip(x,0,1) truncates x to [0,1][0,1], and the character and world labels refer to character and relationship development and world or social expansion. B.5 Human Validation of Judges Consistency and Richness use separate blinded human-reference samples. Annotators receive the trajectory and rubric, but not model, architecture, sampling stratum, or model-Judge outputs. Figure 8: Human validation of the Judges. (a) Design-weighted Consistency classification on 120 trajectories. (b) One-vs-rest observed agreement across the same sample; the pairwise series average three comparisons. (cβd) Richness agreement and error on a separate 36-trajectory sample. Figure 8 summarizes both validation studies. The Consistency sample draws 120 human-labeled cases from 2,447 trajectories with no uncertain Judge verdicts. Inverse-probability weights restore the target distribution, while 14 uncertain cases are kept only for additional analysis. Panel (b) treats each represented inconsistency type as a one-vs-rest judgment over all 120 trajectories, so shared presence and shared absence both count as agreement. The separate Richness sample contains 36 cases stratified by six models and three source families. The three-Judge median provides the closest aggregate match to the human reference, with the lowest MAE and near-zero bias. Across the three breadth dimensions used in R, its ordinal ratings fall within one point of the human reference in 97.8% of weighted comparisons (mean quadratic-weighted ΞΊ=.780ΞΊ=.780). Together, the two studies support the aggregate Judge measures used in the main analysis. B.6 Long-Horizon Sensitivity A shorter trajectory has fewer opportunities to contradict itself or develop. First-contradiction analysis remains at risk until the event or the end of the evaluated prefix, and the trajectories in these experiments end at step 20. A missing continuation is not counted as a contradiction. Richness is instead weighted by observed narrative steps because, unlike Consistency, it is not determined by a single first event. For each shorter horizon, we recompute Generation Coverage GhG_h and Cobsβ(h)C_obs(h) from the same trajectories and give the six models equal weight within each group. Figure 9: Horizon sensitivity for the two six-model groups. Panels (aβb) show equal-model mean generation and contradiction-free survival. Panel (c) reports the timing distribution of first contradictions observed by step 20. Panel (d) recomputes the normalized summaries at h=5,10,15,20h=5,10,15,20. Figure 9 shows that only 63.1% of Qwen-series and 43.9% of Frontier first contradictions observed by step 20 had appeared by step 10. A ten-step evaluation would therefore miss 36.9% and 56.1%, respectively, of the first contradictions eventually observed in these same trajectories. Because Richness Judges assess each observed trajectory as a whole, we do not reuse those ratings to estimate Richness at earlier horizons. Appendix C Architecture Contrasts and Robustness Architecture affects both story quality and the opportunity to reach later steps. Figure 11 therefore shows computational cost alongside Generation Coverage. Extra computation alone does not provide more narrative evidence. Figure 12 also separates valid story endings from generation failures instead of treating the latter as Consistency or Richness outcomes. The directed trace in Figure 6 distinguishes generation failures from possible narrative precursors. Character-Agent, World-Model, and Story-Agent failures prevent the current narrative. A World-Memory failure occurs after a valid narrative, so that step remains canon even though later generation stops. To trace possible precursors, we localize the first conflict and compare it with the player intent, Character-Agent intents, World-Model beats, and memory views. Conservative lexical matches identify possible precursors rather than causal origins. Table 3 separates generation failures from possible narrative precursors. Across both model groups, 1,709 generation failures occur before the current narrative is produced. The 1,021 World-Memory failures instead follow a valid narrative and prevent 12,672 later narrative steps from being produced. Added structure can therefore reduce sustained generation through extra handoffs without contradicting text already shown. Temporal conflicts are the most common category on which Judges agree in both model groups. Identity and relationship conflicts tend to appear earlier and usually contradict InitialWorld commitments. Character-state and world-fact conflicts more often contradict earlier narratives. Strong Character-Agent precursors appear in 89 inconsistent trajectories, including 42 that persist through the World Model. Another 27 trajectories have only a World-Model precursor. Most cases lack a strong localized upstream match. The trace thus shows that errors can pass through multiple layers, but not that added layers are their predominant source. Quantity Qwen Frontier Generation failures Centralized Character Agent failures 1.1% 5.4% Distributed Character Agent failures 1.9% 12.8% World Model failures 33.6% 6.9% Story Agent failures 24.4% 4.8% World Memory Agent failures 42.7% 24.8% Possible narrative precursors Memory precursor 0.8% 0.0% Player-intent precursor 3.2% 0.4% Character-Agent precursor 12.2% 5.8% World-Model precursor 2.8% 2.8% No strong recorded precursor 55.8% 64.1% Violation not localized 9.7% 12.7% Plurality tie 15.6% 14.2% Table 3: Two complementary error-propagation audits. Generation-failure rates use trajectories in which the component is enabled. Precursor rates use majority-inconsistent trajectories and report the plurality origin of conservative lexical matches to upstream content. These matches identify possible precursors, not causal effects or errors repaired downstream. C.1 Per-Model Architecture Effects Table 8, the per-model counterpart to main-text Table 1, uses the same paired measures and 10,000 hierarchical source-cluster/EpisodeSeed bootstrap draws. Its intervals describe seed variation among the models studied. They do not capture decoding or model-population uncertainty and are not adjusted for multiple comparisons. C.2 Base-Conditional Module Effects Table 9 compares each module on versus off within a base while averaging over the other module. These conditional contrasts compare the resulting trajectories as a whole rather than holding the narrative state fixed. The decomposition shows no base with a stable WMEM Consistency gain. WMOD has the clearest topology dependence: Frontier Richness changes by +0.4β[β1.3,+2.1]+0.4\;[-1.3,+2.1] at B1, +4.3β[+2.6,+6.0]+4.3\;[+2.6,+6.0] at B2, and +5.3β[+4.0,+6.6]+5.3\;[+4.0,+6.6] at B3. C.3 Full-Horizon Paired Comparison This sensitivity retains an architecture pair only when both conditions reach step 20 with valid scores. It then averages paired differences within each model and the available model estimates within each group. Figure 10: Architecture contrasts among paired conditions in which both trajectories reach the full horizon. Bars average the six model-specific estimates in each group. Positive values favor the first-named base or module-on condition. This analysis complements the primary analysis, which includes each trajectory up to its observed length. The paired results in Figure 10 preserve the character-agent trade-off and the absence of a joint WMEM gain, showing that the main pattern is not created solely by comparing short and full-horizon prefixes. Figure 11: Generation cost and coverage under four architecture contrasts. Bars average within-model percentage changes across the six models in each group. Positive values favor the first-named base or module-on condition. B3 invokes four isolated Character Agents per active step, and lower coverage can reduce later calls and tokens. Appendix D Model and Scale Effects The models exhibit distinct profiles across Generation Coverage, Consistency, and Richness, with no common ranking across the three outcomes. This model-level variation is separate from scaling, which we assess only among the comparable dense Qwen2.5 checkpoints. Figure 12 shows how often each model sustains generation through the 20-step horizon. Figure 13 then shows the corresponding Qwen trajectories step by step. Figure 12: Generation outcomes by model across all 12 architectures. Each bar represents 504 planned trajectories. Full horizon indicates 20 completed narrative steps. Story ending denotes an accepted ending, such as the playerβs death, permanent incapacity, or a genuinely closed story. Only four occur. Output-format and other technical failures reduce Generation Coverage but are not story outcomes. Figure 13: Finite-horizon survival curves for the Qwen series. Panel (a) shows the probability of generating through step t. Panel (b) shows contradiction-free survival, with trajectories that stop for technical reasons right-censored. Dotted tails have fewer than 30 trajectories at risk. Curves end at the experimental horizon H=20H=20, and their integrals yield the summaries in Figure 14. Qwen3.5-397B-A17B is shown separately as an MoE model. D.1 Qwen-Series Scaling Figure 14 condenses the step-resolved curves into finite-horizon model summaries. Dense fits use log2 _2 total parameters for the five Qwen2.5 checkpoints. Qwen3.5-397B-A17B is excluded because total parameter count cannot be compared directly between its sparse MoE architecture and the dense models. Figure 14: Qwen-series scaling under unequal trajectory coverage. Dense-model fits use log parameter count for Qwen2.5 3Bβ72B. Qwen3.5-397B-A17B is shown separately as an MoE model. Open markers have fewer than 30 trajectories at risk at step 20. Outcome Ξ± Ξ² R2R^2 Generation Coverage G -32.62 19.32 .879 G, excluding 14B -26.46 19.17 .992 Consistency CobsC_obs 45.71 6.55 .639 Richness R 13.86 4.03 .720 Table 4: Model-level fits Y=Ξ±+Ξ²βlog2βPY=Ξ±+Ξ² _2P, where P is the dense-model parameter count in billions. The primary rows use five Qwen2.5 models. The sensitivity row omits 14B, and Qwen3.5 MoE is excluded. The fitted associations apply to the five studied dense Qwen2.5 checkpoints. Table 4 shows that the unusually low Generation Coverage at 14B does not drive its slope: excluding that checkpoint changes the estimate from 19.32 to 19.17 points per parameter doubling. Consistency and Richness vary non-monotonically across the five model sizes. Tables 5 and 6 expand the long-horizon error analysis by separating the first-conflict categories from the location of the prior canonical commitment. First-violation reason Qwen: n (%) / step Frontier: n (%) / step Temporal conflict 131 (25.8%) / 8.0 144 (31.0%) / 11.0 World-fact conflict 52 (10.3%) / 10.0 88 (18.9%) / 13.0 Character-state conflict 70 (13.8%) / 7.0 38 (8.2%) / 9.0 Identity conflict 42 (8.3%) / 2.0 23 (4.9%) / 7.0 Relationship conflict 34 (6.7%) / 2.0 6 (1.3%) / 4.0 Knowledge conflict 2 (0.4%) / 10.0 4 (0.9%) / 11.5 Causal conflict 0 (0.0%) / β 0 (0.0%) / β Other 2 (0.4%) / 2.0 1 (0.2%) / 15.0 No 2-Judge agreement 174 (34.3%) / 10.0 160 (34.4%) / 13.0 Table 5: Adjudicated reasons for the first inconsistency, excluding generation failures. A category requires agreement from at least two inconsistent Judges. Percentages use all majority-inconsistent trajectories in each model panel. The value after each slash is the median first contradictory step. Agreed reason Initial World Episode opening Earlier Narrative Current Narrative Unresolved tied Total Temporal 21 0 82 17 155 275 World fact 2 0 63 2 73 140 Character state 5 0 54 9 40 108 Identity 48 0 0 1 16 65 Relationship 39 0 0 0 1 40 Knowledge 0 0 3 1 2 6 Table 6: Candidate source of the canonical commitment violated at the first inconsistent step, pooled across both model panels. Sources are localized from the Judgesβ stated prior commitment. Ambiguous and tied cases are grouped as unresolved. The table identifies where a commitment was recorded, not which agent caused the conflict. Appendix E Source-Family Inference Table 7 uses 10,000 hierarchical bootstrap draws over source clusters and EpisodeSeed records while retaining models and paired architectures. Both primary CobsC_obs DraCor contrasts include zero, whereas the CadjC_adj sensitivity is more favorable to DraCor. In the leave-one-play-out analysis, all seven DraCorβSmallville contrasts remain positive. Six of the seven DraCorβITP contrasts remain positive; omitting one play gives a difference of β0.27-0.27 points (Figure 15). Figure 15: Leave-one-DraCor-play-out sensitivity for the 12-model CobsC_obs contrasts. Each point compares DraCor with a simulation-native control after omitting one play. All DraCorβSmallville contrasts remain positive; one DraCorβITP contrast is slightly below zero (β0.27-0.27 points). Measure ITP DraCor Smallville Consistency: estimate [95% bootstrap CI] CobsC_obs 82.1 [78.4, 86.1] 83.5 [79.1, 87.9] 78.1 [75.7, 84.0] CadjC_adj 80.3 [77.3, 82.8] 83.5 [77.1, 87.8] 75.2 [72.4, 77.0] DraCor contrast: difference [95% bootstrap CI] CobsC_obs +1.4 [-4.6, +6.9] β +5.4 [-2.2, +9.9] CadjC_adj +3.2 [-3.8, +8.3] β +8.3 [+1.8, +13.5] Realized temporal exposure Observed narrative steps 12.5 13.3 14.0 Day advance (%) 0.7 0.3 1.0 Full horizon (%) 50.4 54.0 60.8 First contradiction step 9.8 10.2 9.6 Relative day at contradiction 1.1 1.1 1.2 Table 7: Source-family Consistency and trajectory timing. Cells report equal-model means with 95% hierarchical bootstrap intervals. DraCor contrasts subtract ITP or Smallville from DraCor. Consistency is judged against the supplied InitialWorld and emerging trajectory, not against a published continuation. Prior knowledge is observable only when it creates an internal contradiction. As Table 7 shows, all families use the same relative-day representation and record whether the story advances to a new day. Such advances are rare and least common in DraCor, while mean first-contradiction steps are similar. The similar timing gives no indication that the relative-day representation drives the family pattern. Generator Contrast ΞβG G ΞβCobs C_obs ΞβCadj C_adj ΞβR R Qwen series Qwen2.5-3B B2 β B1 +1.8β[β0.8,+4.4]+1.8\,[-0.8,+4.4] β37.6β[β64.2,β8.4]-37.6\,[-64.2,-8.4] +0.1β[β24.2,+33.3]+0.1\,[-24.2,+33.3] +5.8β[β1.1,+15.0]+5.8\,[-1.1,+15.0] B3 β B1 +0.0β[β1.7,+1.3]+0.0\,[-1.7,+1.3] β38.9β[β62.0,β9.8]-38.9\,[-62.0,-9.8] β20.0β[β24.3,+6.8]-20.0\,[-24.3,+6.8] +10.8β[β2.3,+20.2]+10.8\,[-2.3,+20.2] B3 β B2 β1.8β[β3.7,β0.3]-1.8\,[-3.7,-0.3] β1.3β[β29.1,+31.6]-1.3\,[-29.1,+31.6] β20.0β[β30.6,+3.2]-20.0\,[-30.6,+3.2] +5.0β[β10.0,+13.9]+5.0\,[-10.0,+13.9] WMEM on β off +0.7β[β1.1,+2.3]+0.7\,[-1.1,+2.3] β9.3β[β33.8,+10.6]-9.3\,[-33.8,+10.6] β25.1β[β33.5,+5.6]-25.1\,[-33.5,+5.6] +1.2β[β5.3,+8.0]+1.2\,[-5.3,+8.0] WMOD on β off β0.8β[β2.5,+0.7]-0.8\,[-2.5,+0.7] β1.2β[β25.0,+27.1]-1.2\,[-25.0,+27.1] β20.0β[β29.9,+9.0]-20.0\,[-29.9,+9.0] β4.9β[β10.4,+3.5]-4.9\,[-10.4,+3.5] Qwen2.5-7B B2 β B1 +4.6β[+0.2,+9.0]+4.6\,[+0.2,+9.0] +7.6β[β2.8,+18.3]+7.6\,[-2.8,+18.3] +7.7β[β3.4,+17.5]+7.7\,[-3.4,+17.5] β2.2β[β4.7,+0.3]-2.2\,[-4.7,+0.3] B3 β B1 +1.5β[β2.8,+5.7]+1.5\,[-2.8,+5.7] +8.3β[β1.9,+19.3]+8.3\,[-1.9,+19.3] +7.1β[β3.2,+17.6]+7.1\,[-3.2,+17.6] +0.2β[β2.1,+2.5]+0.2\,[-2.1,+2.5] B3 β B2 β3.2β[β7.4,+1.2]-3.2\,[-7.4,+1.2] +0.7β[β8.1,+9.8]+0.7\,[-8.1,+9.8] β0.6β[β8.8,+10.1]-0.6\,[-8.8,+10.1] +2.4β[+0.1,+4.6]+2.4\,[+0.1,+4.6] WMEM on β off β22.4β[β26.1,β18.5]-22.4\,[-26.1,-18.5] +0.6β[β6.1,+7.6]+0.6\,[-6.1,+7.6] +0.7β[β8.2,+6.9]+0.7\,[-8.2,+6.9] β0.4β[β2.6,+1.5]-0.4\,[-2.6,+1.5] WMOD on β off β44.7β[β49.2,β40.0]-44.7\,[-49.2,-40.0] +26.0β[+19.9,+31.9]+26.0\,[+19.9,+31.9] β64.9β[β71.3,β56.9]-64.9\,[-71.3,-56.9] β18.8β[β20.8,β16.8]-18.8\,[-20.8,-16.8] Qwen2.5-14B B2 β B1 +1.5β[β1.5,+4.3]+1.5\,[-1.5,+4.3] β3.0β[β27.1,+21.6]-3.0\,[-27.1,+21.6] β6.1β[β12.9,+4.7]-6.1\,[-12.9,+4.7] +6.7β[+2.7,+10.6]+6.7\,[+2.7,+10.6] B3 β B1 +1.9β[β1.9,+5.7]+1.9\,[-1.9,+5.7] β10.4β[β33.0,+13.5]-10.4\,[-33.0,+13.5] β6.4β[β13.0,+6.3]-6.4\,[-13.0,+6.3] +9.2β[+5.7,+12.5]+9.2\,[+5.7,+12.5] B3 β B2 +0.4β[β2.0,+2.9]+0.4\,[-2.0,+2.9] β7.4β[β30.0,+14.2]-7.4\,[-30.0,+14.2] β0.3β[β7.8,+8.6]-0.3\,[-7.8,+8.6] +2.5β[β1.5,+6.7]+2.5\,[-1.5,+6.7] WMEM on β off β7.3β[β9.9,β4.7]-7.3\,[-9.9,-4.7] β16.9β[β31.4,+4.5]-16.9\,[-31.4,+4.5] β3.9β[β12.4,+1.7]-3.9\,[-12.4,+1.7] β1.8β[β5.0,+1.2]-1.8\,[-5.0,+1.2] WMOD on β off β25.2β[β30.4,β19.5]-25.2\,[-30.4,-19.5] +14.8β[β1.6,+34.9]+14.8\,[-1.6,+34.9] β41.3β[β48.4,β31.1]-41.3\,[-48.4,-31.1] β12.0β[β15.1,β9.1]-12.0\,[-15.1,-9.1] Qwen2.5-32B B2 β B1 +5.8β[β0.7,+12.1]+5.8\,[-0.7,+12.1] β8.6β[β14.6,β2.8]-8.6\,[-14.6,-2.8] β8.6β[β14.9,β2.8]-8.6\,[-14.9,-2.8] +3.2β[+0.8,+5.8]+3.2\,[+0.8,+5.8] B3 β B1 +4.4β[β1.8,+10.6]+4.4\,[-1.8,+10.6] β13.1β[β19.0,β6.9]-13.1\,[-19.0,-6.9] β13.1β[β19.8,β6.9]-13.1\,[-19.8,-6.9] +6.4β[+3.5,+9.4]+6.4\,[+3.5,+9.4] B3 β B2 β1.4β[β5.9,+3.2]-1.4\,[-5.9,+3.2] β4.5β[β13.3,+4.4]-4.5\,[-13.3,+4.4] β4.5β[β13.9,+4.5]-4.5\,[-13.9,+4.5] +3.2β[+0.7,+5.8]+3.2\,[+0.7,+5.8] WMEM on β off β38.4β[β44.0,β32.8]-38.4\,[-44.0,-32.8] β1.9β[β7.9,+4.1]-1.9\,[-7.9,+4.1] β1.9β[β8.2,+3.8]-1.9\,[-8.2,+3.8] β1.2β[β3.2,+0.7]-1.2\,[-3.2,+0.7] WMOD on β off β7.0β[β12.1,β2.0]-7.0\,[-12.1,-2.0] β1.4β[β8.0,+5.2]-1.4\,[-8.0,+5.2] β1.4β[β7.7,+5.4]-1.4\,[-7.7,+5.4] β2.4β[β4.5,β0.4]-2.4\,[-4.5,-0.4] Qwen2.5-72B B2 β B1 +0.6β[β3.6,+5.2]+0.6\,[-3.6,+5.2] β3.4β[β9.0,+1.9]-3.4\,[-9.0,+1.9] β3.4β[β9.0,+1.9]-3.4\,[-9.0,+1.9] β0.8β[β2.9,+1.3]-0.8\,[-2.9,+1.3] B3 β B1 β0.2β[β4.8,+4.5]-0.2\,[-4.8,+4.5] β6.7β[β12.5,β0.6]-6.7\,[-12.5,-0.6] β6.7β[β12.5,β0.6]-6.7\,[-12.5,-0.6] β0.6β[β3.3,+2.1]-0.6\,[-3.3,+2.1] B3 β B2 β0.8β[β4.7,+3.0]-0.8\,[-4.7,+3.0] β3.3β[β9.5,+3.2]-3.3\,[-9.5,+3.2] β3.3β[β9.5,+3.2]-3.3\,[-9.5,+3.2] +0.2β[β1.7,+2.0]+0.2\,[-1.7,+2.0] WMEM on β off β18.5β[β23.7,β13.5]-18.5\,[-23.7,-13.5] β1.2β[β5.3,+3.5]-1.2\,[-5.3,+3.5] β1.2β[β5.3,+3.5]-1.2\,[-5.3,+3.5] β1.1β[β3.2,+0.9]-1.1\,[-3.2,+0.9] WMOD on β off β1.5β[β4.9,+2.0]-1.5\,[-4.9,+2.0] β4.7β[β9.8,+0.4]-4.7\,[-9.8,+0.4] β4.7β[β9.8,+0.4]-4.7\,[-9.8,+0.4] β3.8β[β5.5,β1.8]-3.8\,[-5.5,-1.8] Qwen3.5-397B-A17B B2 β B1 β9.7β[β14.9,β4.9]-9.7\,[-14.9,-4.9] β10.2β[β16.9,β3.9]-10.2\,[-16.9,-3.9] β10.2β[β16.9,β3.9]-10.2\,[-16.9,-3.9] +11.6β[+9.3,+13.8]+11.6\,[+9.3,+13.8] B3 β B1 β14.0β[β18.5,β9.7]-14.0\,[-18.5,-9.7] β11.2β[β19.1,β3.5]-11.2\,[-19.1,-3.5] β11.2β[β19.1,β3.5]-11.2\,[-19.1,-3.5] +16.1β[+13.4,+18.6]+16.1\,[+13.4,+18.6] B3 β B2 β4.3β[β9.6,+1.1]-4.3\,[-9.6,+1.1] β1.0β[β7.9,+5.5]-1.0\,[-7.9,+5.5] β1.0β[β7.9,+5.5]-1.0\,[-7.9,+5.5] +4.5β[+2.1,+6.9]+4.5\,[+2.1,+6.9] WMEM on β off β22.0β[β26.6,β17.5]-22.0\,[-26.6,-17.5] β4.9β[β9.9,+0.1]-4.9\,[-9.9,+0.1] β4.9β[β9.9,+0.1]-4.9\,[-9.9,+0.1] β4.4β[β6.0,β2.9]-4.4\,[-6.0,-2.9] WMOD on β off β3.1β[β7.0,+0.7]-3.1\,[-7.0,+0.7] β5.4β[β11.9,+0.7]-5.4\,[-11.9,+0.7] β5.4β[β11.9,+0.7]-5.4\,[-11.9,+0.7] +6.2β[+4.6,+7.8]+6.2\,[+4.6,+7.8] Frontier models Grok-4.3 B2 β B1 +0.7β[β1.4,+3.6]+0.7\,[-1.4,+3.6] β4.9β[β9.2,β1.1]-4.9\,[-9.2,-1.1] β4.9β[β9.2,β1.1]-4.9\,[-9.2,-1.1] +6.1β[+3.3,+9.0]+6.1\,[+3.3,+9.0] B3 β B1 β0.4β[β4.8,+4.5]-0.4\,[-4.8,+4.5] β5.8β[β10.9,β1.2]-5.8\,[-10.9,-1.2] β5.8β[β10.9,β1.2]-5.8\,[-10.9,-1.2] +16.2β[+13.1,+19.5]+16.2\,[+13.1,+19.5] B3 β B2 β1.1β[β4.1,+1.7]-1.1\,[-4.1,+1.7] β0.9β[β6.5,+5.0]-0.9\,[-6.5,+5.0] β0.9β[β6.5,+5.0]-0.9\,[-6.5,+5.0] +10.1β[+6.6,+13.6]+10.1\,[+6.6,+13.6] WMEM on β off β3.0β[β9.9,+1.7]-3.0\,[-9.9,+1.7] β0.9β[β5.1,+3.0]-0.9\,[-5.1,+3.0] β0.9β[β5.1,+3.0]-0.9\,[-5.1,+3.0] β2.6β[β4.3,β0.8]-2.6\,[-4.3,-0.8] WMOD on β off β0.2β[β3.7,+2.8]-0.2\,[-3.7,+2.8] β3.4β[β7.7,+0.5]-3.4\,[-7.7,+0.5] β3.4β[β7.7,+0.5]-3.4\,[-7.7,+0.5] +1.3β[β0.1,+2.8]+1.3\,[-0.1,+2.8] GLM-5.2 B2 β B1 β2.9β[β8.3,+2.3]-2.9\,[-8.3,+2.3] β10.9β[β18.0,β3.1]-10.9\,[-18.0,-3.1] β11.4β[β18.8,β3.8]-11.4\,[-18.8,-3.8] +10.2β[+7.3,+13.1]+10.2\,[+7.3,+13.1] B3 β B1 +1.9β[β4.7,+8.3]+1.9\,[-4.7,+8.3] β3.7β[β11.2,+3.5]-3.7\,[-11.2,+3.5] β3.7β[β12.6,+3.1]-3.7\,[-12.6,+3.1] +15.4β[+13.1,+17.6]+15.4\,[+13.1,+17.6] B3 β B2 +4.9β[β2.0,+11.7]+4.9\,[-2.0,+11.7] +7.1β[β1.0,+14.3]+7.1\,[-1.0,+14.3] +7.7β[β1.3,+15.2]+7.7\,[-1.3,+15.2] +5.1β[+3.0,+7.2]+5.1\,[+3.0,+7.2] WMEM on β off β50.1β[β56.9,β43.5]-50.1\,[-56.9,-43.5] β0.9β[β7.5,+5.5]-0.9\,[-7.5,+5.5] β1.3β[β10.5,+3.9]-1.3\,[-10.5,+3.9] β5.8β[β8.0,β3.9]-5.8\,[-8.0,-3.9] WMOD on β off β5.9β[β10.8,β1.2]-5.9\,[-10.8,-1.2] β13.2β[β20.2,β5.7]-13.2\,[-20.2,-5.7] β12.8β[β19.0,β3.9]-12.8\,[-19.0,-3.9] +9.3β[+6.9,+11.4]+9.3\,[+6.9,+11.4] DeepSeek-V4-Pro B2 β B1 β10.2β[β17.4,β2.8]-10.2\,[-17.4,-2.8] β4.3β[β10.9,+1.9]-4.3\,[-10.9,+1.9] β4.3β[β11.0,+1.9]-4.3\,[-11.0,+1.9] +10.1β[+7.8,+12.2]+10.1\,[+7.8,+12.2] B3 β B1 β11.2β[β18.5,β2.6]-11.2\,[-18.5,-2.6] β2.8β[β8.9,+3.3]-2.8\,[-8.9,+3.3] β2.8β[β8.9,+3.3]-2.8\,[-8.9,+3.3] +14.2β[+11.8,+16.4]+14.2\,[+11.8,+16.4] B3 β B2 β1.0β[β7.9,+5.9]-1.0\,[-7.9,+5.9] +1.6β[β5.4,+9.0]+1.6\,[-5.4,+9.0] +1.6β[β5.4,+9.0]+1.6\,[-5.4,+9.0] +4.1β[+2.3,+6.0]+4.1\,[+2.3,+6.0] WMEM on β off β26.7β[β31.5,β21.8]-26.7\,[-31.5,-21.8] +0.2β[β4.6,+5.1]+0.2\,[-4.6,+5.1] +0.2β[β4.6,+5.1]+0.2\,[-4.6,+5.1] β4.5β[β6.4,β2.8]-4.5\,[-6.4,-2.8] WMOD on β off β12.0β[β17.1,β7.0]-12.0\,[-17.1,-7.0] β11.8β[β18.0,β5.9]-11.8\,[-18.0,-5.9] β11.8β[β18.1,β5.9]-11.8\,[-18.1,-5.9] +12.1β[+10.1,+14.1]+12.1\,[+10.1,+14.1] Kimi-K2.6 B2 β B1 β7.4β[β14.0,β1.0]-7.4\,[-14.0,-1.0] β1.3β[β6.0,+3.9]-1.3\,[-6.0,+3.9] β1.3β[β6.0,+3.9]-1.3\,[-6.0,+3.9] +6.7β[+4.0,+9.3]+6.7\,[+4.0,+9.3] B3 β B1 β25.7β[β32.7,β18.2]-25.7\,[-32.7,-18.2] β2.8β[β8.6,+3.3]-2.8\,[-8.6,+3.3] β2.8β[β8.6,+3.3]-2.8\,[-8.6,+3.3] +12.1β[+9.7,+14.5]+12.1\,[+9.7,+14.5] B3 β B2 β18.3β[β25.7,β11.2]-18.3\,[-25.7,-11.2] β1.4β[β6.8,+3.9]-1.4\,[-6.8,+3.9] β1.4β[β6.8,+3.9]-1.4\,[-6.8,+3.9] +5.4β[+3.0,+7.8]+5.4\,[+3.0,+7.8] WMEM on β off β2.3β[β7.2,+2.2]-2.3\,[-7.2,+2.2] +2.4β[β2.4,+6.9]+2.4\,[-2.4,+6.9] +2.4β[β2.4,+6.9]+2.4\,[-2.4,+6.9] β1.4β[β3.6,+0.7]-1.4\,[-3.6,+0.7] WMOD on β off β10.5β[β14.5,β6.2]-10.5\,[-14.5,-6.2] β4.1β[β8.2,β0.2]-4.1\,[-8.2,-0.2] β4.1β[β8.2,β0.2]-4.1\,[-8.2,-0.2] +2.7β[+1.0,+4.5]+2.7\,[+1.0,+4.5] Gemini-3.1-Pro B2 β B1 β3.3β[β8.8,+1.9]-3.3\,[-8.8,+1.9] +1.5β[β0.8,+4.1]+1.5\,[-0.8,+4.1] +1.5β[β0.8,+4.1]+1.5\,[-0.8,+4.1] +10.5β[+8.1,+12.9]+10.5\,[+8.1,+12.9] B3 β B1 β8.0β[β13.9,β2.3]-8.0\,[-13.9,-2.3] β1.8β[β5.6,+1.6]-1.8\,[-5.6,+1.6] β1.8β[β5.6,+1.6]-1.8\,[-5.6,+1.6] +15.9β[+14.2,+17.7]+15.9\,[+14.2,+17.7] B3 β B2 β4.6β[β10.7,+1.2]-4.6\,[-10.7,+1.2] β3.3β[β6.9,β0.0]-3.3\,[-6.9,-0.0] β3.3β[β6.9,β0.0]-3.3\,[-6.9,-0.0] +5.4β[+3.5,+7.4]+5.4\,[+3.5,+7.4] WMEM on β off β0.9β[β5.3,+3.6]-0.9\,[-5.3,+3.6] +1.8β[β0.5,+4.2]+1.8\,[-0.5,+4.2] +1.8β[β0.5,+4.2]+1.8\,[-0.5,+4.2] β0.9β[β2.1,+0.3]-0.9\,[-2.1,+0.3] WMOD on β off β2.4β[β6.4,+1.6]-2.4\,[-6.4,+1.6] +0.7β[β1.4,+2.8]+0.7\,[-1.4,+2.8] +0.7β[β1.4,+2.8]+0.7\,[-1.4,+2.8] β0.5β[β2.2,+1.2]-0.5\,[-2.2,+1.2] GPT-5.6-sol B2 β B1 β1.4β[β3.5,+0.6]-1.4\,[-3.5,+0.6] +0.5β[β1.4,+2.2]+0.5\,[-1.4,+2.2] +0.5β[β1.4,+2.2]+0.5\,[-1.4,+2.2] +11.3β[+9.0,+13.6]+11.3\,[+9.0,+13.6] B3 β B1 β1.7β[β4.4,+0.7]-1.7\,[-4.4,+0.7] +0.4β[β1.4,+2.1]+0.4\,[-1.4,+2.1] +0.4β[β1.4,+2.1]+0.4\,[-1.4,+2.1] +14.0β[+11.8,+16.3]+14.0\,[+11.8,+16.3] B3 β B2 β0.3β[β2.9,+2.2]-0.3\,[-2.9,+2.2] β0.1β[β1.8,+1.7]-0.1\,[-1.8,+1.7] β0.1β[β1.8,+1.7]-0.1\,[-1.8,+1.7] +2.7β[+0.6,+4.7]+2.7\,[+0.6,+4.7] WMEM on β off β1.7β[β3.5,+0.1]-1.7\,[-3.5,+0.1] β0.7β[β2.0,+0.7]-0.7\,[-2.0,+0.7] β0.7β[β2.0,+0.7]-0.7\,[-2.0,+0.7] +0.6β[β0.9,+2.2]+0.6\,[-0.9,+2.2] WMOD on β off β1.8β[β3.6,β0.0]-1.8\,[-3.6,-0.0] +0.2β[β1.2,+1.7]+0.2\,[-1.2,+1.7] +0.2β[β1.2,+1.7]+0.2\,[-1.2,+1.7] β4.8β[β6.5,β3.0]-4.8\,[-6.5,-3.0] Table 8: Per-model counterpart to main-text Table 1 for both six-model groups. Each block reports the same five marginal contrasts as point estimates with 95% hierarchical bootstrap intervals. Models are fixed and are not resampled. Generator Conditional contrast ΞβG G ΞβCobs C_obs ΞβCadj C_adj ΞβR R Qwen series Qwen2.5-3B WMEM at B1 +0.4β[β3.2,+2.7]+0.4\,[-3.2,+2.7] β11.9β[β47.5,+0.0]-11.9\,[-47.5,+0.0] β44.4β[β49.5,+7.5]-44.4\,[-49.5,+7.5] β1.1β[β12.1,+10.9]-1.1\,[-12.1,+10.9] WMEM at B2 +0.1β[β4.6,+3.9]+0.1\,[-4.6,+3.9] β25.1β[β70.0,+28.2]-25.1\,[-70.0,+28.2] β30.7β[β58.6,+13.5]-30.7\,[-58.6,+13.5] +4.1β[β11.3,+15.7]+4.1\,[-11.3,+15.7] WMEM at B3 +1.5β[β0.1,+3.0]+1.5\,[-0.1,+3.0] +9.0β[β55.1,+64.5]+9.0\,[-55.1,+64.5] β0.1β[β11.9,+9.2]-0.1\,[-11.9,+9.2] +0.5β[β10.7,+15.8]+0.5\,[-10.7,+15.8] WMOD at B1 β0.4β[β3.9,+1.4]-0.4\,[-3.9,+1.4] β11.9β[β47.5,+0.0]-11.9\,[-47.5,+0.0] β39.4β[β45.0,+10.0]-39.4\,[-45.0,+10.0] β2.8β[β11.0,+11.8]-2.8\,[-11.0,+11.8] WMOD at B2 β1.9β[β6.2,+1.7]-1.9\,[-6.2,+1.7] β5.4β[β38.7,+37.9]-5.4\,[-38.7,+37.9] β18.8β[β44.4,+22.8]-18.8\,[-44.4,+22.8] +1.4β[β9.7,+12.4]+1.4\,[-9.7,+12.4] WMOD at B3 β0.1β[β1.5,+1.3]-0.1\,[-1.5,+1.3] +13.7β[β42.1,+74.7]+13.7\,[-42.1,+74.7] β1.8β[β13.0,+6.5]-1.8\,[-13.0,+6.5] β13.3β[β23.2,+0.3]-13.3\,[-23.2,+0.3] Qwen2.5-7B WMEM at B1 β21.4β[β27.4,β14.5]-21.4\,[-27.4,-14.5] +2.9β[β10.7,+17.2]+2.9\,[-10.7,+17.2] +0.6β[β13.6,+13.2]+0.6\,[-13.6,+13.2] β3.1β[β6.8,+0.4]-3.1\,[-6.8,+0.4] WMEM at B2 β25.4β[β31.0,β19.6]-25.4\,[-31.0,-19.6] β0.6β[β13.8,+11.3]-0.6\,[-13.8,+11.3] β0.6β[β15.8,+11.3]-0.6\,[-15.8,+11.3] +1.1β[β2.1,+4.3]+1.1\,[-2.1,+4.3] WMEM at B3 β20.4β[β26.8,β13.8]-20.4\,[-26.8,-13.8] β0.5β[β10.0,+9.8]-0.5\,[-10.0,+9.8] +2.0β[β10.0,+10.7]+2.0\,[-10.0,+10.7] +0.8β[β2.2,+3.7]+0.8\,[-2.2,+3.7] WMOD at B1 β40.4β[β47.3,β33.6]-40.4\,[-47.3,-33.6] +31.7β[+17.4,+47.7]+31.7\,[+17.4,+47.7] β58.6β[β72.2,β41.3]-58.6\,[-72.2,-41.3] β17.4β[β20.7,β14.2]-17.4\,[-20.7,-14.2] WMOD at B2 β50.2β[β56.3,β43.5]-50.2\,[-56.3,-43.5] +26.4β[+17.0,+36.4]+26.4\,[+17.0,+36.4] β63.6β[β75.5,β52.9]-63.6\,[-75.5,-52.9] β19.6β[β22.4,β16.3]-19.6\,[-22.4,-16.3] WMOD at B3 β43.5β[β50.4,β36.8]-43.5\,[-50.4,-36.8] +19.9β[+7.5,+31.7]+19.9\,[+7.5,+31.7] β72.6β[β83.9,β58.2]-72.6\,[-83.9,-58.2] β19.5β[β22.8,β16.0]-19.5\,[-22.8,-16.0] Qwen2.5-14B WMEM at B1 β4.9β[β9.3,β0.5]-4.9\,[-9.3,-0.5] β12.1β[β42.4,+18.9]-12.1\,[-42.4,+18.9] +8.6β[β11.4,+18.6]+8.6\,[-11.4,+18.6] β3.9β[β9.6,+1.8]-3.9\,[-9.6,+1.8] WMEM at B2 β9.3β[β13.2,β5.7]-9.3\,[-13.2,-5.7] β14.7β[β41.9,+15.9]-14.7\,[-41.9,+15.9] β9.4β[β19.3,β1.0]-9.4\,[-19.3,-1.0] β2.2β[β7.9,+3.3]-2.2\,[-7.9,+3.3] WMEM at B3 β7.6β[β12.3,β3.2]-7.6\,[-12.3,-3.2] β23.9β[β53.3,+3.0]-23.9\,[-53.3,+3.0] β10.7β[β23.5,+2.9]-10.7\,[-23.5,+2.9] +0.6β[β3.4,+4.8]+0.6\,[-3.4,+4.8] WMOD at B1 β24.5β[β31.5,β17.3]-24.5\,[-31.5,-17.3] +10.7β[β21.0,+44.3]+10.7\,[-21.0,+44.3] β54.9β[β65.9,β32.1]-54.9\,[-65.9,-32.1] β23.2β[β27.9,β18.5]-23.2\,[-27.9,-18.5] WMOD at B2 β25.7β[β31.7,β19.3]-25.7\,[-31.7,-19.3] +6.4β[β19.9,+36.8]+6.4\,[-19.9,+36.8] β34.8β[β43.1,β26.1]-34.8\,[-43.1,-26.1] β9.6β[β15.2,β4.2]-9.6\,[-15.2,-4.2] WMOD at B3 β25.5β[β31.4,β19.5]-25.5\,[-31.4,-19.5] +27.4β[β6.2,+51.4]+27.4\,[-6.2,+51.4] β34.1β[β48.5,β23.1]-34.1\,[-48.5,-23.1] β3.4β[β8.1,+1.1]-3.4\,[-8.1,+1.1] Qwen2.5-32B WMEM at B1 β29.8β[β38.9,β21.0]-29.8\,[-38.9,-21.0] β3.3β[β10.8,+3.6]-3.3\,[-10.8,+3.6] β3.3β[β10.9,+3.6]-3.3\,[-10.9,+3.6] β2.3β[β5.9,+1.3]-2.3\,[-5.9,+1.3] WMEM at B2 β42.4β[β50.2,β34.6]-42.4\,[-50.2,-34.6] β3.2β[β14.7,+8.0]-3.2\,[-14.7,+8.0] β3.2β[β14.9,+7.9]-3.2\,[-14.9,+7.9] β1.5β[β4.6,+1.7]-1.5\,[-4.6,+1.7] WMEM at B3 β42.9β[β50.4,β35.3]-42.9\,[-50.4,-35.3] +0.7β[β9.1,+10.0]+0.7\,[-9.1,+10.0] +0.7β[β10.5,+9.8]+0.7\,[-10.5,+9.8] +0.1β[β3.7,+3.8]+0.1\,[-3.7,+3.8] WMOD at B1 β23.8β[β33.0,β14.5]-23.8\,[-33.0,-14.5] +1.2β[β7.9,+9.9]+1.2\,[-7.9,+9.9] +1.2β[β7.9,+9.9]+1.2\,[-7.9,+9.9] β2.7β[β5.8,+0.3]-2.7\,[-5.8,+0.3] WMOD at B2 +3.3β[β3.5,+10.0]+3.3\,[-3.5,+10.0] β5.9β[β16.8,+5.2]-5.9\,[-16.8,+5.2] β5.9β[β16.8,+5.6]-5.9\,[-16.8,+5.6] β2.4β[β5.2,+0.2]-2.4\,[-5.2,+0.2] WMOD at B3 β0.7β[β8.9,+7.2]-0.7\,[-8.9,+7.2] +0.5β[β11.8,+13.0]+0.5\,[-11.8,+13.0] +0.5β[β11.5,+14.3]+0.5\,[-11.5,+14.3] β1.9β[β5.0,+1.1]-1.9\,[-5.0,+1.1] Qwen2.5-72B WMEM at B1 β16.0β[β24.0,β8.6]-16.0\,[-24.0,-8.6] β0.9β[β8.4,+8.3]-0.9\,[-8.4,+8.3] β0.9β[β8.4,+8.3]-0.9\,[-8.4,+8.3] +0.2β[β2.7,+3.0]+0.2\,[-2.7,+3.0] WMEM at B2 β19.3β[β26.4,β12.7]-19.3\,[-26.4,-12.7] +2.3β[β6.1,+11.3]+2.3\,[-6.1,+11.3] +2.3β[β6.1,+11.3]+2.3\,[-6.1,+11.3] β3.2β[β6.9,+0.6]-3.2\,[-6.9,+0.6] WMEM at B3 β20.0β[β27.4,β12.8]-20.0\,[-27.4,-12.8] β4.9β[β11.8,+2.2]-4.9\,[-11.8,+2.2] β4.9β[β11.8,+2.2]-4.9\,[-11.8,+2.2] β0.3β[β3.1,+2.5]-0.3\,[-3.1,+2.5] WMOD at B1 β7.0β[β14.4,β0.4]-7.0\,[-14.4,-0.4] β4.3β[β12.5,+3.5]-4.3\,[-12.5,+3.5] β4.3β[β12.5,+3.5]-4.3\,[-12.5,+3.5] β2.3β[β6.4,+1.5]-2.3\,[-6.4,+1.5] WMOD at B2 β2.1β[β8.3,+4.3]-2.1\,[-8.3,+4.3] β5.1β[β11.6,+1.4]-5.1\,[-11.6,+1.4] β5.1β[β11.6,+1.4]-5.1\,[-11.6,+1.4] β5.4β[β8.2,β2.5]-5.4\,[-8.2,-2.5] WMOD at B3 +4.6β[β1.0,+10.9]+4.6\,[-1.0,+10.9] β4.6β[β14.3,+4.2]-4.6\,[-14.3,+4.2] β4.6β[β14.3,+4.2]-4.6\,[-14.3,+4.2] β3.7β[β7.0,β0.5]-3.7\,[-7.0,-0.5] Qwen3.5-397B-A17B WMEM at B1 β19.9β[β26.4,β13.4]-19.9\,[-26.4,-13.4] β2.4β[β9.7,+5.0]-2.4\,[-9.7,+5.0] β2.4β[β9.7,+5.0]-2.4\,[-9.7,+5.0] β1.0β[β3.6,+1.7]-1.0\,[-3.6,+1.7] WMEM at B2 β22.0β[β30.1,β14.2]-22.0\,[-30.1,-14.2] β8.1β[β19.6,+2.2]-8.1\,[-19.6,+2.2] β8.1β[β19.6,+2.2]-8.1\,[-19.6,+2.2] β7.5β[β11.2,β4.1]-7.5\,[-11.2,-4.1] WMEM at B3 β24.1β[β31.3,β17.0]-24.1\,[-31.3,-17.0] β4.2β[β14.6,+5.6]-4.2\,[-14.6,+5.6] β4.2β[β14.6,+5.6]-4.2\,[-14.6,+5.6] β4.7β[β7.4,β1.8]-4.7\,[-7.4,-1.8] WMOD at B1 +0.2β[β4.7,+4.9]+0.2\,[-4.7,+4.9] β0.5β[β7.5,+5.8]-0.5\,[-7.5,+5.8] β0.5β[β7.5,+5.8]-0.5\,[-7.5,+5.8] +1.7β[β1.5,+4.9]+1.7\,[-1.5,+4.9] WMOD at B2 +3.1β[β4.1,+10.0]+3.1\,[-4.1,+10.0] β6.9β[β18.5,+4.5]-6.9\,[-18.5,+4.5] β6.9β[β18.5,+4.5]-6.9\,[-18.5,+4.5] +6.3β[+3.9,+8.8]+6.3\,[+3.9,+8.8] WMOD at B3 β12.6β[β20.9,β4.2]-12.6\,[-20.9,-4.2] β8.7β[β18.8,+1.3]-8.7\,[-18.8,+1.3] β8.7β[β18.8,+1.3]-8.7\,[-18.8,+1.3] +10.7β[+7.5,+13.7]+10.7\,[+7.5,+13.7] Frontier models Grok-4.3 WMEM at B1 β3.5β[β11.4,+0.0]-3.5\,[-11.4,+0.0] β2.1β[β8.2,+4.0]-2.1\,[-8.2,+4.0] β2.1β[β8.2,+4.0]-2.1\,[-8.2,+4.0] β1.7β[β4.5,+1.1]-1.7\,[-4.5,+1.1] WMEM at B2 β2.5β[β10.7,+2.9]-2.5\,[-10.7,+2.9] +0.5β[β6.8,+7.2]+0.5\,[-6.8,+7.2] +0.5β[β6.8,+7.2]+0.5\,[-6.8,+7.2] β2.4β[β5.8,+0.8]-2.4\,[-5.8,+0.8] WMEM at B3 β3.1β[β10.0,+4.2]-3.1\,[-10.0,+4.2] β1.3β[β9.7,+6.6]-1.3\,[-9.7,+6.6] β1.3β[β9.7,+6.6]-1.3\,[-9.7,+6.6] β3.6β[β6.9,β0.4]-3.6\,[-6.9,-0.4] WMOD at B1 β1.2β[β4.8,+0.0]-1.2\,[-4.8,+0.0] β7.2β[β13.4,β1.4]-7.2\,[-13.4,-1.4] β7.2β[β13.4,β1.4]-7.2\,[-13.4,-1.4] β2.0β[β5.0,+1.1]-2.0\,[-5.0,+1.1] WMOD at B2 β0.1β[β3.3,+2.9]-0.1\,[-3.3,+2.9] +1.4β[β6.2,+8.9]+1.4\,[-6.2,+8.9] +1.4β[β6.2,+8.9]+1.4\,[-6.2,+8.9] +1.5β[β1.7,+4.6]+1.5\,[-1.7,+4.6] WMOD at B3 +0.6β[β4.3,+5.6]+0.6\,[-4.3,+5.6] β4.6β[β12.7,+3.0]-4.6\,[-12.7,+3.0] β4.6β[β12.7,+3.0]-4.6\,[-12.7,+3.0] +4.5β[+1.9,+7.3]+4.5\,[+1.9,+7.3] GLM-5.2 WMEM at B1 β53.6β[β62.1,β44.6]-53.6\,[-62.1,-44.6] +5.9β[β4.7,+16.0]+5.9\,[-4.7,+16.0] +5.9β[β6.9,+15.0]+5.9\,[-6.9,+15.0] β7.1β[β10.6,β4.0]-7.1\,[-10.6,-4.0] WMEM at B2 β43.8β[β52.8,β34.3]-43.8\,[-52.8,-34.3] β13.1β[β26.2,+1.7]-13.1\,[-26.2,+1.7] β14.3β[β30.4,β1.1]-14.3\,[-30.4,-1.1] β5.1β[β8.5,β2.1]-5.1\,[-8.5,-2.1] WMEM at B3 β52.9β[β61.8,β43.9]-52.9\,[-61.8,-43.9] +4.4β[β7.2,+16.1]+4.4\,[-7.2,+16.1] +4.4β[β11.6,+14.4]+4.4\,[-11.6,+14.4] β5.1β[β8.1,β2.5]-5.1\,[-8.1,-2.5] WMOD at B1 β12.0β[β17.7,β6.2]-12.0\,[-17.7,-6.2] β13.5β[β23.7,β3.1]-13.5\,[-23.7,-3.1] β13.5β[β24.0,β3.2]-13.5\,[-24.0,-3.2] +8.2β[+5.0,+11.1]+8.2\,[+5.0,+11.1] WMOD at B2 β7.0β[β15.5,+1.3]-7.0\,[-15.5,+1.3] β9.9β[β22.2,+3.9]-9.9\,[-22.2,+3.9] β8.7β[β20.2,+5.8]-8.7\,[-20.2,+5.8] +9.6β[+5.6,+13.4]+9.6\,[+5.6,+13.4] WMOD at B3 +1.4β[β5.4,+8.2]+1.4\,[-5.4,+8.2] β16.2β[β26.0,β6.6]-16.2\,[-26.0,-6.6] β16.2β[β24.9,β1.5]-16.2\,[-24.9,-1.5] +10.0β[+6.3,+13.7]+10.0\,[+6.3,+13.7] DeepSeek-V4-Pro WMEM at B1 β25.4β[β33.3,β17.1]-25.4\,[-33.3,-17.1] β2.2β[β11.0,+6.1]-2.2\,[-11.0,+6.1] β2.2β[β11.0,+6.1]-2.2\,[-11.0,+6.1] β2.0β[β4.4,+0.5]-2.0\,[-4.4,+0.5] WMEM at B2 β34.3β[β43.5,β25.5]-34.3\,[-43.5,-25.5] +1.3β[β8.1,+10.7]+1.3\,[-8.1,+10.7] +1.3β[β8.2,+10.7]+1.3\,[-8.2,+10.7] β6.9β[β10.0,β3.9]-6.9\,[-10.0,-3.9] WMEM at B3 β20.4β[β30.1,β10.4]-20.4\,[-30.1,-10.4] +1.6β[β7.4,+10.5]+1.6\,[-7.4,+10.5] +1.6β[β7.6,+10.5]+1.6\,[-7.6,+10.5] β4.7β[β8.7,β0.9]-4.7\,[-8.7,-0.9] WMOD at B1 β7.4β[β15.9,+0.3]-7.4\,[-15.9,+0.3] β14.9β[β23.9,β6.7]-14.9\,[-23.9,-6.7] β14.9β[β23.9,β6.7]-14.9\,[-23.9,-6.7] +8.3β[+4.8,+11.5]+8.3\,[+4.8,+11.5] WMOD at B2 β16.2β[β24.2,β8.6]-16.2\,[-24.2,-8.6] β14.7β[β23.8,β5.4]-14.7\,[-23.8,-5.4] β14.7β[β23.8,β5.4]-14.7\,[-23.8,-5.4] +13.3β[+9.6,+16.8]+13.3\,[+9.6,+16.8] WMOD at B3 β12.3β[β21.4,β3.5]-12.3\,[-21.4,-3.5] β5.8β[β16.3,+3.7]-5.8\,[-16.3,+3.7] β5.8β[β16.4,+3.7]-5.8\,[-16.4,+3.7] +14.6β[+10.7,+19.1]+14.6\,[+10.7,+19.1] Kimi-K2.6 WMEM at B1 β2.7β[β9.8,+4.5]-2.7\,[-9.8,+4.5] +3.1β[β3.3,+9.2]+3.1\,[-3.3,+9.2] +3.1β[β3.3,+9.2]+3.1\,[-3.3,+9.2] +0.4β[β2.0,+2.8]+0.4\,[-2.0,+2.8] WMEM at B2 β9.8β[β20.6,+0.1]-9.8\,[-20.6,+0.1] +1.9β[β5.2,+8.9]+1.9\,[-5.2,+8.9] +1.9β[β5.2,+8.9]+1.9\,[-5.2,+8.9] β4.4β[β8.9,β0.2]-4.4\,[-8.9,-0.2] WMEM at B3 +5.6β[β2.3,+12.9]+5.6\,[-2.3,+12.9] +2.0β[β5.3,+9.6]+2.0\,[-5.3,+9.6] +2.0β[β5.3,+9.6]+2.0\,[-5.3,+9.6] β0.1β[β3.4,+3.2]-0.1\,[-3.4,+3.2] WMOD at B1 β12.6β[β20.1,β5.4]-12.6\,[-20.1,-5.4] β5.0β[β11.5,+1.3]-5.0\,[-11.5,+1.3] β5.0β[β11.5,+1.3]-5.0\,[-11.5,+1.3] +2.0β[β0.8,+4.8]+2.0\,[-0.8,+4.8] WMOD at B2 β8.8β[β14.6,β2.7]-8.8\,[-14.6,-2.7] +2.3β[β5.1,+9.9]+2.3\,[-5.1,+9.9] +2.3β[β5.1,+9.9]+2.3\,[-5.1,+9.9] +3.4β[+0.1,+6.6]+3.4\,[+0.1,+6.6] WMOD at B3 β10.1β[β19.5,β0.7]-10.1\,[-19.5,-0.7] β9.6β[β19.0,β1.3]-9.6\,[-19.0,-1.3] β9.6β[β19.0,β1.3]-9.6\,[-19.0,-1.3] +2.7β[β0.2,+5.6]+2.7\,[-0.2,+5.6] Gemini-3.1-Pro WMEM at B1 β4.6β[β10.1,+0.8]-4.6\,[-10.1,+0.8] +1.0β[β1.3,+3.6]+1.0\,[-1.3,+3.6] +1.0β[β1.3,+3.6]+1.0\,[-1.3,+3.6] +1.0β[β1.2,+3.2]+1.0\,[-1.2,+3.2] WMEM at B2 β7.8β[β16.2,+0.3]-7.8\,[-16.2,+0.3] β2.3β[β4.7,β0.3]-2.3\,[-4.7,-0.3] β2.3β[β4.7,β0.3]-2.3\,[-4.7,-0.3] β4.0β[β6.4,β1.4]-4.0\,[-6.4,-1.4] WMEM at B3 +9.6β[β2.1,+20.8]+9.6\,[-2.1,+20.8] +6.5β[+0.5,+13.5]+6.5\,[+0.5,+13.5] +6.5β[+0.5,+13.5]+6.5\,[+0.5,+13.5] +0.3β[β2.0,+2.6]+0.3\,[-2.0,+2.6] WMOD at B1 β3.6β[β9.9,+2.3]-3.6\,[-9.9,+2.3] β1.8β[β6.2,+1.7]-1.8\,[-6.2,+1.7] β1.8β[β6.2,+1.7]-1.8\,[-6.2,+1.7] β1.5β[β4.1,+1.1]-1.5\,[-4.1,+1.1] WMOD at B2 β3.2β[β10.7,+4.2]-3.2\,[-10.7,+4.2] β0.9β[β3.5,+1.4]-0.9\,[-3.5,+1.4] β0.9β[β3.5,+1.4]-0.9\,[-3.5,+1.4] +0.0β[β3.3,+3.1]+0.0\,[-3.3,+3.1] WMOD at B3 β0.4β[β6.6,+6.0]-0.4\,[-6.6,+6.0] +4.6β[β0.1,+10.0]+4.6\,[-0.1,+10.0] +4.6β[β0.1,+10.0]+4.6\,[-0.1,+10.0] +0.1β[β2.5,+2.7]+0.1\,[-2.5,+2.7] GPT-5.6-sol WMEM at B1 β2.7β[β5.7,+0.0]-2.7\,[-5.7,+0.0] β1.0β[β3.6,+1.7]-1.0\,[-3.6,+1.7] β1.0β[β3.6,+1.7]-1.0\,[-3.6,+1.7] +2.1β[β1.1,+5.5]+2.1\,[-1.1,+5.5] WMEM at B2 +1.1β[β2.9,+5.0]+1.1\,[-2.9,+5.0] β1.1β[β3.7,+0.5]-1.1\,[-3.7,+0.5] β1.1β[β3.7,+0.5]-1.1\,[-3.7,+0.5] +0.1β[β2.9,+3.1]+0.1\,[-2.9,+3.1] WMEM at B3 β3.5β[β6.7,β0.6]-3.5\,[-6.7,-0.6] β0.0β[β2.2,+2.7]-0.0\,[-2.2,+2.7] β0.0β[β2.2,+2.7]-0.0\,[-2.2,+2.7] β0.2β[β2.5,+2.1]-0.2\,[-2.5,+2.1] WMOD at B1 β0.8β[β3.9,+2.2]-0.8\,[-3.9,+2.2] β1.0β[β3.6,+1.6]-1.0\,[-3.6,+1.6] β1.0β[β3.6,+1.6]-1.0\,[-3.6,+1.6] β12.3β[β15.6,β8.9]-12.3\,[-15.6,-8.9] WMOD at B2 β3.1β[β7.1,+0.8]-3.1\,[-7.1,+0.8] +1.4β[β0.0,+4.1]+1.4\,[-0.0,+4.1] +1.4β[β0.0,+4.1]+1.4\,[-0.0,+4.1] β1.8β[β5.1,+1.4]-1.8\,[-5.1,+1.4] WMOD at B3 β1.7β[β6.0,+2.3]-1.7\,[-6.0,+2.3] β0.0β[β2.2,+2.7]-0.0\,[-2.2,+2.7] β0.0β[β2.2,+2.7]-0.0\,[-2.2,+2.7] β0.2β[β2.9,+2.5]-0.2\,[-2.9,+2.5] Table 9: Per-model module effects within each base for both six-model groups. Each module is compared on versus off while averaging over the other. Entries are point estimates with 95% hierarchical bootstrap intervals. Models are fixed and are not resampled.