Paper deep dive
EvoBelief: Falsifiable World Models for Self-Improving LLM Agents
Janet Wang, Jiang Guo, Yi Xiang, Jiaqing Yuan, Mingwen Dong, Hideo Kobayashi, Shuaichen Chang, Chandan K. Reddy, Jiarong Jiang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/12/2026, 6:20:59 AM
Summary
The paper introduces EvoBelief, a self-evolving world model for Large Language Model (LLM) agents that improves performance by maintaining and updating falsifiable hypotheses about environment dynamics rather than updating model parameters. EvoBelief operates by forming hypotheses when predictions mismatch observations or when decision-relevant unknowns exist, validating these hypotheses through passive or active evidence, and promoting them to reusable rules. This approach allows agents to adapt to static, drifting, and scientific discovery environments, demonstrating significant improvements over baselines like ReAct, WorldEvolver, and Frozen World Models on benchmarks such as ALFWorld, MiniGrid, and DiscoveryWorld.
Entities (13)
Relation Signals (11)
EvoBelief → evaluatedon → ALFWorld
confidence 98% · Across four benchmarks ... On ALFWorld ... EvoBelief improves task performance
EvoBelief → evaluatedon → ScienceWorld
confidence 98% · ScienceWorld and ALFWorld (static, largely prior-aligned dynamics) ... EvoBelief improves task performance
EvoBelief → evaluatedon → DiscoveryWorld
confidence 98% · In DiscoveryWorld's Reactor Lab, it discovers and validates quantitative laws
EvoBelief → evaluatedon → MiniGrid
confidence 98% · On MiniGrid, it revises stale rules and recovers after unannounced dynamics changes.
Janet Wang → affiliatedwith → Tulane University
confidence 95% · Janet Wang 1∗† ... 1 Tulane University
Jiang Guo → affiliatedwith → Amazon
confidence 95% · Jiang Guo 2∗ ... 2 Amazon
EvoBelief → outperforms → WorldEvolver
confidence 95% · Table 1 shows EvoBelief achieving higher success rates than WorldEvolver on ALFWorld and ScienceWorld.
EvoBelief → outperforms → Frozen World Model
confidence 95% · On ALFWorld without actor-side transition memory, it improves success over the matched Frozen World Model by 23.9% and 31.3%
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Self-evolving LLM agents typically improve how they act through accumulated experience, reflection, or skills. Yet even procedurally capable agents can fail when their beliefs about environment dynamics are incomplete, incorrect, or outdated. We study a complementary axis of self-evolution: evolving those beliefs during deployment. Such beliefs can support multiple tasks, but incorrect ones can also propagate across them. We introduce EvoBelief, a self-evolving world model that represents environment knowledge as falsifiable hypotheses. Unexpected outcomes and decision-relevant unknowns trigger hypothesis formation. Later observations and controlled experiments provide independent validation before a rule can guide predictions or plans within its supported scope. Counterexamples narrow or retire rules as the environment changes. The language model remains frozen, and each rule remains linked to its evidence and the decisions it changes. Across four benchmarks spanning static dynamics, hidden drift, and scientific discovery, EvoBelief improves task performance and reuses learned rules across tasks. On ALFWorld without actor-side transition memory, it improves success over the matched Frozen World Model by 23.9% and 31.3% with Claude Haiku 4.5 and GPT-5.4, respectively. On MiniGrid, it revises stale rules and recovers after unannounced dynamics changes. In DiscoveryWorld's Reactor Lab, it discovers and validates quantitative laws, completing 83.3% of Normal-tier and 58.3% of Challenge-tier instances, versus 0% for each evaluated baseline. These results demonstrate self-improvement through testable, revisable environment knowledge rather than model-parameter updates.
Tags
Links
- Source: https://arxiv.org/abs/2610.12789v1
- Canonical: https://arxiv.org/abs/2610.12789v1
Trouble viewing inline? Open PDF directly →
Full Text
103,845 characters extracted from source content.
Expand or collapse full text
Preprint. EVOBELIEF:FALSIFIABLE WORLD MODELS FOR SELF-IMPROVING LLM AGENTS Janet Wang 1∗† Jiang Guo 2∗ Yi Xiang 2 Jiaqing Yuan 2 Mingwen Dong 2 Hideo Kobayashi 2 Shuaichen Chang 2 Chandan K. Reddy 2 Jiarong Jiang 2 1 Tulane University 2 Amazon ABSTRACT Self-evolving LLM agents typically improve how they act through accumulated experience, reflection, or skills. Yet even procedurally capable agents can fail when their beliefs about environment dynamics are incomplete, incorrect, or out- dated. We study a complementary axis of self-evolution: evolving those beliefs during deployment. Such beliefs can support multiple tasks, but incorrect ones can also propagate across them. We introduce EvoBelief, a self-evolving world model that represents environment knowledge as falsifiable hypotheses. Unex- pected outcomes and decision-relevant unknowns trigger hypothesis formation. Later observations and controlled experiments provide independent validation be- fore a rule can guide predictions or plans within its supported scope. Counterex- amples narrow or retire rules as the environment changes. The language model remains frozen, and each rule remains linked to its evidence and the decisions it changes. Across four benchmarks spanning static dynamics, hidden drift, and sci- entific discovery, EvoBelief improves task performance and reuses learned rules across tasks. On ALFWorld without actor-side transition memory, it improves success over the matched Frozen World Model by 23.9% and 31.3% with Claude Haiku 4.5 and GPT-5.4, respectively. On MiniGrid, it revises stale rules and re- covers after unannounced dynamics changes. In DiscoveryWorld’s Reactor Lab, it discovers and validates quantitative laws, completing 83.3% of Normal-tier and 58.3% of Challenge-tier instances, versus 0% for each evaluated baseline. These results demonstrate self-improvement through testable, revisable environ- ment knowledge rather than model-parameter updates. 1INTRODUCTION Existing self-evolving LLM agents mostly adapt to new or changing environments by improving how they act. They learn from past interaction trajectories, refine behavior through reflection, accu- mulate memories, or distill experience into reusable skills (Zhao et al., 2024; Shinn et al., 2023; Park et al., 2023; Wang et al., 2023). These mechanisms are effective in addressing failures arising from inadequate strategy or procedure. But an agent also acts on its prior beliefs about how the environ- ment works, and in a new environment these beliefs can be wrong. Consider the agent in Figure 1, whose task is to synthesize a compound X. It already knows how to heat, purify, and analyze sam- ples, yet it fails because its prior predicts that X is stable up to 120 ◦ C, while X in fact decomposes at 100 ◦ C. A task-specific correction, such as “use a lower temperature,” can address the immediate failure but does not capture the underlying property. The agent should instead test how X’s stability depends on temperature and retain the resulting evidence-supported constraint. This belief can then guide not only synthesis, but also related tasks such as shipping, storage, and analysis. This motivates belief evolution, a complementary form of self-evolution that updates a persistent model of environment dynamics rather than only adapting task-specific behavior. A policy specifies how an agent acts toward a goal, π(a | s,g), whereas a world model captures how the environment * Corresponding authors: swang47@tulane.edu, gujiang@amazon.com. † Work done during an internship at Amazon. 1 arXiv:2610.12789v1 [cs.AI] 8 Oct 2026 Preprint. Execute (following the prior) compound X heat to 100°C purify decomposed products Observed outcome X is stable up to 120°C (frozen prior belief) agent (a) Same capable agent, wrong belief Goal: synthesize and isolate compound X Controlled experiments Temperature Observation Result 60°C 70°C 80°C 90°C 100°C stable stable stable decomposes decomposes (b) Turn surprise into evidence Predicted: compound X Observed: decomposed products mismatch hypothesis: X decomposes above a certain temperature Influences diverse future tasks Synthesis (existing skill) Shipping & Storage (new task) Analysis (new task) choose low- temperature route (e.g., ≤ 70°C) require temperature- controlled storage and shipping avoid high- temperature GC; use LC instead (c) Verified belief changes future action Persistent world belief W verified update: under the tested conditions, compound X decomposes above ~80°C Figure 1: Illustrative example. (a) Following its frozen prior, a capable agent heats compound X to 100 ◦ C and gets decomposition products. (b) The mismatch triggers a hypothesis, which controlled experiments test. (c) The verified belief is stored in the persistent world model W and changes later actions in the original task and in new ones, without any parameter update. Based on the updated belief, the agent avoids gas chromatography (GC), whose operating temperature exceeds the supported range, and correctly chooses to use liquid chromatography (LC) instead. responds, W (s ′ | s,a). When the world model is wrong, even a capable agent may choose actions based on incorrect predicted outcomes. Conversely, correcting an environmental belief can improve decisions across multiple tasks without separately updating a procedure or skill for each task. This raises a natural question: how can an LLM agent form, test, revise, and reuse beliefs about its environment while keeping the underlying language model frozen? A growing line of work has begun to adapt world models for LLM agents. WALL-E and WALL-E 2.0 induce rules from collected trajectories to align an LLM world model with the environment, and keep the rules that correct the most prediction errors in the collected trajectories (Zhou et al., 2024; 2026). Imagine-then-plan approaches such as ITP-I use offline-trained world models to sim- ulate future outcomes during planning, while keeping the world model itself fixed (Liu et al., 2026). WorldEvolver is closest to our setting, updating world-model knowledge online from prediction mismatches with a memory of past episodic transitions and semantic facts (Zhang et al., 2026b). These works demonstrate the value of environment-specific world models for agents, but leave open three challenges central to continual adaptation: validation, maintenance, and application. Newly inferred rules must first be validated against additional evidence before being trusted, and once estab- lished, they must be maintained as the environment changes, by being revised, narrowed, or retired when contradicted by new evidence. Finally, validated beliefs should influence future predictions and decisions, including tasks different from the one in which they were acquired. We introduce EvoBelief, a self-evolving world model that addresses these challenges through an explicit cycle of hypothesis formation, validation, and revision (Figure 2). When an outcome con- tradicts the model’s prediction, or when an unknown matters for a decision, EvoBelief proposes fal- sifiable hypotheses about the local environment dynamics involved. It tests these hypotheses against later observations or through controlled experiments, promotes evidence-supported hypotheses into reusable rules, and revises, narrows, or retires rules when later evidence contradicts them. Only validated rules are allowed to influence the agent’s future predictions and plans. EvoBelief maintains a two-timescale belief representation, capturing episode-local situational state (z t ) and persistent, evidence-linked beliefs about environment dynamics (W t ). The underlying LLM remains frozen, and all adaptation occurs in this external world model, separating the model’s fixed parametric prior from an environment-specific belief that evolves through test-time interaction. This explicit representation also makes belief evolution inspectable. Each rule is linked to the evidence that supports it, the conditions under which it applies, and subsequent observations that may chal- lenge it. This design allows us to trace how a validated rule guided a prediction or plan, and evalu- ate how learned beliefs transfer across tasks, influence decisions, and adapt when the environment changes. Our contributions are: • We introduce belief evolution as a complementary form of self-improvement for LLM agents, in which a persistent world model captures environment-specific beliefs that can evolve at test time through interaction and generalize across tasks without updating the underlying language model. 2 Preprint. (a) Execute with the current belief o t+1 o t Observe a 1 a 2 a 3 Candidate actions formed without W EvoBelief z t situation W t world belief validated rules only Predict and plan a t Select + act Environment (b) Evolve the belief with evidence mismatch or unknown Hypothesize Gather evidence Passive · later task transitions Active · live experiments controlled treatment / control Verify W t+1 Evolve belief validated rules change later predictions/plans Figure 2: Verified belief revision closes the loop from experience to later behavior. EvoBelief uses the current z t and W t to predict action consequences. A grounded prediction mismatch or a decision-critical unknown opens an evidence loop that forms falsifiable hypotheses, gathers evi- dence, and verifies their scope. Only validated, applicable rules guide predictions and plans. • We develop a belief lifecycle for validation, maintenance, and application: hypotheses are val- idated before being trusted, established rules can be revised or retired as evidence changes, and validated beliefs are applied to relevant future predictions and decisions. • We evaluate EvoBelief on four benchmarks covering static and changing dynamics, and scientific discovery, showing improved task success, cross-task transfer of learned beliefs, adaptation to shifts in dynamics, and discovery of quantitative laws through active experimentation. 2RELATED WORK Belief states and world models. In a POMDP, the belief state is a distribution over latent environ- ment states (Kaelbling et al., 1998), and predictive state representations characterize state through predictions of future observations (Littman & Sutton, 2001). World models address the complemen- tary question of how actions change the world, from planning with learned dynamics to recurrent and latent models (Sutton, 1990; Ha & Schmidhuber, 2018; Hafner et al., 2019; Assran et al., 2025; Wang et al., 2026). Recent language-agent work externalizes state uncertainty or maintains linguistic beliefs under partial observability (Singh et al., 2026; Murphy, 2026; Jiang et al., 2026; Liao et al., 2026), and the belief-based world model of Kumar et al. (2026) exposes a hand-specified belief over the current state to the policy but does not learn it from interaction. EvoBelief separates the two roles across timescales: z t estimates the current situation, and W t is a persistent belief over local environment dynamics learned from interaction. Related world-model agents plan with the model as a simulator, inject world knowledge into planning, learn environment rules from experience, or evolve transition memories (Hao et al., 2023; Qiao et al., 2024; Zhou et al., 2024; 2026; Liu et al., 2026; Fang et al., 2025; Zhang et al., 2026b); others explore curiosity-driven exploration, deliber- ate environment probing, theory formation, and closed-loop scientific discovery (Pathak et al., 2017; Levy et al., 2025; Song & Cai, 2026; Kabra et al., 2026; Jin et al., 2026), and interactive benchmarks now require such discovery (Wang et al., 2022; Jansen et al., 2024). EvoBelief differs in that only validated, scoped claims may change a prediction or a plan. Experience, memory, and self-evolving agents. LLM agents improve from experience through re- flections, episodic memories, reusable lessons, procedures, and skills (Shinn et al., 2023; Zhao et al., 2024; Park et al., 2023; Wang et al., 2023; Ouyang et al., 2026b;a; Chen et al., 2024; Ma et al., 2026; Shang & Yang, 2026), and a broader self-evolving literature modifies prompts, scaffolds, search pro- cedures, or agent code while keeping the model fixed (He et al., 2026; Li et al., 2026; Zhang et al., 2026a; Robeyns et al., 2025; Lee et al., 2026; Xu et al., 2026). ProPlay learns procedural world models, HDSO optimizes skills through hypothesis-driven testing, and AutoManual builds environ- ment instruction manuals from interaction (Ma et al., 2026; Shang & Yang, 2026; Chen et al., 2024). EvoBelief instead maintains predictive claims about environment dynamics with explicit conditions, evidence-supported scope, and counterexample-driven revision. These approaches primarily change how the agent acts; EvoBelief instead changes what the agent believes about the deployed environ- 3 Preprint. ment and separates belief existence from behavioral authority, which a learned claim gains only after independent validation and can lose to later counterexamples. That lifecycle is also what continual adaptation asks for when transition dynamics change and earlier knowledge goes stale (Xie et al., 2021; Wołczyk et al., 2021; Kessler et al., 2023). Belief-calibrated optimization is closest in vocab- ulary (Chen et al., 2026), but its beliefs concern how an evaluator responds to edits of the agent’s own scaffold rather than the deployed environment’s dynamics. 3METHOD 3.1PROBLEM FORMULATION At time t, an environment has latent state s t ∈ S , emits an observation o t ∈ O, and transitions after action a t ∈ A. A task is specified by goal g, and the agent observes the interaction history h t = (o 0 ,a 0 ,o 1 ,...,a t−1 ,o t ). The environment may contain local dynamics that disagree with the frozen language model’s pretrained prior, and those dynamics may change during deployment. With the model parameters θ fixed, EvoBelief maintains two externalized states, b t = (z t ,W t ).(1) Here z t answers the local question of what the agent currently takes to be true about this situation. It records observed entities, relations, quantities, and task progress, and is reset at each episode bound- ary. W t answers the persistent question of how this environment responds to actions. It survives episode resets and changes only as new evidence is incorporated. The frozen LLM therefore sup- plies a general prior, while W t stores environment-specific knowledge learned during deployment. 3.2WORLD KNOWLEDGE AS TESTABLE HYPOTHESES The basic unit of knowledge in W t is a falsifiable hypothesis H about an observable action conse- quence. We write its claim as H := (C∧ A⇒ Y ),(2) which states that when condition C holds, taking action or procedure A should produce observable consequence Y . This form makes the claim directly testable, since one can establish C, execute A, and check whether Y occurs. Each hypothesis also stores the scope in which it is supported, a recipe for testing it, and an evidence record. A may be a single action, a multi-step procedure, or a measurement intervention, while Y may be a state change or a measured quantity. A hypothesis may contain both fixed entities and abstract object roles. When it is used, those roles are bound to concrete objects (i.e. bindings) in the current context, and broader scope is earned only from evidence across varied bindings rather than assumed in advance. For example, the claim that heating above about 80 ◦ C decomposes the sample is first bound to compound X, and it covers other compounds only after evidence from them. Hypotheses are formed from the agent’s own interaction, starting from an empty W 0 . After an ac- tion, EvoBelief compares the realized outcome with the prediction made before acting. A grounded prediction mismatch suggests that the current model of the environment is wrong, and a decision- critical unknown, one whose resolution would change what the agent should do next, suggests that it is incomplete. Because one surprising outcome can have several explanations, EvoBelief may keep rival hypotheses until later evidence distinguishes them. In the compound X example, decom- position could depend on temperature, heating duration, or the vessel. Forming a hypothesis is not evidence for it. An unvalidated hypothesis cannot justify a task-directed action as established knowl- edge, but it may motivate an experiment. Details on hypothesis identity and admissible evidence are in Appendix B. 3.3TESTING HYPOTHESES WITH PASSIVE AND ACTIVE EVIDENCE Evidence can arrive in two ways. Passive evidence comes from an ordinary task transition that hap- pens to test an existing hypothesis, as when a later task heats compound X and records whether it remains intact. Active evidence comes from an experiment run specifically to resolve an uncertainty or distinguish among rival hypotheses. Both routes update the same evidence record. Active exper- iments use live environment actions and therefore consume the task’s interaction budget. EvoBelief 4 Preprint. admits one only when its outcome can change a decision the agent must make, no validated rule already resolves the question, and the required setup is executable. Before the experiment runs, its support and falsification criteria are fixed. The realized outcome then settles the test as supported, counterexampled, or inconclusive. An inconclusive test means that the required condition was not actually witnessed, or that the outcome was missing or ambiguous, and it counts as neither support nor counterevidence. Unresolved hypotheses remain pending and can be tested later. Experiment admission and counting details are in Appendix B. 3.4FROM EVIDENCE TO AUTHORITY: THE RULE LIFECYCLE Evidence determines what a hypothesis is allowed to do. We use authority to mean permission for a learned claim to guide a later prediction or plan. A newly formed hypothesis has none. Permission to reuse must be earned from independent evidence across episodes, bindings, and tests. For example, one batch of compound X that decomposed at 90 ◦ C is not enough, and the claim must hold again in a later episode, on a different batch, before it may guide heating decisions. Let S be the supports of a hypothesis, and let N ep (S), N bind (S), and N test (S) count the distinct episodes, bindings, and independent tests represented in those supports. A hypothesis is committed to W t as a validated rule only when Commit(H)⇐⇒ N ep (S)≥ τ ep ∧ N bind (S)≥ τ bind ∧ N test (S)≥ τ test .(3) The thresholds are set before any benchmark run, are never tuned on benchmark results, and are listed in Appendix B. Each generalized role must vary across supporting bindings, and the event that first formed a hypothesis can never by itself grant reusable authority. Counterevidence can later remove that authority. EvoBelief first narrows the rule to the scope still supported by its surviving evidence and recomputes the commitment condition. If the surviving evidence no longer satisfies Eq. 3, the rule is demoted. It loses its reusable authority but keeps its evidence, so later support can restore it. A rule is retired and no longer used only when repeated counterexamples leave it with no supporting evidence. Because the evidence record is preserved, every change in status can be traced to the observations that caused it. The same lifecycle handles drift in environment dynamics. A previously useful rule can lose authority under changed dynamics, while new evidence supports a successor. 3.5USING THE WORLD MODEL FOR PREDICTION AND PLANNING A validated rule matters only when it applies to the current situation. At each decision, the agent first forms candidate actions A t without access to W t . For each candidate a ∈ A t , the frozen predictor supplies an anchor prediction p 0 t (a) = F θ (a,g,z t ),(4) which captures what the agent would expect without self-learned world knowledge. EvoBelief then retrieves validated rules relevant to the candidate and checks whether their conditions hold and whether the current binding lies inside their evidence-supported scope. This check uses the state estimate z t when assessing candidates and the state predicted after each earlier step when working out a sequence of actions. Retrieval alone has no effect. Only an applicable validated rule may change the anchor prediction. Those rule-supported consequences affect behavior in two ways. First, they can revise the expected outcome of a candidate action, changing its ranking or ruling it out when a required precondition is violated. For example, once the decomposition rule for compound X is validated, it changes the predicted outcome of heating X to 100 ◦ C, so that candidate falls below a lower-temperature route. A rule whose condition does not hold makes no prediction, so it cannot rule out an action by itself. Second, compatible rules can be chained into a short plan when the consequence of one rule establishes the condition of another, and the agent can execute the plan’s first step even if it is not among the candidates in A t . A validated quantitative relation can likewise set the parameter values needed to obtain a target consequence (for example, heating compound X below 80 ◦ C, where it stays intact). If no validated rule applies, EvoBelief falls back to the same W -blind choice it would otherwise have made. If an unresolved claim is decision-relevant, an admissible experiment may instead be 5 Preprint. Table 1: Success rate (%) with and without actor-side transition memory, on ALFWorld and Science- World for both backbones. All methods are run through the same harness, interface, and budgets. ALFWorldScienceWorld SettingMethodHaiku 4.5 GPT-5.4 Haiku 4.5 GPT-5.4 No memory ReAct33.5846.2755.5641.11 ITP-I (Liu et al., 2026)29.8539.5542.2236.67 WorldEvolver (Zhang et al., 2026b)35.0761.9452.2265.56 Frozen WM42.5444.7846.6753.33 EvoBelief66.4276.1264.4471.11 Memory ReAct45.5250.0050.0046.67 ITP-I (Liu et al., 2026)35.0740.3047.7842.22 WorldEvolver (Zhang et al., 2026b)58.9672.3964.4467.78 Frozen WM57.4655.2247.7858.89 EvoBelief68.6676.8768.8972.22 selected to test it (§3.3). Whenever a validated rule changes an executed action, EvoBelief records the rule, the prediction or plan it changed, and the realized outcome. A success caused by using the rule is recorded for analysis but does not count as new independent support for the same rule. Appendix C gives the benchmark-specific interfaces and the model-call inventory. 4EXPERIMENTS 4.1SETUP We evaluate on four benchmarks spanning three deployment regimes: ScienceWorld and ALFWorld (static, largely prior-aligned dynamics) (Wang et al., 2022; Shridhar et al., 2020), MiniGrid (hid- den dynamics drift) (Chevalier-Boisvert et al., 2023), and DiscoveryWorld (scientific discovery) (Jansen et al., 2024). Splits, budgets, and configurations are in Appendix E. We compare against ReAct, WorldEvolver, and Frozen WM. Frozen WM uses the same W -blind actor and predictor as EvoBelief but lets no learned rule guide a prediction or a plan. The comparison with Frozen WM evaluates the complete EvoBelief system relative to this matched prior-driven control. ALFWorld and ScienceWorld are evaluated without and with actor-side transition memory, provided identically to every method. EvoBelief runs without actor-side transition memory on MiniGrid and Discovery- World. Claude Haiku 4.5 and GPT-5.4 are the backbones used. Within each comparison all methods share the benchmark interface, task order, seeds, and budgets, and each task is attempted once. Implementation. On ALFWorld, MiniGrid, and ScienceWorld, action–effect rules are induced from typed transitions and applied to predict state changes for planning and veto, without a rule- conditioned model call. On DiscoveryWorld, quantitative and categorical laws are fitted to measure- ments and observations, and validated by a held-out prediction frozen before the intervention. Only DiscoveryWorld’s frozen predictor engine, the fixed prediction component shared with Frozen WM, makes rule-conditioned model calls. EvoBelief uses the same role-level model interfaces on every benchmark. Its representations and control components are fixed before evaluation (Appendix C). 4.2REGIME I: STATIC, LARGELY PRIOR-ALIGNED ENVIRONMENTS ScienceWorld and ALFWorld test whether belief evolution helps when the environment is station- ary and much of its dynamics remain compatible with the pretrained prior. Table 1, which also includes the world-model lookahead baseline ITP-I, shows that EvoBelief has the highest task suc- cess rate in every setting shown, but the route by which the belief layer helps differs across the two environments. On ALFWorld, persistent validated rules are the main behavioral route. They correct local action mechanics that the prior gets wrong or leaves underspecified, supply missing prerequisites, and prevent the agent from repeating failed procedures. In the trace of Figure 3, for example, the learned cooling rule changes the predicted consequence of cooling, inserts the missing move to the fridge, and completes the task. 6 Preprint. Task 79: put a cool mug in cabinet. Both agents hold mug 3 at countertop 1; the fridge is across the kitchen. Validated rule in W t (committed before this task) formed on bread (ep. 74); supported on lettuce (ep. 76) cool OBJECT with fridge → OBJECT becomes cool, only if OBJECT is in hand and the agent stands at the open fridge Frozen WM (same actor; no rule may act) predicts “the mug becomes cool” acts cool mug 3 with fridge 1 observes “Nothing happens.” outcomenever recovers; stops at the 50-decision cap EvoBelief (planner supplies the missing step) predicts “no effect: not at the open fridge” acts go to fridge 1, then cool mug 3 with fridge 1 observes “mug 3 is cool” outcometask done in 10 decisions Figure 3: A rule-induced decision change on a real ALFWorld trace (Haiku 4.5, unseen task). Frozen WM predicts that cooling succeeds, executes the draft to no effect, and never recovers. EvoBelief uses a validated cooling rule, moves to the fridge first, and completes the task in ten decisions. 150100134 Deployment episodes 0 50 100 EvoBelief Frozen WM WorldEvolver (a) Tasks solved 150100134 Deployment episodes 0 1k 2k vs Frozen WM vs WorldEvolver (b) Decisions saved Haiku no mem Haiku mem GPT no mem GPT mem 0 20 40 60 Frozen WM WorldEvolver EvoBelief (c) Decisions per episode Haiku no mem Haiku mem GPT no mem GPT mem 0 100 200 (d) LLM calls per episode Figure 4: Learning over deployment and efficiency on ALFWorld. (a) Tasks solved so far over the 134 unseen tasks (Haiku 4.5, no transition memory). Shaded blocks are the six task types, and dotted and dashed lines mark when the generic and the clean, cool, and heat rules were committed. (b) Decisions saved by EvoBelief over Frozen WM and WorldEvolver. (c, d) Mean decisions and model calls per episode for both backbones, without and with memory (lower is better). Because W t persists across tasks, generic household rules learned early are reused across later task families, while process-specific rules such as clean, heat, and cool are reused across new object bindings within their supported family (Tables 8 and 9). Across the four ALFWorld configurations, over 80% of the decisions under a validated rule fall in a family other than the one where the rule was validated (Table 8). This reuse also reduces repeated exploration, yielding more successes with fewer decisions and model calls (Figure 4). On ALFWorld, EvoBelief’s margin over Frozen WM is largest without transition memory for both backbones and remains positive with it, consistent with episodic recall and explicit world rules supplying partly overlapping cross-episode information. ScienceWorld exercises the same belief lifecycle but through a different route. Live experiments account for most direct behavioral intervention, while validated rules guide a smaller number of later decisions. The experiments resolve decision-relevant unknowns using the same evidence and validation machinery that stores persistent rules, and Table 6 summarizes the resulting experiment steps and rule-induced decisions. Thus the two static environments test complementary uses of the same world belief, persistent rule reuse across later tasks in ALFWorld and targeted evidence gathering with more selective rule consumption in ScienceWorld. Family breakdowns and paired tests are in Appendix F. 4.3REGIME I: HIDDEN DYNAMICS DRIFT MiniGrid tests whether a learned world model stays useful when the environment changes silently during deployment. In every 40-episode stream the key that opens the door switches after episode 20, and MG-C and MG-M add preconditions the prior does not expect (Figure 5). EvoBelief solves most episodes before and after the switch (Table 2), while Frozen WM and WorldEvolver rarely solve MG-C or MG-M, where success requires the hidden rule. On MG-R, a less adversarial control 7 Preprint. episodes 1-20 drift episodes 21-40 MG-Cbox lifts only with the door shut and hands empty starttoggle with key: unlocks and opens; enter drop key, close door lift box: door shut, hands empty MG-Mthe key only unlocks; an empty-hand toggle opens starttoggle with key: unlocks, stays shut drop key, toggle again: door opens enter, lift box Figure 5: The two central MiniGrid drift families. Each episode is a two-room grid with a locked door, three keys, and a target box in the far room. Left: the drift shared by every family. Only the ringed key unlocks the door, blue for episodes 1–20 and green for 21–40. The switch is silent, and the red key that matches the door is a decoy. Right: the preconditions each family requires to complete the task of lifting the box. In MG-C the box lifts only with the door shut and hands empty. In MG-M the toggle only unlocks the door, and a second, empty-handed toggle opens it. Dark red marks steps the pretrained prior does not expect. Figure 8 in the appendix shows all three families. Table 2: MiniGrid task success (%) before and after the hidden key switch. Each cell pools 60 episodes from three 40-episode streams per family; MG-C and MG-M use development seeds and MG-R uses fresh control seeds. Haiku 4.5, without transition memory. Higher is better. MG-CMG-MMG-R MethodPre-driftPost-driftPre-driftPost-driftPre-driftPost-drift Frozen WM0.000.001.670.0048.3350.00 WorldEvolver0.000.006.6716.6720.0026.67 EvoBelief61.6795.0078.3381.6783.3378.33 where the prior solves about half of the episodes, EvoBelief still improves substantially, so the gain is not explained by a weak prior or a floor effect. Over the deployment (Figure 6), early episodes go to experiments on uncertain dynamics, after which validated rules carry short plans reused across episodes. When the silent key switch makes the door rule fail, the counterevidence retires the rule and reopens experimentation. EvoBelief then validates a successor rule, and success returns to its pre-switch level. On three held-out seeds EvoBelief solves 69.6% of episodes against 1.3% for Frozen WM, and 80.8% against 7.1% when every color and room name is permuted, so the rules are grounded in observed effects rather than entity names (Table 10). Mechanism ablations (Table 11) show that both sides of the loop matter. Without active experiments the critical rules never gain independent support, and without learned rules the planner falls back to the prior, so both variants stay at the prior’s level. Keeping validated rules in shadow, with the same experiments but no executed plan step, also lowers success, most on MG-C, whose solution is a multi-step chain. Lifecycle traces and the remaining ablations are available in Appendix F.4. 4.4REGIME I: SCIENTIFIC DISCOVERY DiscoveryWorld tests whether EvoBelief can acquire environment laws that neither prior knowledge nor episodic recall can supply. We evaluate two themes at the Normal and Challenge tiers. Reactor Lab hides a quantitative relation between a measured crystal property and the reactor’s resonance, so the target resonance must be computed from a law inferred from the agent’s own measurements. Plant Nutrients hides a categorical rule for which nutrient settings make a planted seed grow. 8 Preprint. 110203040 Deployment episodes 0 50 100 Success (%) key switch EvoBelief WorldEvolver Frozen WM (a) Success per episode 110203040 Deployment episodes 0 12 24 Decisions / episode Frozen WM: 24 every episode prior experiment committed plan (b) Decision sources Figure 6: Adaptation over deployment on MiniGrid. (a) Success by episode. The dashed line marks the silent key switch after episode 20. (b) EvoBelief decisions per episode by source: pretrained prior, active experiment, or committed plan. Both panels average the six development streams from MG-C and MG-M. 04080 Success rate (%) ReAct z t only (no W) WorldEvolver (transitions only) WorldEvolver (full) EvoBelief 41.1 57.8 60.0 65.6 71.1 (a) Substrate ablation 0.60.70.80.9 Held-out score State-Δ F1 Feasibility EM Token F1 ScienceWorld 0.60.70.80.9 ALFWorld (b) Held-out prediction Frozen WMWorldEvolverEvoBelief Figure 7: Substrate ablation and held-out prediction. (a) Success on the ScienceWorld tasks of Table 1 with GPT-5.4 and no transition memory, by persistent substrate. Hatched bars store no explicit rules. (b) Structured state-change prediction on 1,500 held-out transitions per benchmark with Claude Haiku 4.5. State-∆ F1 measures how well the predicted state changes match the realized ones, Feasibility whether the action is correctly predicted to execute, and exact match (EM) and token F1 compare the predicted and realized state records. Exact prediction scores are in Table 15. Table 3: DiscoveryWorld results with GPT-5.4 on twelve fresh task instances per task family and difficulty tier. Completion, Knowledge, and Procedural are the benchmark’s task-completion, blind knowledge, and subgoal-progress scores, in percent. NormalChallenge MethodCompletion Knowledge Procedural Completion Knowledge Procedural Reactor Lab ReAct0.000.0070.330.000.0064.66 WorldEvolver0.000.0072.670.000.0069.54 Hypothesizer0.000.0074.000.000.0068.39 EvoBelief83.3379.1789.0058.3362.5084.20 Plant Nutrients ReAct41.6716.6776.8550.000.0077.78 WorldEvolver16.678.3370.3775.000.0083.33 Hypothesizer50.000.0079.6375.000.0084.26 EvoBelief75.0075.0082.4183.3333.3385.19 In Reactor Lab, no baseline (including the benchmark’s native Hypothesizer) completes a task in- stance or receives knowledge credit, whereas EvoBelief completes most Normal and a majority of Challenge task instances (Table 3). Every completed Reactor Lab task instance follows the intended lifecycle (Tables 12 and 13), in which rival quantitative laws are tested, the survivor is validated by a prediction frozen before a held-out crystal is tried, and the validated law configures the target reactor. Plant Nutrients tests the same loop with a categorical rule whose consequence is realized only through a multi-step planting-and-growth procedure. EvoBelief completes the most task instances 9 Preprint. in both tiers and is the only method that earns knowledge credit in both. The reported confidence in- terval for its knowledge-score difference from each baseline excludes zero, while the interval for its completion difference from the strongest baseline of each tier includes zero (Table 7), so discovering the rule and completing the downstream task are related but distinct challenges. Per-instance laws, the paired knowledge margins, failure cases, and the development protocol are in Appendix F.5. 4.5ABLATION AND ANALYSIS To understand what drives these gains, we examine which persistent substrate helps most, whether the learned belief predicts held-out transitions, and how it changes behavior in each environment. In Figure 7 (a), explicit falsifiable rules outperform WorldEvolver with transition memory alone or with its full memory substrate, as well as the z t -only variant and ReAct. In Figure 7 (b), EvoBelief leads on State-∆ F1, exact match, and token F1 on both benchmarks, while WorldEvolver has higher feasibility accuracy on ScienceWorld but lower content scores. The behavioral route differs by environment (Table 6). Passively acquired rules drive planning and veto across tasks on ALFWorld, validated plans dominate on MiniGrid, where retirements track the hidden switch, and validated laws set plan parameters on DiscoveryWorld. ScienceWorld records many live experiment steps but comparatively few decisions guided by validated rules, so most direct behavioral intervention there comes from targeted experiments. Gap sizes and design notes are in Appendix F. 5CONCLUSION & DISCUSSION Agent self-improvement need not begin by changing the agent. When failures come from misun- derstanding the world, the missing component is an environment belief that evidence can challenge. EvoBelief keeps that belief as falsifiable hypotheses, promotes them into rules only through in- dependent evidence, and lets counterexamples revoke that authority when the environment drifts. Validated rules guide predictions and plans, while unresolved hypotheses can motivate live experi- ments. Across the evaluated regimes, the results demonstrate improved task performance, reuse of learned rules, adaptation to hidden dynamics changes, and quantitative law discovery, while keeping the belief state open to inspection. Where episodic memory already supplies what the prior lacks, rules and memory act partly as substitutes. Because a validated rule describes the environment rather than a task, one verified correction can improve many tasks at once, including tasks the agent has never seen. Retiring stale rules complements learning new ones, because the ability to withdraw au- thority keeps accumulated knowledge an asset rather than a liability when the environment changes. More broadly, language models make hypotheses cheap to generate but not to verify, so an agent should act on a belief only after it has survived a test it could have failed. Carrying this discipline to multimodal, embodied, and real-world software agents is a next step. REFERENCES Mido Assran, Adrien Bardes, David Fan, Quentin Garrido, Russell Howes, Matthew Muckley, Am- mar Rizvi, Claire Roberts, Koustuv Sinha, Artem Zholus, et al. V-jepa 2: Self-supervised video models enable understanding, prediction and planning. arXiv preprint arXiv:2506.09985, 2025. Minghao Chen, Yihang Li, Yanting Yang, Shiyu Yu, Binbin Lin, and Xiaofei He. AutoManual: Con- structing instruction manuals by LLM agents via interactive environmental learning. Advances in Neural Information Processing Systems, 37:589–631, 2024. Yuhan Chen, Zhihua Tian, Mahavir Dabas, Charith Peris, Rahul Gupta, Ming Jin, Feiyang Kang, Siyuan Zhang, Nan Wang, and Ruoxi Jia. Belief-calibrated optimization: An explicit world model for agentic optimization. arXiv preprint arXiv:2609.01861, 2026. Maxime Chevalier-Boisvert, Bolun Dai, Mark Towers, Rodrigo Perez-Vicente, Lucas Willems, Salem Lahlou, Suman Pal, Pablo Samuel Castro, and J K Terry. Minigrid & miniworld: Modular & customizable reinforcement learning environments for goal-oriented tasks. Advances in Neural Information Processing Systems, 36:73383–73394, 2023. Tianqing Fang, Hongming Zhang, Zhisong Zhang, Kaixin Ma, Wenhao Yu, Haitao Mi, and Dong Yu. Webevolver: Enhancing web agent self-improvement with co-evolving world model. In 10 Preprint. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, p. 8970–8986, 2025. David Ha and Jürgen Schmidhuber. Recurrent world models facilitate policy evolution. Advances in Neural Information Processing Systems, 31, 2018. Danijar Hafner, Timothy Lillicrap, Jimmy Ba, and Mohammad Norouzi. Dream to control: Learning behaviors by latent imagination. arXiv preprint arXiv:1912.01603, 2019. Shibo Hao, Yi Gu, Haodi Ma, Joshua Hong, Zhen Wang, Daisy Wang, and Zhiting Hu. Reasoning with language model is planning with world model. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, p. 8154–8173, 2023. Yufei He, Juncheng Liu, Yue Liu, Yibo Li, Tri Cao, Zhiyuan Hu, Xinxing Xu, and Bryan Hooi. Evotest: Evolutionary test-time learning for self-improving agentic systems. In International Conference on Learning Representations, volume 2026, p. 104166–104202, 2026. Peter Jansen, Marc-Alexandre Côté, Tushar Khot, Erin Bransom, Bhavana Dalvi Mishra, Bod- hisattwa Prasad Majumder, Oyvind Tafjord, and Peter Clark. Discoveryworld: A virtual environ- ment for developing and evaluating automated scientific discovery agents. Advances in Neural Information Processing Systems, 37:10088–10116, 2024. Haitao Jiang, Lin Ge, Hengrui Cai, and Rui Song. Pabu: Progress-aware belief update for efficient llm agents. arXiv preprint arXiv:2602.09138, 2026. Jiajie Jin, Yuyang Hu, Kai Qiu, Qi Dai, Chong Luo, Guanting Dong, Xiaoxi Li, Tong Zhao, Xiaolong Ma, Gongrui Zhang, et al. Toward generalist autonomous research via hypothesis-tree refinement. arXiv preprint arXiv:2606.11926, 2026. Sanchit Kabra, Nikhil Abhyankar, Saaketh Desai, Prasad Iyer, and Chandan K Reddy.Llm- autoscilab: Closed-loop scientific discovery via active experimentation with llms. arXiv preprint arXiv:2605.24043, 2026. Leslie Pack Kaelbling, Michael L Littman, and Anthony R Cassandra. Planning and acting in partially observable stochastic domains. Artificial Intelligence, 101(1-2):99–134, 1998. Samuel Kessler, Mateusz Ostaszewski, MichałPaweł Bortkiewicz, Mateusz ̇ Zarski, Maciej Wolczyk, Jack Parker-Holder, Stephen J Roberts, Piotr Mi, et al. The effectiveness of world models for con- tinual reinforcement learning. In Conference on Lifelong Learning Agents, p. 184–204. PMLR, 2023. Shubham Kumar, Harshit Kumar, Narendra Ahuja, and Saurabh Jha. Towards a belief-based world model for llm agents. arXiv preprint arXiv:2609.00455, 2026. Yoonho Lee, Roshen Nair, Qizheng Zhang, Kangwook Lee, Omar Khattab, and Chelsea Finn. Meta- harness: End-to-end optimization of model harnesses. arXiv preprint arXiv:2603.28052, 2026. Guillaume Levy, Cedric Colas, Pierre-Yves Oudeyer, Thomas Carta, and Clement Romac. Worldllm: Improving llms’ world modeling using curiosity-driven theory-making. arXiv preprint arXiv:2506.06725, 2025. Yibo Li, Zijie Lin, Ailin Deng, Xuan Zhang, Yufei He, Shuo Ji, Tri Cao, and Bryan Hooi. Just-in- time reinforcement learning: Continual learning in llm agents without gradient updates. arXiv preprint arXiv:2601.18510, 2026. Junfeng Liao, Qizhou Wang, Jianing Zhu, Bo Du, Rui Yan, and Xiuying Chen. Belief memory: Agent memory under partial observability. arXiv preprint arXiv:2605.05583, 2026. Michael Littman and Richard S Sutton. Predictive representations of state. Advances in Neural Information Processing Systems, 14, 2001. Youwei Liu, Jian Wang, Hanlin Wang, Beichen Guo, and Wenjie Li. Imagine-then-plan: Agent learning from adaptive lookahead with world models. arXiv preprint arXiv:2601.08955, 2026. 11 Preprint. Yijun Ma, Zehong Wang, Yiyang Li, Ziming Li, Xiaoguang Guo, Weixiang Sun, Chuxu Zhang, and Yanfang Ye. ProPlay: Procedural world models for self-evolving LLM agents. arXiv preprint arXiv:2606.12780, 2026. Kevin Murphy. Agentic forecasting using sequential bayesian updating of linguistic beliefs. arXiv preprint arXiv:2604.18576, 2026. Siru Ouyang, Jun Yan, Yanfei Chen, Rujun Han, Zifeng Wang, Bhavana Dalvi Mishra, Rui Meng, Chun-Liang Li, Yizhu Jiao, Kaiwen Zha, et al. Skillos: Learning skill curation for self-evolving agents. arXiv preprint arXiv:2605.06614, 2026a. Siru Ouyang, Jun Yan, I Hsu, Yanfei Chen, Ke Jiang, Zifeng Wang, Rujun Han, Long Le, Samira Daruki, Xiangru Tang, et al. Reasoningbank: Scaling agent self-evolving with reasoning mem- ory. In International Conference on Learning Representations, volume 2026, p. 94327–94354, 2026b. Joon Sung Park, Joseph O’Brien, Carrie Jun Cai, Meredith Ringel Morris, Percy Liang, and Michael S Bernstein. Generative agents: Interactive simulacra of human behavior. In Proceedings of the 36th Annual ACM Symposium on User Interface Software and Technology, p. 1–22, 2023. Deepak Pathak, Pulkit Agrawal, Alexei A Efros, and Trevor Darrell. Curiosity-driven exploration by self-supervised prediction. In International Conference on Machine Learning, p. 2778–2787. PMLR, 2017. Shuofei Qiao, Runnan Fang, Ningyu Zhang, Yuqi Zhu, Xiang Chen, Shumin Deng, Yong Jiang, Pengjun Xie, Fei Huang, and Huajun Chen. Agent planning with world knowledge model. Ad- vances in Neural Information Processing Systems, 37:114843–114871, 2024. Maxime Robeyns, Martin Szummer, and Laurence Aitchison. A self-improving coding agent. arXiv preprint arXiv:2504.15228, 2025. Fangxin Shang and Yehui Yang. Hypothesis-driven skill optimization for LLM agents. arXiv preprint arXiv:2606.22330, 2026. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Language agents with verbal reinforcement learning. Advances in Neural Information Processing Systems, 36:8634–8652, 2023. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre Côté, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. Alfworld: Aligning text and embodied environments for interactive learning. arXiv preprint arXiv:2010.03768, 2020. Joykirat Singh, Zaid Khan, Archiki Prasad, Justin Chih-Yao Chen, Akshay Nambi, Hyunji Lee, Elias Stengel-Eskin, and Mohit Bansal. Agent-brace: Decoupling beliefs from actions in long-horizon tasks via verbalized state uncertainty. arXiv preprint arXiv:2605.11436, 2026. Xinyuan Song and Zekun Cai. Ask the world before acting: Budgeted environment probing for world-model calibration. arXiv preprint arXiv:2606.31422, 2026. Richard S Sutton. Integrated architectures for learning, planning, and reacting based on approxi- mating dynamic programming. In Machine Learning Proceedings 1990, p. 216–224. Elsevier, 1990. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023. Ruoyao Wang, Peter Jansen, Marc-Alexandre Côté, and Prithviraj Ammanabrolu. Scienceworld: Is your agent smarter than a 5th grader? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 11279–11298, 2022. Ying Wang, Oumayma Bounou, Yann LeCun, and Mengye Ren. Adajepa: An adaptive latent world model. arXiv preprint arXiv:2606.32026, 2026. 12 Preprint. Maciej Wołczyk, Michał Zaj ̨ac, Razvan Pascanu, Łukasz Kuci ́ nski, and Piotr Miło ́ s. Continual world: A robotic benchmark for continual reinforcement learning. Advances in Neural Informa- tion Processing Systems, 34:28496–28510, 2021. Annie Xie, James Harrison, and Chelsea Finn. Deep reinforcement learning amidst continual struc- tured non-stationarity. In International Conference on Machine Learning, p. 11393–11403. PMLR, 2021. Tianshi Xu, Huifeng Wen, and Meng Li. Adapting the interface, not the model: Runtime harness adaptation for deterministic llm agents. arXiv preprint arXiv:2605.22166, 2026. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. Jenny Zhang, Shengran Hu, Cong Lu, Robert Lange, and Jeff Clune. Darwin gödel machine: open- ended evolution of self-improving agents. In International Conference on Learning Representa- tions, volume 2026, p. 104223–104294, 2026a. Xuan Zhang, Wenxuan Zhang, See-Kiong Ng, and Yang Deng. Self-evolving world models for llm agent planning. arXiv preprint arXiv:2606.30639, 2026b. Andrew Zhao, Daniel Huang, Quentin Xu, Matthieu Lin, Yong-Jin Liu, and Gao Huang. Expel: Llm agents are experiential learners. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, p. 19632–19642, 2024. Siyu Zhou, Tianyi Zhou, Yijun Yang, Guodong Long, Deheng Ye, Jing Jiang, and Chengqi Zhang. Wall-e: World alignment by rule learning improves world model-based llm agents. arXiv preprint arXiv:2410.07484, 2024. Siyu Zhou, Tianyi Zhou, Yijun Yang, Guodong Long, Deheng Ye, Jing Jiang, and Chengqi Zhang. Wall-e: World alignment by neurosymbolic learning improves world model-based llm agents. Advances in Neural Information Processing Systems, 38:65352–65391, 2026. 13 Preprint. ALIMITATIONS AND FUTURE WORK Our experiments use text-based environments, where transitions can be logged and predictions checked when the required outcomes are observable. The belief lifecycle itself is not tied to text, and extending it to multimodal, embodied, and real-world software agents is a natural next step. Such settings bring stochastic effects, which call for statistical tests in place of count thresholds when committing a rule. They also involve interventions that can be costly or irreversible, where live experiments would run under explicit risk limits or human approval. When episodic memory already supplies what the prior lacks, it overlaps with explicit beliefs, so learning when to rely on each is a promising direction. Finally, because validated beliefs carry their evidence records, they could be shared across agents or distilled into model weights, turning what one agent has verified into the prior of the next. BMETHOD DETAILS This appendix collects implementation-level mechanisms that §3.2–§3.3 reference but do not require for the main argument. Hypothesis identity and the evidence ledger. Hypothesis identity is goal-free: a canonical sig- nature over (C,A,Y ) merges rediscoveries of the same claim across different tasks and evidence polarities, so evidence gathered under one goal accrues to the same hypothesis when it resurfaces under another. The per-hypothesis evidence record of §3.2 is implemented as an append-only evi- dence ledger; entries are never rewritten, each entry retains the episode, binding, and test it came from, and every lifecycle transition cites the ledger entries that caused it. Hypotheses cover action- conditioned transitions, object properties and measurement procedures, multi-step procedures, and conditional constraints, the negative counterpart that says what an action cannot do while its condi- tion is witnessed. Evidence admissibility. Transitions whose outcome reflects the harness rather than the environ- ment, such as parser rejections, interface errors, and transport failures, are excluded from hypothesis formation and from settlement. A prediction mismatch is admissible only when it is grounded in the realized environment observation itself, not in the agent’s own summary of it. Discriminating tests and counting. Rival hypotheses are settled against the same realized evi- dence, each against the prediction it froze before the test ran, so one measurement can support one rival and counterexample the others. An outcome that every rival predicts discriminates nothing and supports no conditional claim. All readouts of one experimental design count as a single indepen- dent test, so a multi-step experiment cannot inflate the support counts of the commitment gate in §3.4. Per-episode budgets bound hypothesis formation, the number of experiments, and the steps they may spend, and a hypothesis formed in episode k can be validated only by an independent later occurrence. Experiment admission. Beyond the admission conditions of §3.3, an experiment is admitted only when at least two plausible outcomes would lead to different next actions and no public readout al- ready answers the question. Admitted experiments are executed in the order in which their questions become decision-relevant, and an experiment whose question is resolved by other evidence is closed without running. Commitment thresholds. Every instantiation fixes the thresholds of Eq. 3 before evaluation and never tunes them on a target. Reuse across episodes requires supports on at least two distinct bind- ings in at least two distinct episodes, at least one of which is independent of any decision the hy- pothesis itself influenced. Where a task instance is a single long episode, as in DiscoveryWorld, a hypothesis is committed by one pre-registered held-out test whose prediction was frozen before the test ran. 14 Preprint. Table 4: Hypothesis source, behavioral route, experiment route, and use of the frozen engine’s rule- conditioned model call on each benchmark. Calls per decision by role are in Table 5. BenchmarkHypothesis sourceBehavioral routeExperiment routeRule- conditioned call ALFWorldTyped transitions; code induction Rule-based planning; precondition veto Passive evidence only None MiniGridTyped transitions; code induction Validated plan steps replace the draft Live experiments scheduled by code None ScienceWorldPublic observations and typed transitions; code induction Rule-based control, observation obligations, and veto Live experiments admitted by code None DiscoveryWorld Laws fitted by code from measurements and observations Code-projected consequences; law-derived execution plans Compiled live experiments Conditional CLANGUAGE-MODEL CALLS, PROMPTS, AND RETRIEVAL Division of labor. The framework in Section 3 specifies a shared belief lifecycle, and the four benchmarks instantiate that lifecycle through different structured interfaces. The frozen language model proposes task actions, supplies anchor predictions, and, where needed, extracts typed ob- servations. The benchmark adapter turns public transitions or measurements into the hypothesis representation used by W t , compiles admissible tests, and enforces evidence accounting, commit- ment, applicability, retirement, and rule-based planning. Thus the rule lifecycle and its authority checks are implemented outside the actor rather than delegated to a prompt. On ALFWorld, Min- iGrid, and ScienceWorld, rule-supported consequences are computed directly from the structured representation. DiscoveryWorld is the only benchmark whose frozen predictor engine may also make a rule-conditioned model call. The actor itself never receives the validated rule text. Ta- ble 4 summarizes the benchmark-specific instantiations, and the remainder of this appendix lists the model calls and prompts. This appendix reproduces the actor, candidate-generation, predictor, and transition-analysis prompts, quoted verbatim from the frozen code of the sealed runs, and describes the DiscoveryWorld knowledge judge. User messages are described by their input fields. Auxiliary engine calls are included in the call inventory. In DiscoveryWorld’s frozen predictor engine, the rule-conditioned prediction is p W t (a) = F θ (a,g,z t ,R E t (a)),(5) where R E t (a) contains the engine’s own applicable rules, distinct from the validated rules R t (a) of Algorithm 1; only rule-supported changes are retained. This call is separate from the law layer’s direct consequence prediction and law-derived execution plans. C.1ROLES AND CALLS Table 5 lists the calls per decision by role. On every benchmark, the actor prompt contains no rule, prediction, or plan; it receives the goal, the observation, the action interface, its own history, and, in the transition-memory setting, the shared memory block of Section C.3. Where a rule-conditioned model call is used, it uses the same predictor prompt as the anchor call, with the rule field populated: p 0 t (a) = predictor, rules omitted, p W t (a) = predictor, applicable rules supplied. The model’s hypothesis-proposal, distillation, and binding calls feed the frozen predictor engine’s own rule store, which is consumed only on DiscoveryWorld. Benchmark adapters. ALFWorld makes exactly two calls per decision, the actor draft and one batched anchor-prediction call over the whole candidate slate (6,172 logical calls over 3,086 envi- ronment steps in the sealed Haiku run with transition memory, a ratio of 2.000 in every episode); its candidates are instantiated from the public action grammar by code, and its typed public state is projected by code rather than by a model. MiniGrid issues one anchor-prediction call per candidate 15 Preprint. Table 5: Language-model calls per decision, by role. N is the number of W-blind alternative can- didates (at most 4 on ScienceWorld, 2 on DiscoveryWorld). Per-episode auxiliary calls are a state seed at episode start, hypothesis proposal after a mismatch, trajectory distillation after a success, and role binding for the engine’s own experiments. Call counts are taken from one configuration per benchmark, Haiku 4.5 on ALFWorld and MiniGrid and GPT-5.4 on ScienceWorld and Discovery- World; the roles and calls per decision do not depend on the backbone. BenchmarkActorAlternativesAnchorRule-conditionedAnalysis ALFWorld1code1 (batched)nonecode MiniGrid1code1 per candidate (5)none1 ScienceWorld111 + Nnone1 DiscoveryWorld111 + Nup to 1 + N1 and one evidence-analysis call per transition, about 8.5 calls per decision in a sealed episode. Sci- enceWorld adds one alternatives call, about six calls in a routine decision. DiscoveryWorld uses the benchmark’s own ReAct actor prompt unchanged and averages 5.02 logical calls per executed envi- ronment action; it is the one benchmark on which the frozen engine’s rule-conditioned prediction is requested, and the call is skipped for a candidate when no retrieved rule applies to it. The Knowl- edge score on DiscoveryWorld is computed after the episode by the benchmark’s judge prompt, which never feeds back into the agent. Decoding settings are matched within each benchmark, except for the Hypothesizer’s higher output limit: temperature 0 everywhere, with per-role output ceilings fixed in the frozen configuration (actor 256 tokens on ALFWorld, 768 on MiniGrid, 160 on ScienceWorld, and 3,072 on DiscoveryWorld; predictor 768, or 2,048 for the batched ALFWorld call; evidence analysis 768 on MiniGrid, 1,536 on ScienceWorld, and 8,192 on DiscoveryWorld; hypothesis proposal 1,536) and a 60,000-character input ceiling per role. C.2CANONICAL PROMPTS BY ROLE Actor. The actor chooses one executable action toward the goal from observable information only. Actor prompt You are an embodied task agent. Use only the public goal, current public observation, realized history, and any explicitly labeled method-visible memory. Never assume hidden evaluator state. Return strict JSON. Input, in order: the action interface (the grammar or the currently available actions), the optional memory block, the goal, the current observation, and the last 24 rows of the agent’s own history as Observation: and Action: lines. Output: one JSON object "action": "...". On ScienceWorld the same instruction is issued in ReAct form (“Use only observable information. Never assume hidden state or use reward/scorer internals.”) and answered in two lines, Thought: and Action:, with a 160-token budget; on DiscoveryWorld the benchmark’s distributed ReAct prompt is used unchanged, pinned by its digest. When the EvoBelief layer replaces the draft, the executed action enters this history as an ordinary Action: row with its realized observation; the actor is never told that its draft was overridden. Candidate alternatives. Where alternatives are proposed by the model (ScienceWorld and Dis- coveryWorld), the call is W-blind by construction. Candidate-alternatives prompt You generate a small finite set of alternative task actions. You have no world-model rules and must not assume hidden state. Return exactly JSON candidates:[action:"...", reason:"..."]. Use only the supplied goal, current observation, grounded state claims, recent executed history, baseline_action, and action grammar. Propose at most the supplied limit, only actions distinct from baseline_action. Maximize useful candidate coverage: include distinct grounded action templates when appropriate, and for a multi-argument grammar action consider distinct grounded argument assignments or orientations instead of paraphrasing one choice. Prioritize an explicit missing goal prerequisite, recovery from a blocked/repeated action, or an evidence-grounded terminal 16 Preprint. action. Copy every concrete referent from the supplied surfaces. Do not invent an entity, action, observation, scientific fact, rule, or valid-action oracle. An empty list is valid. Input: goal, observation, state_claims, the last six executed steps, baseline_action (the actor’s draft), action_grammar, limit. On ALFWorld and MiniGrid the alternatives are instantiated from the public action grammar over already-observed entities by code. Predictor. The predictor evaluates exactly the supplied action and never proposes or selects one. Predictor prompt Evaluate exactly the supplied draft_action; do not propose, edit, select, authorize, or recommend an action. Learned world-model rules may affect prediction only. Return exactly JSON expected_outcome:"...",prediction_rule_ids:[...],action_rule_ids:[], critical_unknown:null|"...",predicted_observable_change:null|"...", action_feasibility:"feasible|blocked|unknown", goal_alignment:"advances|neutral|conflicts|unknown". prediction_rule_ids may cite only supplied prediction-authority rules that materially support this prediction. action_rule_ids MUST always be empty: W has no action-proposal or action-authority surface. Use only observable state and supplied rules; never use hidden state, score, reward, or a valid-action oracle. Input: goal, observation, state (the typed claims of z t ), unknowns, draft_action, and validated_rules, which is empty for p 0 t (a) and holds the rules that passed the applicability check for p W t (a). Output contract: expected outcome, observable change, feasibility, goal effect, a critical unknown if any, and the rules cited. On ALFWorld the same contract is requested once for the whole candidate slate, with one entry per candidate and the observable change expressed as a typed delta over the benchmark’s public schema. Transition and evidence analysis. After every executed transition the model updates the typed state estimate z t from observation spans and reports whether the anchor prediction matched. Its verdict is recorded; whether it counts as evidence for any rule is decided by code. Transition and evidence analysis prompt After a realized transition, use only the supplied observation_spans and return exactly JSON with state_update:added_claims:[text,span_id], superseded_claims:[new_text,span_id,supersedes:[claim_id]], maintenance_pruned_claims:[claim_id], prediction_verdict:"match|mismatch|unscorable", evidence_span_id:null|"OBS_N". Every non-null span ID must be copied from observation_spans. Do not retype quotes. Validate each state-update list independently: when no row is grounded, return an empty list. Never infer hidden state, reward, score, completion, action validity, or unobserved effects. Input: the prior claims, the observation before, the executed action, the observation after, the predic- tion, and the observation spans with identifiers. A repair variant re-requests only the JSON framing of a malformed reply. At episode start a seed call of the same form returns the initial claims and unknowns (at most 32 and 12) from the first observation. On ALFWorld the typed public state is projected by code and this role is not called. Knowledge judge (DiscoveryWorld). The benchmark’s own scoring prompt, used unchanged after the episode: it receives the task description, the agent’s knowledge, and one critical question, and answers 1 or 0 with an explanation; the benchmark asks two critical questions per task instance, one call each. For EvoBelief the agent’s knowledge is the deterministic rendering of the store, in which a statement is listed as established only if its prediction was frozen before the test, the realized evidence settled it as passed, and no counterexample stands. The rendered knowledge never names an arm, and every prompt is checked for that before dispatch. 17 Preprint. C.3ACTOR-SIDE TRANSITION MEMORY The setting “with transition memory” installs one shared factor on the actor of every method. Before the actor is asked for an action, the factor retrieves at most four realized public transitions from ear- lier episodes, ranked by lexical overlap between the current goal and public state and the stored goal and pre-state, with a per-item limit of 1,200 characters and a total budget of 4,800. Only transitions of completed earlier episodes are retrievable, every realized transition is stored regardless of its out- come, no outcome label enters the store, and the candidate action never enters the query. Each item is rendered as its goal, the observation before, the action, the observation after, and the public delta, under the header Method-visible memory: (Cross-episode public transition memory (previous episodes only): on ALFWorld), spliced into the actor prompt be- tween the action interface and the history. The factor reads no rule, no store, and no arm name, and a run refuses to start if its budget differs from the frozen one. On MiniGrid the EvoBelief runs use the setting without memory. DDECISION-STEP ALGORITHM Algorithm 1 summarizes one decision step of §3.5. Candidates are formed without access to W t ; each receives an anchor prediction and an effective prediction ˆp t (a) that retains only rule-supported changes to p 0 t (a); where a rule-conditioned model output (Eq. 5) is available, the changes it proposes are retained only where a validated rule supports them. An experiment may be admitted at the cur- rent step when a decision-relevant claim is unresolved, its setup is executable, and budget remains (§3.3); admission freezes the test criteria and does not by itself give the experiment priority. SELECT follows the fixed admission, retention, and priority rules of the instantiation: it returns a candidate action, the first step of a validated plan, or the next step of an admitted experiment, together with the basis d t for that choice, and it preserves the W -blind choice when no permitted intervention applies. PLAN checks rule conditions and scope at every predicted intermediate state, and its first step need not lie in A t . The decision record d t is formed before execution and preserves the predictions and evidence provenance needed to distinguish independent tests from rule-caused confirmations. After the environment step, ABSORB updates the experiment’s progress from the executed transition, set- tles tests only when their outcome can be assessed, and recomputes the lifecycle of §3.4; unfinished tests remain pending. Every route spends the same task budget. The implementations of CONSE- QUENCE, PLAN, and SELECT, including the use of rule-conditioned model calls, are specified in §4.1 and Appendix C. Algorithm 1 One EvoBelief decision step Require: goal g, frozen model F θ , history h t , state estimate z t , world belief W t , pending experi- ment E t (possibly empty), remaining environment-action budget B t > 0 1: A t ← CANDIDATES(g,h t ,z t )▷ formed without access to W t 2: for all a∈ A t do 3: p 0 t (a)← F θ (a,g,z t )▷ anchor prediction 4: R t (a)← APPLICABLE(W t ,z t ,a) ▷ validated rules whose condition holds and whose scope covers the binding 5:ˆp t (a)← CONSEQUENCE(z t ,a,g,p 0 t (a),R t (a)) ▷ rule-supported changes only; p 0 t (a) if R t (a) =∅ 6: end for 7: E t ← ADMIT(W t ,z t ,g,B t ,E t ) ▷ open or keep an experiment: decision-relevant, executable, within budget; criteria frozen 8: π t ← PLAN(W t ,z t ,g) ▷ intermediate states predicted by validated rules; empty if no rule applies 9: (a t ,d t )← SELECT(A t ,p 0 t (a), ˆp t (a),π t ,E t ,z t ,g) ▷ a candidate, the first step of π t , or the next step of E t , with its basis 10: o t+1 ← ENV(a t ); B t+1 ← B t − 1▷ every route spends the task budget 11: (z t+1 ,e t )← ENCODE(z t ,a t ,o t+1 ) 12: (W t+1 ,E t+1 )← ABSORB(W t ,E t ,e t ,d t ) ▷ form; advance or settle tests; commit, narrow, demote, retire 13: record d t and the realized outcome o t+1 18 Preprint. EEXPERIMENTAL SETUP DETAILS Benchmarks. The static, prior-aligned regime uses ScienceWorld (Wang et al., 2022), a text- based science laboratory whose mechanics are stationary but partially hidden (90 episodes per method), and ALFWorld (Shridhar et al., 2020), a household suite testing beliefs about object state, affordances, and procedures (the 134 unseen test tasks, each run once per method). The dynamic regime uses MiniGrid (Chevalier-Boisvert et al., 2023), where the agent must identify the key that actually opens a locked door, cross into the far room, and lift a target box within 24 steps (Figures 5 and 8 describe the families). A seed fixes the sequence of layouts of a stream: key, agent, and box locations are re-sampled every episode, with the first three episodes on the three fixed layouts of the original benchmark. The campaign runs MG-C and MG-M on three development seeds, on three held-out seeds fixed before the runs, on the same held-out seeds with every color and room name permuted in the observations, and on a third seed set reserved for the mechanism ablations, giving 240 matched episodes per method per set; MG-R runs on three further fresh seeds, 120 matched episodes per method. The discovery regime uses DiscoveryWorld (Jansen et al., 2024) on two themes. In Reactor Lab, each reactor activates only when tuned to within±2 Hz of a resonance fre- quency governed by a hidden parametric law of a crystal property measurable through instruments, so the held-out resonance of a new reactor cannot be retrieved from memory and must be computed from a fitted law. In Plant Nutrients, a field’s soil controller sets nutrient levels, a planted seed grows only under the hidden nutrient rule, and the Challenge tier hides that rule in a logical combination of two nutrients. Both themes run at their Normal and Challenge tiers on GPT-5.4: twelve fresh task instances per tier, frozen before any run, four methods on the same task instances, one attempt each (provider-side failures were re-run under the pre-declared transport policy), 320 environment actions and 1,650 model calls per task instance, and no transition memory. On Reactor Lab, EvoBelief’s experiments draw on a science budget of 208 of the 320 actions. Completion and the subgoal- based procedural score are the benchmark’s own, and Knowledge is the benchmark’s LLM-judged score over two sealed critical questions, judged blind to the method with one judge configuration shared by both themes. The benchmark’s own Hypothesizer agent is included as a baseline; its rows in Table 3 come from a separately frozen re-run whose only change is a raised output ceiling (Ap- pendix F.5). A frozen-belief diagnostic and the Plant Nutrients protocol and mechanism are reported in Appendix F.5. Actor settings. ALFWorld and ScienceWorld run under two actor settings, without and with tran- sition memory; on MiniGrid the EvoBelief runs use the setting without it; no method uses it on DiscoveryWorld. Transition memory is specified in Appendix C.3. Baselines. ReAct keeps no persistent cross-episode belief (Yao et al., 2022). ITP-I consults imagine-then-plan lookahead with a learned world model (Liu et al., 2026). WorldEvolver evolves a world model online over stored transitions and semantic memory, a closely related self-evolving world-model baseline (Zhang et al., 2026b). WorldEvolver and ITP-I are the official released imple- mentations, run through the same harness, interface, and budgets as every other method. WALL-E 2.0’s published ALFWorld result uses a different rule-learning protocol and backbone and is not di- rectly comparable to our single-pass benchmark evaluation. Frozen WM is the same W-blind actor with the same anchor predictor and selector, but its predictions and choices follow the pretrained prior alone: no learned rule may alter a prediction, a ranking, or a plan. It does not run EvoBelief’s belief store. Frozen WM and EvoBelief are independent live runs paired by task, not replays. For rule-guided interventions we record the rule, the prediction or plan it changed, and the realized out- come. The paired comparison with Frozen WM evaluates the complete EvoBelief system, including its experiments, relative to the matched prior-driven control. Protocol and fairness. WorldEvolver’s original evaluation reports best-of-L success over L=5 trials per task, so a task counts as solved when any one of the five trials succeeds (Zhang et al., 2026b). In the main benchmark comparisons, each task is attempted once. All methods, including the baselines we rerun, share one harness with the same task set, task order, seeds, action interface, context limits, retry policy, and scoring. Both backbones run the identical protocol, and success is task completion as scored by the benchmark. On ScienceWorld with GPT-5.4, EvoBelief’s run without transition memory was preregistered, executed blind, and sealed first. The Frozen WM 19 Preprint. and WorldEvolver comparators, and then all three runs with transition memory, were authorized afterwards and executed blind on the same 90 tasks, each group unblinded once. Metrics and controls. MiniGrid is split at the hidden changepoint into pre-drift and post-drift halves. Mechanism diagnostics count rules formed, tested, supported, counterexampled, and retired, and decision flips, decisions where the anchor prediction alone would have selected differently and a validated rule’s authorized change is the recorded cause. Held-out structured state-change prediction uses an identical frozen prediction interface for every method. The frozen predictor engine that supplies the anchor prediction, shared with the Frozen WM control, also runs sandboxed tests of its own typed rules on ScienceWorld and ALFWorld. These tests do not replace live task actions, and their outcomes do not enter EvoBelief’s store. FADDITIONAL RESULTS F.1MECHANISM SUMMARY Table 6 collects, in one vocabulary, what the belief machinery did on each benchmark, and §4.5 summarizes how the routes of §3.5 carry different loads by regime. The counts are reported in the unit each implementation records, so the columns are comparable in kind rather than in magnitude: an ALFWorld hypothesis is a rule branch over object roles, a MiniGrid hypothesis is a guarded rule instance that is re-validated after every epoch, a ScienceWorld hypothesis is a typed rule, and a DiscoveryWorld hypothesis is a candidate law. Table 6: What the belief machinery did, by benchmark. One setting per benchmark: ALF- World, Haiku 4.5 without transition memory (134 tasks); MiniGrid, the six development streams (240 episodes); ScienceWorld, GPT-5.4 without transition memory (90 tasks); DiscoveryWorld Re- actor Lab and Plant Nutrients, GPT-5.4 (24 task instances each). Each row counts the unit named in its label, and the rows are not stages of one funnel. ALFWorld: 10 validated rules over 43 validation events. MiniGrid: 220 distinct validated rules over 2,185 validated rule versions, and counterexam- ples are negative witness records. ScienceWorld: 54 experiments opened, 39 completed, yielding 573 support records, while the 4,031 passive supports come from ordinary task transitions. Sup- ports are counted per occurrence, so a claim observed repeatedly in one context adds many records but little independent evidence, which is why validated rules are far fewer than support records. Confirmed: the predicted effect held in the next observation, over the decisions a rule changed on ALFWorld, all plan steps on MiniGrid, the scorable rule predictions on ScienceWorld, law-derived plans on Reactor Lab, and positive consequence verifications on Plant Nutrients. On Discovery- World, the counterexample row counts rival laws falsified. ScienceWorld’s procedural safeguards for task milestones and irreversible steps, fixed before evaluation (§4.1), are not belief actions and are not counted. A dash means the implementation does not record the quantity. ALFWorld MiniGrid ScienceWorld DW ReactorDW Plant Hypotheses formed (claims)12639236669324 Passive supports (records)2302,1564,031– Live experiments (runs)0458549651 Validated rules (distinct)10220361838 Counterexamples (records)910,6452129612 Retirements (rule versions)284570–15 Decisions under validated rules707886142091,129 unitactions plan stepsactionsactionsactions Experiment steps (live actions)01,051424– Confirmed265/324830/88613/1417/1836 unitdecisions plan stepspredictionsplans verifications F.2PAIRED TESTS AND CONFIDENCE INTERVALS Because every comparison is paired, Table 7 reports, for each matched comparison, the exact two- sided McNemar test on the discordant pairs and a percentile bootstrap interval for the difference in success rate, resampling paired episodes ten thousand times. The DiscoveryWorld rows are the sealed campaign’s own instance-paired analysis. 20 Preprint. Table 7: Paired tests for the matched comparisons. Success rates in percent; ∆ is EvoBelief minus the baseline in percentage points with its 95% percentile bootstrap interval over paired episodes; Ours and Base are the two success rates, Disc. counts the episodes only EvoBelief solved against those only the baseline solved; p is the exact two-sided McNemar test on those pairs. ALFWorld pairs the 134 unseen tasks, WorldEvolver being the authors’ release; ScienceWorld pairs its 90 tasks; MiniGrid pairs episodes by stream and position, 240 per seed set, and the post-versus-pre rows compare EvoBelief’s second half with its own first half, pairing episode i + 20 with episode i of the same stream; MiniGrid episodes within a stream share an evolving belief state, and the episode-level tests and intervals do not account for this dependence; DiscoveryWorld pairs the 12 fresh task instances per tier of each theme, with the Hypothesizer at its raised output ceiling. Benchmark, settingBaselinenOursBase∆95% CIDisc.p ALFWorld, Haiku, no memoryFrozen WM134 66.42 42.54 23.8816.42, 31.3434 / 2 <0.001 WorldEvolver 134 66.42 35.07 31.3421.64, 41.0451 / 9 <0.001 ALFWorld, Haiku, memoryFrozen WM134 68.66 57.46 11.192.99, 20.15 25 / 100.017 WorldEvolver 134 68.66 58.969.701.49, 17.91 24 / 110.041 ALFWorld, GPT-5.4, no memoryFrozen WM134 76.12 44.78 31.3423.13, 40.3045 / 3 <0.001 WorldEvolver 134 76.12 61.94 14.185.22, 23.13 29 / 100.003 ALFWorld, GPT-5.4, memoryFrozen WM134 76.87 55.22 21.6413.43, 29.8533 / 4 <0.001 WorldEvolver 134 76.87 72.394.48 −2.99, 12.69 18 / 120.362 ScienceWorld, Haiku, no memoryFrozen WM90 64.44 46.67 17.785.56, 30.0025 / 90.009 WorldEvolver90 64.44 52.22 12.221.11, 23.3319 / 80.052 ScienceWorld, Haiku, memoryFrozen WM90 68.89 47.78 21.117.78, 34.44 29 / 100.003 WorldEvolver90 68.89 64.444.44 −7.78, 16.67 18 / 140.597 ScienceWorld, GPT-5.4, no memory Frozen WM90 71.11 53.33 17.784.44, 31.11 28 / 120.017 WorldEvolver90 71.11 65.565.56 −4.44, 15.5614 / 90.405 ScienceWorld, GPT-5.4, memoryFrozen WM90 72.22 58.89 13.333.33, 23.3317 / 50.017 WorldEvolver90 72.22 67.784.44 −6.67, 15.56 16 / 120.572 MiniGrid, development seedsFrozen WM240 79.170.42 78.7573.33, 83.75 189 / 0 <0.001 WorldEvolver 240 79.175.83 73.3367.50, 79.17 178 / 2 <0.001 MiniGrid, held-out seedsFrozen WM240 69.581.25 68.3362.08, 74.17 165 / 1 <0.001 MiniGrid, permuted labelsFrozen WM240 80.837.08 73.7568.33, 79.17 177 / 0 <0.001 MiniGrid, development, post vs prepre-switch120 88.33 70.00 18.3310.83, 25.8324 / 2 <0.001 MiniGrid, held-out, post vs prepre-switch120 75.83 63.33 12.503.33, 22.50 25 / 100.017 MiniGrid, permuted, post vs prepre-switch120 87.50 74.17 13.335.83, 21.6721 / 50.002 DiscoveryWorld Reactor, NormalReAct12 83.330.00 83.3358.33, 100.0010 / 00.002 WorldEvolver12 83.330.00 83.3358.33, 100.0010 / 00.002 Hypothesizer12 83.330.00 83.3358.33, 100.0010 / 00.002 DiscoveryWorld Reactor, Challenge ReAct12 58.330.00 58.3333.33, 83.337 / 00.016 WorldEvolver12 58.330.00 58.3333.33, 83.337 / 00.016 Hypothesizer12 58.330.00 58.3333.33, 83.337 / 00.016 DiscoveryWorld Plant, NormalReAct12 75.00 41.67 33.330.00, 66.675 / 10.219 WorldEvolver12 75.00 16.67 58.3316.67, 91.678 / 10.039 Hypothesizer12 75.00 50.00 25.00 −16.67, 66.675 / 20.453 DiscoveryWorld Plant, ChallengeReAct12 83.33 50.00 33.338.33, 58.334 / 00.125 WorldEvolver12 83.33 75.008.33 −16.67, 33.332 / 11.000 Hypothesizer12 83.33 75.008.33 −16.67, 33.332 / 11.000 F.3REGIME I: STATIC-ENVIRONMENT DETAILS Regime I details. On ALFWorld the gain concentrates on the task families whose procedures the prior misstates, above all cooling, heating, and two-object placement, where a wrong belief about what an action does, or about the order in which two actions must be taken, cannot be repaired by recall alone. Over the deployment (Figure 4), the curves begin to separate after the first rules are committed. The generic household dynamics, that reaching a receptacle reveals its contents, that taking requires an empty hand, that placing moves an object, are committed within the first task type and consumed in every later one, so each new type begins with most of its dynamics already validated; the one process rule a type adds, cooling, heating, or cleaning, is committed within its first few tasks and reused on the rest. The cumulative gap therefore widens with deployment, fastest on the process types where the prior misstates the procedure. A validated rule removes the exploratory and repeated actions the prior spends on rediscovering a precondition it has already met, so Evo- Belief solves more tasks with fewer decisions and fewer model calls, whereas WorldEvolver pays for its world model with far more calls than the agents that consult none. Because the benchmark enumerates its task types in blocks, later episodes are also harder for the prior: over the second half of the deployment, where the enumeration reaches the process types, Frozen WM solves fewer tasks than in the first half while EvoBelief solves more. This is why the comparison is drawn on matched tasks: the widening is measured against the same agent on the same task, not against an easier past. 21 Preprint. Table 8: Where committed rules are used on ALFWorld. Rule-use decisions: decisions whose executed action a validated rule licensed. Later task: a task after the one in which the rule was first validated. Other family: the task family differs from that of the task in which the rule was validated. Type unseen at formation: the executed action binds at least one role to an object type absent from every binding that formed the rule (instance numbers ignored). Changed: decisions under a validated rule whose executed action differs from the controller’s draft after normalizing the surface forms of put. Confirmed: the transition was not rejected and every atom the rule predicted appears in the next observation. Percentages are of decisions under a validated rule, and of changed decisions for the last column. SettingRules Rule-use decisions Later task Other family Type unseen at formation Changed Confirmed Haiku, no memory1070799.8682.4688.1232481.79 Haiku, memory875399.8785.7990.1725078.40 GPT, no memory877499.8785.6691.3430483.22 GPT, memory878999.7586.6990.8723781.43 Table 9: The rules of one setting, GPT-5.4 without transition memory. Validation task: the index of the task in the deployment order at which the rule first earned authority, with that task’s family. Tasks used: distinct later tasks with a decision taken under the rule. Bindings: distinct object and receptacle instances the rule was bound to. The two go-to rules differ in their effect: one predicts that reaching a receptacle reveals its contents, the other only the new location. Clean, heat, and cool rules can only be used in their own family. RuleValidation taskTasks used Families Bindings Changed Confirmed go to (contents) 1, look at in light115626221170 take2, look at in light1096832020 put7, look at in light9156344 open24, pick and place635888 go to (location) 21, pick and place335103333 clean44, clean then place2511644 heat95, heat then place18191111 cool76, cool then place131933 Of the 1,115 decisions a validated rule changed across the four settings, 907 had their predicted effect fully confirmed in the next observation. The store classified 199 outcomes as inconclusive: the go-to rule predicted arrival at a receptacle and a description of its contents, the agent arrived, and the receptacle turned out to be closed, so the second atom was unobservable. In the remaining 9, the environment refused the action, and the store recorded the refusal as evidence against the rule’s condition or as an inconclusive refusal. None of the 1,115 was classified as an effect mismatch. ALFWorld: rules travel across tasks. The belief persists across the 134 unseen tasks, so a rule validated early is available to every later task. Table 8 counts, for every setting, where decisions under a validated rule were made relative to the task in which the rule earned its authority, and Table 9 lists the rules of one setting. Uses in the task where the rule was validated are rare by construction, since validation needs independent evidence from at least two tasks. F.4REGIME I: DRIFT DETAILS Regime I details. The two families MG-C and MG-M were designed so that the solution requires one step the pretrained prior does not expect, and the 24-step budget leaves no room to discover it by chance. The same prior-governed agent is competent where the environment mostly agrees with its prior and helpless where one hidden rule contradicts it, so the near-zero rows of MG-C and MG-M in Table 2 measure the hidden rules, not a weak baseline. In the first episode of the development stream on MG-C (seed 256), ReAct picks up the color-matched red key, toggles the door three times without effect, cycles through the other keys, opens the door with the blue key on its fourteenth step, enters the far room, and then fails three lifts in a row because the door it opened is still open; it never closes it. On MG-M the same agent unlocks the door on its fifteenth step, sees it closed but unlocked, and spends the remaining steps trying to walk through it. The frozen predictor does not help here: it predicts with the same prior the actor drafts from, so it expects the decoy key to 22 Preprint. Table 10: MiniGrid on held-out seeds (%). Haiku 4.5, actor without transition memory. Three seeds on both families, fixed before any run and never used while building the method, 240 episodes per method per block; the permuted block renames every color and room in the observations of both methods. Higher is better. SeedsMethodPre-switchPost-switchAll Held-outFrozen WM2.500.001.25 EvoBelief63.3375.8369.58 Held-out, permuted labelsFrozen WM10.004.177.08 EvoBelief74.1787.5080.83 Table 11: What the MiniGrid gain needs (%). Each block compares variants of EvoBelief on one seed set, 240 episodes per row. The planner without learned rules never leaves the start state of its search and defers to the prior on every decision; without active experiments, rules never gather inde- pendent support; in the shadow variant rules are learned and validated but no plan step is executed; without the commitment gate a rule may act on its first supported instance. Higher is better. SeedsVariantPre-switchPost-switchAll DevelopmentPlanner without learned rules0.830.000.42 No active experiments0.830.000.42 EvoBelief70.0088.3379.17 MechanismRules never executed (shadow)50.0064.1757.08 No commitment gate66.6770.0068.33 EvoBelief61.6777.5069.58 work just as the actor does and never overrules the draft, and all prior-governed methods share a single success. MG-R is the control for this design: it keeps the decoy key and the silent switch, so the prior stops at about half of the episodes, whereas EvoBelief reaches a similar level on all three families, so the learned rules erase a difficulty gap that the prior cannot. On MG-R EvoBelief loses nothing before the switch, the two agents are level only in the first episodes after it, and EvoBelief pulls furthest ahead in the late post-switch episodes once the successor rule is committed, so the gain is not an artifact of a floor. On the development streams of Figure 6, the last dozen episodes of every stream are at or above their pre-switch level, and Frozen WM uses the full budget of 24 decisions in every episode and makes 178 model calls per episode against 98 for EvoBelief. Every method runs under the same harness, budget, seeds, and step limit. Development seeds were used while building the generic instantiation; the held-out seeds, their label permutation, the mechanism seeds, and the MG-R seeds were fixed before those runs. Experiments double as exploration on MiniGrid. In the shadow variant of Table 11 the belief forms, tests, and validates rules exactly as in the full method and the planner computes the same plans, but no plan step is executed: every executed action comes from the prior or from an experi- ment. In episode 7 of MG-C seed 797 twelve of its fourteen decisions are experiment steps, which pick up the working key, open the door, close it again from the far room, and leave the agent one drop away from the box; the prior’s draft finishes the task. In episode 16 the full method spends three experiment steps and then executes a four-step committed plan, entering the far room, closing the door, dropping the key, and lifting the box, while the shadow variant needs twelve decisions, eleven of them experiment steps, and succeeds only because the prior’s last draft coincides with the plan step it was not allowed to take. The value of executing committed rules is therefore concentrated on MG-C, whose solution is a chain no single experiment completes, and it also shortens episodes and lowers cost. Acting on rules at their first supported instance, without the independent confirmation the commitment gate demands, leaves success and the rate of failed authorized steps close to the full method’s on this benchmark, whose public transitions are deterministic; we report it as a null result rather than as support for the gate. One rule’s life, across the silent changepoint. The stream on MG-C seed 749 shows the lifecycle doing what drift demands. In episode 5 an active experiment validates a first door rule, that toggling opens the door while the blue key is held; the same episode refutes it, because the rule ignored the door’s own state, and it is retired within three decisions. In episode 6 an experiment validates its successor, which conditions on the door being locked and on where the other keys lie, and that rule 23 Preprint. All three families: the key that opens the door switches silently after episode 20 episodes 1-20: blue keyepisodes 21-40: green keythe red key matches the door and never works MG-R startkey toggle: unlocks and opens enter, lift box MG-C startkey toggle: unlocks and opens; enter drop key, close door lift box: needs door shut, hands empty MG-M starttoggle with key: unlocks, stays shut drop key, toggle again: door opens enter, lift box Figure 8: What each MiniGrid family requires. One row per family, each frame a state of the two-room grid: the robot is the agent, a key drawn next to it is the key it carries, the box drawn next to it is the lifted target. A padlock marks a locked door, a plain door is shut but unlocked, and a door drawn aside is open. In every family the red key matches the door and never works, the blue key opens the door for the first twenty episodes, and after episode 20 the green key does instead; the ring marks the key that opens the door and the switch is never announced. In MG-R (R for rebind) one toggle with the working key unlocks and opens the door, and nothing more is asked. In MG-C (C for conjunctive) the door behaves the same, but the box lifts only with the door shut and the agent’s hands empty, so after entering the agent must drop the key and close the door it just opened. In MG-M (M for multi-stage) toggling with the working key only unlocks the door, which stays shut until a second toggle with empty hands. The switch is the only change within a stream; the family rules hold in both halves. The steps in dark red are the ones the pretrained prior does not expect. carries every plan through episode 20. The switch makes it fail at the first decision of episodes 21 and 22; two refutations in distinct episodes open a new evidence epoch for the claim, releasing the pre-switch evidence, and success resumes in episode 24 on experiment and prior steps. In episode 26 an active experiment validates the post-switch successor, that toggling opens the door while the green key is held, which then carries the remaining plans. No grant is ever archival: authority is always backed by currently surviving evidence, so the first generation does not survive its own counterexamples and the successor earns commitment with freshly built supports. On MG-M the same machinery commits the two-stage door as two rules, that the working key unlocks the door and that an empty-handed toggle then opens it, and relearns the first of them after the switch. F.5REGIME I: DISCOVERY DETAILS Table 12 counts what happened on the twenty-four fresh Reactor Lab task instances, and Table 13 lists the validated law of every task instance. A calibration law learned on disjoint instances did not solve the evaluated instances without local experimentation. A frozen belief preloaded with such a law and run without experiments on four Normal and six Challenge task instances names the variable and earns half of the knowledge credit but never the coefficients, and completes nothing. Plant Nutrients: protocol. The theme’s hidden rule is categorical. A first version of the Plant Nu- trients instantiation was run on six fresh task instances under a pre-declared performance gate and failed it: it completed no task instance, although its rules had recovered the hidden nutrient rule on every Normal task instance and a table consistent with the hidden combination on every Challenge task instance. The failure had three named causes. A correct rule was retired by a counterexam- ple that was not one, because growth also requires a seed in the tile and seeds inside tiles are not published; the validated rule set the soil controller but did not carry the rest of the planting chain, 24 Preprint. Table 12: What EvoBelief did on the fresh task instances. Counts from the sealed science records of the twelve task instances per tier; the baselines have no such record by construction. A hypothesis is a candidate law with a prediction frozen before any experiment. One executed experiment settles every rival whose frozen prediction its realized outcome contradicts, so fewer experiments falsify more rivals, and a law is validated only by a held-out prediction the reactor confirms, never by fit quality alone. Model calls and environment actions are totals over the tier. NormalChallenge Hypotheses frozen236433 Experiments executed4848 Rivals falsified157139 Laws validated108 Task instances with a validated law108 Task instances whose law-derived plan passed107 Task instances completed107 Actions taken under a validated law11594 Model calls, EvoBelief27444988 Model calls, ReAct1955918790 Model calls, WorldEvolver61086949 Environment actions, EvoBelief20262737 Environment actions, ReAct38393540 Environment actions, WorldEvolver30513472 which the actor took without the rule; and the knowledge surface described what appeared on the soil rather than what a planted seed would do, which the judge did not credit. The version reported in Table 3 is a revised implementation that addresses those three failure modes. A counterexample by absence is admitted only after the plan has publicly created the causal opportunity, that is, after the deposit, the refill, and a pre-declared number of maturation ticks; a validated rule controls the whole plan, from configuring the controller through binding, acquiring and depositing the seed, waiting for maturation, and observing; and the knowledge surface renders the causal chain and is scoped to the task instance. Development used only task instances previously used in the pilot. The code was then frozen and run once on the twelve Normal and twelve Challenge task instances of a fresh set that had been enumerated blind before the pilot ran and never executed by any method. One repair, the identification of the actuator on a previously used development Challenge instance, was imple- mented after the development checks passed and was re-validated offline and on one previously used development instance; the pilot is never pooled with the reported result. Plant Nutrients: mechanism. The DW Plant column of Table 6 counts what happened on the twenty-four fresh task instances. Of the nineteen completed task instances, seventeen completed through a validated task plan; one completed through a discriminating plan whose task plan fell short, and one completed with no plan. The retirements are informative: fifteen committed rules were retired by counterexamples, nine of them after the plan itself had created the opportunity, and thirty-five absence observations that would have counted as counterexamples in the first version were deferred because the opportunity had not yet been created. Knowledge margins over the baselines, in percentage points with paired bootstrap intervals: on Normal, 58.3 [33.3, 83.3] over ReAct, 66.7 [25.0, 100.0] over WorldEvolver, and 75.0 [50.0, 100.0] over the Hypothesizer; on Challenge, 33.3 [8.3, 58.3] over ReAct, WorldEvolver, and the Hypothesizer. Completion contrasts are in Table 7. The Hypothesizer baseline at two output ceilings. The Hypothesizer consolidates its knowledge into one long structured reply per step. Under the output ceiling frozen for the original campaigns, 3,072 tokens, that reply was truncated on both Challenge tiers, three bounded resends failed the same way, and the predeclared attempt policy classifies a failure at a frozen input or output admission as a method failure that is not retried: the Reactor Lab run sealed two Challenge task instances and the Plant Nutrients run one. Table 14 reports those original runs. For a complete baseline we reran the full Hypothesizer evaluation under a separately fixed configuration whose only change is that ceiling, raised to 8,192 tokens, on the same task instances by hash, one attempt per cohort, blind until every cohort had sealed. Prompt, policy, backbone, budgets, populations, and judge were unchanged, and no other method was re-run. Under the raised ceiling eleven consolidation replies exceeded the old ceiling and none approached the new one, so each of them would have been a 25 Preprint. Table 13: The law validated on each fresh task instance. f is the reactor resonance in Hz; c 5 , d, t, and q are the crystal’s spectrometer channel 5, density, temperature, and quantum gap. x is the held-out crystal’s reading, Predicted the prediction frozen before the reactor was dialed, and Dialed the frequency at which the reactor then activated. Rivals is the number of rival laws falsified on that task instance, Plan the outcome of the plan derived from the law for the target crystal (a passed plan completes the task), and Knowledge the judge’s score on that task instance. Normal seeds 13 and 18 and Challenge seeds 9, 14, and 19 hide the variable in a radiation reading with random background; on Challenge seed 20 no rival survived within budget, and on seed 17 the law was validated but its plan never settled. Seed Validated lawx PredictedDialed Rivals PlanKnowledge Normal s9f = 104.99 c 5 + 107.2530.503309.50 3310.0018 pass1.00 s10 f = 99.01 d + 102.2461.316172.69 6173.0014 pass1.00 s11 f = 105.00 t + 95.9617.811966.05 1966.0518 pass1.00 s12 f = 105.01 q + 90.4049.725311.60 5312.0016 pass1.00 s13none (radiation seed)7 –0.00 s14 f = 102.99 c 5 + 91.9239.094118.27 4118.2712 pass1.00 s15 f = 108.00 d + 99.6934.313805.48 3805.4812 pass1.00 s16 f = 106.01 t + 102.6446.755058.50 5059.0018 pass1.00 s17 f = 96.00 q + 90.9369.786789.88 6790.0010 pass0.50 s18none (radiation seed)9 –0.00 s19 f = 105.99 c 5 + 98.2316.281823.68 1824.0012 pass1.00 s26 f = 105.99 t + 100.1048.045192.24 5192.2411 pass1.00 Challenge s9none (radiation seed)4 –0.00 s10 f = 14.00 t 2 + 32.99 t + 568.9911.502800.00 2800.0017 pass1.00 s11 f = 17.00 d 2 + 25.85 d + 510.4815.615058.63 5059.0021 pass1.00 s12 f = 17.01 q 2 + 20.72 q + 783.455.491409.67 1410.0015 pass1.00 s13 f = 17.99 c 2 5 + 29.04 c 5 + 645.76 12.843985.97 3986.0018 pass1.00 s14none (radiation seed)7 –0.00 s15 f = 19.00 t 2 + 29.88 t + 212.7818.577321.15 7321.1515 pass1.00 s16 f = 17.99 d 2 + 33.16 d + 41.0811.852960.66 2961.005 pass0.50 s17 f = 12.89 q 2 + 23.02 q + 790.497.121608.01 1608.559 unsettled0.50 s18 f = 14.99 c 2 5 + 24.03 c 5 + 581.88 17.275470.28 5470.2718 pass1.00 s19none (radiation seed)3 –0.00 s20none7 –0.50 Table 14: The Hypothesizer at its original 3,072-token output ceiling. Same metrics and task instances as Table 3, in percent; the Challenge runs ended on output truncation, and their means are over the task instances sealed before the failure. FamilyTierSealedCompletionKnowledgeProcedural Reactor LabNormal12 of 120.000.0074.33 Reactor LabChallenge2 of 120.000.0074.14 Plant NutrientsNormal12 of 1266.670.0082.41 Plant NutrientsChallenge1 of 120.000.0066.67 truncation before. Those re-runs are the Hypothesizer rows of Table 3. The two Normal re-runs also show the run-to-run variation of the backbone: the Plant Nutrients completion count moved from eight to six on the same twelve task instances with the same prompt. F.6HELD-OUT PREDICTION EVALUATION Held-out prediction evaluation. Table 15 evaluates 1,500 transitions per benchmark from episodes disjoint from belief formation. On ScienceWorld the belief is formed on 90 development episodes, and on ALFWorld on the 140 seen-split development episodes, with the evaluated transi- tions drawn from the unseen split. Each transition is scored once, with Haiku 4.5 as the backbone for every row, one frozen prediction prompt, and greedy decoding. State-∆ F1 is the set F1 between predicted and realized canonical state-change atoms, averaged over transitions, with a correct pre- diction of no change scoring one; Feasibility is agreement on whether the action executes; EM and 26 Preprint. Table 15: Held-out structured state-change prediction on ScienceWorld and ALFWorld, 1,500 tran- sitions per benchmark, disjoint from belief formation. State-∆ F1 measures canonical state changes; Feasibility measures action executability; EM and Token F1 compare structured state. Haiku 4.5. ModelState-∆ F1↑ Feasibility↑EM↑Token F1↑ ScienceWorld Frozen WM (no evolving W )0.72580.75470.60130.7866 WorldEvolver (semantic-only)0.70610.78270.60270.7814 WorldEvolver (full)0.66800.87530.59930.7820 EvoBelief0.75300.81130.64070.8045 ALFWorld Frozen WM (no evolving W )0.75720.84530.65670.8080 WorldEvolver (semantic-only)0.68620.85000.57330.7607 WorldEvolver (full)0.72500.84000.60600.7849 EvoBelief0.84730.85130.76130.8698 Token F1 compare one deterministic rendering of the predicted and the realized state record. Frozen WM is the controller without W ; the WorldEvolver rows run the official implementation with its semantic memory alone and with both memories. The ALFWorld block is a post-hoc rescoring of sealed structured predictions under the ScienceWorld metric definitions, which were fixed before any ALFWorld aggregate under them was computed. Under a paired bootstrap over transitions (10,000 resamples), EvoBelief’s lead over every other row on State-∆ F1, EM, and Token F1 has a 95% in- terval excluding zero on both benchmarks; on Feasibility, WorldEvolver-full’s lead on ScienceWorld excludes zero, while every ALFWorld difference has an interval that includes zero. 27