Paper deep dive

When History Fails to Become Experience: Action Calibration in Language Agents

Jingyu Liu, Zhiwen Wang, Yuxin Jing, Huanyu Zhou, Yong Liu

Year: 2026Venue: arXiv preprintArea: cs.CLType: PreprintEmbeddings: 62

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 10/5/2026, 6:52:41 AM

Summary

This paper investigates why language agents often fail to improve decision-making by using interaction history, a phenomenon termed 'action calibration.' The authors find that while history generally helps, shuffling action-observation pairs causes only modest performance drops, suggesting agents do not reliably link actions to outcomes. Explicitly labeling observations as action outcomes ('Causal Transition') improves task success and reduces repetition. However, having agents generate their own experiences ('Experience') often degrades performance due to unsupported inferences. To address this, the authors propose a 'Learned Calibrator' that selectively records and retains useful experiences based on their impact on future actions, outperforming both ordinary history and simple outcome labeling.

Entities (21)

Action Calibration · concept · 95%AlfWorld · environment · 95%AppWorld · environment · 95%ByteDance · organization · 95%Gaoling School of Artificial Intelligence · organization · 95%Huanyu Zhou · person · 95%Jingyu Liu · person · 95%Learned Calibrator · modelcomponent · 95%Renmin University of China · organization · 95%ScienceWorld · environment · 95%WebShop · environment · 95%Yong Liu · person · 95%Yuxin Jing · person · 95%Zhiwen Wang · person · 95%Causal Transition · method · 90%DeepSeek V4 Pro · model · 90%GPT 5.6 Sol · model · 90%Qwen3-32b · model · 90%Qwen3.5-Plus · model · 90%Qwen3-8b · model · 90%Experience · method · 85%

Relation Signals (33)

Jingyu Liu → affiliatedwith → Renmin University of China

confidence 95% · Jingyu Liu 1 ... 1 Gaoling School of Artificial Intelligence, Renmin University of China

Yuxin Jing → affiliatedwith → ByteDance

confidence 95% · Yuxin Jing 2 ... 2 ByteDance

Zhiwen Wang → affiliatedwith → ByteDance

confidence 95% · Zhiwen Wang 2 ... 2 ByteDance

Huanyu Zhou → affiliatedwith → ByteDance

confidence 95% · Huanyu Zhou 2 ... 2 ByteDance

Yong Liu → affiliatedwith → Renmin University of China

confidence 95% · Yong Liu 1,3,4† 1 Gaoling School of Artificial Intelligence, Renmin University of China

Yong Liu → affiliatedwith → Gaoling School of Artificial Intelligence

confidence 95% · Yong Liu 1,3,4† 1 Gaoling School of Artificial Intelligence

Jingyu Liu → affiliatedwith → Gaoling School of Artificial Intelligence

confidence 95% · Jingyu Liu 1 ... 1 Gaoling School of Artificial Intelligence

Causal Transition → improves → task success

confidence 90% · Figure 4 shows that Causal Transition improves overall task success over Raw.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Language agents should draw on prior attempts and environmental feedback to improve subsequent decisions within the same task. However, providing additional interaction history can sometimes reduce task success, suggesting that agents do not consistently use this information effectively. To investigate this limitation, we examine how agents use history. We find that history improves task completion overall, yet much of this benefit persists even when past actions are shuffled. Disrupting the correspondence between actions and observations causes only a modest decline in task success. We therefore hypothesize that agents do not reliably connect past actions with their outcomes when deciding how to proceed. To test this hypothesis, we explicitly label each returned observation as the outcome of the preceding action. This simple annotation improves task success and reduces next-action repetition without introducing new environmental information. Building on this insight, we introduce a learned calibrator that explicitly reassesses past actions and selectively records experience to guide subsequent decisions, improving task success beyond outcome labeling alone.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

59,312 characters extracted from source content.

Expand or collapse full text

Preprint WHEN HISTORY FAILS TO BECOME EXPERIENCE: ACTION CALIBRATION IN LANGUAGE AGENTS Jingyu Liu 1 Zhiwen Wang 2 Yuxin Jing 2 Huanyu Zhou 2 Yong Liu 1,3,4† 1 Gaoling School of Artificial Intelligence, Renmin University of China, Beijing, China 2 ByteDance 3 Beijing Key Laboratory of Research on Large Models and Intelligent Governance 4 Engineering Research Center of Next-Generation Intelligent Search and Recommendation, MOE liujy1016@ruc.edu.cn † liuyonggsai@ruc.edu.cn ABSTRACT Language agents should draw on prior attempts and environmental feedback to improve subsequent decisions within the same task. However, providing additional interaction history can sometimes reduce task success, suggesting that agents do not consistently use this information effectively. To investigate this limitation, we examine how agents use history. We find that history improves task completion overall, yet much of this benefit persists even when past actions are shuffled. Disrupting the correspondence between actions and observations causes only a modest decline in task success. We therefore hypothesize that agents do not reliably connect past actions with their outcomes when deciding how to proceed. To test this hypothesis, we explicitly label each returned observation as the outcome of the preceding action. This simple annotation improves task success and reduces next- action repetition without introducing new environmental information. Building on this insight, we introduce a learned calibrator that explicitly reassesses past actions and selectively records experience to guide subsequent decisions, improving task success beyond outcome labeling alone. 1INTRODUCTION A line of work on language agents and self-evolving systems distills past trajectories into reusable lessons and guidelines for subsequent tasks (Ding et al., 2026; Xu et al., 2026). Within an ongoing task, however, the agent’s own actions and environmental feedback are already available as an interaction history (Yao et al., 2023). These attempts can reveal what worked, what failed, and what needs to change. This history therefore provides task-specific evidence that should help the model make better decisions. However, interaction history can also reinforce unsuccessful behavior. Prior work links conversational inertia to imitation of agents’ own responses (Wan et al., 2026), and finds that failed action strings can sometimes outweigh corrective feedback in small models (Gumaan, 2026). The same history that encourages an agent to repeat an unsuccessful action also contains feedback indicating its failure. This raises the question of how agents use that evidence within an ongoing task. We therefore test whether additional history helps the agent. We restore an early environment state from a failure trajectory and compare continuations with the original prefix alone or with additional future actions and feedback from the same trajectory, provided as reference history. The reference history is expected to help the model better understand the environment and identify what might help and what might hurt. But overall recovery is lower with the additional reference history: in this setting, more attempts and feedback make it harder for the agent to complete the task instead of helping it understand the environment. To better understand how models use interaction history, we shuffle past actions while leaving observations in their original order, disrupting the original action–outcome pairings. Overall task success nevertheless remains close to that with the original history. These results suggest that models make limited use of the original action–outcome pairings in ordinary history. Together with the recovery results, this motivates our hypothesis that ordinary history does not reliably elicit action 1 arXiv:2610.02769v1 [cs.CL] 2 Oct 2026 Preprint A Ordinary history History Action Aₜ Observation Oₜ₊₁ Calibration gap Feedback without reliable reassessment B Outcome labeling History Action Aₜ Outcome of Aₜ Observation Oₜ₊₁ Only the label changes Higher task success, less repetition C Learned calibration History Aₜ Oₜ₊₁ Learned calibrator Reassess the action using its outcome Write & retain experience NO_UPDATE Actor: next action Figure 1: Action calibration within an ongoing task. (A) Ordinary history simply retains the observation returned by the environment. (B) Outcome annotates the observation as the outcome of the previous action. (C) A calibrator selectively retains experience to guide later decisions. calibration: using an action’s outcome to decide whether an earlier approach should be continued, adjusted, or abandoned. One possibility is that the model treats an observation primarily as context for the action taken in response, rather than as the outcome of the previous action. So it fails to calibrate the action based on its outcome. To test this hypothesis, we annotate each returned observation as the outcome of the action that produced it (outcome of action A: new observation). This change draws attention to what a past action actually achieved, without supplying new environmental information or an additional judgment about whether the action was appropriate. This annotation improves overall task success and reduces action repetition. These gains further support the hypothesis that agents underuse existing action–outcome feedback in ordinary history. Outcome labels leave the agent to implicitly decide what to learn from each attempt. We therefore ask the model to explicitly decide whether calibration is needed and perform calibration by summarizing and recording experience from its interaction history. Yet the expected improvement does not follow: retaining self-generated experiences throughout a task lowers overall success relative to ordinary history. Inspecting the trajectories reveals that some experiences merely restate available feedback, while others even introduce unsupported judgments. These results motivate learning which experiences to retain. Their effects on next-action quality across multiple actors provide supervision for a calibrator that learns when an update is warranted and what experience to write. The learned calibrator improves task completion on held-out tasks over both ordinary history and outcome labeling. Figure 1 summarizes the calibration gap, outcome labeling, and learned calibration. Our contributions are fourfold: 1. We show that additional reference history can reduce recovery. 2.History interventions further reveal that historical information helps task completion, and much of this benefit persists when historical action strings are shuffled. 3.We show that annotating observations as action outcomes improves task success and reduces next-action repetition on fixed histories. 4.We train a calibrator to select and generate experiences, improving task completion on held-out tasks. 2RELATED WORK Summarizing and using experience. Agents extract reusable guidance from past interactions, ranging from verbal reflections across trials (Shinn et al., 2023) to procedural skills from trajectory memories (Ding et al., 2026). Learning also shapes how this guidance is managed and reused for future tasks: agents can evolve memory operations and learn which to apply (Zhang et al., 2026a), or 2 Preprint use environmental feedback to update experience utility and guide retrieval (Zhang et al., 2026b). These methods explicitly construct guidance from previous trajectories. We study whether agents use feedback already in an ongoing task’s history to revise earlier approaches. Negative effects of interaction history. Conversational and contextual inertia can reinforce prior responses and reasoning despite new constraints (Wan et al., 2026; Chen et al., 2026). Failed tool calls can encourage repetition despite error feedback (Gumaan, 2026). EVU (Wang et al., 2026) identifies belief inertia: observations that contradict an agent’s expectations may fail to update its internal understanding of the environment in small models. However, these studies do not examine how the negative effects of history interact with the additional information it provides, or the reason why current models fail to take advantage of feedback. 3MORE HISTORY DOES NOT ENSURE BETTER RECOVERY As interaction proceeds, an agent accumulates more information about the environment. This information should help the agent avoid ineffective choices and reassess its approach. We test this expectation by restoring an early state of each unsuccessful trajectory and then providing the model with additional future interaction history as a reference to show how the extra information affects performance. Specifically, for a source trajectory of lengthL, we restore the environment to⌊L/3⌋, and instruct the model to continue the task with the following two settings. 1. Raw provides only the action–observation history through that point. 2.Reference additionally provides the recorded segment from⌊L/3⌋to⌊2L/3⌋in a separate reference block, explicitly marked as not executed in the restored environment. For example, if a source trajectory fails after 30 actions, both continuations start from the state after action 10; the second also receives actions 11–20 and their outcomes as reference information. The actor receives the actual current observation and available actions, together with any newly executed continuation history. Both conditions use the source actor and allow up to 30 new actions. Recovery means terminal task success within this budget. Raw Prefix Reference PrefixRef. History through 혓/ퟥ Reference: 혓/ퟥ to ퟤ혓/ퟥ Each: 30 new actions ALF World Science World Web Shop App World 0 10 20 30 40 Recovery (%) 33.2 20.4 4.0 8.8 26.4 18.0 2.0 6.4 RawReference GPT 5.6-Sol DeepSeek V4 Pro Qwen 3.5 Plus Qwen3 32B Qwen3 8B 0 10 20 30 Recovery (%) 27.5 18.0 16.5 12.0 9.0 25.0 13.5 11.0 9.0 7.5 RawReference A Shared restartB By environmentC By model Same state at 혓/ퟥ Overall recovery 16.60% → 13.20% (−3.40 p) Figure 2: Recovery from the same restored state. For a failed trajectory of lengthL, Panel A contrasts Raw (L/3history) with Reference (adding the recordedL/3–2L/3segment, not executed in the restored environment). Both conditions resume atL/3with a budget of 30 new actions, using the same 1,000 source trajectories. Panels B–C pool recovery by environment and model, respectively. We evaluate GPT-5.6-Sol, DeepSeek-V4-Pro, Qwen3.5-Plus, Qwen3-32B, and Qwen3-8B in ALF- World, ScienceWorld, WebShop, and AppWorld (Shridhar et al., 2021; Wang et al., 2022; Yao et al., 2022; Trivedi et al., 2024). 3 Preprint We can observe from Figure 2 that adding the reference history lowers recovery, despite providing more attempts and feedback from the same task. Recovery is lower in all four environment aggregates and all five model aggregates. Appendix D.6 gives the prompt format. The same pattern holds with separate user and assistant messages: adding reference history reduces recovery from 15.8% to 11.4% (Appendix F.2). Because adding history also lengthens the input, we examine prompt lengths with and without reference history. As shown in Figure 3, prompts with additional reference history have a median length of approximately 1.6k tokens. We retain only historical observations and actions, omit intermediate reasoning, and format the history as in Feng et al. (2025). This length is way below the context capacity of current models, especially state-of-the-art models such as GPT-5.6-Sol, showing that the performance decline cannot be explained solely by context length. 1k4k16k64k Prompt tokens at resumption 0 50 100 Requests at or below (%) 97.0% below 16k Reference Raw Reference 1k4k16k32k Prompt tokens (Reference) ALFWorld ScienceWorld WebShop AppWorld A Prompt-length distributionB By environment Figure 3: Prompt lengths at the first request after resumption, for 1,000 source trajectories. Panel A compares Raw with Reference; Panel B shows Reference medians (diamonds) and 10th–90th percentiles (lines) by environment. Both horizontal axes are logarithmic. The dashed line at 16,384 tokens is a visual reference, not a model context limit. 4HOW AGENTS USE THEIR HISTORY Additional reference history does not, by itself, yield better recovery from these unsuccessful trajectories. We therefore ask how much agents benefit from retaining historical information and how much their performance depends on preserving its original action–outcome organization. We compare ordinary history (Raw) with four interventions in full rollouts from the initial task state (Table 1). Recent-3-Only removes all but the three latest history entries; Observation-Only removes historical action strings but retains all observations in order. Shuffle-All retains both kinds of content but randomly reassigns actions among all historical action positions. Shuffle-Old instead preserves the three latest entries and reassigns older actions only among earlier positions. Both shuffle conditions keep observations in chronological order while changing the placement of action strings and their original correspondences with observations. Table 1: Five history views at the same point in an interaction.O t is an observation; actionA t responds to it and producesO t+1 . Orange denotes reassigned actions; the permutations are illustrative. ConditionEarlier historyRecent history Raw(O 1 , A 1 ), (O 2 , A 2 ), (O 3 , A 3 ) (O 4 , A 4 ), (O 5 , A 5 ), (O 6 , A 6 ) Recent-3-OnlyEntire entries omitted(O 4 , A 4 ), (O 5 , A 5 ), (O 6 , A 6 ) Observation-Only O 1 , O 2 , O 3 O 4 , O 5 , O 6 Shuffle-All(O 1 , A 4 ), (O 2 , A 5 ), (O 3 , A 6 ) (O 4 , A 1 ), (O 5 , A 2 ), (O 6 , A 3 ) Shuffle-Old(O 1 , A 2 ), (O 2 , A 3 ), (O 3 , A 1 ) (O 4 , A 4 ), (O 5 , A 5 ), (O 6 , A 6 ) Current observation O 7 is retained in all five conditions. Each of the five history conditions contains 500 tasks per model–environment cell, or 10,000 episodes per condition across the five actors and four environments. 4 Preprint Removing earlier history substantially lowers overall success (Table 2). Recent context is therefore not generally a substitute for accumulated information. Observation-Only retains much of Raw’s success and outperforms Recent-3-Only overall: historical observations preserve useful task information even when the actions that accompanied them are omitted. We show full results in Appendix C. Table 2: Task success (%) under history interventions. The two blocks summarize the same evaluation grid: 2,500 tasks per environment, 2,000 per model, and 10,000 overall, per condition. Table 4 gives the full breakdown. GroupRaw Recent-3- Only Observation- Only Shuffle-AllShuffle-Old By environment ALFWorld67.0852.4461.8864.6468.08 ScienceWorld39.1231.8835.8834.8438.56 WebShop27.7227.2820.6825.4427.88 AppWorld51.6432.3645.2850.4452.48 By model GPT-5.6-Sol73.3060.2068.8570.7573.45 DeepSeek-V4-Pro63.0546.4056.0057.6562.35 Qwen3.5-Plus48.3037.1541.1048.0049.80 Qwen3-32B29.0522.9522.3027.2029.80 Qwen3-8B18.2513.2516.4015.6018.35 Overall46.3935.9940.9343.8446.75 Shuffle-All retains most of Raw’s performance, with an overall decrease of only 2.55 percentage points (Table 2). Task success is thus only modestly sensitive to disrupting the original action–outcome pairings. Shuffle-Old improves over Shuffle-All and stays close to Raw: when recent interactions remain intact, preserving earlier pairings provides little additional overall benefit. Preserving recent interactions may also make the immediate context more coherent and familiar, so this gain need not reflect action calibration. Historical content remains useful despite disruption of its original action–outcome organization. Together with poorer recovery from additional reference history (Section 3), this motivates our hypothesis that ordinary history does not reliably elicit action calibration. Agents may extract useful information without sufficiently using outcomes to reassess whether earlier approaches should be continued, adjusted, or abandoned. 5TURNING RECORDED FEEDBACK INTO USEFUL EXPERIENCE 5.1MAKING ACTION OUTCOMES EXPLICIT To test this hypothesis, we ask whether explicitly identifying an action’s outcome helps the agent use feedback already in its history to reassess previous attempts. We therefore use Causal Transition to make each action’s outcome explicit. We label each returned observation as the outcome of its preceding action:[Outcome of A t−1 : O t , A t ], followed by [Outcome of A t : O t+1 , A t+1 ]. The underlying observations and their order are unchanged. Figure 4 shows that Causal Transition improves overall task success over Raw. Gains appear in most model– environment combinations and in every model-level and environment-level aggregate. Explicitly reminding the actor to recheck its history yields a similar overall success rate to Causal Transition (Appendix D.1). We hypothesize that this difficulty arises in part from the next-action objective. Language agents are typically asked to answer a question or choose the next action from the current observation. This objective may encourage them to respond directly to the latest observation without relating it back to the action that produced it. 5 Preprint ALF World Science World Web Shop App World Model mean +0.2+1.4+0.4+2+1 +1.4+0.8+0.4+0.4+0.75 +5.6+5.2+2.6+1.2+3.65 +1.2+5.2+1.2+1+2.15 0+2.4+3.8+0.8+1.75 +1.68+3+1.68+1.08 +1.86 ALF World Science World Web Shop App World Model mean −1.4 −1.8 −2.4 −1.2 −1.7 −0.8 −2.2 −2.2 −0.4 −1.4 +6.2+0.4+1.2 −3.8+1 −7.8+2.2 −2.4 −1 −2.25 −12.2+2+0.2 −0.2 −2.55 −3.2+0.12 −1.12 −1.32 −1.38 −13013 Model GPT-5.6-Sol DeepSeek-V4-Pro Qwen3.5-Plus Qwen3-32B Qwen3-8B Env. mean A Causal Transition Overall change +1.86 p B Experience Overall change −1.38 p Success change from Raw (p) Figure 4: Task-success changes from the matching model–environment Raw baseline, in percentage points (500 tasks per cell). Both panels use the same color scale. The final row and column give environment and model means; the lower-right cell gives the pooled change over 10,000 episodes per condition. To test whether outcome annotation changes action reuse, we measure next-action repetition on fixed histories. We use exact-action repetition as a directly measurable indicator of action reuse (Gumaan, 2026). We select histories ending in two identical actions, filtered for feedback that does not support another repetition (judged by GPT-5.6-Sol). The source actor generates one next action from each prefix under Raw and Causal, and we check whether the actor repeats the action (Appendix B.1). Figure 5 shows lower next-action repetition under Causal for all five models. The annotation supplies neither new evidence nor advice about which action to take, yet it changes action reuse on the same preceding history. Together with the rollout gains, this result shows that identifying existing feedback as an action’s outcome can affect both the next decision and task completion. These results support our hypothesis that ordinary history does not fully elicit outcome-based reassessment of past actions. 0 25 50 8.7 25.2 25.5 18.9 51.4 6.5 22.9 21.8 15.7 45.3 4.3 19.1 19.1 17.1 25.2 GPT-5.6-Sol N = 46 DeepSeek-V4-Pro N = 131 Qwen3.5-Plus N = 188 Qwen3-32B N = 492 Qwen3-8B N = 958 Next-action repetition (%) RawCausalExperience Figure 5: Probability of repeating the same action a third time. Each model receives the sameN histories in all three conditions, ending in two identical actions and screened for feedback that does not support another repetition. Lower is better. 5.2FROM OUTCOME LABELS TO EXPLICIT EXPERIENCE Annotating an observation as an action’s outcome tells the actor what happened, but leaves it to decide what to learn from that result. Since outcome labeling already helps, we ask whether explicitly stating that lesson as an experience provides further benefit. Under Experience, we retain the Raw history format and append actor-generated experiences to the history. After each completed nonterminal interaction, we make a separate review call to the same actor, denotedC actor . Its inputx t contains the task and raw history through actionA t and its returned observationO t+1 . We instruct the model to generate an experience based on the given action and returned observation when needed. The experience-generation prompt is provided in Appendix G.2. 6 Preprint The call produces either a task-relevant experience E t or NOUPDATE: y t = C actor (x t )∈ NOUPDATE, (CALIBRATE, E t ) .(1) A generated experienceE t remains in the actor’s history for subsequent decisions;NOUPDATE leaves that history unchanged. This review asks the actor to use environmental feedback to explicitly decide whether an earlier approach should be continued, adjusted, or abandoned. Appendix D.3 reports how often each actor generates an experience or returns NOUPDATE in each environment. However, experience does not deliver the expected gain in task completion. In full rollouts, its overall success rate is lower than both Raw and Causal (Figure 4). It reduces next-action repetition relative to Raw across all five models in the fixed-history probe, with a lower pooled rate than Causal (Figure 5). But this reduction in repetition does not translate into improved task success. These results are consistent with prior findings that generated reflections can be ineffective or counterproductive for interactive agents (Li et al., 2025; Lippmann et al., 2025). To understand what causes the performance degradation, we inspect the experiences generated during full rollouts. We find that current models can produce poor experiences: some simply repeat the observation, while others introduce unsupported claims. Figure 6 shows examples. The GPT-5.6-Sol experience simply restates an inventory change already reported in the observation. The Qwen3.5- Plus experience introduces a claim unsupported by the environment feedback. Appendix D.5 reports further examples. Appendix D.4 further shows that generated experiences can degrade next-action quality, including experiences generated by GPT-5.6-Sol. CaseAction (A) and observation (O)Generated experience A Outcome restatement GPT-5.6-Sol · ALFWorld A: take potato 1 from fridge 1 O: “You pick up the potato 1 from the fridge 1.” B Unsupported inference Qwen3.5-Plus · ALFWorld A: put mug 1 on stoveburner 1 O: “Nothing happens.” “Potato 1 is now in the agent's inventory.” “...the mug must be placed inside a pot that is already on the stoveburner to become hot.” Figure 6: An outcome restatement and an unsupported inference in actor-generated experience. Highlighting marks the repeated fact or unsupported rule. Quotations come from real ALFWorld trajectories; the second experience is excerpted. Causal labels existing feedback as an action’s outcome. Experience goes further by adding the actor’s interpretation and retaining it as guidance for later decisions. Because this guidance persists, redundant or unsupported experiences can affect more than the immediately following action. The next question is therefore which experiences should be retained. 6LEARNING WHEN AND HOW TO CALIBRATE ACTIONS 6.1CONSTRUCTING LABELS FROM PAIRED NEXT-ACTION COMPARISONS The Experience baseline already lets the actor skip an update, yet it can retain redundant or unsup- ported guidance. Learning which experiences to retain therefore calls for supervision grounded in their effects on decisions. We use changes in next-action quality across multiple actors as a signal for when to retain an experience and what to write. We sample 10,000 real decision contexts evenly across the five source actors and four environments, using rollouts on tasks disjoint from the final evaluation. GPT-5.6-Sol reviews each context and returns eitherNOUPDATEor a candidate experienceE. A directNOUPDATEresponse becomes a NOUPDATEtraining target. Then, a GPT judge screens candidate experiences to exclude unsupported claims and experiences that merely repeat the observation. If a candidate fails this screening, its context is retained with a NOUPDATE training target. Only candidates that pass this content screening are evaluated with all five actors. Each actor generates a next action twice from the same decision context: once from the original prompt and once with Eadded. The two generations form a within-actor pair that differs only in whether the candidate 7 Preprint experience is present. In a separate judging call, GPT-5.6-Sol compares each pair and classifies the change in next-action quality as improved, unchanged, or degraded. The judge considers whether the actions respect execution constraints, plausibly advance the task, and avoid attempts already shown to be unproductive by the recorded feedback. Letn ↑ (E)andn ↓ (E)be the numbers of actors whose next-action quality improves or degrades with E. We define the cross-actor quality margin as q(E) = n ↑ (E)− n ↓ (E).(2) Ifq(E) > 0orn ↓ (E) = 0, the training target isCALIBRATEfollowed byE. Otherwise, the target is NOUPDATE. Thus, a candidate experience is retained when improvements outnumber degradations or when no actor’s next-action quality degrades. Candidates that leave next-action quality unchanged for all actors are retained because their guidance may become useful later in the trajectory. We fine-tune a Qwen3-8B calibrator on these prompt–target pairs using supervised fine-tuning for 3 epochs with learning rate 1e-5. Each target contains eitherNOUPDATEorCALIBRATEfollowed by the selected experience. The model therefore learns both whether to write an update and what to write. During rollout, the frozen calibrator reviews each completed nonterminal interaction and either adds an experience to the history or returnsNOUPDATEwithout changing it. Generated experiences persist in the history, while the original actor continues to generate actions. Appendix G records the core interfaces. 6.2TASK COMPLETION EVALUATION We compare the calibrator with Raw and Causal Transition using the same five actors and four environments, with 500 tasks per model–environment cell. The rollout tasks are distinct from the calibrator’s training tasks. GPT-5.6 Sol DeepSeek V4 Pro Qwen3.5 Plus Qwen3 32B Qwen3 8B 0 20 40 60 80 Task success (%) +5.6 +4.5 +6.2 +3.2 +6.0 ALFWorld Science World WebShop AppWorld +5.2 +7.3 +6.4 +1.2 A By actor modelB By environment Overall 46.39% → 51.45% (+5.06 p vs Raw) RawCausal TransitionCalibrator Figure 7: Full-rollout success (%) for Raw, Causal Transition, and Calibrator. Panel A pools 2,000 tasks per actor; Panel B pools 2,500 per environment (500 per model–environment cell). Labels show Calibrator gains over Raw in percentage points; Overall pools all 10,000 tasks per condition. Figure 7 shows that the calibrator improves task success over Raw in every model-level and environment-level aggregate. Qwen3.5-Plus shows the largest performance gain and also bene- fits most from labeling observations as action outcomes (Figure 4). We hypothesize that these larger gains reflect weaker action-calibration ability than in the other models. A calibrator trained with content screening alone, without paired next-action comparisons across actors, achieves lower overall success than the full pipeline (Appendix E.2). This supports assessing experiences through their effects on actor decisions. 6.3FAILURE RECOVERY To test whether our method and outcome annotation help the model use historical information, we return to the same-state recovery comparison in Section 3. 8 Preprint Can changing how feedback is presented and reviewed improve recovery when reference history is available? Alongside Raw and Reference, we evaluate three reference-augmented conditions. Causal + Reference labels observations as action outcomes in both the actual history and the reference segment. Experience generates an experience once at resumption and retains it, while Calibrator uses the learned update rule during continuation. All five conditions share the source trajectories, restored L/3 state, actor, and 30-new-action budget. All three interventions raise pooled recovery above both unmodified Reference and the Raw prefix- only baseline, with Calibrator achieving the highest recovery (Figure 8). The recovery gain from Experience shows that reviewing failed histories can produce useful guidance for subsequent decisions. In general full rollouts, however, our trajectory inspection also identifies redundant or unsupported experiences that can misguide later decisions and lead to worse overall performance. Thus, the recovery loss from supplying unannotated reference history can be reversed by labeling action outcomes or adding explicit guidance. This shows that these methods can effectively suppress the negative effects of history and take advantage of the extra information. Appendix D.6 provides further details. 6.4INTEGRATING CALIBRATION INTO THE ACTOR Can the actor learn to generate this guidance alongside its action, without a separate calibration call? We further integrate the calibrator into the actor by training two Qwen3-8B variants. The action-only baseline is trained on successful trajectories generated by GPT-5.6-Sol. The joint variant uses successful trajectories generated by GPT-5.6-Sol with our calibrator and combines calibration and action in each target. It therefore produces both in a single generation. Both variants use equal amounts of training data and five random seeds. We evaluate each trained model on 500 tasks per environment. The joint actor achieves higher task success in all four environments (Figure 9). Averaged across environments and seeds, success rises from 47.21% to 52.21%. Calibration-augmented distillation therefore improves task completion performance. Raw Reference Causal Experience Calibrator 0 5 10 15 20 Recovery (%) 16.6 13.2 18.1 17.8 19.9 With reference history Figure 8: Recovery from the sameL/3state (1,000 continuations/condition). All but Raw add reference history; the dashed line marks Raw. Experience is generated once at resumption and retained; Calibrator runs during continuation. ALFWorld Science World WebShopAppWorld 0 20 40 60 80 100 Task success (%) 85.1 51.6 32.9 19.2 89.9 56.6 38.0 24.4 Action onlyCalibration + action Figure 9: Qwen3-8B actor success. The joint actor generates calibration and action in one re- sponse. Bars show mean±SD across five train- ing seeds, with 500 tasks per environment and seed. 7CONCLUSION Interaction history provides evidence for revising earlier attempts, but retaining it does not ensure better decisions. Labeling observations as action outcomes improves task success without new environmental information, whereas self-generated experience can introduce redundant or unsup- ported guidance. Learning which experiences to retain improves task completion and recovery when unsuccessful attempts are supplied as reference. Effective history use therefore depends on using past outcomes to guide later decisions. 9 Preprint AI USE STATEMENT We used generative AI tools to assist with literature search, discussion and refinement of research hypotheses and experimental designs, code implementation and debugging, data analysis, and figure preparation. We also used these tools to revise, polish, and review the manuscript. The authors reviewed and verified the AI-assisted work. REFERENCES Xingwu Chen, Zhanqiu Zhang, Steven Y. Guo, and Difan Zou. Breaking contextual inertia: Reinforcement learning with single-turn anchors for stable multi-turn interaction. In Findings of the Association for Computational Linguistics: ACL 2026, p. 6302–6318. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026.findings-acl.313. URLhttps://aclanthology.org/2026.findings-acl. 313/. Jie Ding, Rui Sun, Xinyuan Zhang, Zeyu Zhang, and Xin Liu. APEx: Distillation of agent procedural experience for adaptive deep research question answering. arXiv preprint arXiv:2609.02253, 2026. doi: 10.48550/arXiv.2609.02253. URL https://arxiv.org/abs/2609.02253. Lang Feng, Zhenghai Xue, Tingcong Liu, and Bo An.Group-in-group policy optimization for LLM agent training. In Advances in Neural Information Processing Systems, volume 38, 2025. doi: 10. 52202/085713-1544. URLhttps://proceedings.neurips.c/paper_files/paper/2025/ hash/420c9f777c0b4f78d515e53cf74d58b2-Abstract-Conference.html. Esmail Gumaan. Feedback that backfires: Why small language model agents repeat the call they just watched fail. arXiv preprint arXiv:2608.23651, 2026. doi: 10.48550/arXiv.2608.23651. URLhttps://arxiv. org/abs/2608.23651. Kun Li, Tingzhang Zhao, Wei Zhou, and Songlin Hu. DORA: Dynamic optimization prompt for contin- uous reflection of LLM-based agent. In Proceedings of the 31st International Conference on Compu- tational Linguistics, p. 7546–7557. Association for Computational Linguistics, 2025. URLhttps: //aclanthology.org/2025.coling-main.504/. Philip Lippmann, Matthijs T. J. Spaan, and Jie Yang. Positive experience reflection for agents in interactive text environments. In Proceedings of the 1st Workshop for Research on Agent Language Models (REALM 2025), p. 131–142. Association for Computational Linguistics, 2025. doi: 10.18653/v1/2025.realm-1.10. URL https://aclanthology.org/2025.realm-1.10/. Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. Reflexion: Lan- guage agents with verbal reinforcement learning. In Advances in Neural Information Processing Systems, volume 36, p. 8634–8652, 2023. doi: 10.52202/075280-0377. URLhttps://doi.org/10.52202/ 075280-0377. Mohit Shridhar, Xingdi Yuan, Marc-Alexandre C ˆ ot ́ e, Yonatan Bisk, Adam Trischler, and Matthew Hausknecht. ALFWorld: Aligning text and embodied environments for interactive learning. In International Conference on Learning Representations, 2021. URL https://openreview.net/forum?id=0IOX0YcCdTn. Harsh Trivedi, Tushar Khot, Mareike Hartmann, Ruskin Manku, Vinty Dong, Edward Li, Shashank Gupta, Ashish Sabharwal, and Niranjan Balasubramanian. AppWorld: A controllable world of apps and people for benchmarking interactive coding agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 16022–16076, 2024. doi: 10.18653/v1/2024. acl-long.850. URL https://aclanthology.org/2024.acl-long.850/. Yang Wan, Zheng Cao, Zhenhao Zhang, Zhengwen Zeng, Shuheng Shen, Changhua Meng, and Linchao Zhu. Mitigating conversational inertia in multi-turn agents. arXiv preprint arXiv:2602.03664, 2026. URL https://arxiv.org/abs/2602.03664. Hanlin Wang, Chak Tou Leong, Jian Wang, and Wenjie Li. Seeing isn’t believing: Mitigating belief inertia via active intervention in embodied agents. In Findings of the Association for Computational Linguistics: ACL 2026, p. 37796–37815. Association for Computational Linguistics, 2026. doi: 10.18653/v1/2026. findings-acl.1884. URL https://aclanthology.org/2026.findings-acl.1884/. Ruoyao Wang, Peter Jansen, Marc-Alexandre C ˆ ot ́ e, and Prithviraj Ammanabrolu. ScienceWorld: Is your agent smarter than a 5th grader? In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, p. 11279–11298, 2022. doi: 10.18653/v1/2022.emnlp-main.775. URLhttps: //aclanthology.org/2022.emnlp-main.775/. 10 Preprint Ruiyao Xu, Tiankai Yang, and Wei-Chieh Huang. HyperSkill: Self-evolving LLM agents via hypergraph- structured skill memory. arXiv preprint arXiv:2608.16114, 2026. doi: 10.48550/arXiv.2608.16114. URL https://arxiv.org/abs/2608.16114. Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. WebShop: Towards scalable real-world web interaction with grounded language agents. In Advances in Neural Information Processing Systems, volume 35, p. 20744–20757, 2022. doi: 10.52202/068431-1508. URLhttps://doi.org/10.52202/ 068431-1508. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Syner- gizing reasoning and acting in language models. In International Conference on Learning Representations, 2023. URL https://openreview.net/forum?id=WE_vluYUL-X. Haozhen Zhang, Quanyu Long, Jianzhu Bao, Tao Feng, Weizhi Zhang, Haodong Yue, and Wenya Wang. MemSkill: Learning and evolving memory skills for self-evolving agents. arXiv preprint arXiv:2602.02474, 2026a. doi: 10.48550/arXiv.2602.02474. URL https://arxiv.org/abs/2602.02474. Shengtao Zhang, Jiaqian Wang, Ruiwen Zhou, Junwei Liao, Yuchen Feng, Zhuo Li, Yujie Zheng, Weinan Zhang, Ying Wen, Zhiyu Li, Feiyu Xiong, Yutao Qi, Bo Tang, and Muning Wen. MemRL: Self-evolving agents via runtime reinforcement learning on episodic memory. arXiv preprint arXiv:2601.03192, 2026b. doi: 10.48550/arXiv.2601.03192. URL https://arxiv.org/abs/2601.03192. 11 Preprint ASHARED EXPERIMENTAL PROTOCOL Interaction and termination.Full rollouts for the history interventions and calibration comparisons allow at most 30 actions and terminate upon success, terminal failure, or exhaustion of this budget. Continuations restore the environment toL/3and receive 30 additional actions. Success is determined by the environment’s finalwonor equivalentsuccessflag, not intermediate progress scores. Full- rollout Experience uses separate calibration and action calls, each capped at 8,192 output tokens. BFIXED-HISTORY AUDITS B.1NEXT-ACTION REPETITION ON FIXED HISTORIES Exact repetition can be a valid part of a task. To focus on reuse despite contrary feedback, this probe reuses a fixed cohort of 1,815 histories ending in two identical actions, selected by screening for feedback that does not support another execution of the same action (judged by GPT-5.6-Sol). The source pool consists of 500 natural rollouts per model–environment cell. We select at most one pair from each trajectory: its earliest nonterminal repeat pair. The same actor that produced a history receives it under each condition. This fixes the task, preceding actions, and environmental observations across the three conditions discussed in Sections 5.1 and 5.2. Three renderings of the same prefix. Raw is the vanilla interaction history. Causal preserves this sequence and labels each returned observation in place as the outcome of its preceding action. Experience first asks the source actor to review the Raw prefix in a separate calibration call. A generated experience is inserted beside the latest action–outcome transition before the next decision; NOUPDATE or an invalid update leaves the Raw input unchanged. Table 3: Next-action repetition on fixed histories. Entries show repeated-action counts with rates in parentheses. N is the shared number of histories for each condition. Figure 5 plots these rates. ModelNRawCausalExperience GPT-5.6-Sol464 (8.70%)3 (6.52%)2 (4.35%) DeepSeek-V4-Pro13133 (25.19%)30 (22.90%)25 (19.08%) Qwen3.5-Plus18848 (25.53%)41 (21.81%)36 (19.15%) Qwen3-32B49293 (18.90%)77 (15.65%)84 (17.07%) Qwen3-8B958492 (51.36%)434 (45.30%)241 (25.16%) Pooled1,815 670 (36.91%) 585 (32.23%) 388 (21.38%) CCOMPLETE DIAGNOSTIC SUCCESS RESULTS Table 4 combines the history interventions and explicit-calibration diagnostics from Sections 4 and 5. All seven conditions use the same 500 tasks per model–environment cell and share the Raw baseline. 12 Preprint Table 4: Complete diagnostic success counts, out of 500 tasks per model–environment cell. Totals aggregate 10,000 episodes per condition. “Obs.-Only” denotes Observation-Only; “Exp.” denotes Experience. ModelEnvironmentRaw Recent-3- Only Obs.- Only Shuffle- All Shuffle- OldCausalExp. GPT-5.6-Sol ALFWorld448421456456453449441 ScienceWorld403338384364400410394 WebShop188171143169186190176 AppWorld427274394426430437421 DeepSeek-V4-Pro ALFWorld350280318327349357346 ScienceWorld293235267255285297282 WebShop185168109146180187174 AppWorld433245426425433435431 Qwen3.5-Plus ALFWorld327255349374362355358 ScienceWorld169134136139169195171 WebShop163177110161166176169 AppWorld307177227286299313288 Qwen3-32B ALFWorld308196223270302314269 ScienceWorld756473747510186 WebShop92987891939880 AppWorld10610172109126111101 Qwen3-8B ALFWorld244159201189236244183 ScienceWorld38263739355048 WebShop65687769728466 AppWorld18121315242217 Total successes / 10,0004,6393,5994,0934,3844,6754,8254,501 Pooled success rate46.39%35.99% 40.93% 43.84% 46.75% 48.25% 45.01% 13 Preprint DCAUSAL TRANSITION AND EXPERIENCE DIAGNOSTICS D.1EXPLICIT HISTORY RECHECKING Check retains the Raw history format and adds the following reminder to the actor prompt: [History caution] Earlier actions, reasoning, plans, and interpretations in the interaction history may be wrong, incomplete, or stale. Re-check them against the task goal, constraints, and recorded environment or tool observations before choosing the next action. Repetition is not evidence of correctness. [/History caution] Table 5 reports the full model–environment results. Overall, Check improves on Raw and achieves a similar success rate to Causal Transition. Table 5: Task success with an explicit history-check reminder. Entries are successful-task counts out of 500 per model–environment cell. Overall totals pool 10,000 tasks per condition; the final row gives the corresponding success rates. Causal denotes Causal Transition. ModelEnvironmentRawCausalCheck GPT-5.6-Sol ALFWorld448449467 ScienceWorld403410410 WebShop188190195 AppWorld427437417 DeepSeek-V4-Pro ALFWorld350357359 ScienceWorld293297305 WebShop185187190 AppWorld433435425 Qwen3.5-Plus ALFWorld327355362 ScienceWorld169195200 WebShop163176179 AppWorld307313290 Qwen3-32B ALFWorld308314315 ScienceWorld75101107 WebShop929895 AppWorld106111117 Qwen3-8B ALFWorld244244267 ScienceWorld385052 WebShop658485 AppWorld182223 Overall successes4,6394,8254,860 Overall success (%)46.3948.2548.60 D.2RETAINING EXPERIENCE IN LATER PROMPTS After each nonterminal transition, a separate call to the same actor model receives the environment, task, preceding raw history, latest action, and returned observation. Earlier generated experiences, calibration reasoning, rewards, success labels, and future actions are excluded. Its final JSON is one of "decision":"NOUPDATE","experience":null "decision":"CALIBRATE","experience":"...". 14 Preprint The prompt asks for a new task-relevant correction or durable progress fact that should affect later decisions. There is no calibration before the initial action; terminal transitions are recorded and skipped. Each generated experience is linked to its source transition and rendered beside it in every subsequent action prompt. D.3EXPERIENCE GENERATION RATES We measure how often actors write an experience in an additional set of full rollouts under the same Experience protocol, with 500 tasks per model–environment cell. Table 6 reports the numbers of generated experiences andNOUPDATEresponses, together with their percentages among valid review responses. Table 6: Experience generation andNOUPDATEdecisions across five actors and four environments. Entries give counts with percentages of valid review responses in parentheses. Each cell covers 500 tasks; Overall pools responses across all 10,000 episodes. ModelEnvironment Valid reviewsExperienceNOUPDATE GPT-5.6-Sol ALFWorld5,8484,182 (71.51)1,666 (28.49) ScienceWorld7,0444,723 (67.05)2,321 (32.95) WebShop3,4531,886 (54.62)1,567 (45.38) AppWorld5,6365,482 (97.27)154 (2.73) DeepSeek-V4-Pro ALFWorld8,6051,980 (23.01)6,625 (76.99) ScienceWorld8,7572,471 (28.22)6,286 (71.78) WebShop3,5351,385 (39.18)2,150 (60.82) AppWorld4,8992,326 (47.48)2,573 (52.52) Qwen3.5-Plus ALFWorld8,117861 (10.61)7,256 (89.39) ScienceWorld8,9901,167 (12.98)7,823 (87.02) WebShop3,288876 (26.64)2,412 (73.36) AppWorld5,1682,182 (42.22)2,986 (57.78) Qwen3-32B ALFWorld10,2411,338 (13.07)8,903 (86.93) ScienceWorld8,8451,223 (13.83)7,622 (86.17) WebShop2,199643 (29.24)1,556 (70.76) AppWorld5,0292,744 (54.56)2,285 (45.44) Qwen3-8B ALFWorld11,5466,851 (59.34)4,695 (40.66) ScienceWorld10,5086,961 (66.24)3,547 (33.76) WebShop2,2492,155 (95.82)94 (4.18) AppWorld7,6346,053 (79.29)1,581 (20.71) Overall131,591 57,489 (43.69) 74,102 (56.31) D.4CANDIDATE EXPERIENCE AND NEXT-ACTION QUALITY This comparison examines whether a generated experience helps an actor choose a better next action from the same interaction history. GPT-5.6-Sol generates candidate experiences from real rollout contexts. For each context, each of the five recipient actors generates two next actions: one with the candidate experience and one without it. These actions form a within-actor pair for evaluation. Pairwise evaluation. When the parsed executable actions are identical, the pair is assigned Tie without judging; reasoning text need not match. Other pairs are evaluated by GPT-5.6-Sol in two blind comparisons with the A/B order reversed, at temperature 0. Win denotes better next-action quality with the experience; Loss denotes worse quality. These outcomes require agreement across both orders. Two Tie judgments or inconsistent judgments are recorded as Tie. 15 Preprint Effect on next-action quality. Across all completed comparisons, the paired judgments favor the action generated with an experience in 14.0% of cases and favor the Raw action in 13.0% (Figure 10). All five recipient actors show both outcomes. Under this evaluation, helpful guidance coexists with guidance that leads to worse next actions, even though all candidates are generated by GPT-5.6-Sol. Explicit experience generation therefore does not reliably yield an improvement in next-action quality. GPT-5.6 Sol DeepSeek V4 Pro Qwen3.5 Plus Qwen3-32BQwen3-8BOverall 0 5 10 15 20 7.6 13.6 10.8 18.1 19.9 14.0 8.0 12.4 12.4 15.5 16.9 13.0 Next-action comparisons (%) ImprovedDegraded Figure 10: LLM-judged changes in next-action quality with candidate experiences generated by GPT-5.6-Sol. Bars compare actions generated by each recipient actor with and without an experience on the same history. Rates use all completed comparisons, including ties; ties are omitted from the bars. D.5GENERATED EXPERIENCES AND THEIR SUBSEQUENT USE Table 7 contrasts a redundant restatement, an unsupported claim, and an experience that calls for further verification, drawn from additional Experience rollouts. We verify that each experience appears in the next actor prompt alongside its source transition; steps are zero-based. Figure 6 presents two additional ALFWorld examples from GPT-5.6-Sol and Qwen3.5-Plus, contrasting a restated outcome with an unsupported operational rule. 16 Preprint Table 7: Examples of retained judgments and subsequent actions, with verbatim experience excerpts. CaseTask evidence and retained experienceSubsequent behavior Restatement ALFWorld Qwen3-8B The task requires cleaning an egg and placing it in a fridge. put egg 2 in bowl 1 returns “Nothing happens.” Experience: “Putting the egg in a bowl does not successfully place it in the fridge.” The action and identical experience recur at steps 11–13, with earlier judgments visible. Repeated writes restate an already recorded failure. Unsupported claim WebShop Qwen3.5- Plus A 3.4-oz, paraben-free shampoo below $40 is requested. The opened page shows a 3.5-oz product at $29.99 but does not verify the paraben-free attribute. Experience: “The selected 3.5 oz dry shampoo is paraben-free and priced under $40, confirming that product detail pages provide the necessary attribute verification to satisfy the user’s specific constraints.” The actor buys the product next without inspecting features or description. The judgment overlooks the size mismatch and endorses an unverified attribute. Verification WebShop Qwen3.5- Plus A black, non-slip PU leather organizer below $80 is requested. The page verifies color, material, and a $45.99 price, but not non-slip functionality. Experience excerpt: “. . . though non-slip functionality requires further verification via features or description.” The actor selects black, then inspects features at step 3 and description at step 5. The experience notes that the non-slip property still needs verification. D.6REFERENCE-HISTORY RECOVERY FROM A SHARED STATE Shared restored state and history conditions. All five conditions use the 1,000 unsuccessful source trajectories selected in Section 3, with 50 sources per model–environment cell. Every continuation restores the environment to⌊L/3⌋and uses the source actor with a budget of 30 new actions after restoration. Raw supplies only the history through this point. Reference adds the recorded actions and observations between⌊L/3⌋and⌊2L/3⌋as reference information, without executing them in the restored environment. The three intervention conditions use the same reference segment, giving 1,000 continuations per condition and 5,000 in total. Separating reference history from actual interaction. The additional segment is placed in a delimited reference block. The prompt then distinguishes newly executed continuation history from that reference and supplies the actual current observation and available actions. At the first continuation step, the newly executed history isNone. The reference-augmented prompt has the following structure: Normal history through L/3 Additional reference history (not executed in the current environment; the actual environment was restored to L/3): Reference history from L/3 to 2L/3 End of additional reference history. Actual interaction history after restoring the environment to L/3: Actual continuation history; None at the first step Current observation: Actual current observation Available actions: Actual current actions 17 Preprint Feedback organization and calibration.Reference retains the ordinary observation–action format. Causal + Reference instead labels each returned observation as the outcome of its preceding action throughout the actual history, reference segment, and newly executed continuation. Reference records are explicitly labeled asReference ActionandOutcome of Reference Action; their boundaries remain separate from the actual interaction. Experience retains the ordinary history format, generates an experience, and keeps that experience available for subsequent decisions. Calibrator applies the frozen model and update rule from Section 6 to the reference-augmented continuation, retaining generated updates while the source actor continues to choose actions. EADDITIONAL EVALUATION OF THE CALIBRATOR E.1FULL TASK-SUCCESS RESULTS: 500 TASKS PER CELL Table 8 reports Raw, Causal Transition, and the trained calibrator with 500 tasks per cell. Entries report task-success rates (%); Figure 7 gives the model and environment marginals. Table 8: Method-evaluation success rates (%), with 500 tasks per cell. The three conditions each contain 10,000 episodes. ModelSettingALFWorld ScienceWorld WebShop AppWorld GPT-5.6-Sol Raw89.6080.6037.6085.40 Causal Transition89.8082.0038.0087.40 Calibrator94.2091.8042.4087.00 DeepSeek-V4-Pro Raw70.0058.6037.0086.60 Causal Transition71.4059.4037.4087.00 Calibrator75.0067.4043.4084.20 Qwen3.5-Plus Raw65.4033.8032.6061.40 Causal Transition71.0039.0035.2062.60 Calibrator80.4037.8038.6061.00 Qwen3-32B Raw61.6015.0018.4021.20 Causal Transition62.8020.2019.6022.20 Calibrator60.8022.0023.0023.20 Qwen3-8B Raw48.807.6013.003.60 Causal Transition48.8010.0016.804.40 Calibrator51.2013.2023.409.00 E.2ABLATING PAIRED NEXT-ACTION COMPARISONS The content-screening-only ablation uses GPT’s content screening of candidate experiences for unsupported claims and redundant restatements of observations to construct training targets. It omits the subsequent generation and comparison of next actions with and without experience for each actor. Table 9 compares the two trained calibrators on task success, with 500 tasks per model–environment cell. Pooled success increases from 49.61% with content screening alone to 51.45% with the full pipeline, with gains in every model-level and environment-level aggregate. 18 Preprint Table 9: Training-data ablation of paired next-action comparisons. Entries are successful-task counts out of 500 per model–environment cell. Content screening only retains GPT’s review of the experience text but omits the subsequent action comparisons; Full Calibrator uses both stages. The final two rows pool 10,000 tasks per condition. ModelTraining dataALFWorld ScienceWorld WebShop AppWorld GPT-5.6-Sol Content screening only452451204424 Full Calibrator471459212435 DeepSeek-V4-Pro Content screening only364312205417 Full Calibrator375337217421 Qwen3.5-Plus Content screening only397197182293 Full Calibrator402189193305 Qwen3-32B Content screening only284105107108 Full Calibrator304110115116 Qwen3-8B Content screening only2526711129 Full Calibrator2566611745 Overall Content screening only4,961 / 10,000 (49.61%) Full Calibrator5,145 / 10,000 (51.45%) FHISTORY AS SEPARATE USER AND ASSISTANT MESSAGES The main experiments present accumulated interactions in a single history block. Here we test the history interventions and same-state continuation using separate chat messages. Fixed instructions remain in the system message; historical observations and executed actions alternate as user and assistant messages. The final user message contains the current observation and any currently available actions. Historical available-action lists are not replayed. F.1FULL-ROLLOUT HISTORY INTERVENTIONS Rollouts allow at most 30 actions, with an explicit 8,192-token output limit per actor request. The seven conditions use the same 100 tasks in each model–environment cell, totaling 2,000 episodes per condition across five actors and four environments. The history interventions follow Section 4, applied to the corresponding user and assistant messages. Under Causal Transition, returned observations are labeled in their existing user messages; Experience uses the separate review calls and retains generated experiences as described in Appendix D.2. 19 Preprint Table 10: Native-chat success counts out of 100 per model–environment cell (2,000 episodes per condition). “Obs. Only” denotes Observation-Only; “Exp.” denotes Experience. ModelEnvironmentRaw Recent-3 Only Obs. Only Shuffle All Shuffle OldCausalExp. GPT-5.6-Sol ALFWorld95809393959590 ScienceWorld83637279798580 WebShop45413141394545 AppWorld81407380838482 DeepSeek-V4-Pro ALFWorld67596668616567 ScienceWorld44274845524651 WebShop34392426393131 AppWorld76447677757975 Qwen3.5-Plus ALFWorld85747981838581 ScienceWorld42223437364338 WebShop33362733383837 AppWorld52294245555850 Qwen3-32B ALFWorld55424951535752 ScienceWorld20151719172122 WebShop19211719202117 AppWorld19181112152017 Qwen3-8B ALFWorld48364242455045 ScienceWorld6235397 WebShop13151212121814 AppWorld32351155 Total successes / 2,000920705819870911955906 Pooled success rate46.00%35.25% 40.95% 43.50% 45.55% 47.75% 45.30% 20 Preprint The pooled results retain the main ordering: Causal improves over Raw, whereas Experience does not. Retaining earlier content remains useful, and Shuffle-Old stays close to Raw while outperforming Recent-3-Only. F.2REFERENCE-HISTORY CONTINUATION To test whether the recovery decline in Section 3 also occurs with this message format, we compare Raw and Reference on 50 paired unsuccessful sources per model–environment cell, giving 1,000 continuations per condition. Both conditions restore the environment to⌊L/3⌋and allow up to 30 new actions by the source actor. Raw supplies only the actual prefix. Reference adds the recorded segment from⌊L/3⌋to⌊2L/3⌋as a separate user message after that prefix, explicitly marked as not executed in the restored environment. A subsequent user message provides the actual current observation and available actions. Newly executed interactions continue as user and assistant messages. Reference lowers pooled recovery from 158/1,000 (15.8%) to 114/1,000 (11.4%) (Table 11). Recovery declines in all five model aggregates and all four environment aggregates. The decline therefore also appears when actual interaction history is represented as separate chat messages. Table 11: Recovery success counts with user–assistant history. Each model–environment cell contains 50 paired sources; the Total column is out of 200 per model. Overall counts are out of 250 per environment and 1,000 in total. ModelHistoryALFWorldScienceWorldWebShopAppWorldTotal GPT-5.6-Sol Raw25204554 Reference24174348 DeepSeek-V4-Pro Raw10129536 Reference695323 Qwen3.5-Plus Raw11129537 Reference584421 Qwen3-32B Raw1032419 Reference743014 Qwen3-8B Raw901212 Reference60208 Overall Raw65472521158 Reference48381810114 GCORE PROMPT INTERFACES This section records the exact text that defines the paper’s central history and Experience interventions. G.1HISTORY RENDERING The following abstract templates match the single history block used in the main evaluation. Raw preserves the ordinary observation–action sequence. Causal Transition changes only the label that assigns the returned observation to its preceding action. Experience writes an actor-generated review beside its source transition and retains it in later actor prompts; NOUPDATE inserts no text. Raw. [Observation t: ’O t’, Action t: ’At’] [Observation t+1: ’Ot+1’, Action t+1: ’At+1’] 21 Preprint Causal Transition. [Initial Observation t: ’O t’, Action t: ’At’] [Outcome of Action t: ’Ot+1’, Action t+1: ’At+1’] Experience. [Observation t: ’Ot’, Action t: ’At’] [Experience derived from Action t -> its returned observation: E t] [Observation t+1: ’O t+1’, Action t+1: ’At+1’] G.2EXPERIENCE REVIEW The full-rollout Experience condition and candidate-experience construction use the same review interface. The input contains the environment, task, raw history, latest action, and returned observation. Previously generated experiences are excluded. You are calibrating your own prior interaction history for the same task. Inspect the task, the raw action-observation history, and the latest completed action and returned observation. Return exactly one JSON object as the final answer. Use "decision":"CALIBRATE","experience":"..." only when the latest observation establishes a new task-relevant correction or durable progress fact that should change how the preceding history guides later decisions in this task. Otherwise return "decision":"NOUPDATE","experience":null. The experience must be one concise sentence grounded in the latest observation. State a general implication of the observed outcome, not a plan or a concrete next action. Do not output Markdown or commentary around the final JSON. The only valid outputs are"decision":"NOUPDATE","experience":nulland "decision":"CALIBRATE","experience":"...". G.3JUDGING AND JOINT GENERATION For each within-actor comparison, GPT-5.6-Sol receives the original actor prompt and two candidate actions without actor, method, arm, or experience identifiers. Each changed pair is judged twice with A/B order reversed; inconsistent results are ties. Identical executable actions are ties without a judge call. The joint Qwen3-8B actor emits exactly one calibration block followed immediately by the environ- ment action. Its calibration output is either<calibration>"decision":"NOUPDATE", "experience":null</calibration>or aCALIBRATEdecision with an experience string. The initial step usesNOUPDATE; later outputs review the immediately preceding com- pleted transition. 22

Instant research discovery