Paper deep dive
What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction
Jinnan Li, Zheren Fu, Yue Wang, Jinzhe Li, Yuan Wu, Yi Chang
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 9/9/2026, 5:13:23 AM
Summary
This paper investigates causal history effects in multi-turn LLM interactions, specifically how assistant-generated responses influence subsequent behavior. Using a sharded simulation framework (FULL vs. SHARDED), the authors analyze degradation across six task families and five models. Key findings include that multi-turn degradation is task- and model-dependent, and that neutralizing assistant history improves performance (+.027) beyond simple context shortening. 'Turn Surgery' interventions reveal that 63.7% of degraded trajectories contain at least one beneficial intervention point, and 48.4% of binary tasks admit a fail-to-success reversal. Internal state analysis on Qwen3-14B shows these effects are task-dependent rather than universal.
Entities (18)
Relation Signals (16)
LLMs → experiences → Multi-turn Interaction
confidence 95% · Multi-turn interaction creates a feedback process in which an LLM's previous responses become context for later behavior.
Summary → partof → Study
confidence 95% · The six task families are Actions (105), Code (100), Data-to-Text (120), Database (107), Math (103), and Summary (92).
Actions → partof → Study
confidence 95% · The six task families are Actions (105), Code (100), Data-to-Text (120), Database (107), Math (103), and Summary (92).
Database → partof → Study
confidence 95% · The six task families are Actions (105), Code (100), Data-to-Text (120), Database (107), Math (103), and Summary (92).
Data-to-Text → partof → Study
confidence 95% · The six task families are Actions (105), Code (100), Data-to-Text (120), Database (107), Math (103), and Summary (92).
Code → partof → Study
confidence 95% · The six task families are Actions (105), Code (100), Data-to-Text (120), Database (107), Math (103), and Summary (92).
Math → partof → Study
confidence 95% · The six task families are Actions (105), Code (100), Data-to-Text (120), Database (107), Math (103), and Summary (92).
Laban et al. → proposed →
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Multi-turn interaction creates a feedback process in which an LLM's previous responses become context for later behavior. Prior work shows substantial multi-turn degradation and that assistant-generated history can affect later behavior. However, it remains unclear how these effects manifest across models, tasks, turns, and inside a model. We study these gaps across six task families and five models. Degradation from fully specified single-turn input (FULL) to progressively revealed multi-turn interaction (SHARDED) is clearly task- and model-dependent, and stronger one-shot performance does not imply greater interaction robustness. We then retrospectively analyze completed SHARDED conversations by replaying the user messages already observed in each trajectory while editing only assistant-generated history. Replacing prior assistant responses with neutral content (termed neutralization) changes downstream min-max normalized performance by +.027 across 2,973 trajectories. On a prespecified length-controlled subset, short and length-matched neutralization yield nearly identical effects (+.069 versus +.068), showing that simple context shortening is insufficient to explain the effect of history editing. Turn Surgery further intervenes on one assistant turn at a time. Among 237 selected degraded trajectories, 63.7% contain at least one beneficial intervention, while most tested positions remain unchanged; for binary tasks, 48.4% admit a fail-to-success reversal. An open-weight case study links behaviorally consequential history changes to measurable downstream state differences, but finds task-dependent rather than universal internal signatures. Overall, assistant-generated history has active but selective effects on multi-turn performance, motivating selective rather than uniform history management.
Tags
Links
- Source: https://arxiv.org/abs/2609.05882v1
- Canonical: https://arxiv.org/abs/2609.05882v1
Trouble viewing inline? Open PDF directly →
Full Text
63,460 characters extracted from source content.
Expand or collapse full text
What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction Jinnan Li jnli23@mails.jlu.edu.cn School of Artificial Intelligence Jilin University Changchun, China Zheren Fu fzr@ustc.edu.cn School of Cyber Science and Technology, University of Science and Technology of China Hefei, China Yue Wang wangyue@email.unc.edu School of Information and Library Science, University of North Carolina at Chapel Hill Chapel Hill, North Carolina, USA Jinzhe Li jinzhe25@mails.jlu.edu.cn School of Artificial Intelligence Jilin University Changchun, China Yuan Wu ∗ yuanwu@jlu.edu.cn School of Artificial Intelligence Jilin University Changchun, China Yi Chang ∗ yichang@jlu.edu.cn School of Artificial Intelligence Jilin University Changchun, China Abstract Multi-turn interaction creates a feedback process in which an LLM’s previous responses become context for later behavior. Prior work shows substantial multi-turn degradation and that assistant- generated history can affect later behavior. However, it remains unclear how these effects manifest across models, tasks, turns, and inside a model. We study these gaps across six task families and five models. Degradation from fully specified single-turn input (FULL) to progressively revealed multi-turn interaction (SHARDED) is clearly task- and model-dependent, and stronger one-shot per- formance does not imply greater interaction robustness. We then retrospectively analyze completedSHARDEDconversations by re- playing the user messages already observed in each trajectory while editing only assistant-generated history. Replacing prior assistant responses with neutral content (termed neutralization) changes downstream min-max normalized performance by+.027 across 2,973 trajectories. On a prespecified length-controlled subset, short and length-matched neutralization yield nearly identical effects (+.069 versus+.068), showing that simple context shortening is insufficient to explain the effect of history editing. Turn Surgery fur- ther intervenes on one assistant turn at a time. Among 237 selected degraded trajectories, 63.7% contain at least one beneficial inter- vention, while most tested positions remain unchanged; for binary tasks, 48.4% admit a fail-to-success reversal. An open-weight case study links behaviorally consequential history changes to measur- able downstream state differences, but finds task-dependent rather than universal internal signatures. Overall, assistant-generated his- tory has active but selective effects on multi-turn performance, motivating selective rather than uniform history management. 1 Keywords large language models, multi-turn interaction, conversational reli- ability, counterfactual intervention, evaluation ∗ Corresponding authors. 1 Code and data are available at: https://github.com/jinnanli/llms-ate-their-words 1 Introduction Large language models (LLMs) are increasingly used as interactive assistants and agents. In task-driven settings, users provide require- ments over multiple turns, such as revising earlier constraints and continuing the task based on the model’s previous responses [7,34]. Unlike single-turn interaction, multi-turn interaction requires mod- els to repeatedly reason over an evolving context, creating new reliability challenges when earlier interaction states influence later decisions. We refer to this model-generated and cumulatively re- tained context as self-generated assistant history. This raises a natural question: Is what a model writes into the conversation merely an interaction record, or can it actively influence subsequent task performance? Figure 1 illustrates this idea: assistant responses are not only the outputs in the current turn, but also become future input context to influence later behavior. Prior work introduces the multi-turn degradation as a substantial reliability problem. Laban et al. [15]proposed sharded simulation, which compares presenting a complete instruction at once (FULL) with progressively revealing the same information across user– assistant turns (SHARDED). They show thatSHARDEDinteraction can substantially reduce performance and produce behavioral failures such as premature answers, unsupported assumptions, reliance on previous responses, and loss of intermediate information. These findings do not isolate whether the model’s own previous responses contribute to such failures. Some other work suggests that a model’s own previous responses can become harmful context and bias later behavior [11,17,36]. However, it remains unclear how these history effects vary across tasks and models, which individual assistant turns are consequential, and whether they are accompanied by measurable changes in internal model states. We address these gaps through retrospective analysis of observed conversations. Starting from completed conversations, we replay the same user messages while modifying the assistant history. This allows us to test how previous assistant responses affect later model behavior. We then compare these effects across tasks and models, identify which assistant turns matter most, and examine whether they are reflected in the internal states of an open-weight model. Specifically, (RQ1) we characterize how multi-turn degradation varies across tasks and models. (RQ2) we test whether modifying arXiv:2609.05882v1 [cs.CL] 5 Sep 2026 Li et al. (a) Assistant responses become future context (b) Performance often degrades from FULL to SHARDED Does self-generated assistant history contribute to multi-turn degradation? Figure 1: Motivation: self-generated assistant history as the future context. (a) Assistant responses become part of later context. (b) Performance often degrades fromFULLtoSHARDED, motivating whether this self-generated history contributes to multi-turn degradation. assistant-generated history changes outcomes beyond the effect of context shortening. (RQ3) we introduce Turn Surgery, which mod- ifies one assistant turn at a time and replays recorded user messages to identify influential history turns. (RQ4) we examine whether behaviorally consequential history interventions have measurable internal-state signatures. Figure 2 summarizes these research ques- tions and their corresponding analyses. Our main contributions and findings are as follows: •Structured multi-turn robustness landscape. Across six task families and five models,FULL–SHARDEDdegradation exhibits a task×model structure, and one-shot rankings do not predict interaction robustness. • History effects persist under length control. Across 2,973 paired trajectories, short neutralization produces an aggregate effect of+.027. On a prespecified length-controlled subset, short and length-matched neutralization yield nearly identical effects (+.069 versus+.068), showing that simple context shortening is insufficient to explain the history effect. •History effects are turn-selective and distributed across trajectories. Single-turn counterfactual replay shows that 63.7% of selected degraded trajectories contain at least one beneficial surgery, while 57.6% of tested positions leave the final outcome unchanged. These differences across turns motivate future work on selective, turn-level history man- agement. Figure 2: Overview of our research work, which covers (1) degradation landscape, (2) assistant-generated history effects, (3) influential turns, and (4) internal model states. •History effects have measurable but task-dependent internal signatures. Open-weight analysis associates in- terventions that change behavior with measurable down- stream state divergence. Complementary analyses show that these internal effects vary across tasks. 2 Related Work 2.1 Multi-turn LLM Evaluation and Underspecified Interaction LLM evaluation has increasingly extended from single-turn prompts to interactive settings. MT-Bench [37], MINT [32], and subsequent benchmarks such as MT-Eval [13], MT-Bench-101 [1], and Multi- Challenge [9] extend evaluation toward multi-turn capability, tool use, refinement, and consistency across interaction histories. Re- lated work on clarification and underspecified interaction further shows that models may answer prematurely or make unsupported assumptions when user intent is incomplete [10, 20]. The most direct foundation for our study is the sharded simu- lation framework of Laban et al. [15]. By progressively revealing shards of the same fully specified task, the framework enables controlled comparison betweenFULLandSHARDEDinteraction and demonstrates substantial multi-turn degradation, together with behavioral signatures such as premature answer attempts, unsup- ported assumptions, and reliance on previous responses. Recent follow-up work has explored mitigating this degradation through curriculum reinforcement learning, adaptive context resetting, and explicit intent mediation [12,16,18]. These studies show that multi- turn degradation can be reduced by changing training or context handling. We use the same controlled setting to ask a comple- mentary retrospective question: given the user messages already observed in a completed conversation, how much of subsequent be- havior is attributable to the model’s accumulated assistant history, and do consequential history changes leave measurable internal signatures? What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction 2.2 Context and Memory in Interactive Foundation Models Long-context and conversational-memory research shows that re- taining more context does not guarantee that models use it effec- tively. Studies of long-context utilization identify strong sensitivity to information position and relevance [19], while memory sys- tems such as MemGPT [23], LongMemEval [33], and LoCoMo [22] study how models retain and organize information across longer interactions. In agentic systems, the context additionally contains model-generated plans, reflections, tool interpretations, and inter- mediate responses [29]. Future behavior therefore depends on both externally supplied information and internally generated history. However, existing studies often evaluate context retention or mem- ory organization at the whole-history level, leaving open how indi- vidual assistant-generated states within a trajectory contribute to downstream outcomes. Related analyses likewise show that earlier model outputs and conversational roles can systematically shape later behavior [24, 31]. Huang et al. [11]study assistant-side history directly by compar- ing full-context prompting with configurations that omit, summa- rize, or otherwise reduce previous assistant responses. They find that assistant history can be unnecessary or harmful and further evaluate history omission with length controls on sharded GSM8K examples derived from Laban et al. [15]. Other work reduces harm- ful carry-over across turns through history-cleaned or canonical- context supervision [17,36], sequential history condensation, or alignment to stronger single-turn anchors [5, 30]. These mitigation-oriented studies and ours are complementary in objective and granularity. Existing methods show that managing assistant history can improve multi-turn behavior, but mainly oper- ate through whole-history omission or transformation, or through training-level policies. We ask which assistant turns in observed conversations are causally consequential. Whole-history neutraliza- tion tests history effects beyond surface length, and Turn Surgery then neutralizes one intermediate assistant turn at a time while re- playing the recorded user messages, yielding position-specific sensi- tivity profiles across tasks and models. These profiles are diagnostic rather than a mitigation method, but can provide finer-grained evi- dence for future selective history management that distinguishes turns worth retaining from those that interfere with later behavior. 2.3 Behavioral and Counterfactual Analysis of Interaction Trajectories A growing line of work analyzes failures at the trajectory level rather than only through terminal task success. MAST [3] and TRAIL [8] develop structured taxonomies and annotated traces for diagnosing failures in multi-agent and agentic workflows, while AgentDebug [38] identifies critical errors and uses targeted feed- back to rerun failed trajectories. These approaches highlight that the step at which a failure becomes visible need not be the step that most strongly determines the final outcome. More recent work moves from observational diagnosis toward intervention-based attribution. DoVer [21] validates failure hy- potheses by modifying candidate states and rerunning downstream execution. CausalFlow [2] uses step-level counterfactual interven- tions to identify failure-inducing steps and construct minimal re- pairs, while Causal Agent Replay [28] estimates how interven- tions on individual agent steps shift the final outcome distribu- tion. Turn Surgery shares this intervene-and-replay principle but studies a different object: ordinary assistant turns in controlled multi-turn conversations rather than heterogeneous agent execu- tion traces. Starting from a completed conversation, it changes one self-generated response and replays the downstream user messages already observed in that trajectory. This estimates position-specific downstream outcome sensitivity conditional on the recorded user- message history and reveals whether consequential positions are concentrated, distributed, or explained by simple recency. 3 Method: Evaluation and Intervention Framework We first characterize multi-turn degradation usingFULLandSHARDED, then use retrospective replay of completedSHARDEDtrajectories to test whether assistant-generated history affects final outcomes, in- cluding a length-matched control. Turn Surgery applies the same replay logic to one assistant turn at a time to localize turn-level history effects, and RQ4 analyzes internal signatures associated with these interventions in an open-weight model. Throughout, causal and counterfactual history effects denote intervention-defined changes conditional on the user messages already observed in the original trajectory and exclude effects mediated through later user adaptation. 3.1 Sharded Multi-Turn Evaluation Foundation We build on the sharded multi-turn evaluation framework intro- duced by Laban et al. [15]. The benchmark contains 627 fully speci- fied instructions across six task families. For each instruction, we compare two ways of presenting the same task information.FULLprovides the complete instruction in one user message, whereasSHARDEDreveals the information across multiple turns while retaining earlier assistant responses as context. We write aSHARDEDtrajectory as휏= (푈 1 ,퐴 1 , . . .,푈 푇 ,퐴 푇 ), where 푈 푡 and 퐴 푡 are the user and assistant messages at turn 푡 . The six task families are Actions (105), Code (100), Data-to-Text (120), Database (107), Math (103), and Summary (92). They are derived respectively from BFCL-V3 Parallel, HumanEval, ToTTo, Spider, GSM8K, and Summary of a Haystack [4, 6, 14, 25, 26, 35]. In the originalSHARDEDinteraction, a user simulator selects and rephrases the next unrevealed shard based on the current conversa- tion state. Our interventions are retrospective: after a trajectory is generated, we replay its observed user messages rather than rerun- ning the simulator. This isolates changes due to assistant history during replay rather than constraining the original user path. Ef- fects therefore condition on the observed user-message sequence and exclude later user adaptation. All conditions and interventions are scored on the final assistant output using the benchmark’s original task-specific evaluators: function-call correctness for Actions, test-based pass@1 for Code, SacreBLEU/100 for Data-to-Text [27], SQL execution match for Database, normalized exact match for Math, and coverage and citation score for Summary. Li et al. 3.2 Model Panel and Statistical Evaluation We evaluate GPT-5.6 Luna, Grok 4.5, GPT-4o, DeepSeek V3.2, and Qwen3-14B. All primary generations and replay experiments use temperature=0 and a maximum output length of 1,000 tokens. Qwen3-14B additionally serves as the open-weight model for the internal analysis in RQ4. The 627 instructions and five models yield 3,135 instruction– model pairs; 2,989 have valid scores under bothFULLandSHARDED and enter RQ1; replay analyses use the trace-specific subsets defined below. Incomplete requests are retained as missing observations rather than converted into failures. Let푆 푐 푖 denote the normalized task-specific evaluator score for case 푖 under condition 푐. To prevent larger tasks from dominating, model-level summaries macro-average task means with equal weight. Our degradation measure is푆 FULL − 푆 SHARDED ; we report model×task estimates in addition to model-level summaries. Uncertainty is estimated with 10,000 paired percentile-bootstrap resamples, resampling instruc- tions within task for task-macro statistics. 3.3 Neutralizing Self-Generated Assistant History To test whether assistant-generated history affects the final out- come, we retrospectively replay the user messages from each ob- servedSHARDEDtrajectory while replacing every non-final assistant response with the fixed neutral placeholder푁= Acknowledged. This short neutralization removes task-specific content while pre- serving turn structure; it is a causal diagnostic, not a correction or simulated better response. The same model regenerates the final response, which is scored with the original task-specific evaluator. For trajectory푖, the history-intervention effect is푆 SHORT 푖 −푆 SHARDED 푖 . Because it also shortens history, Section 3.4 tests that alternative explanation. Among 3,033SHARDEDtrajectories with readable traces and valid scores, 2,973 yield paired short-neutralization outcomes for RQ2. Our primary estimate averages paired effects within the 30 model ×task cells with equal cell weight, with uncertainty from 10,000 within-cell paired-bootstrap resamples. We additionally report a trajectory-weighted mean and, for binary tasks, fail-to-success and success-to-failure transitions. 3.4 Controlling for History Length Short neutralization changes both history content and length, leav- ing context shortening as a potential explanation. To control for this, length-matched neutralization repeats푁to match the surface length of each original non-final assistant response. Length is measured as the number of non-whitespace spans matched by the regular expression +; the same estimator and replacement rule are used for all models. All 1,250 positive-length replacements in the final controlled subset are matched exactly under this estimator; one additional replacement has zero original length. From trajectories with valid paired outcomes underSHARDEDand short neutralization, we prespecified 10 candidates per model× task cell (300 total) before observing any length-matched outcomes. Of these, 264 yield valid outcomes with an actual intermediate- history replacement and form the common paired subset in Figure 4. Within this subset, we compare short neutralization withSHARDED, length-matched neutralization withSHARDED, and length-matched with short neutralization. Overall, task-level, and model-level es- timates use model×task macro aggregation, paired comparisons, and 10,000 percentile-bootstrap resamples. 3.5 Turn Surgery: Turn-Level Counterfactual Replay Whole-history neutralization cannot identify which earlier assis- tant responses matter. Turn Surgery therefore intervenes on one intermediate assistant turn at a time to measure its downstream influence. Intervention protocol. For a target assistant turn퐴 푖푡 , preceding history is unchanged and퐴 푖푡 is replaced with the neutral place- holder푁. We replay the recorded downstream user messages while the same model autoregressively regenerates subsequent assistant responses. The final response is scored with the original evalua- tor; its difference from the originalSHARDEDoutcome measures the effect of intervening at that history position, conditional on the recorded user messages. Analysis set. Turn Surgery focuses on trajectories with clear SHARDEDdegradation. Of 3,135 instruction–model pairs, 3,033 have validSHARDEDtraces and 2,989 have completeFULLandSHARDED scores. For Actions, Code, Database, and Math, a trajectory is el- igible when푆 FULL =1 and푆 SHARDED =0; for Data-to-Text and Summary, eligibility requires푆 FULL − 푆 SHARDED ≥0.10. This yields 977 replay-eligible trajectories. Before observing surgery outcomes, we select up to eight per model×task cell using a fixed deterministic ordering; one cell has only five eligible cases, yielding 237 selected trajectories. Trajectory- level reach statistics therefore characterize this selected set rather than the full eligible population. Neutralizing each non-final assis- tant turn separately produces 1,472 planned branches, of which 1,440 yield valid outcomes and enter the analysis. Effects and inference. For branch푗of trajectory푖, the surgery effect isΔ 푖푗 = 푆 surgery 푖푗 − 푆 SHARDED 푖 . Primary classes use exact sign (휖=0): positive ifΔ 푖푗 >0, no-change ifΔ 푖푗 =0, and negative if Δ 푖푗 <0. For continuous-score tasks, we additionally assess robust- ness with휖= .01, treating|Δ 푖푗 | ≤ .01 as no-change and values outside this range by their sign; binary outcomes remain exact. Because trajectories contain different numbers of valid surgery branches, our primary summary uses a trajectory-equal estimand: Δ traj = 1 푛 traj 푛 traj ∑︁ 푖=1 1 푘 푖 푘 푖 ∑︁ 푗=1 Δ 푖푗 ! ,(1) where푘 푖 is the number of valid surgery branches for trajectory 푖and푛 traj is the number of trajectories included in the analysis. Thus, branch effects are first averaged within each trajectory and the resulting trajectory-level means are then weighted equally. We additionally report a branch-weighted mean as a robustness check. Turn-level outcome composition follows the same trajectory-equal principle: branch proportions are computed within each trajectory and then averaged equally across trajectories. Trajectory-level reach records whether a trajectory contains at least one positive surgery. What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction For binary tasks, a fail-to-success branch changes the final score from 0 underSHARDEDto 1 after surgery. We report the trajectory- level proportion containing at least one such reversal; this analysis covers 157 binary-task trajectories. We additionally characterize where positive surgery effects oc- cur and how many such positions appear within each trajectory. For positional analysis, relative position denotes the normalized location of the intervened assistant turn, while distance-to-final measures the number of subsequent user turns between that turn and the final user request. These branch effects form a turn-level sensitivity signal over historical positions. Inference uses 10,000 trajectory-clustered bootstrap resamples, each retaining all valid surgery branches from a sampled trajectory. We additionally assess repeatability on a frozen subset of 60 tra- jectories. For each, we select the intervention with the largest first- run effect magnitude and rerun it twice under the same temperature- 0 configuration. Exact outcome-class consistency requires all three runs to share the same positive, no-change, or negative class; effect- sign agreement is the proportion of the three run pairs in the same class, treating no-change separately. This stability analysis con- cerns selected high-magnitude interventions rather than all Turn Surgery branches. 3.6 Open-Weight Internal Analysis To examine how behaviorally consequential history interventions are reflected inside the model, we conduct an open-weight case study on Qwen3-14B. RQ4 asks whether positive-effect surgeries produce larger downstream state changes, whether restoring those states toward the original-history state can partially reverse the behavioral effect, and whether complementary probes show a con- sistent pattern. The primary internal analyses cover Actions, Code, and Data-to-Text, the three task families for which exact benchmark evaluation of newly generated outputs is locally available. Probe definitions were frozen on a 24-trajectory discovery set spanning these tasks; primary estimates use a disjoint held-out set of 118 trajectories and 603 surgery branches, including 46 trajectories with both positive and no-change interventions. Branch labels are re- computed from Qwen3-14B task-score deltas under the same Turn Surgery protocol; primary held-out probe contrasts compare posi- tive with no-change branches. A separate prospective Data-to-Text (D2T) population is used only for replication and is not pooled with the held-out set. State propagation. We first ask whether positive-effect surgeries produce larger changes in downstream model states. At each de- coder layer, we extract the residual stream, the representation passed between transformer blocks, at the final content token of each replayed downstream user message. These user messages are already part of the observed trajectory and are replayed un- changed while the assistant-history intervention varies. We call these positions downstream user anchors and the last one the final downstream anchor. Original-versus-surgery representation diver- gence is measured as one minus cosine similarity and summarized both at the final anchor and by a normalized trapezoidal area over all downstream anchors (propagation AUC). Because positive and no-change branches differ in task composition, we report pooled, equal-task-macro, and within-trajectory contrasts, with stricter po- sitional controls using nearest relative position or distance-to-final difference at most one. State restoration. We next ask whether these downstream state changes are directionally related to the behavioral effect. At the final downstream anchor, we perform residual-state patching that moves a surgery context back toward its corresponding original-history state. At each tested layer, the surgery-context residual vector is replaced once during prefill with the corresponding original-history vector, after which deterministic greedy generation proceeds nor- mally. These restoration patches measure how much of the surgery benefit is undone by restoring this downstream state. The held-out cohort uses one positive–no-change pair from each of 46 mixed trajectories; Figure 6B reports the 33 D2T pairs alongside 43 prospec- tive D2T pairs. D2T restoration is evaluated over predefined layer ranges 3–12 and 16–31; the cohorts are analyzed separately, and patching is treated as a bounded causal intervention rather than a mediation test. Additional internal probes. Finally, we ask whether simpler inter- nal signals provide a consistent signature of behavioral influence. We measure static target-turn attention and original-to-surgery redistribution of aggregate attention to prior assistant history. For attention knockout, we block the target assistant turn either only from final generated tokens (direct-only) or from all downstream query positions (full-downstream); their difference in absolute ef- fect is the propagation advantage. We also measure the continuation margin, the difference between length-normalized teacher-forced log-likelihoods of successful and failed continuations, and examine how it changes between original and surgery contexts. These probes are treated as complementary diagnostics of task dependence rather than evidence for a single universal mechanism. 4 Results We organize our analysis around four progressively focused re- search questions. RQ1 characterizes theFULL–SHARDEDdegrada- tion landscape across models and tasks. RQ2 tests whether inter- vening on self-generated assistant history changes downstream performance and uses a length-matched control to test whether the observed history effect can be explained by simple context shorten- ing. RQ3 localizes this history sensitivity to specific trajectories and intermediate turns through turn-level counterfactual intervention. RQ4 examines internal signatures of behaviorally consequential history interventions in an open-weight model. 4.1 RQ1: Multi-turn Degradation Forms a Structured Task× Model Landscape All five models degrade when moving from fully specified input to progressively sharded interaction, but the magnitude of this loss varies substantially across models and tasks. As shown in Figure 3A, the absoluteFULL–SHARDEDgaps are .137 for Luna, .164 for Grok, .193 for GPT-4o, .211 for DeepSeek, and .224 for Qwen. Notably, model rankings underFULLdo not match rankings byFULL–SHARDEDdegradation. Grok achieves the highestFULL score but does not have the smallest gap, whereas Luna does not lead underFULLyet exhibits the smallest absolute gap. Although Li et al. GPT-5.6 Luna Grok-4.5 GPT-4o DeepSeek-V3.2 Qwen3-14B 0.40 0.45 0.50 0.55 0.60 0.65 0.70 0.75 0.80 Task-macro score A Overall performance FULLSHARDED GPT-5.6 Luna Grok-4.5 GPT-4o DeepSeek-V3.2 Qwen3-14B Actions Database Data-to-Text Code Math Summary 0.300.270.200.380.38 0.360.360.330.440.31 0.090.180.170.120.17 0.06†0.03†0.180.160.27 -0.01†0.100.160.150.18 0.020.030.110.030.04 B FULL − SHARDED gap by task and model 0.0 0.1 0.2 0.3 0.4 Score gap Figure 3: Multi-turn degradation exhibits a structured task×model landscape. (A) Overall task-macro performance underFULL andSHARDED, showing the aggregate degradation from one-shot input to multi-turn interaction. (B) The correspondingFULL- SHARDEDgaps across tasks and models, revealing heterogeneous degradation patterns. Error bars denote 95% paired-bootstrap confidence intervals;† indicates intervals including zero. absolute gaps may still depend partly on starting performance, stronger one-shot performance therefore does not necessarily imply greater multi-turn robustness. Figure 3B further shows that degradation follows a structured task×model pattern. Actions and Database exhibit relatively large FULL–SHARDEDlosses across models. Data-to-Text shows moderate degradation with comparatively consistent behavior across mod- els, whereas Code and Math exhibit stronger model dependence. Summary is the most stable overall. Thus, multi-turn degradation is not a uniform property of a model but depends systematically on the task and model combination. Together, RQ1 shows that multi-turn robustness is distinct from one-shot capability and exhibits substantial task×model structure. BecauseSHARDEDinteraction also requires models to repeatedly condition on assistant responses they previously wrote into the conversation, RQ2 asks whether this self-generated history is itself a causal contributor to downstream performance. 4.2 RQ2: Self-Generated Assistant History Shapes Downstream Performance 4.2.1 Large-Scale History Intervention Changes Downstream Out- comes. Assistant-history neutralization changes downstream out- comes on average, showing that self-generated assistant responses are causally consequential under retrospective replay of the recorded user-message sequence. Among 3,033SHARDEDtrajectories with readable traces and valid benchmark scores, short neutralization yields 2,973 paired outcomes. Across this population, the equal-cell model×task macro score is .550 underSHARDEDand .577 under short neutralization, corresponding to an overall effect of+.027 with a 95% CI of[.013, .041]. Assigning equal weight to paired tra- jectories yields a similar effect of+.033 with a 95% CI of[.018, .048]. This average masks substantial heterogeneity across models and tasks. Luna, Grok, GPT-4o, and Qwen show positive effects under short neutralization, whereas DeepSeek exhibits a clear negative ef- fect of−.149[−.186,−.111]. At the task level, Actions shows a large positive effect of+.223[.181, .265], whereas Code shows a clear negative effect of−.133[−.180,−.087]. Database has a point esti- mate of+.036[−.007, .080], while the remaining tasks show smaller positive, near-zero, or mildly negative effects. Overall, these results show that assistant-generated history has measurable downstream effects, but the direction and magnitude of those effects vary sub- stantially across models and tasks. Because short neutralization changes both history content and length, we next test whether this history effect can instead be explained by shorter surface history. 4.2.2The History Effect Persists Under Length Control. The history effect persists when neutralized histories are matched to their orig- inal surface length, indicating that simple context shortening is insufficient to explain it. From trajectories paired underSHARDED and short neutralization, we prespecified 300 length-control candi- dates using model×task stratification. Of these, 264 yield valid out- comes with an actual intermediate-history replacement and form the common paired subset for all three conditions. As described in Section 3.4, every positive-length replacement is matched exactly under the shared whitespace-span estimator. On this common subset, Figure 4A shows effects of+.069[.024, .115] for short neutralization and+.068[.027, .110]for length-matched neutralization, both relative toSHARDED. The difference between the two neutralization conditions is only−.001[−.039, .035]. Because the full-population and controlled estimates use different popula- tions, the length-control inference rests on this paired 264-trajectory comparison. The nearly identical effects within this subset show that restoring surface history length does not remove the observed history effect. The controlled task-level results remain heterogeneous. Under length-matched neutralization, the effect is+.260[.160, .380]for Actions and+.140[.020, .280]for Database, whereas Data-to-Text shows a small negative effect of−.022[−.061, .017]. Together, the What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction 0.000.020.040.060.080.100.12 Effect vs SHARDED SHORT MATCHED +0.069 +0.068 MATCHED − SHORT -0.001 [-0.039, +0.035] Common subset, n=264 A Length control 0%20%40%60%80%100% Conditional percentage within baseline state SHARDED failure (n=595) SHARDED success (n=1329) 50.9% 303/595 49.1% 292/595 14.7% 85.3% 1134/1329 B Binary transitions (conditional) green: rescuedred: harmedgray: unchanged Figure 4: History effects persist under surface-length match- ing. (A) Effects of short and length-matched neutralization relative toSHARDEDon the common subset of 264 trajecto- ries. (B) Conditional binary outcome transitions under short neutralization among trajectories that initially succeed or fail underSHARDED. Error bars denote 95% paired-bootstrap confidence intervals. full-population and length-controlled analyses show that the con- tent of self-generated assistant history affects downstream behavior beyond what can be explained by surface history length alone. 4.2.3History Intervention Produces Heterogeneous Outcome Transi- tions. The positive aggregate effect also masks bidirectional trajectory- level changes. Figure 4B examines binary outcomes under short neutralization across the retrospective replay population. Among 1,924 binary-task trajectories, 303 of 595 baseline failures are res- cued (50.9%), whereas 195 of 1,329 baseline successes are harmed (14.7%). The same uniform history intervention can therefore improve some trajectories while degrading others. This trajectory-level het- erogeneity motivates a more selective question: which specific his- tory positions help or hurt downstream behavior? RQ3 addresses this by intervening on individual assistant turns. 4.3 RQ3: Turn Surgery Reveals Broad but Structured History Sensitivity Turn Surgery reveals that history sensitivity is broad across de- graded trajectories but selective at the level of individual assistant turns. As described in Section 3.5, from 977 replay-eligible degraded trajectories, we select 237 model×task-stratified trajectories be- fore observing surgery outcomes and neutralize each intermediate assistant turn separately. All trajectory-level proportions below characterize this selected analysis set. Figure 5 summarizes the average effects, trajectory-level reach, turn-level selectivity, and positional structure of these interventions. 4.3.1Average Effect and Trajectory Reach. Within the selected de- graded set, Turn Surgery has a positive average effect, and positive interventions occur across many trajectories rather than being driven by a small number of cases. Using the trajectory-equal es- timand, Figure 5A shows a mean surgery effect of+.148, with a 95% CI of[.116, .180]relative to the originalSHARDEDtrajectory. Assigning equal weight directly to all surgery branches yields a branch-weighted effect of+.110 with a 95% CI of[.077, .151], show- ing that the positive average is robust to aggregation choice. Figure 5B shows that 151 of 237 selected degraded trajectories (63.7%) contain at least one positive surgery. Among these 151 trajectories, 100 (66.2%) contain multiple positive surgery positions, indicating that sensitivity is often distributed across several history locations rather than concentrated in a unique pivot turn. This pattern is robust to an휖= .01 practical tolerance for continuous- score tasks: 148 of 237 trajectories remain any-positive, and 95 of these 148 contain multiple positive positions. For binary outcomes, 76 of 157 trajectories (48.4%) contain at least one 0→1 fail-to-success surgery. Thus, in nearly half of the selected binary trajectories, intervening on a single intermediate assistant turn can produce a counterfactual branch that reverses the final outcome. 4.3.2 Turn-Level Selectivity and Position. Broad trajectory-level reach coexists with strong turn-level selectivity. Under trajectory- equal weighting, 31.5% of tested positions produce a positive effect, 57.6% produce no change, and 10.8% produce a negative effect. A positive surgery means that neutralizing the original turn improves the downstream score, whereas a negative surgery means that neutralization degrades it. Thus, historical assistant turns are not interchangeable: many produce no change under this interven- tion, neutralizing some improves downstream performance, while neutralizing others harms it. Figure 5C shows how positive and negative effects vary across normalized assistant-turn positions. Relative position has only a weak negative association with surgery effect (휌= −.139). Fig- ure 5D further shows that distance to the final user message is nearly unrelated to surgery effect (휌=−.005). History sensitivity therefore has positional structure but is not reducible to a simple recency rule, so proximity to the final request alone cannot identify the turns that matter. 4.3.3Stability of Selected Surgery Effects. Selected high-magnitude interventions are reasonably, but not perfectly, repeatable. On the frozen 60-trajectory stability subset, 43 of 60 interventions retain the same effect class across all three runs, yielding an exact outcome- class consistency of 71.7%. Across the 180 within-intervention run pairs, 146 share the same effect class, yielding an effect-sign agree- ment of 81.1%. Thus, these selected interventions exhibit some run-to-run variation even under the recorded temperature-0 config- uration, while their dominant effect direction remains reasonably stable. Taken together, RQ3 shows that history sensitivity is broad at the trajectory level but uneven and bidirectional across individual turns, without following a simple recency rule. These findings argue against treating all assistant history uniformly. Turn Surgery also provides turn-level sensitivity labels that could support future methods for selectively retaining, downweighting, or discarding prior assistant responses. Li et al. Trajectory (equal) Branch (weighted) 0.08 0.10 0.12 0.14 0.16 0.18 Final-score effect (Surgery − SHARDED) +0.148 +0.110 A Average effect Any positive (151/237) Any 0→1 (76/157) 0.40 0.45 0.50 0.55 0.60 0.65 0.70 Trajectory proportion 63.7% 48.4% B Trajectory reach 0.00.20.40.60.81.0 Normalized assistant-turn position 0.00 0.05 0.10 0.15 0.20 0.25 0.30 0.35 0.40 Branch fraction 1,440 valid branches C Relative-position profile PositiveNegative 123456–89–1213+ Distance to final (user turns) 0.000 0.025 0.050 0.075 0.100 0.125 0.150 0.175 0.200 Mean surgery effect Spearman ρ = -0.005 95% CI [-0.126, +0.150] 57/237 positive ≥6 user turns before final D Distance to final Figure 5: Turn Surgery reveals broad but structured history sensitivity within the selected degraded trajectories. (A) Average surgery effects under two aggregation schemes. (B) Trajectory-level reach of positive interventions and fail-to-success reversals. (C) Turn-level positive and negative effects across normalized assistant-turn positions, where 0 and 1 correspond to the earliest and latest intermediate assistant turns within each trajectory, respectively. (D) Surgery effects across distance-to-final bins. 4.4 RQ4: Internal Signatures Accompany Behavioral History Sensitivity RQ1–RQ3 establish causal history effects that are selective across individual turns. RQ4 asks what changes inside the model when such a turn matters. We find larger downstream state changes for influential interventions, while state-restoration effects and other internal signatures vary across tasks and probes, with no single signature recurring consistently across tasks. Overall, the internal consequences of history sensitivity are measurable but task dependent. Figure 6 summarizes the held-out and prospective Qwen3-14B analyses. 4.4.1Positive-Effect Interventions Produce Larger Downstream State Changes. Controlled comparisons show larger downstream residual- state divergence for positive than for no-change surgeries. The pooled AUC contrast is reversed at−.00430 because the groups differ in task composition, but task-balanced and within-trajectory comparisons recover the opposite direction (Figure 6A). The task- macro AUC contrast across Actions, Code, and D2T is+.00547 with a 95% CI of[.00069, .01390], and the within-trajectory com- parison over 193 matched pairs from 46 trajectories yields+.00337 [.00116, .00565]. The pattern remains under stricter positional con- trols: nearest-position matching yields+.00271[.00056, .00497], while matching with distance-to-final difference≤1 yields+.00641 [.00344, .00996]. A separate prospective Data-to-Text analysis like- wise shows greater divergence at the final downstream anchor, with a contrast of+.00123 [.00079, .00167]. 4.4.2State Restoration Provides a Bounded Causal Follow-Up. Residual- state patching tests whether restoring a surgery-induced down- stream state toward its original-history value undoes part of the surgery benefit (Figure 6B). For D2T, the held-out and prospective cohorts contain 33 and 43 pairs, respectively. Across 40 layers, their restoration profiles correlate at휌= .456, with 29 layers sharing the same sign. However, prospective confidence intervals within the predefined regions cross zero. Patching is therefore consistent with a D2T-specific directional link between downstream residual state and behavior, but does not establish mediation or a task-general restoration mechanism. 4.4.3No Single Internal Probe Is Consistent Across Tasks. Figure 6C shows that the held-out probes do not yield a single signature across tasks. Static target-turn attention does not distinguish positive from no-change interventions. Assistant-history attention redistribution is significant only in Actions, while state-propagation AUC is sig- nificantly positive only in Code; among the held-out knockout probes, only full-downstream intervention in Actions is significant. In the prospective D2T analysis, attention reallocation is approxi- mately null, whereas direct-only and full-downstream knockouts show sensitivity; the propagation advantage remains uncertain. Continuation-margin shifts depend on the matching scheme and likewise do not provide a universal predictor. Together, the internal signatures vary across tasks, probes, and evaluation cohorts rather than defining a task-general predictor. 5 Discussion Our results distinguish interaction robustness from one-shot capa- bility and identify self-generated assistant history as one source of this distinction. Model rankings underFULLdo not align with FULL–SHARDEDdegradation, so stronger isolated-task performance does not imply greater robustness to progressively revealed in- formation. Evaluation of assistants and agents should therefore complement fully specified benchmarks with controlled multi-turn evaluation [9, 13, 15]. Retrospective neutralization shows that this self-authored con- text has measurable downstream consequences even after restor- ing surface history length, with effects varying across models and tasks. Assistant-generated history is therefore not merely a tran- script of prior interaction: earlier model outputs become future inputs that can support or interfere with subsequent behavior. This view is consistent with recent evidence on context pollution, self- contamination, and conversational inertia [11, 31, 36]. What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction 0510 Positive − no-change divergence (×10³) Task-macro final Task-macro AUC Within traj. final Within traj. AUC A Controlled state divergence Task-macro (118 trajectories) Within-trajectory (46 traj. / 193 pairs) 05101520253035 Layer −10 −5 0 5 10 D2T restoration effect (Δ score ×10³) B State restoration Held-out (33 pairs) Prospective (43 pairs) ActionsCodeD2T Static target attention History attention shift State propagation AUC Direct-only Full- downstream ○ −○ ○+○ ○ +○ C Held-out task probes significant positivesignificant negativeCI includes zero Figure 6: Internal changes accompany positive-effect history interventions, but their signatures are task dependent rather than universal. (A) Controlled state divergence between positive and no-change interventions. (B) State restoration profiles under held-out and prospective evaluations. (C) Held-out task-level outcomes across complementary probes for Actions, Code, and D2T. Symbols in panel C denote significant positive/negative effects or confidence intervals including zero. Turn Surgery further shows why history management should be selective rather than uniform. Neutralizing individual assistant turns produces positive, no-change, and negative downstream ef- fects distributed across positions and not reducible to simple re- cency. These intervention-derived signals could supervise future selectors that decide which earlier responses to retain, downweight, or discard. Because uniform removal would discard helpful turns alongside harmful ones, Turn Surgery connects causal diagnosis to an actionable systems direction: identifying which pieces of self-generated context should continue to influence later computa- tion [2, 21, 28]. The open-weight analysis links behaviorally consequential his- tory interventions to downstream representational differences, while state restoration provides bounded, D2T-specific causal evidence rather than a bidirectional or task-general recovery mechanism. Attention redistribution, state propagation, and restoration like- wise remain task-dependent, supporting structured internal effects without implying a single universal mechanism. Our study remains diagnostic but points to a concrete engineer- ing path. Turn Surgery neither trains nor validates a history selector, and its signals are not context-independent labels of turn quality; instead, they motivate selective history management from causal turn-level feedback whose value may depend on the surround- ing trajectory, task, and model. More broadly, reliable interactive models must account for the context they help create and later consume [7, 29, 34]. 5.1 Limitations Our study has four main limitations. First, replay interventions ret- rospectively alter assistant history within an observed conversation while preserving the user messages already present in that trajec- tory. The resulting effects are therefore conditional on the observed user-message history and do not estimate the full adaptive interac- tion process: had the assistant originally responded differently, a real user might also have changed subsequent messages. Second, length-matched neutralization uses a shared whitespace- span estimator rather than exact model-specific tokenization, and repeated neutral filler introduces a highly regular replacement pat- tern. This control therefore rules out simple surface shortening rather than isolating every semantic or structural property of as- sistant history. The stability analysis also evaluates one selected intervention per trajectory rather than population-level repeatabil- ity across all Turn Surgery branches. Third, Turn Surgery effects are obtained only after counterfac- tual intervention and downstream evaluation. They therefore pro- vide retrospective turn-level causal signals, not an online mech- anism for predicting which prior responses should be retained, downweighted, or removed. Learning and validating such a history- management policy remains future work. Fourth, Turn Surgery is evaluated on a selected set of degraded trajectories from six task families and five models in a single bench- mark. Its prevalence in the broader trajectory population and gener- alization to open-ended, longer-horizon agent interactions remain to be established. The internal analysis is additionally limited to Qwen3-14B, so whether similar signatures emerge across model families remains an open question. 6 Conclusion We study self-generated assistant history as a source of multi-turn reliability effects through cross-task evaluation, whole-history in- tervention, Turn Surgery, and open-weight internal analysis. Multi- turn degradation exhibits a clear task×model structure, and retro- spectively altering assistant history within observed conversations changes downstream outcomes even after controlling for surface history length. Turn Surgery further shows that this sensitivity is broad across selected degraded trajectories but selective across individual history positions. Open-weight analysis associates behav- iorally consequential interventions with measurable downstream state differences while revealing task-dependent rather than univer- sal internal signatures. Together, these results show that assistant history is not merely a passive record but part of the computational Li et al. context that interactive models must manage selectively, motivat- ing future methods that learn from intervention-derived turn-level signals which prior responses to retain, downweight, or discard. 7 Ethical Considerations This work studies multi-turn language-model behavior through controlled benchmark evaluation and counterfactual replay. It does not involve deployment to real users, human-subject experimen- tation, or the collection of personal or sensitive user data. Our interventions retrospectively modify self-generated assistant his- tory within observed benchmark trajectories while preserving the user messages already present in those histories, with the goal of diagnosing downstream history effects. The resulting causal claims are intervention-defined effects con- ditional on the observed user-message histories. They do not esti- mate fully adaptive interactions in which a changed assistant re- sponse could also change subsequent user behavior, and benchmark history sensitivity should not be interpreted as direct evidence of safety, reliability, or failure rates in deployed systems. Turn Surgery signals are likewise retrospective diagnostic quantities rather than validated online history-management decisions. The internal anal- yses are additionally limited to the evaluated open-weight setting and should not be interpreted as complete explanations of model computation. We therefore view these analyses as tools for studying interactive model behavior rather than as sufficient evidence for high-stakes deployment or automated decision-making. References [1]Ge Bai, Jie Liu, Xingyuan Bu, Yancheng He, Jiaheng Liu, Zhanhui Zhou, Zhuoran Lin, Wenbo Su, Tiezheng Ge, Bo Zheng, and Wanli Ouyang. 2024. MT-Bench-101: A Fine-Grained Benchmark for Evaluating Large Language Models in Multi- Turn Dialogues. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2024, Bangkok, Thailand, August 11-16, 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, 7421–7454. doi:10.18653/V1/2024. ACL-LONG.401 [2]Akash Bonagiri, Devang Borkar, Gerard Janno Anderias, Setareh Rafatirad, and Houman Homayoun. 2026. CausalFlow: Causal Attribution and Counterfactual Repair for LLM Agent Failures. arXiv preprint arXiv:2605.25338 (2026). [3]Mert Cemri, Melissa Z Pan, Shuyi Yang, Lakshya A Agrawal, Bhavya Chopra, Rishabh Tiwari, Kurt Keutzer, Aditya Parameswaran, Dan Klein, Kannan Ram- chandran, Matei A Zaharia, Joseph Gonzalez, and Ion Stoica. 2025. Why Do Multi-Agent LLM Systems Fail?. In Advances in Neural Information Process- ing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghas- semi, and N. Chen (Eds.), Vol. 38, Main Conference. Curran Associates, Inc. doi:10.52202/085713-4082 [4]Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde De Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al.2021. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374 (2021). [5]Xingwu Chen, Zhanqiu Zhang, Steven Y Guo, and Difan Zou. 2026. Breaking contextual inertia: Reinforcement learning with single-turn anchors for stable multi-turn interaction. In Findings of the Association for Computational Linguistics: ACL 2026. 6302–6318. [6]Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al.2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168 (2021). [7]Yang Deng, Xuan Zhang, Wenxuan Zhang, Yifei Yuan, See Kiong Ng, and Tat- Seng Chua. 2024. On the multi-turn instruction following for conversational web agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 8795–8812. [8]Darshan Deshpande, Varun Gangal, Hersh Mehta, Jitin Krishnan, Anand Kannap- pan, and Rebecca Qian. 2025. Trail: Trace reasoning and agentic issue localization. arXiv preprint arXiv:2505.08638 (2025). [9]Kaustubh Deshpande, Ved Sirdeshmukh, Johannes Baptist Mols, Lifeng Jin, Ed- Yeremai Hernandez-Cardona, Dean Lee, Jeremy Kritz, Willow E. Primack, Sum- mer Yue, and Chen Xing. 2025. MultiChallenge: A Realistic Multi-Turn Con- versation Evaluation Benchmark Challenging to Frontier LLMs. In Findings of the Association for Computational Linguistics, ACL 2025, Vienna, Austria, July 27 - August 1, 2025 (Findings of ACL), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, 18632–18702. https://aclanthology.org/2025.findings-acl.958/ [10] Christine Herlihy, Jennifer Neville, Tobias Schnabel, and Adith Swaminathan. 2024. On Overcoming Miscalibrated Conversational Priors in LLM-based Chat- Bots. In Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelli- gence (Proceedings of Machine Learning Research, Vol. 244), Negar Kiyavash and Joris M. Mooij (Eds.). PMLR, 1599–1620. https://proceedings.mlr.press/v244/ herlihy24a.html [11]Jenny Y. Huang, Leshem Choshen, Wei Sun, Omar Khattab, Ramón Fernandez Astudillo, Mehul Damani, Tamara Broderick, and Jacob Andreas. 2026. Do LLMs Benefit From Their Own Words? arXiv:2602.24287 [cs.CL] https://arxiv.org/abs/ 2602.24287 [12]Haziq Mohammad Khalid, Athikash Jeyaganthan, Timothy Do, Yicheng Fu, Vasu Sharma, Sean O’Brien, and Kevin Zhu. 2025. ERGO: Entropy-guided Resetting for Generation Optimization in Multi-turn Language Models. In Proceedings of the 2nd Workshop on Uncertainty-Aware NLP (UncertaiNLP 2025). 273–286. [13]Wai-Chung Kwan, Xingshan Zeng, Yuxin Jiang, Yufei Wang, Liangyou Li, Lifeng Shang, Xin Jiang, Qun Liu, and Kam-Fai Wong. 2024. MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 20153–20177. doi:10.18653/v1/ 2024.emnlp-main.1124 [14] Philippe Laban, Alexander Richard Fabbri, Caiming Xiong, and Chien-Sheng Wu. 2024. Summary of a haystack: A challenge to long-context llms and rag systems. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing. 9885–9903. [15]Philippe Laban, Hiroaki Hayashi, Yingbo Zhou, and Jennifer Neville. 2026. Llms get lost in multi-turn conversation. In International Conference on Learning Representations, Vol. 2026. 54738–54778. [16] Ming Li, Pei Chen, Zhenhao Zhang, Tao Yang, Xinyang Zhang, Han Li, Tianyu Cao, Ming Zeng, Zhuofeng Wu, Meng Jiang, et al.2026. Mitigating Lost in Multi-turn Conversation via Curriculum RL with Verifiable Accuracy and Ab- stention Rewards. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 33364–33381. [17]Zizhuo Lin, Quanling Liu, Jinsheng Quan, Chao Zhang, Yifan Zhu, Xing Shi, Jingtao Xu, Zhihui Li, and Yawei Luo. 2026. Same Evidence, Different Answers: Canonical-Context On-Policy Distillation for Multi-Turn Language Models. arXiv preprint arXiv:2605.30251 (2026). [18] Geng Liu, Fei Zhu, Rong Feng, Changyi Ma, Shiqi Wang, and Gaofeng Meng. 2026. Intent Mismatch Causes LLMs to Get Lost in Multi-Turn Conversation. arXiv:2602.07338 [cs.CL] https://arxiv.org/abs/2602.07338 [19] Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. Lost in the middle: How language models use long contexts. Transactions of the association for computational linguistics 12 (2024), 157–173. [20]Sichun Luo, Yi Huang, Mukai Li, Shichang Meng, Fengyuan Liu, Zefa Hu, Junlan Feng, and Qi Liu. 2025. ClarifyMT-Bench: Benchmarking and Improving Multi- Turn Clarification for Conversational Large Language Models. arXiv preprint arXiv:2512.21120 (2025). [21]Ming Ma, Jue Zhang, Fangkai Yang, Yu Kang, Qingwei Lin, Saravan Rajmohan, and Dongmei Zhang. 2026. Dover: Intervention-driven auto debugging for llm multi-agent systems. In International Conference on Learning Representations, Vol. 2026. 28059–28087. [22]Adyasha Maharana, Dong-Ho Lee, Sergey Tulyakov, Mohit Bansal, Francesco Barbieri, and Yuwei Fang. 2024. Evaluating very long-term conversational memory of llm agents. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). 13851–13870. [23]Charles Packer, Sarah Wooders, Kevin Lin, Vivian Fang, Shishir G Patil, Ion Stoica, and Joseph E Gonzalez. 2023. Memgpt: Towards llms as operating systems. arXiv preprint arXiv:2310.08560 (2023). [24]Xu Pan, Jingxuan Fan, Zidi Xiong, Ely Hahami, Jorin Overwiening, and Ziqian Xie. 2026. User-Assistant Bias in LLMs. In Findings of the Association for Computational Linguistics: ACL 2026. 9218–9241. [25]Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. ToTTo: A controlled table-to-text generation dataset. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP). 1173–1186. [26]Shishir G Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E Gonzalez. 2025. The berkeley function calling leaderboard (bfcl): From tool use to agentic evaluation of large language models. In Forty- second International Conference on Machine Learning. What if LLMs Ate Their Words: Causal History Effects in Multi-Turn Interaction [27]Matt Post. 2018. A call for clarity in reporting BLEU scores. In Proceedings of the third conference on machine translation: Research papers. 186–191. [28]Jaineet Shah. 2026. Causal Agent Replay: Counterfactual Attribution for LLM- Agent Failures. arXiv preprint arXiv:2606.08275 (2026). [29]Noah Shinn, Federico Cassano, Ashwin Gopinath, Karthik Narasimhan, and Shunyu Yao. 2023. Reflexion: Language agents with verbal reinforcement learn- ing. Advances in neural information processing systems 36 (2023), 8634–8652. [30]Jyotika Singh, Fang Tu, Miguel Ballesteros, Weiyi Sun, Sandip Ghoshal, Michelle Yuan, Yassine Benajiba, Sujith Ravi, and Dan Roth. 2026. Mt-osc: Path for llms that get lost in multi-turn conversation. In Findings of the Association for Compu- tational Linguistics: ACL 2026. 27137–27160. [31]Yang Wan, Zheng Cao, Zhenhao Zhang, Zhengwen Zeng, Shuheng Shen, Changhua Meng, and Linchao Zhu. 2026. Mitigating Conversational Inertia in Multi-Turn Agents. arXiv preprint arXiv:2602.03664 (2026). [32]Xingyao Wang, Zihan Wang, Jiateng Liu, Yangyi Chen, Lifan Yuan, Hao Peng, and Heng Ji. 2024. MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net. https://openreview.net/forum?id=jp3gWrMuIZ [33]Di Wu, Hongwei Wang, Wenhao Yu, Yuwei Zhang, Kai-Wei Chang, and Dong Yu. 2024. Longmemeval: Benchmarking chat assistants on long-term interactive memory. arXiv preprint arXiv:2410.10813 (2024). [34]Shunyu Yao, Noah Shinn, Pedram Razavi, and Karthik Narasimhan. 2024.휏- bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv preprint arXiv:2406.12045 (2024). [35]Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al.2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. In Proceedings of the 2018 conference on empirical methods in natural language processing. 3911–3921. [36] Haoyu Zheng, Yun Zhu, Shu Yuan, Shangming Chen, Qing Wang, Wenqiao Zhang, Jun Xiao, and Yueting Zhuang. 2026. MAIGO: Mitigating Lost-in- Conversation with History-Cleaned On-Policy Self-Distillation. arXiv preprint arXiv:2605.27186 (2026). [37]Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric P. Xing, Hao Zhang, Joseph E. Gonzalez, and Ion Stoica. 2023. Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. In Advances in Neural Information Pro- cessing Systems 36: Annual Conference on Neural Information Processing Sys- tems 2023, NeurIPS 2023, New Orleans, LA, USA, December 10 - 16, 2023, Alice Oh, Tristan Naumann, Amir Globerson, Kate Saenko, Moritz Hardt, and Sergey Levine (Eds.). http://papers.nips.c/paper_files/paper/2023/hash/ 91f18a1287b398d378ef22505bf41832-Abstract-Datasets_and_Benchmarks.html [38]Kunlun Zhu, Zijia Liu, Bingxuan Li, Muxin Tian, Yingxuan Yang, Jiaxun Zhang, Pengrui Han, Qipeng Xie, Fuyang Cui, Weijia Zhang, et al.2025. Where llm agents fail and how they can learn from failures. arXiv preprint arXiv:2509.25370 (2025).