Paper deep dive

Transformers Remember First, Forget Last: Dual-Process Interference in LLMs

Sourav Chattaraj, Kanak Raj

Year: 2026Venue: arXiv preprintArea: cs.IRType: PreprintEmbeddings: 53

Intelligence

Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 92%

Last extracted: 7/20/2026, 6:25:35 AM

Summary

This study investigates memory interference in 39 Large Language Models (LLMs) by adapting cognitive psychology paradigms. It finds that LLMs exhibit a universal pattern where Proactive Interference (PI) dominates Retroactive Interference (RI), meaning early information is protected at the expense of recent information, which is the inverse of human memory. RI and PI are uncorrelated, suggesting distinct mechanisms: RI is capacity-dependent (scaling with model size) and involves passive retrieval failure, while PI is an architectural constraint involving active primacy intrusion. Reasoning models excel at RI but not PI.

Entities (9)

LLMs · model-class · 99%Proactive Interference · concept · 98%Retroactive Interference · concept · 98%PIES · metric · 95%RIES · metric · 95%Transformer Attention · mechanism · 95%Model Size · hyperparameter · 92%Claude-3.5-sonnet · model-instance · 90%o1 · model-instance · 90%

Relation Signals (7)

Proactive Interference → dominates → Retroactive Interference

confidence 98% · Every model shows the same pattern: proactive interference (PI) dominates retroactive interference (RI) universally

Retroactive Interference → isuncorrelatedwith → Proactive Interference

confidence 95% · RI and PI are uncorrelated (R^2 = 0.044)

Model Size → predicts → Retroactive Interference Resistance

confidence 92% · Model size predicts RI resistance (R2=0.49)

Transformer Attention → creates → Primacy Bias

confidence 90% · suggesting that transformer attention creates a primacy bias

Model Size → doesnotpredict → Proactive Interference Resistance

confidence 90% · Model size predicts RI resistance ... but not PI

o1 → exhibits → Low PI Resistance

confidence 90% · o1 ranks #35 for PI (PIES = 21.9)

o1 → exhibits → High RI Resistance

confidence 90% · o1 ranks #1 for RI (RIES = 186.4)

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:When large language models encounter conflicting information in context, which memories survive -- early or recent? We adapt classical interference paradigms from cognitive psychology to answer this question, testing 39 LLMs across diverse architectures and scales. Every model shows the same pattern: proactive interference (PI) dominates retroactive interference (RI) universally (Cohen's d = 1.73, p < 0.0001), meaning early encodings are protected at the cost of recent information -- the opposite of human memory, where RI typically dominates. Three findings indicate that RI and PI reflect separate memory mechanisms. RI and PI are uncorrelated (R^2 = 0.044), rejecting a unified "memory capacity." Model size predicts RI resistance (R^2 = 0.49) but not PI (R^2 = 0.06, n.s.) -- only RI is capacity-dependent. And error analysis reveals distinct failure modes: RI failures are passive retrieval failures (51%), while PI failures show active primacy intrusion (56%); both show <1% hallucination. These patterns parallel the consolidation-retrieval distinction in cognitive science, suggesting that transformer attention creates a primacy bias with direct implications for interference-heavy applications.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

51,360 characters extracted from source content.

Expand or collapse full text

Transformers Remember First, Forget Last: Dual-Process Interference in LLMs Sourav Chattaraj Kanak Raj Thomson Reuters, Bangalore, India sourav.chattaraj@thomsonreuters.comkanak.raj@thomsonreuters.com Abstract When large language models encounter conflicting information in context, which memories survive—early or recent? We adapt classical interference paradigms from cognitive psychology to answer this question, testing 39 LLMs across diverse architectures and scales. Every model shows the same pattern: proactive interference (PI) dominates retroactive interference (RI) universally (Cohen’s d=1.73d=1.73, p<0.0001p<0.0001), meaning early encodings are protected at the cost of recent information—the opposite of human memory, where RI typically dominates. Three findings indicate that RI and PI reflect separate memory mechanisms. RI and PI are uncorrelated (R2=0.044R^2=0.044), rejecting a unified “memory capacity.” Model size predicts RI resistance (R2=0.49R^2=0.49) but not PI (R2=0.06R^2=0.06, n.s.)—only RI is capacity-dependent. And error analysis reveals distinct failure modes: RI failures are passive retrieval failures (51%), while PI failures show active primacy intrusion (56%); both show <<1% hallucination. These patterns parallel the consolidation–retrieval distinction in cognitive science, suggesting that transformer attention creates a primacy bias with direct implications for interference-heavy applications. Transformers Remember First, Forget Last: Dual-Process Interference in LLMs Sourav Chattaraj†thanks: sourav.chattaraj@thomsonreuters.com Kanak Raj†thanks: kanak.raj@thomsonreuters.com Thomson Reuters, Bangalore, India 1 Introduction Figure 1: LLMs Show Opposite Interference Pattern to Humans. Retroactive interference (RI, left) maintains near-ceiling accuracy through N=50 (96%), then drops sharply to 27% at N=300. Proactive interference (PI, right) collapses accuracy from 85% to 2%—near-complete failure. All 39 models show PI >> RI, contrasting with human memory where RI dominates. Shaded regions: 95% CI. When humans learn new information that conflicts with prior knowledge, the new typically disrupts recall of the old—retroactive interference (RI). The reverse, where prior knowledge blocks new learning, is proactive interference (PI). A century of research establishes that human memory shows RI >> PI: recent information overwrites earlier memories more readily than earlier memories block recent ones (Jenkins and Dallenbach, 1924; Underwood, 1957). Consider a medical AI tracking a patient’s blood pressure across a 12-hour emergency visit: 120 mmHg at triage, 135 after 15 minutes, 128 post-medication, 118 at discharge. When the physician asks “What was the initial blood pressure?” (RI task), the system must recall the first value despite three subsequent updates. When asked “What is the current blood pressure?” (PI task), it must access the most recent value despite earlier readings competing for attention. These scenarios demand opposite memory operations—and our findings suggest LLMs handle them very differently. Despite remarkable advances in few-shot learning and reasoning (Brown et al., 2020; Achiam et al., 2023), and context windows expanding to millions of tokens, how LLMs handle memory interference remains poorly understood. We adapted classical interference paradigms from cognitive psychology to test 39 large language models. Every single model shows the opposite pattern: PI dominates RI across all architectures, scales, and training regimes (Cohen’s d=1.73d=1.73, p<0.0001p<0.0001). Transformers protect early encodings at the cost of recent information—the inverse of biological memory (Figure 1). This inversion is not merely surprising—it suggests distinct computational structure. If RI and PI reflected unified “memory capacity,” they would correlate strongly. Instead, they are essentially independent (R2=0.044R^2=0.044). Model size correlates with RI resistance but not PI resistance. Error analysis reveals distinct failure modes: RI failures are passive (models cannot retrieve), while PI failures are active (earlier values intrude). Both show minimal hallucination—models confuse positions rather than fabricate. These dissociations suggest RI and PI engage computationally distinct mechanisms, paralleling the consolidation–retrieval distinction in cognitive science (Wixted, 2004). RI appears to test whether initial encodings can resist overwriting—a capacity-limited process scaling with parameters. PI tests whether attention can favor recent over competing earlier information—an architectural constraint independent of scale. Prior LLM memory research has focused on PI (Wang and Sun, 2025), leaving RI unexplored. Yet applications from medical records to legal documents require recalling initial states after updates. Understanding how these interference types differ is essential for principled deployment. Contributions. 1. Evidence consistent with dual-process memory: RI and PI are uncorrelated (R2=0.044R^2=0.044) with opposite scaling relationships, suggesting consolidation-based RI (capacity-dependent) versus retrieval-based PI (capacity-independent). 2. Inverted interference profile: All tested LLMs show PI >> RI (d=1.73d=1.73), contrasting with human memory where RI typically dominates—suggesting transformer attention produces primacy protection absent in biological systems. 3. Cognitive failure taxonomy: RI failures are passive (retrieval failure), PI failures are active (primacy intrusion), both with minimal hallucination—position confusion, not fabrication. 2 Background Interference in Human Memory. Retroactive interference (RI) and proactive interference (PI) have been studied since the 1920s as fundamental constraints on human memory (Jenkins and Dallenbach, 1924; Underwood, 1957). RI occurs when new learning disrupts recall of prior material; PI occurs when prior learning blocks recall of new material. In serial recall, humans typically show both primacy and recency effects, with recency dominating immediate recall (Murdock, 1962)—a pattern our LLM results will contrast with. Importantly, the relative magnitude of RI versus PI is not universal—it depends on experimental conditions including semantic similarity, retention interval, degree of original learning, and individual differences (MacLeod, 2024). While RI often dominates in immediate paired-associate learning (the paradigm closest to our experiments), some conditions favor PI dominance, particularly with extended prior learning or category shifts (Wickens, 1970). Working memory capacity predicts interference resistance (Engle, 2002; Kane and Engle, 2000). Modern theories, building on dual-store models of memory (Atkinson and Shiffrin, 1968), distinguish two mechanisms (Wixted, 2004): consolidation-based interference (RI), where new learning overwrites insufficiently stabilized memories, and retrieval-based interference (PI), where competing traces disrupt cue-driven recall, with stronger memories actively suppressing weaker competitors (Anderson et al., 1994). This consolidation–retrieval distinction provides our framework for interpreting LLM interference patterns. Memory Interference in LLMs. We distinguish in-context interference—competition between items within a single forward pass—from catastrophic forgetting, where new training overwrites prior knowledge in model weights (McCloskey and Cohen, 1989; French, 1999). Our work addresses the former: can models maintain access to information when semantically similar content creates competition? Existing long-context benchmarks primarily test search—whether models can locate specific information within extended contexts (Kamradt, 2023; Bai et al., 2024; Modarressi et al., 2025; Fu et al., 2025; Ling et al., 2025). These tasks confound retrieval difficulty with context length, leaving open whether models can maintain information under semantic conflict. Transformer attention also exhibits known positional biases—models struggle with middle positions (Liu et al., 2024) and disproportionately attend to initial tokens (Xiao et al., 2024)—but how these biases interact with memory interference remains untested. Wang and Sun (2025) conducted the first systematic study of PI in LLMs, finding that model size predicts resistance (R2=0.26R^2=0.26) while context length does not. However, retroactive interference—whether LLMs can recall initial information after conflicting updates—remains unexplored. More broadly, while scaling laws predict general capability improvements with size (Kaplan et al., 2020; Hoffmann et al., 2022), whether RI and PI scale differently has not been tested. This gap matters both theoretically and practically: if RI and PI tap distinct mechanisms (consolidation vs. retrieval), models may show different vulnerability profiles; if they reflect unified capacity, performance should correlate. We address this by adapting classical AB-AC interference paradigms (Underwood, 1957) to test both RI and PI on identical stimulus sequences, enabling direct comparison across 39 LLMs. 3 Methods Task Design. We adapt the classic AB-AC interference paradigm from cognitive psychology (Underwood, 1957) to probe LLM memory. Models first learn 46 category-value pairs (e.g., visual art: impressionism), then process N interleaved updates per category (N∈3,10,50,100,200,300N∈\3,10,50,100,200,300\), where each update assigns a new value to a previously seen category (e.g., visual art: baroque). We then query either the initial value (retroactive interference, RI) or the most recent value (proactive interference, PI). Critically, RI and PI use identical stimulus sequences—only the query target differs—enabling controlled comparison (Figure 2). Dataset. We constructed a dataset of 46 semantic categories with real-world meaningful values per category. Updates are randomly interleaved across categories without grouping updates for the same key contiguously—mimicking concurrent updates in real-world data logs. This setup ensures interference arises from semantic competition, not positional proximity. Details in Appendix A. Figure 2: Experimental Paradigm. Both RI and PI use identical sequences: 46 initial facts followed by N updates. RI queries initial values (green); PI queries final values (red). This isolates interference from search difficulty. Metrics. To capture overall interference resistance (not just performance at one level), we compute the Retroactive Interference Endurance Score (RIES)—the area under the accuracy-vs-interference curve, log-scaled to weight performance equally across interference magnitudes: RIES=∫N=3300ARI(N)d(log10⁡(N+1))RIES= _N=3^300A_RI(N)\,d( _10(N+1)) (1) where ARI(N)A_RI(N) is retrieval accuracy for initial values at interference level N. A model scoring 100% accuracy at all levels achieves RIES ≈ 200; a model at 50% throughout scores ≈ 100. PIES (Proactive Interference Endurance Score) uses the identical formula applied to accuracy for most-recent values. Higher scores indicate stronger resistance to interference. Thus, RIES >> PIES means a model resists RI better than PI—equivalently, PI causes more performance degradation than RI. Models. We tested 39 LLMs with complete data across all interference levels, spanning 1B–2.5T parameters. Models include open-weight (Llama, Mistral, Qwen) and proprietary (GPT-4o, Claude-3.5, o1, o3) architectures, with context windows from 8K to 2M tokens. Both RI and PI experiments used 3 independent runs per model; results showed high consistency (RI: 73.7% zero variance, 90.2% SD << 5; PI: 64.8% zero variance, 90.4% SD << 5; see Figure 10 for distribution). Full specifications in Appendix B. 4 Results 4.1 Retroactive Interference in LLMs All 39 tested models exhibit substantial retroactive interference. At minimal interference (N=3N=3 updates per category), mean accuracy is 93.5% (SD=4.96%). Accuracy remains high through N=50N=50 (96.3%, reflecting ceiling effects), then drops sharply: 74.5% at N=100N=100, 43.5% at N=200N=200, and 27.4% at N=300N=300. This 66-point drop from peak confirms that LLMs struggle to recall initial information when subsequent updates create semantic conflict—establishing RI as a robust phenomenon warranting mechanistic investigation. Performance varies substantially across models (Table 1). Top performers (o1, Claude-3.5-sonnet) maintain above 60% accuracy even at maximum interference, while bottom performers (smaller Llama variants) drop below 10%. This 6× range motivates our investigation of what factors predict resistance. Table 1: RI Resistance Varies Widely. Top 5 and bottom 5 models by RIES, showing accuracy at low (N=3) and high (N=300) interference. Model RIES Acc@3 Acc@300 Top 5 o1 186.4 100% 86% o3 185.8 100% 83% o3-mini 183.3 100% 79% Claude-3.5-sonnet 178.2 98% 62% GPT-4o 176.1 97% 58% Bottom 5 Llama-3.2-3b 113.2 82% 4% Mistral-7b 121.5 85% 8% Qwen-2.5-7b 128.7 88% 12% Gemma-2-9b 131.4 89% 15% Llama-3.1-8b 134.6 90% 18% 4.2 Model Size Correlates with RI Resistance What determines whether a model resists retroactive interference? We tested two candidate predictors: model size (parameter count) and context window length. Size correlates with RI resistance. Linear regression reveals that parameter count accounts for 49% of variance in RIES—our aggregate measure of accuracy maintained across all interference levels (R2=0.491R^2=0.491, β=0.70β=0.70, p<0.0001p<0.0001; Figure 3). Models in the largest size tier (>>500B parameters) achieve mean RIES of 171.2, compared to 142.8 for the smallest tier (<<10B)—a 28-point advantage, with ANOVA confirming a significant monotonic size effect (F(3,35)=11.83F(3,35)=11.83, p<0.0001p<0.0001). Context length does not. Context window size shows no relationship with RIES (R2=0.003R^2=0.003, p=0.746p=0.746). A model with 32K context and 100B parameters outperforms a model with 1M context and 10B parameters. This dissociation suggests that interference resistance reflects representational capacity, not buffer size. Critically, size does NOT predict PI resistance. When we apply the same regression to PIES (the equivalent metric for PI), model size explains only 6% of variance (R2=0.06R^2=0.06, p=0.14p=0.14, n.s.). This asymmetry—size predicting RI but not PI—provides first evidence that the two interference types engage different mechanisms. Figure 3: Size Predicts RI Resistance, Not PI. (A) Model size vs RIES shows strong positive relationship (R2=0.49R^2=0.49, p<0.0001p<0.0001). (B) Context length vs RIES shows no relationship (R2=0.003R^2=0.003, n.s.). Size-PI relationship: R2=0.06R^2=0.06, n.s.—size does not predict PI resistance (see Figure 6 in Appendix). Mechanistic interpretation. We attribute this asymmetry to superposition (Elhage et al., 2022): smaller models represent features with overlapping vectors, creating interference; larger models allocate more orthogonal representations. For RI, this representational room lets initial encodings survive alongside updates. For PI, the bottleneck is attention selection, not representation—an architectural constraint unaffected by capacity (see Discussion). 4.3 Reasoning Models Excel at RI—With a Hidden Cost Do architectural innovations beyond scale affect interference resistance? We compared chain-of-thought reasoning models (Wei et al., 2022; OpenAI, 2024) (o1, o3, o4-mini, DeepSeek-R1; n=6) against standard models (n=33). Reasoning models show 18% higher RIES. Mean RIES for reasoning models is 178.0 (SD=12.6) versus 150.3 (SD=18.2) for non-reasoning models—an 18.4% advantage (independent t(37)=3.54t(37)=3.54, p=0.001p=0.001, Cohen’s d=1.76d=1.76). Five of six reasoning models rank in the top 10 for RI resistance. Extended inference-time computation appears to enhance memory consolidation. But this advantage does not transfer to PI. The same reasoning models that excel at RI show no advantage—and often disadvantage—for PI. Most strikingly, o1 ranks #1 for RI (RIES = 186.4) but #35 for PI (PIES = 21.9), an 8.5× asymmetry. This dissociation further demonstrates that RI and PI tap different mechanisms: reasoning aids consolidation but does not help—and may hurt—recency access. Practical implication: Reasoning models are optimal for applications requiring robust historical memory (document analysis, knowledge retrieval), but suboptimal for tasks requiring recent-state tracking (dialogue, real-time updates). 4.4 Decay Patterns Reveal Architectural Diversity How does accuracy degrade as interference increases? Unlike PI, which follows a universal log-linear decline across all models (Wang and Sun, 2025), RI decay shows striking architectural diversity. Fitting decay functions to each model’s accuracy curve reveals: 35% follow exponential decay (rapid early forgetting), 30% polynomial (gradual degradation), 26% log-quadratic (initial resistance then collapse), and 9% power-law (O-series reasoning models). This heterogeneity suggests different architectures implement memory consolidation differently, while retrieval competition (PI) operates through shared attention mechanisms. The architectural fingerprint in decay patterns provides further evidence that RI and PI engage distinct computational processes. 4.5 RI and PI Engage Distinct Mechanisms If RI and PI reflect a unified “memory capacity,” models good at one should be good at the other. We tested this by correlating RIES and PIES across all 39 models. RI and PI are uncorrelated. Pearson correlation yields r=0.21r=0.21 (R2=0.044R^2=0.044, p=0.201p=0.201; Figure 4). Knowing a model’s RI performance tells you essentially nothing about its PI performance. This rejects the unified-capacity hypothesis and suggests RI and PI tap computationally distinct processes. All models show PI >> RI. Despite this independence, a striking asymmetry emerges: every tested model (39/39, 100%) shows higher RIES than PIES. The mean difference is 69.5 points (paired t(38)=10.71t(38)=10.71, p<0.0001p<0.0001, Cohen’s d=1.73d=1.73). This is a large effect—LLMs find proactive interference substantially harder than retroactive interference. This contrasts with typical human findings. In human memory, retroactive interference typically dominates (Underwood, 1957): new information disrupts old more than old disrupts new. LLMs show the opposite. Transformer attention mechanisms appear to create “primacy protection”—early encodings are preserved at the cost of recency access. Figure 4: RI and PI: Uncorrelated with Systematic Bias. (A) Scatter plot shows weak correlation (R2=0.044R^2=0.044, p=0.20p=0.20); all points above diagonal indicate universal RIES >> PIES. (B) Bland-Altman plot reveals systematic bias of +69.5 points (95% limits: −9.4-9.4 to +148.5+148.5), confirming RI and PI are not interchangeable measures. 4.6 Asymmetry Magnitude Varies by Architecture While all models show PI >> RI, the magnitude of asymmetry varies dramatically across model families, revealing distinct architectural trade-offs in memory system design: • O-Series (Extreme): o1 ranks #1 for RI (RIES = 186.4) but #35 for PI (PIES = 21.9)—an 8.5× asymmetry. Chain-of-thought reasoning optimizes consolidation at the cost of recency. • Claude (Minimal): Claude-4.5-opus shows RIES = 178.4 and PIES = 170.3—only 1.05× asymmetry. Seven of the top 10 PI performers are Claude models, suggesting balanced memory design. • GPT (Moderate): GPT-5 shows 1.9× asymmetry, occupying the middle ground between extremes. This 8-fold range in asymmetry magnitude demonstrates that the universal PI >> RI pattern admits substantial architectural variation. Reasoning architectures appear to sacrifice recency for historical accuracy; Claude’s balanced design suggests deliberate optimization for both. These patterns provide principled guidance for model selection: applications requiring robust historical memory should prefer O-series, while those requiring recent-state tracking should prefer Claude. 4.7 Error Analysis Reveals Distinct Failure Modes To understand why RI and PI engage different mechanisms, we analyzed 5,409 RI errors and 9,753 PI errors across all models. We classified errors into five categories: (1) retrieval failure—no value returned; (2) same-key interference—value from correct category, wrong position; (3) cross-key interference—value from different category; (4) hallucination—value never presented; (5) partial match—close but not exact match. RI and PI show distinct error signatures (Table 2). RI failures are predominantly retrieval failures (50.8%)—models cannot access any value for the queried category. PI failures are predominantly same-key intrusions (56.1%)—models return earlier values from the correct category, suggesting active competition from prior encodings. Minimal hallucination in both. Critically, hallucination rates are below 1% for both RI (0.8%) and PI (0.6%). Models do not fabricate values—they confuse positions within seen information. This suggests interference is a retrieval/selection problem, not a generation problem. Metacognitive awareness is PI-specific. Not all retrieval failures are silent. Using LLM-as-judge classification, we find that 8.6% of PI retrieval failures involve explicit refusals (“I’m sorry, but I can’t determine that from the provided text”), compared to 0% for RI. This asymmetry is striking: models recognize when old memories compete with recent information, but fail to detect when recent information overwrites earlier ones. In transformer terms, PI creates multiple attention candidates for the same query—the model can “sense” competing activations and refuse. RI simply dilutes attention to the target position; with no competing signal, the model confidently returns whatever value receives strongest (but incorrect) attention. Interpretation. RI failures appear “passive”: the initial encoding was overwritten or inaccessible, leaving nothing to retrieve. PI failures appear “active”: earlier encodings intrude on attempts to access recent information. This passive/active distinction aligns with the consolidation (RI) vs. retrieval competition (PI) framework from cognitive science. Position bias confirms the mechanism. When same-key intrusions occur, where do the incorrect values come from? RI intrusions show recency bias: errors come from middle-to-late positions (mean normalized position = 0.51, where 0 = first update, 1 = most recent). The interfering content “wins”—models return recent values when asked for initial ones. PI intrusions show primacy bias: errors come from early positions, with 14% of errors returning the first value presented. The initial encoding persists and blocks access to updates. This contrast is sharpest at minimal interference (N=3N=3): RI errors come from position 0.72 (late) versus PI errors from 0.12 (early). This opposing directionality—RI errors from late, PI errors from early—provides direct mechanistic evidence that the two interference types engage opposite retrieval dynamics. Table 2: Error Distribution: RI vs PI. Error Type RI (%) PI (%) Retrieval failure 50.8 42.5 Same-key intrusion 46.0 56.1 Cross-key intrusion 0.6 0.7 Hallucination 0.9 0.2 Partial match 1.8 0.7 5 Discussion Evidence Consistent with Dual-Process Memory. The central finding of this work is that retroactive and proactive interference appear to engage computationally distinct mechanisms in LLMs. Three lines of evidence support this claim. First, RIES and PIES are uncorrelated (R2=0.044R^2=0.044)—knowing a model’s RI performance tells you almost nothing about its PI performance. Second, model size correlates with RI resistance (R2=0.49R^2=0.49) but not PI resistance (R2=0.06R^2=0.06, n.s.)—only RI appears capacity-dependent. Third, error profiles differ qualitatively: RI failures are passive (retrieval failure dominates), while PI failures show active intrusion from earlier positions—mirroring retrieval-induced forgetting (Anderson et al., 1994), where stronger memory traces actively suppress weaker competitors from the same category. This pattern parallels the consolidation–retrieval distinction in cognitive science (Wixted, 2004). RI appears to test whether initial encodings can resist overwriting—a process requiring representational capacity that scales with parameters. PI tests whether attention mechanisms can favor recent over competing earlier information—an architectural constraint independent of scale. Superposition May Explain the Capacity Dependence. Why does size predict RI but not PI? The superposition hypothesis (Elhage et al., 2022) provides a mechanistic answer. When neural networks must represent more features than they have dimensions, they encode features as nearly-orthogonal vectors that partially overlap—creating interference between similar representations. Smaller models, with fewer parameters, are forced into greater superposition; larger models can allocate more orthogonal representations (Scherlis et al., 2022). For RI, where the initial category-value pair must survive alongside subsequent updates, representational overlap causes the initial encoding to be “blurred” by later ones. Larger models reduce this overlap, preserving the initial encoding. For PI, the challenge is fundamentally different: all values may be distinctly represented, but attention must select the correct one. Attention’s primacy bias—reinforcing early tokens throughout the forward pass—is an architectural property that additional parameters cannot override. This is consistent with RI being capacity-limited (benefiting from reduced superposition) while PI is architecture-constrained (limited by attention dynamics). Why Transformers Differ from Humans. Human memory consistently shows RI >> PI (Underwood, 1957)—new information disrupts old more than old disrupts new. LLMs show the universal opposite: all 39 models exhibit PI >> RI. Where human serial position curves show recency dominance in immediate recall (Murdock, 1962), LLMs exhibit primacy dominance—inverting the canonical pattern. We attribute this to self-attention’s inherent bias toward early positions, consistent with the positional biases introduced earlier: the “lost in the middle” effect, where models struggle with information in middle positions (Liu et al., 2024), and “attention sinks,” where initial tokens receive disproportionate attention regardless of content (Xiao et al., 2024). In causal attention, early tokens accumulate attention from all subsequent positions, while later tokens can only attend backward. This creates asymmetric representational strength: early encodings are reinforced throughout the forward pass; recent tokens lack this cumulative advantage. The result is “primacy protection”—a feature, not a bug, of the architecture—that preserves distant context at the cost of recency. Practical Implications. Our findings suggest one empirically-supported principle: parameter count matters more than context length for interference resistance—a 100B model with 32K context outperformed a 10B model with 1M context in our paradigm. Table 3 summarizes key empirical observations; however, we emphasize that (1) these derive from a single experimental paradigm, (2) the exact nature of each application determines which interference type matters more, and (3) many factors beyond RI/PI characteristics affect model selection. Application-specific validation is essential before deployment. Table 3: Empirical Observations from Our Paradigm. Key findings; generalization to specific applications requires validation. Model Characteristic Observed Pattern Reasoning models 18% higher RI resistance; lowest PI resistance Large dense models Strong RI and PI; balanced performance Parameter count Predicts RI (R2R^2=0.49), not PI (R2R^2=0.06) Context length No predictive value for either RI or PI 6 Conclusion We presented the first systematic comparison of retroactive and proactive interference in LLMs, providing evidence that these phenomena may engage distinct computational mechanisms rather than reflecting unified memory capacity. Three findings inform understanding of transformer memory: (1) RI and PI are uncorrelated (R2=0.044R^2=0.044) with opposite scaling relationships—RI appears capacity-dependent, PI does not; (2) all 39 tested models show PI >> RI (d=1.73d=1.73), contrasting with the typical human pattern where RI dominates; (3) error analysis supports the distinction—RI failures are passive (retrieval failure), PI failures are active (primacy intrusion), both with minimal hallucination. These results suggest that transformers exhibit fundamentally different interference profiles from biological memory: attention mechanisms appear to protect early encodings at the cost of recency. Future Directions. Three directions extend this work. First, ecologically valid benchmarks: our synthetic category-value paradigm should be complemented by long-running narratives where facts evolve organically (Kočiský et al., 2018) and semi-structured evolving documents—medical case logs, legal case files, news timelines—where factual updates are critical and ground truth remains verifiable. Second, mechanistic validation: attention probing on open-weight models can directly test whether early tokens accumulate disproportionate attention, and analysis of training checkpoints (Biderman et al., 2023) can reveal whether the PI >> RI asymmetry is architectural or learned. Third, targeted mitigation: once mechanisms are understood, principled interventions—recency-weighted attention, structured context ordering, positional debiasing (Liu et al., 2024)—become feasible. Limitations Our findings derive from a single experimental paradigm—the AB-AC paired-associate task. While this classic design enables controlled comparison between RI and PI, it remains unclear whether the observed patterns generalize to other interference manipulations (e.g., varied retention intervals, intervening tasks) or alternative paradigms. The dual-process interpretation, while consistent with our behavioral dissociations, requires validation through additional experimental designs. Additional limitations include: (1) parameter counts for closed-source models are estimates; (2) synthetic category-value stimuli may not capture naturalistic interference dynamics (see Future Directions); (3) our evidence is behavioral—mechanistic validation via attention probing is discussed in Future Directions; (4) all tested models are transformer-based, excluding state-space models and base versus fine-tuned comparisons; (5) English-only testing limits cross-linguistic generalization. Ethical Considerations This work evaluates memory interference in LLMs through controlled experiments that do not involve human subjects or personal data. All experiments used synthetic category-value pairs specifically designed for this research. Our findings on model-specific vulnerabilities (e.g., PI susceptibility in reasoning models) aim to inform responsible deployment rather than enable misuse. We note potential dual-use concerns: understanding interference patterns could theoretically be exploited to craft adversarial prompts that confuse model memory. However, we believe the defensive value—enabling practitioners to select appropriate models for interference-sensitive applications and anticipate failure modes—outweighs this risk. Model parameter counts for proprietary systems are estimates; we make no claims about internal architectures beyond publicly available information. We encourage replication and extension of these findings across additional languages and model families to ensure generalizability. References J. Achiam, S. Adler, S. Agarwal, L. Ahmad, I. Akkaya, F. L. Aleman, D. Almeida, J. Altenschmidt, S. Altman, S. Anadkat, et al. (2023) GPT-4 technical report. arXiv preprint arXiv:2303.08774. Cited by: §1. M. C. Anderson, R. A. Bjork, and E. L. Bjork (1994) Remembering can cause forgetting: retrieval dynamics in long-term memory. Journal of Experimental Psychology: Learning, Memory, and Cognition 20 (5), p. 1063–1087. Cited by: §2, §5. R. C. Atkinson and R. M. Shiffrin (1968) Human memory: a proposed system and its control processes. In Psychology of Learning and Motivation, Vol. 2, p. 89–195. Cited by: §2. Y. Bai, X. Lv, J. Zhang, H. Lyu, J. Tang, Z. Huang, Z. Du, X. Liu, A. Zeng, L. Hou, et al. (2024) LongBench: a bilingual, multitask benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, Cited by: §2. S. Biderman, H. Schoelkopf, Q. G. Anthony, H. Bradley, K. O’Brien, E. Hallahan, M. A. Khan, S. Purohit, U. S. Prashanth, E. Raff, et al. (2023) Pythia: a suite for analyzing large language models across training and scaling. International Conference on Machine Learning, p. 2397–2430. Cited by: §6. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, et al. (2020) Language models are few-shot learners. Advances in neural information processing systems 33, p. 1877–1901. Cited by: §1. N. Elhage, T. Hume, C. Olsson, N. Schiefer, T. Henighan, S. Kravec, Z. Hatfield-Dodds, R. Lasenby, D. Drain, C. Chen, et al. (2022) Toy models of superposition. Transformer Circuits Thread. Cited by: §4.2, §5. R. W. Engle (2002) Working memory capacity as executive attention. Current directions in psychological science 11 (1), p. 19–23. Cited by: §2. R. M. French (1999) Catastrophic forgetting in connectionist networks. Trends in cognitive sciences 3 (4), p. 128–135. Cited by: §2. H. Y. Fu, A. Shrivastava, J. Moore, P. West, C. Tan, and A. Holtzman (2025) AbsenceBench: language models can’t tell what’s missing. arXiv preprint arXiv:2506.11440. Cited by: §2. J. Hoffmann, S. Borgeaud, A. Mensch, E. Buchatskaya, T. Cai, E. Rutherford, D. d. L. Casas, L. A. Hendricks, J. Welbl, A. Clark, et al. (2022) Training compute-optimal large language models. arXiv preprint arXiv:2203.15556. Cited by: §2. J. G. Jenkins and K. M. Dallenbach (1924) Obliviscence during sleep and waking. The American journal of psychology 35 (4), p. 605–612. Cited by: §1, §2. G. Kamradt (2023) Needle in a haystack - pressure testing llms. External Links: Link Cited by: §2. M. J. Kane and R. W. Engle (2000) Working-memory capacity, proactive interference, and divided attention: limits on long-term memory retrieval. Journal of Experimental Psychology: Learning, Memory, and Cognition 26 (2), p. 336–358. Cited by: §2. J. Kaplan, S. McCandlish, T. Henighan, T. B. Brown, B. Chess, R. Child, S. Gray, A. Radford, J. Wu, and D. Amodei (2020) Scaling laws for neural language models. arXiv preprint arXiv:2001.08361. Cited by: §2. T. Kočiský, J. Schwarz, P. Blunsom, C. Dyer, K. M. Hermann, G. Melis, and E. Grefenstette (2018) The NarrativeQA reading comprehension challenge. Transactions of the Association for Computational Linguistics 6, p. 317–328. Cited by: §6. Z. Ling, K. Liu, K. Yan, Y. Yang, W. Lin, T. Fan, L. Shen, Z. Du, and J. Chen (2025) LongReason: a synthetic long-context reasoning benchmark via context expansion. arXiv preprint arXiv:2501.15089. Cited by: §2. N. F. Liu, K. Lin, J. Hewitt, A. Paranjape, M. Bevilacqua, F. Petroni, and P. Liang (2024) Lost in the middle: how language models use long contexts. Transactions of the Association for Computational Linguistics 12, p. 157–173. Cited by: §2, §5, §6. C. M. MacLeod (2024) Interference theory: history and current status. In The Oxford Handbook of Human Memory, p. 1173–1208. Cited by: §2. M. McCloskey and N. J. Cohen (1989) Catastrophic interference in connectionist networks: the sequential learning problem. Psychology of learning and motivation 24, p. 109–165. Cited by: §2. A. Modarressi, H. Deilamsalehy, F. Dernoncourt, T. Bui, R. A. Rossi, S. Yoon, and H. Schütze (2025) NoLiMa: long-context evaluation beyond literal matching. In Proceedings of the 42nd International Conference on Machine Learning, Cited by: §2. B. B. Murdock (1962) The serial position effect of free recall. Journal of Experimental Psychology 64 (5), p. 482–488. Cited by: §2, §5. OpenAI (2024) Learning to reason with llms. Technical report OpenAI. External Links: Link Cited by: §4.3. A. Scherlis, K. Sachan, A. S. Jermyn, J. Benton, and B. Shlegeris (2022) Polysemanticity and capacity in neural networks. arXiv preprint arXiv:2210.01892. Cited by: §5. B. J. Underwood (1957) Interference and forgetting. Psychological review 64 (1), p. 49–60. Cited by: §1, §2, §2, §3, §4.5, §5. C. Wang and J. V. Sun (2025) Unable to forget: proactive interference reveals working memory limits in llms beyond context length. In ICML 2025 Workshop on Long Context Foundation Models, Cited by: §1, §2, §4.4. J. Wei, X. Wang, D. Schuurmans, M. Bosma, F. Xia, E. Chi, Q. V. Le, D. Zhou, et al. (2022) Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems 35, p. 24824–24837. Cited by: §4.3. D. D. Wickens (1970) Encoding categories of words: an empirical approach to meaning. Psychological Review 77 (1), p. 1–15. Cited by: §2. J. T. Wixted (2004) The psychology and neuroscience of forgetting. Annual review of psychology 55, p. 235–269. Cited by: §1, §2, §5. G. Xiao, Y. Tian, B. Chen, S. Han, and M. Lewis (2024) Efficient streaming language models with attention sinks. In International Conference on Learning Representations, Cited by: §2, §5. Appendix A Experimental Setup Stimuli. We used 46 semantic categories with 50–70 real-world meaningful values per category: Arts: visual art, dance style, literary genre, painting medium. Science: chemical element, constellation, mathematical concept. Nature: bird species, flower species, tree species, weather phenomenon. Food: cheese variety, wine variety, pasta shape, culinary herb, fruit variety. Culture: ancient civilization, martial art, board game. Technology: programming language, telescope type, photography technique. Example values for “fruit variety”: mangosteen, bergamot, persimmon, mirabelle, kiwi, grape, etc. Prompt Structure. Table 4 shows the three-phase RI prompt structure. Phase Content Instruction You will learn initial facts about categories, then see updates. Later, recall the INITIAL values you learned first. Initial Learning visual art: abstract expressionism tools: screwdriver … (46 categories) Updates Following facts have been updated: visual art: baroque tools: hammer … (N×46N× 46 updates, randomized order) Query What was the INITIAL value of visual art? Table 4: RI Prompt Structure. The prompt presents initial category–value pairs, followed by N rounds of updates, then queries a single category. PI Prompt: Identical structure, except the query asks “What was the LAST value of visual art?” Both conditions use the same stimulus sequence—only the retrieval target differs. Metric Definitions. Both RIES and PIES are computed as the area under the accuracy-vs-interference curve using trapezoidal integration with log-scaled x-axis. Let Δi=log10⁡(Li+1+1)−log10⁡(Li+1) _i= _10(L_i+1\!+\!1)- _10(L_i\!+\!1) for levels Li∈3,10,50,100,200,300L_i∈\3,10,50,100,200,300\: RIES=∑i=1n−1AiRI+Ai+1RI2⋅ΔiRIES= _i=1^n-1 A^RI_i+A^RI_i+12· _i (2) PIES=∑i=1n−1AiPI+Ai+1PI2⋅ΔiPIES= _i=1^n-1 A^PI_i+A^PI_i+12· _i (3) where AiRIA^RI_i and AiPIA^PI_i are RI and PI accuracy at level LiL_i. The +1+1 offset accounts for the initial value presentation. Log-scaling ensures equal weighting across orders of magnitude. Higher scores indicate greater resistance. Appendix B Model Specifications Table 8 lists all 39 models tested with complete data across all interference levels, sorted by parameter count (descending). RIES measures resistance to retroactive interference; PIES measures resistance to proactive interference. Architecture indicates Dense or Mixture-of-Experts (MoE); Reasoning indicates chain-of-thought models. Note: Models with optional extended thinking (e.g., Claude-4-opus, Gemini-2.5-pro) were tested with thinking disabled. Only models that use chain-of-thought by default (o-series, DeepSeek-R1) are classified as reasoning. Models are grouped into four size tiers (Table 5) for decay curve analysis. The testbed spans seven model families (OpenAI, Anthropic, Google, Meta, Amazon, Alibaba, DeepSeek) covering parameter counts from 1B to 2,500B, both Dense and MoE architectures, and both standard and reasoning configurations. Tier Parameters n Examples XS ≤ 10B 5 Llama-3.2-1b, Nova-micro S 11–99B 10 Llama-3.2-90b, o4-mini M 100–670B 13 Claude-4.5-opus, Llama-3.1-405b L ≥ 671B 11 GPT-5, o1, DeepSeek-R1 Table 5: Size Tier Definitions. Parameter-based grouping used for decay curve analysis (Figures 5–6). Appendix C Error Type Taxonomy All incorrect responses (5,409 total across 39 models and 6 interference levels) were classified into five mutually exclusive error types (Table 6). Examples assume the query asks for the initial value of “visual art” (correct: “abstract expressionism”), with N updates reassigning it to “baroque,” “cubism,” etc. Table 7 shows how the distribution shifts across interference levels. Error Type % Definition & Example Same-Key Interf. 46.0 Value from correct category, wrong position. “baroque” Retrieval Failure 50.8 No value produced; refusal or empty. “I’m not sure” Partial Match 1.8 Truncated or recombined. “abstract express” Hallucination 0.9 Plausible but never presented. “impressionism” Cross-Key Interf. 0.6 Value from a different category. “screwdriver” Table 6: Error Type Taxonomy. Five mutually exclusive error categories (n=5,409 total errors). Sorted by frequency. Error Type Low Moderate High N=3,10 N=50,100 N=200,300 Same-Key Interf. 6.4% 25.8% 55.8% Cross-Key Interf. 1.8% 0.3% 0.5% Hallucination 4.5% 1.1% 0.1% Retrieval Failure 80.1% 72.9% 42.0% Table 7: Error Distribution by Interference Stage. At low interference, errors are dominated by retrieval failures. As interference increases, same-key intrusions progressively dominate, indicating that competing memories become strong enough to override the target rather than simply blocking retrieval. Model B Arch Rea. RIES PIES Model B Arch Rea. RIES PIES GPT-5 2500 MoE N 186.4 99.2 Gemini-2.0-flash 100 MoE N 162.5 101.6 GPT-5-mini 2500 MoE N 162.5 109.1 Gemini-2.0-flash-lite 100 MoE N 143.9 98.6 GPT-5-nano 2500 MoE N 160.6 21.0 Llama-3.2-90b 90 Dense N 134.6 87.1 o3 2000 MoE Y 185.8 76.0 Nova-pro 70 Dense N 157.5 63.7 o3-mini 2000 MoE Y 183.3 75.6 Llama-3.1-70b 70 Dense N 134.6 87.1 o1 1760 MoE Y 186.4 22.0 Llama-3.3-70b 70 Dense N 138.7 100.3 o1-preview 1760 MoE Y 186.4 76.0 Gemini-2.5-flash-lite 40 MoE N 146.3 91.3 GPT-4.1 1760 MoE N 176.9 118.6 o4-mini 30 MoE Y 170.5 22.0 GPT-4.1-mini 1760 MoE N 157.6 116.9 Qwen-3-coder-30b 30 MoE N 166.4 89.3 GPT-4.1-nano 1760 MoE N 128.2 73.9 Claude-4.5-haiku 15 Dense N 142.6 128.5 DeepSeek-R1 671 Dense Y 155.4 55.6 Llama-3.2-11b 11 Dense N 138.4 47.6 Llama-3.1-405b 405 Dense N 141.7 57.2 Claude-3.5-haiku 10 Dense N 139.8 126.4 Llama-4-maverick 400 MoE N 178.3 67.9 Llama-3.1-8b 8 Dense N 128.3 57.1 Claude-4.5-opus 300 Dense N 178.4 170.3 Nova-lite 7 Dense N 130.7 53.2 Gemini-2.5-pro 300 MoE N 162.5 123.6 Llama-3.2-3b 3 Dense N 113.6 16.8 Claude-4.5-sonnet 250 Dense N 162.5 124.2 Nova-micro 1 Dense N 126.1 76.8 Qwen-3-VL-235b 235 MoE N 177.3 120.2 Llama-3.2-1b 1 Dense N 126.1 1.0 Claude-4-opus 200 Dense N 139.6 131.0 Claude-4-sonnet 200 Dense N 162.5 137.8 Claude-3-opus 175 Dense N 145.2 131.5 B = parameters in billions GPT-3.5-turbo 175 MoE N 156.1 120.0 Rea. = reasoning model (Y/N) Llama-4-scout 109 MoE N 153.9 40.2 Table 8: Complete Model Specifications. All 39 models with RIES and PIES scores. Models sorted by parameter count (descending). See Section B for details. Appendix D Detailed Results by Model and Condition D.1 Top and Bottom Performers Table 9 shows RI and PI accuracy at each interference level for the five highest- and lowest-scoring models. RI top performers are reasoning models maintaining near-ceiling accuracy; PI top performers are exclusively Claude models. Notably, o1 ranks #1 for RI but appears in the PI bottom 5 at 0%—the starkest dual-process dissociation. RI Accuracy (%) PI Accuracy (%) Model 3 10 50 100 200 300 Model 3 10 50 100 200 300 Top 5 (highest RIES) Top 5 (highest PIES) o1 100 97.8 98.6 100 100 100 Claude-4.5-opus 100 97.8 91.3 80.4 84.8 71.7 o1-preview 100 97.8 100 100 98.6 99.3 Claude-4-sonnet 100 97.8 93.5 34.8 23.9 8.7 GPT-5 100 97.8 100 100 98.6 38.4 Claude-3-opus 100 84.8 95.7 32.6 28.3 2.2 o3 100 97.8 100 99.3 99.3 100 Claude-4-opus 100 91.3 91.3 30.4 21.7 4.3 o3-mini 100 97.8 100 65.9 43.5 68.1 Claude-4.5-haiku 100 93.5 95.7 19.6 10.9 4.3 Bottom 5 (lowest RIES) Bottom 5 (lowest PIES) Llama-3.1-8b 58.7 97.8 89.1 43.5 16.7 13.8 o1 100 0 0 0 0 0 GPT-4.1-nano 82.6 71.0 87.7 56.5 23.9 6.5 o4-mini 100 0 0 0 0 0 Nova-micro 94.9 89.1 69.6 35.5 15.2 2.2 GPT-5-nano 95.7 0 0 0 0 0 Llama-3.2-1b 73.9 97.8 81.9 56.5 0 2.2 Llama-3.2-3b 73.9 0 0 0 2.2 0 Llama-3.2-3b 0 89.1 97.8 21.7 17.4 76.1 Llama-3.2-1b 2.2 0 0 0 2.2 0 Table 9: RI and PI Accuracy (%) at Each Interference Level. Top 5 and bottom 5 models by RIES (left) and PIES (right). D.2 Reasoning vs Non-Reasoning Models The six reasoning models (o1, o1-preview, o3, o3-mini, o4-mini, DeepSeek-R1) exhibit the most extreme RI–PI dissociation. Table 10 compares mean accuracy by interference level; Table 11 provides the per-model breakdown. 3 10 50 100 200 300 RI Accuracy (%) Reasoning 100 97.8 99.8 94.1 82.0 73.4 Non-reasoning 92.8 92.9 95.3 69.1 38.9 15.7 Δ +7.2 +4.9 +4.4 +25.0 +43.1 +57.7 PI Accuracy (%) Reasoning 99.6 59.1 0 0 0 0 Non-reasoning 93.5 69.4 53.6 10.9 9.9 3.4 Δ +6.2 −-10.3 −53.6-53.6 −-10.9 −-9.9 −-3.4 Table 10: Reasoning vs Non-Reasoning Models. Mean accuracy by interference level. Reasoning models show a widening RI advantage (+57.7% at N=300) but catastrophic PI failure (0% from N=50 onward). RI Accuracy (%) PI Accuracy (%) Model 3 10 50 100 200 300 3 10 50 100 200 300 o1 100 97.8 98.6 100 100 100 100 0 0 0 0 0 o1-preview 100 97.8 100 100 98.6 99.3 100 97.8 0 0 0 0 o3 100 97.8 100 99.3 99.3 100 100 97.8 0 0 0 0 o3-mini 100 97.8 100 65.9 43.5 68.1 97.8 97.8 0 0 0 0 o4-mini 100 97.8 100 100 55.1 5.8 100 0 0 0 0 0 DeepSeek-R1 100 97.8 100 99.3 95.7 67.4 100 60.9 0 0 0 0 Table 11: Reasoning Models: RI vs PI Accuracy at Each Interference Level. Most models maintain high RI accuracy through N=100, though o3-mini and o4-mini degrade earlier. All six collapse to 0% PI by N=50, with o1 and o4-mini failing at N=10—the earliest collapse point. Why 0% PI accuracy? The 0% scores do not reflect refusals or empty responses. These models respond confidently with all 46 category values—but consistently return values from earlier positions rather than the most recent ones. The chain-of-thought process appears to consolidate earlier encodings so strongly that recent updates become inaccessible, even when the prompt explicitly requests the “LAST value.” This is primacy intrusion, not retrieval failure. Table 12 shows a representative example. Field Value Model o1 Condition PI, N=300 Category visual art Expected (last) land art (update #300) Returned “neo pop 176” (update #176) The model responds confidently with position-indexed values: “visual art → neo pop 176”—returning update #176 out of 300, not the last. Despite explicitly tracking updates (“I have carefully tracked every update…”), it retrieves mid-sequence values. Accuracy: 0/46 (0%). The model’s reasoning process cannot overcome primacy bias. Table 12: Example PI Failure: o1 at N=300. The model returns earlier update values (with position indices) instead of the most recent ones, demonstrating primacy intrusion despite explicit chain-of-thought reasoning. Field Value Model Claude-4.5-haiku Condition RI, N=300 Category literary genre Expected (initial) memoir Returned hard sci fi The model returns “hard sci fi” (a later update value) instead of the initial value “memoir.” At N=300, with 300 intervening updates per category, the initial encoding has been overwritten by subsequent information. Accuracy: 4/46 (8.7%). Unlike PI failures where earlier values intrude, RI failures show later values displacing the original—recency overwrites primacy. Table 13: Example RI Failure: Claude-4.5-haiku at N=300. The model returns a later update value instead of the initial one, demonstrating how retroactive interference overwrites early encodings with recent information. Appendix E Additional Figures This section presents supplementary visualizations that complement the tabular results above. Figures 5–6 show RI and PI decay curves stratified by model size tier. Figure 7 contrasts reasoning and non-reasoning models. Figures 8–9 provide detailed error pattern visualizations. Figure 5: RI Decay Curves by Model Size Tier. Larger models (>100B parameters) maintain higher accuracy across all interference levels. The decay pattern shows that size provides consistent protection against retroactive interference, with the largest models maintaining >50% accuracy even at N=300. Figure 6: PI Decay Curves by Model Size Tier. Unlike RI (Figure 5), PI decay shows no consistent relationship with model size (R2=0.06R^2=0.06, n.s.). Medium-sized models (M tier, blue) actually outperform large models (L tier, green) at high interference levels. This lack of size-dependence, contrasting with RI’s strong size correlation (R2=0.49R^2=0.49), provides evidence that PI is architecture-constrained rather than capacity-limited. Figure 7: RI Decay Curves: Reasoning vs Non-Reasoning Models. Chain-of-thought reasoning models (solid lines) show markedly slower decay compared to standard models (dashed lines). Extended inference-time computation appears to enhance memory consolidation, with reasoning models maintaining higher accuracy at all interference levels. Figure 8: Detailed Error Pattern Analysis. Distribution of error types across RI and PI conditions. RI errors are predominantly retrieval failures (50.8%), while PI errors show primacy intrusion (56.1% same-key interference). Both conditions show minimal hallucination (<1%), confirming that models confuse positions rather than fabricate values. Figure 9: Position-Based Error Analysis. When same-key intrusions occur, RI errors come from middle-to-late positions (recency bias), while PI errors come from early positions with 14% returning the first value (primacy bias). This opposing directionality provides mechanistic evidence for distinct retrieval dynamics. Figure 10: Run-to-Run Variance Distribution. Standard deviation of accuracy across 3 independent runs, aggregated by interference level for RI (blue) and PI (orange). Both conditions show high reproducibility: the majority of model-level measurements exhibit zero or near-zero variance (RI: 73.7% exact zero, 90.2% SD << 5; PI: 64.8% exact zero, 90.4% SD << 5). Comparable variance distributions confirm that the asymmetric RI vs. PI findings reflect genuine model behavior rather than measurement noise.

Instant research discovery