Paper deep dive
Is Memorization Context-Sensitive? Prefix-Based Extraction Beyond Isolated Prefixes
Ali Satvaty, Narjes Sharafi, Jirui Qi, Suzan Verberne, Fatih Turkmen
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 10/10/2026, 3:01:18 AM
Summary
This paper investigates whether contextual conditioning in Large Language Models (LLMs) mitigates memorization or merely alters the set of extractable samples. Using prefix-based extraction attacks on three open-weight models, the authors find that memorization consists of a context-robust core (samples extractable regardless of context) and a context-sensitive boundary (samples whose extractability changes with context). While context can suppress some boundary cases, it does not erase memorization, and robustly extractable samples persist, especially with longer prefixes. This challenges the assumption that Retrieval-Augmented Generation (RAG) significantly reduces memorization risk.
Entities (10)
Relation Signals (7)
Falcon-7B-Instruct → evaluatedin → Study
confidence 99% · We evaluate three open-weight instruction-tuned models: ... Falcon-7B-Instruct
OLMo-2-1124-7B-Instruct → evaluatedin → Study
confidence 99% · We evaluate three open-weight instruction-tuned models: ... OLMo-2-1124-7B-Instruct
Llama-3-8B-Instruct → evaluatedin → Study
confidence 99% · We evaluate three open-weight instruction-tuned models: Llama-3-8B-Instruct
LLM → exhibits → Memorization
confidence 95% · Large language models (LLMs) can expose memorized training sequences under prefix-based extraction
Context → doesnoterase → Memorization
confidence 93% · We find that context does not simply erase memorization. Instead, extractable memorization consists of a context-robust core and a context-sensitive boundary.
RAG → doesnotfullymitigate → Memorization Risk
confidence 92% · These findings qualify the view that RAG reduces memorization risk. Context can lower aggregate extraction by suppressing boundary cases, yet robustly extractable samples persist
Prefix Length → increasesstabilityof → Memorization across contexts
confidence 90% · Many samples that are extractable without context remain extractable under the retrieved context, especially as the prefix length increases.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) can expose memorized training sequences under prefix-based extraction: given a prefix from a training example, the model may assign high probability to the original continuation. In deployed systems, however, prefixes are rarely evaluated in isolation. They often appear together with instructions, retrieved documents, or other task-specific context, as in retrieval-augmented generation (RAG). This motivates examining whether contextual conditioning mitigates memorization or merely changes the set of memorized samples that become extractable. We investigate this issue through paired item-level measurements of probabilistic suffix extraction. For each prefix-suffix pair, we score the target suffix under an empty prompt and under retrieved contexts of varying relevance, across three open-weight instruction-tuned models. We find that context does not simply erase memorization. Instead, extractable memorization consists of a context-robust core and a context-sensitive boundary. Many samples that are extractable without context remain extractable under the retrieved context, especially as the prefix length increases. At the same time, context mainly affects marginal samples near the extraction threshold: it suppresses some exposures, but also enables new ones that are missed by prefix-only evaluation. These findings qualify the view that RAG reduces memorization risk. Context can lower aggregate extraction by suppressing boundary cases, yet robustly extractable samples persist, and context-enabled extractability remains security-relevant.
Tags
Links
- Source: https://arxiv.org/abs/2610.12085v1
- Canonical: https://arxiv.org/abs/2610.12085v1
Trouble viewing inline? Open PDF directly →
Full Text
77,422 characters extracted from source content.
Expand or collapse full text
Is Memorization Context-Sensitive? Prefix-Based Extraction Beyond Isolated Prefixes Ali Satvaty Affiliation: University of Groningen Narjes Sharafi Affiliation: University of Groningen Jirui Qi Affiliation: University of Groningen Suzan Verberne Affiliation: Leiden UniversityCorrespondence: a.satvaty@rug.nl Fatih Turkmen Affiliation: University of Groningen Abstract Large language models (LLMs) can expose memorized training sequences under prefix-based extraction: given a prefix from a training example, the model may assign high probability to the original continuation. In deployed systems, however, prefixes are rarely evaluated in isolation. They often appear together with instructions, retrieved documents, or other task-specific context, as in retrieval-augmented generation (RAG). This motivates examining whether contextual conditioning mitigates memorization or merely changes the set of memorized samples that become extractable. We investigate this issue through paired item-level measurements of probabilistic suffix extraction. For each prefix-suffix pair, we score the target suffix under an empty prompt and under retrieved contexts of varying relevance, across three open-weight instruction-tuned models. We find that context does not simply erase memorization. Instead, extractable memorization consists of a context-robust core and a context-sensitive boundary. Many samples that are extractable without context remain extractable under the retrieved context, especially as the prefix length increases. At the same time, context mainly affects marginal samples near the extraction threshold: it suppresses some exposures, but also enables new ones that are missed by prefix-only evaluation. These findings qualify the view that RAG reduces memorization risk. Context can lower aggregate extraction by suppressing boundary cases, yet robustly extractable samples persist, and context-enabled extractability remains security-relevant. 11 1 The code is available at: https://github.com/alistvt/context-sensitive-memorization 1 Introduction Language models can memorize parts of their training data, raising privacy and copyright concerns when memorized content can be reproduced at inference time Carlini et al. (2019); Henderson et al. (2023); Mueller et al. (2024); Freeman et al. (2024). The memorized content may be extracted when an adversary supplies a matching prefix of the original sequence Carlini et al. (2021); Carlini et al. (2023); Hartmann et al. (2023); Satvaty et al. (2024); Tirumala et al. (2022). Prefix–suffix extraction attacks measure this behavior directly: given a prefix from a training sequence, the model may assign unusually high probability to the corresponding suffix, enabling verbatim recovery of memorized text. In many deployed systems, a model is not prompted with an isolated prefix, but with additional text such as retrieved documents, instructions, or task-specific context Wei et al. (2022); Gao et al. (2023). This is especially relevant for retrieval-augmented generation (RAG), where external documents are attached to the input of the model Lewis et al. (2020); Borgeaud et al. (2022). Intuitively, one might expect such context to reduce memorization-based leakage: if the model is grounded in retrieved evidence, it may rely less on parametric memory. This intuition has been explored in recent work suggesting that RAG can reduce leakage Zeng et al. (2024). However, it remains unclear how contextual conditioning changes prefix-based extraction at the level of individual samples. In this work, we study how extra context affects memorization under the prefix–suffix extraction setting. For each prefix–suffix sample, we measure the probability assigned to the target suffix when the prefix is evaluated alone and when it is evaluated with different types of preceding context. We consider semantically related retrieved context, and irrelevant context, and compare all conditions through paired item-level measurements. This lets us examine not only whether aggregate extraction changes under context, but also how context changes the set of samples that become extractable. Our results show that contextual conditioning does not simply remove memorization. Instead, extractable memorization has a context-robust core of samples that remain extractable regardless of the surrounding context, and a context-sensitive boundary consisting of samples near the extraction threshold whose exposure status flips when context is added. Many core samples remain extractable under retrieved contexts, especially as the prefix length increases. At the same time, context changes the exposure status of marginal samples: some prefix-suffix pairs are suppressed by context, while others become extractable only after context is added. This distinction matters for interpreting claims that retrieval reduces memorization. We find that context can lower aggregate extraction by suppressing boundary cases, but this aggregate reduction should not be interpreted as eliminating memorization risk. Robustly extractable samples remain extractable across contexts, and context-enabled samples are missed by prefix-only evaluation. Thus, contextual conditioning is better understood as reshaping the exposure boundary rather than erasing memorized content. Taken together, our contributions are: • We formulate prefix–suffix memorization under contextual conditioning and conduct paired item-level comparisons across empty, relevant, and irrelevant contexts. • We show that extractable memorization is composed of a high-confidence core that persists across contexts and weaker extractions that are more sensitive to surrounding context, with robustness increasing as prefix length grows. • We show that aggregate reductions in exposure can hide security-relevant sample-level behavior, including robust exposures that persist across contexts and examples that become extractable only after context is added. 2 Background and Related Work 2.1 Memorization in LLMs Prior work has shown that LLMs can reproduce verbatim training examples under extraction attacks, where an adversary prompts the model and searches generated outputs for memorized training text Nasr et al. (2025); Kim et al. (2023); Carlini et al. (2021). Definition 1 (Verbatim suffix extraction). Let (p,s)(p,s) denote a prefix–suffix pair from a text sequence, where p is given to the model and s is the target continuation. A suffix s is verbatim extracted if a model generates s exactly after being prompted with p. Later studies examined how memorization changes with model scale, data duplication, training dynamics, and language Carlini et al. (2023); Biderman et al. (2023); Kiyomaru et al. (2024); Tirumala et al. (2022); Satvaty et al. (2025). Our work focuses on verbatim, extractable memorization: given a prefix from a training sequence, we study whether the model assigns high probability to the corresponding suffix. 2.2 Measuring Memorization Memorization is commonly measured through prefix–suffix extraction: given a prefix p from a training sequence, the model is tested on whether it reproduces the corresponding suffix s. Much prior work relies on greedy decoding, treating a sample as memorized only when the decoded continuation matches the training suffix Satvaty et al. (2024). While intuitive, this binary view can be brittle: a suffix may receive high probability even if it is not produced by a single greedy decode. Recent work, therefore, measures memorization probabilistically, using the probability assigned to the target suffix rather than only exact decoded matches Hayes et al. (2025). Definition 2 (Probabilistic extraction). A suffix s is probabilistically extractable if the model assigns high probability to s conditioned on p. We measure this by the suffix log-probability ℓ(p,s)=logPM(s∣p). (p,s)= P_M(s p). (1) Under contextual conditioning, we instead measure ℓc(p,s)=logPM(s∣c,p), ^c(p,s)= P_M(s c,p), (2) where c is the surrounding context. We adopt this view because it gives a continuous per-sample score and supports paired comparisons across contexts. For each prefix–suffix sample, we can measure the suffix probability when the prefix is evaluated alone and when it is evaluated with surrounding context. Prior work has largely measured extractable memorization from isolated prefixes, where the model is prompted only with a prefix and evaluated on whether it assigns high probability to the original continuation. Closest to our setting is work on privacy in retrieval-augmented generation, which studies whether contextual prompts can reveal sensitive information and finds that retrieval can reduce some forms of leakage. Our setting differs from this line of work in two ways. First, we study verbatim prefix-suffix exposure rather than entity-level disclosure, which lets us evaluate whether the model assigns probability mass to a specific target continuation. Second, our unit of analysis is the individual prefix-suffix pair: we track whether the same sample remains exposed, is suppressed, or becomes newly exposed when context is added. This sample-level view is necessary because aggregate leakage rates can decrease even when a robust subset of exposed samples remains extractable. 3 Problem Formulation Memorization in language models is commonly evaluated using a prefix–suffix setup: given a prefix from a training sequence, the model is tested on how likely it is to reproduce the corresponding suffix. Most such evaluations implicitly treat memorization as a property of the pair (p,s)(p,s): if the model assigns high probability to the suffix s after seeing the prefix p, then the sample is considered memorized. In this work, we ask whether this view remains valid when the same prefix is preceded by additional context, i.e. retrieved documents. Prefix–suffix samples. Let M be a language model with parameters θ. We consider a collection of samples =(pi,si)i=1n,D=\(p_i,s_i)\_i=1^n, (3) where pip_i is a prefix and sis_i is the corresponding target suffix from the same source sequence. The standard prefix-only extraction setting evaluates the model’s likelihood of sis_i conditioned only on pip_i. Contextualized extraction. We generalize this setting by introducing a context variable cic_i. The model is then evaluated on the likelihood of the same suffix sis_i conditioned on both the context and the prefix: Pθ(si∣ci,pi).P_θ(s_i c_i,p_i). (4) The context may be empty, semantically related to the prefix, or intentionally irrelevant. Importantly, the target prefix–suffix pair (pi,si)(p_i,s_i) remains fixed; only the preceding context changes. This lets us isolate whether memorization is stable under contextual conditioning. Context conditions. We study the extraction rate of suffix, when the prefix is preceded by three context conditions: C.1 Empty context (c=∅c= ): no context is preceded. C.2 Relevant context (c=relc=rel): the model receives the nearest retrieved document from the retrieval collection. C.3 Irrelevant context (c=irrc=irr): the model receives the farthest retrieved document, used as a low-relevance context. These context conditions allow us to ask whether memorization is a property of the prefix–suffix pair alone, or whether it depends on the surrounding text in which the prefix is evaluated. In particular, we use them to distinguish between memorized samples that remain extractable across contexts and samples whose exposure changes when contextual information is introduced. We refer to the former as the context-robust core (samples exposed under all context conditions) and to the latter as the context-sensitive boundary (samples whose membership in the exposed set Ec(τ)E_c(τ) depends on c, which we show in Section 5.2 are concentrated near the threshold τ). 4 Experiments We evaluate memorization under contextual conditioning: the same prefix–suffix sample is scored either in isolation or in the presence of an additional document context. This setup (more details in Appendix F) lets us isolate how context changes prefix-based extraction while keeping the target prefix and suffix fixed. Our evaluation has three components: a set of prefix–suffix samples used for memorization measurement, a document collection from which contextual documents are selected, and a retrieval procedure that determines which document is placed before the prefix. For each sample, we compare the empty-context condition against two contextualized conditions: semantically related and irrelevant contexts. All comparisons are paired at the item level: for a given sample, only the preceding context changes. Models. We evaluate three open-weight instruction-tuned models: Llama-3-8B-Instruct Grattafiori et al. (2024), Falcon-7B-Instruct Almazrouei et al. (2023), and OLMo-2-1124-7B-Instruct Walsh et al. (2025). The choice of these models is mainly motivated by the transparency of their pretraining data, which makes them suitable for controlled memorization experiments over known data sources. Memorization Dataset. We use prefixes sampled from the CommonCrawl subset of the Pile (Pile-C) Gao et al. (2020), which reflects general web text. Specifically, we evaluate on a subset of 50,000 prefixes drawn uniformly at random from Pile-C, without any filtering or pre-selection for memorization. This sampling strategy avoids selection bias toward highly extractable content and follows the standard practice in memorization measurement studies Hayes et al. (2025); Lasy et al. (2025), which quantify memorization over sequences sampled directly from the pretraining corpus rather than over curated sets of known-memorized examples. As a result, our exposure sets emerge naturally from the model’s behavior on a representative sample of training data, rather than from a regime in which extraction is easiest. We evaluate prefix lengths 32,64,128\32,64,128\ tokens with a fixed suffix length of 3232 tokens. Retriever and retrieval collection. We use E5 dense retriever Wang et al. (2024) for document retrieval. We use this retriever to choose the most relevant and the least relevant documents for each of the prefixes. The retrieval collection is a mixture of three subsets datasets of the Pile Gao et al. (2020): StackExchange, PubMed, and Wikipedia subsets were used; for a total collection of 67,655 documents. This mixture is designed to cover distinct text genres (community Q&A, biomedical abstracts, and encyclopedic prose) so that the retrieved context is not topically biased toward any single domain. Measurement framework We adopt the probabilistic extraction approach of Hayes et al. (2025). Rather than measuring memorization only through exact greedy continuation, we estimate how likely the model is to generate the target suffix when conditioned on a prefix, with or without additional context. This allows us to study memorization as a graded likelihood signal instead of a binary generation outcome. For each sample i, we split a text sequence into a prefix xiprex_i^pre and a suffix xisufx_i^suf. Given a context condition c, we measure the suffix log-probability: ℓic=logPθ(xisuf∣c,xipre), _i^c= P_θ (x_i^suf c,x_i^pre ), (5) where PθP_θ is the language model. The context c may be empty, semantically related, or irrelevant. We denote the corresponding log-probabilities as: ℓi∅,ℓirel,ℓiirr, _i , _i^rel, _i^irr, (6) where ℓi∅ _i is the suffix log-probability under the empty-context condition, ℓirel _i^rel under the nearest retrieved context, and ℓiirr _i^irr under the farthest retrieved context. To analyze whether a sample is extractable under a given condition, we define an exposed set at threshold τ: Ec(τ)=i:ℓic>τ.E_c(τ)=\i: _i^c>τ\. (7) This formulation lets us compare memorization across contexts at both the aggregate and sample levels. At the aggregate level, we compare the distributions of ℓic _i^c across context conditions. At the sample level, we analyze how membership in Ec(τ)E_c(τ) changes when context is added. In particular, we distinguish between samples that remain exposed across contexts, samples that are suppressed by context, and samples that become exposed only under contextual conditioning. 5 Results Figure 1: Suffix log-probability distributions by prefix length. Dashed lines show empty-context prompting, while solid lines show prompting with retrieved context. Across all models, longer prefixes substantially increase suffix likelihood. The gap between empty and contextualized prompting is largest for shorter prefixes and becomes smaller as prefix length increases, suggesting that longer prefixes make memorization more context-independent. Figure 2: Jaccard overlap between exposed sets at τ=−16τ=-16. Each cell reports the overlap between two exposure sets for a given model and prefix length. Exposure-set overlap increases sharply with prefix length across all models and context comparisons. This indicates that longer prefixes make the set of extractable samples more stable across contexts. The results support a consistent picture: retrieved context does not erase prefix-based memorization, but reshapes the exposure boundary. We first show that prefix length dominates contextual variation: longer prefixes make suffix likelihoods and exposed sets increasingly stable across contexts. We then show that this stability comes from a robust core of samples exposed under multiple contexts, while context-induced changes are concentrated near the exposure threshold. Finally, we separate the generic effect of adding context from the semantic effect of retrieving a related document. We report four observations, which we describe in the following subsections. Throughout this section, we use the measurement framework defined in Section 4. Recall that ℓi∅ _i , ℓirel _i^rel, and ℓiirr _i^irr denote the suffix log-probability of sample i under empty context, relevant retrieved context, and irrelevant retrieved context, respectively. We define the exposed set under condition c as Ec(τ)=i:ℓic>τ.E_c(τ)=\i: _i^c>τ\. (8) We use τ=−16τ=-16 in the main analysis22 2 Equivalent to 0.5 per-token drop of logprob for 32 tokens of suffix. and report threshold-sensitivity results in the appendix A. 5.1 Observation 1: Longer prefixes make memorization increasingly context-independent We first ask whether contextual conditioning changes prefix-based memorization measurements, and whether this effect depends on prefix length. Figure 1 shows the distribution of suffix log-probability scores for different prefix lengths under empty and retrieved-context prompting. Across all three models, increasing the prefix length from 32 to 64 and 128 tokens shifts the distribution toward higher suffix likelihoods. This confirms that prefix length is a dominant factor in extraction: longer prefixes provide a stronger conditioning signal and make the memorized continuation more likely. Importantly, the difference between empty-context and contextualized prompting becomes smaller as the prefix length increases. For short prefixes, contextual conditioning can noticeably shift the suffix likelihood distribution. For longer prefixes, the empty-context and contextualized curves become much closer. This suggests that longer prefixes make extraction increasingly context-independent: the model relies more on the prefix itself and less on the preceding context. We quantify this stability using Jaccard overlap between exposed sets: J(Ea,Eb)=|Ea∩Eb||Ea∪Eb|.J(E_a,E_b)= |E_a∩ E_b||E_a∪ E_b|. (9) Figure 2 shows that exposed-set overlap increases consistently with prefix length for all three models. This trend holds for empty versus relevant/irrelevant context and irrelevant versus relevant context. This provides a direct answer to our motivating question: memorization is not uniformly context-independent, but it becomes increasingly context-independent as the prefix becomes more informative. Short-prefix extraction is more context-sensitive, while long-prefix extraction is more stable across contextual conditions. Figure 3: Exposure-set decomposition at τ=−16τ=-16. Green bars show samples exposed under all three conditions. Purple bars show samples exposed without context but suppressed by context. Red and peach bars show samples that become exposed only under relevant or irrelevant context. Robust exposure dominates across models and grows substantially with prefix length, indicating a context-robust memorization core. 5.2 Observation 2: Strongly exposed samples persist across contexts, while weaker exposures are context-sensitive The previous result shows that exposure sets become more stable with longer prefixes. We now ask whether this stability comes from a shared robust core of memorized samples or whether the context completely reshuffles which samples are exposed. Figure 3 decomposes exposed samples into four groups: samples exposed under all three conditions, samples exposed without context but suppressed by context, samples enabled by relevant context, and samples enabled only by irrelevant context. The dominant category is robust exposure which consistently increases with the prefix length for all three models. This indicates that context does not erase memorization. A large fraction of exposed samples remain extractable under empty, relevant, and irrelevant contexts. The robust core becomes especially pronounced as the prefix length increases, consistent with the Jaccard-overlap results in Figure 2.33 3 See Table 2 in Appendix C for additional results. Model Prefix n Mean G Median G Mean S Median S % S<0S<0 Llama-3-8B-Instruct 32 318 0.52 -0.34 -3.35 -1.24 70.1 64 593 0.52 -0.06 -1.11 -0.12 59.7 128 858 0.05 -0.03 -0.32 -0.04 57.5 OLMo-2-1124-7B-Instruct 32 131 -0.42 -0.23 -2.05 -0.75 62.6 64 390 -0.32 -0.06 -0.04 0.06 47.2 128 645 0.14 -0.01 -0.30 0.01 47.3 Falcon-7B-Instruct 32 69 -0.94 -1.02 -1.70 -0.64 60.9 64 321 0.56 0.10 -0.23 -0.13 56.1 128 617 0.42 0.19 -0.16 -0.06 54.3 Table 1: Generic and semantic context effects for exposed or boundary samples. We report the number of boundary samples n, the generic context effect G=ℓirr−ℓ∅G= ^irr- , and the semantic effect S=ℓrel−ℓirrS= ^rel- ^irr. The semantic effect is generally negative, especially for shorter prefixes, indicating that relevant context tends to suppress extraction relative to irrelevant context near the exposure boundary. The generic effect is more model-dependent. Meanwhile, Figure 3 also shows that context-sensitive samples are non-negligible. Some samples are removed from the exposed set when context is added, while others become exposed only under contextual conditioning. To understand where these changes occur, we examine the distance of each sample from the empty-context exposure threshold: mi=|ℓi∅−τ|.m_i=| _i -τ|. (10) We then compute how often samples flip exposure status after context is added. We find that context-induced flips are highly concentrated near the exposure boundary (details in Figure 6 in Appendix B). Samples in the closest bin to the threshold have the highest flip rates, while samples far from the threshold almost never flip. This pattern appears across models and prefix lengths. As the prefix length increases, the relevant and irrelevant context curves also become more similar, especially near the boundary, indicating that longer prefixes reduce both the number of flips and the dependence of flips on the type of retrieved context. Together, these results suggest that memorization of strongly exposed samples remains stable across contexts, whereas less memorized samples are more sensitive to preceding context. 5.3 Observation 3: Context mostly suppresses, but context-enabled extraction remains security-relevant The exposure decomposition also allows us to revisit prior claims that retrieval or contextualization reduces memorization. Our results are consistent with this claim at the aggregate level: context more often suppresses exposure than enables it. In Figure 3, the suppressed category is generally larger than the context-enabled categories. However, the security interpretation is more subtle. A reduction in average exposure does not imply that memorization risk has been removed. First, many samples remain robustly exposed across all contexts, as shown by the green bars in Figure 3. Second, context-enabled exposure is consistently nonzero. That is, some samples that are not extractable under empty-context prompting become extractable when preceded by some context.44 4 Appendix H shows several examples of such cases. This matters for evaluating RAG and other context-augmented systems. Our results show that contextualization can act as a filter: it suppresses some marginal examples, preserves a robust core, and occasionally enables new exposure. Context changes the exposure boundary. This reconciles the observation that RAG can reduce aggregate memorization with the persistence of robustly extractable samples. 5.4 Observation 4: Relevant context adds a semantic effect, but generic context also matters Finally, we ask whether the effect of context is caused by semantic relevance or by the generic presence of additional context. We decompose the effect into two quantities: Gi G_i =ℓiirr−ℓi∅, = _i^irr- _i , (11) Si S_i =ℓirel−ℓiirr. = _i^rel- _i^irr. (12) Here, GiG_i captures the generic effect of adding irrelevant context, while SiS_i captures the additional semantic effect of replacing irrelevant context with relevant retrieved context. Table 1 (also shown as violin plots in Figure 11 in Appendix G.1) shows this decomposition for exposed samples (if a sample exceeds the threshold in any of the three prefix scenarios, it is included). The semantic effect S is often negative, especially for shorter prefixes. This suggests that relevant retrieved context often suppresses extraction relative to irrelevant context. The generic effect G, however, is more model-dependent. In some settings, irrelevant context suppresses extraction; in others, it has little effect or even increases likelihood. This indicates that contextual effects are not purely semantic. Adding context itself can perturb the conditional distribution, while semantic relevance can add an additional suppressive bias for boundary samples. Summary. Our results show that memorization is neither fully context-independent nor fully context-dependent. Instead, exposed memorization consists of a high-confidence core that persists across contexts and a weaker set of extractions that is more sensitive to surrounding text. Longer prefixes enlarge the persistent core, while context mainly affects lower-confidence samples. Thus, contextualization can reduce aggregate extraction by suppressing weaker extractions, but it does not eliminate robustly extractable memorization and can also enable new exposure. Item-level analysis of suffix log-probabilities under relevant and irrelevant context is available in Appendix G.2. 5.5 Ablation analysis Choice of memorization threshold The probabilistic extraction framework assigns each item a continuous suffix log-probability score rather than a binary memorized/non-memorized label. For threshold-based analyses, we use τ=−16τ=-16 as the exposure threshold. This choice is intentionally less strict than only counting very high-confidence extractions, allowing us to capture weaker memorized samples that may still be relevant for contextual exposure. Appendix A gives a security interpretation of this threshold in terms of adversary sampling budgets and shows that our conclusions are stable across thresholds spanning five orders of magnitude of extraction probability. Instruction-Framed Contextualization Figure 4: Jaccard overlap between exposed sets under relevant and irrelevant retrieved context in a fixed instruction-framed setting. Exposure-set overlap increases with prefix length across all models. We repeat part of our analysis under an instruction-framed prompt. We find that the Jaccard overlap between exposed sets under relevant and irrelevant retrieved contexts is already moderate for prefix length 32 and increases substantially with prefix length in all three models (Figure 4). This suggests that under instruction-framed prompting, retrieval choice still affects some boundary cases, but most exposed samples overlap between context types (see Appendix D for more quantitative results). Effect of Chat Templates In our experiments we consider working without the chat templates, since we try to mainly measure the effect of context preceding prefix. We notice that experimenting with a setting where we place the prefix inside a full chat template requires a somewhat different evaluation setup and introduces a different measurement regime: For the sequence (p,s)(p,s) that appeared in training, the model is no longer asked to continue p with s, but instead to generate s after an assistant tag. We find that this has a substantial effect: across all models and prefix lengths, adding the chat template substantially lowers measured memorization. The implication is that chat-template formatting makes the results harder to interpret by mixing the effect of preceding context with a different response-formatting regime. More details are in Appendix E. 6 Discussion Applicability to real-world systems Our setup deliberately isolates the effect of retrieved context on memorization by controlling the chat template, the retriever, and the collection. This isolation does not narrow the applicability of the findings: the core mechanism we study – adding a retrieved document to the prompt of an instruction-tuned model – is the same operation that real-world RAG systems perform. Additional components of production pipelines (prompt rewriting, multi-turn dialogue, output filtering, post-hoc safety classifiers) operate around this core operation rather than replacing it. The conclusion that memorized items remain extractable under retrieval, and that retrieval surfaces items the LLM alone does not, therefore carries over to systems built on top of the same retrieval-and-prompting pattern. Memorization as a proxy for leakage We measure targeted (prefix-suffix) memorization instead of leakage, because leakage itself does not afford a fair comparison between conditions: leakage can occur inadvertently during benign use, and it can be elicited by deliberate extraction attacks, so two systems with identical memorization profiles may yield very different leakage rates depending on the attack regime they are tested under. The probabilistic memorization metric factors this confound out: it captures the underlying tendency of the model to assign non-zero probabilities to a memorized continuation. This probabilistic memorization effect can lead to both inadvertent and adversarial leakage and is therefore a neutral measurement. Items whose memorization is preserved in both plain-LLM and RAG conditions therefore remain equally leakable in the operational sense, even if the specific leakage rate observed in any given deployment will depend on the specific attack model. 7 Conclusion We studied whether prefix-based memorization remains unchanged when the prefix is evaluated with surrounding context. Our results show that contextual conditioning does not simply remove memorization. Across three instruction-tuned models, many extractable samples remain exposed under related and irrelevant contexts, especially as prefix length increases. Context-induced changes are concentrated near the exposure threshold, while samples far from the threshold are largely stable. These findings nuance the claim that retrieval reduces memorization. Context can reduce aggregate extraction by suppressing boundary cases, but robustly exposed samples remain, and some samples become extractable only after context is added. Thus, context should be viewed less as a memorization shield and more as a filter that reshapes which samples are exposed. Future work should extend these analyses to additional datasets and experimental settings. Limitations Our experiments focus on open-weight models from three families, Llama-3, OLMO-2, and Falcon, at approximately the 10B-parameter scale, and use a randomly sampled subset of the CommonCrawl portion of Pile-C as prefix data. Although we do not evaluate the largest open-weight models or closed APIs, the models studied here are already large enough to exhibit substantial memorization behavior and to support meaningful analysis of extraction dynamics. Evaluating larger models could further clarify how absolute memorization rates change with scale, but larger models are not necessary for establishing our main qualitative finding: memorized continuations remain largely context-independent across diverse prompting conditions. Similarly, domains outside Pile-C, such as code, biomedical text, or copyrighted creative writing, may show different absolute memorization rates. Nevertheless, CommonCrawl-derived text is widely used in memorization studies, spans diverse sources and writing styles, and appears across the pre-training corpora of the model families we evaluate. This makes it a suitable basis for cross-model comparison, and the consistency of our findings across model families suggests that domain variation is unlikely to overturn the central conclusion. Our probabilistic extraction scores are computed with temperature T=1T=1 and top-p=1p=1. This setting preserves the model’s original probability distribution and avoids truncating suffix tokens, ensuring that every sample receives a finite and comparable memorization score. We also report retrieval-augmented results using the E5 dense retriever under both nearest and farthest retrieval, reflecting common real-world RAG settings. Preliminary experiments suggest that replacing E5 with other lexical or dense retrievers has only a limited effect on memorization, but a broader retriever sweep remains an important direction for future work. Other decoding or retrieval choices may affect surface-level generations, yet they are unlikely to change the core probabilistic comparison studied here. Finally, we evaluate multiple prefix lengths, 32,64,128\32,64,128\, with a fixed suffix length of 3232. Longer suffixes may lead to different quantitative patterns because token-level probability gaps can compound over longer continuations. A systematic sweep over suffix length is therefore left to future work. Ethical Considerations Our prefixes are drawn from a publicly available corpus (Pile-C); we do not target private or proprietary training data, closed API models, or deployed systems. Our quantitative results are reported as aggregate metrics. For qualitative analysis we include a limited number of illustrative examples in the appendix, consisting of prefix–suffix pairs and their retrieved contexts drawn from Pile-C; these are already public web text, and we do not release the full set of per-example extraction outputs. We release our code to support reproducibility. Our findings argue against treating RAG as a defense for memorization-based leakage and motivate defense efforts targeted at parametric memory itself. Acknowledgements This publication is part of the project LESSEN with project number NWA.1389.20.183 of the research program NWA ORC 2020/21 which is (partly) financed by the Dutch Research Council (NWO). We would like to thank KPN for hosting part of this research. Part of this work was carried out by Narjes during a research internship at KPN, and we thank Gianluigi Bardelloni for his supervision and support. References Almazrouei et al. (2023) E. Almazrouei, H. Alobeidli, A. Alshamsi, A. Cappelli, R. Cojocaru, M. Debbah, É. Goffinet, D. Hesslow, J. Launay, Q. Malartic, D. Mazzotta, B. Noune, B. Pannier, and G. Penedo The falcon series of open language models. arXiv preprint arXiv:2311.16867. Cited by: §4. Biderman et al. (2023) S. Biderman, U. Prashanth, L. Sutawika, H. Schoelkopf, Q. Anthony, S. Purohit, and E. Raff Emergent and predictable memorization in large language models. Advances in Neural Information Processing Systems 36, p. 28072–28090. Cited by: §2.1. Borgeaud et al. (2022) S. Borgeaud, A. Mensch, J. Hoffmann, T. Cai, E. Rutherford, K. Millican, G. van den Driessche, J. Lespiau, B. Damoc, A. Clark, D. de Las Casas, A. Guy, J. Menick, R. Ring, T. Hennigan, S. Huang, L. Maggiore, C. Jones, A. Cassirer, A. Brock, M. Paganini, G. Irving, O. Vinyals, S. Osindero, K. Simonyan, J. W. Rae, E. Elsen, and L. Sifre Improving language models by retrieving from trillions of tokens. In International conference on machine learning, p. 2206–2240. Cited by: §1. Carlini et al. (2023) N. Carlini, D. Ippolito, M. Jagielski, K. Lee, F. Tramer, and C. Zhang Quantifying memorization across neural language models. In The Eleventh International Conference on Learning Representations, Cited by: §1, §2.1. Carlini et al. (2019) N. Carlini, C. Liu, Ú. Erlingsson, J. Kos, and D. Song The secret sharer: evaluating and testing unintended memorization in neural networks. In 28th USENIX security symposium (USENIX security 19), p. 267–284. Cited by: §1. Carlini et al. (2021) N. Carlini, F. Tramèr, E. Wallace, M. Jagielski, A. Herbert-Voss, K. Lee, A. Roberts, T. B. Brown, D. Song, Ú. Erlingsson, A. Oprea, and C. Raffel Extracting training data from large language models. In 30th USENIX security symposium (USENIX Security 21), p. 2633–2650. Cited by: §1, §2.1. Freeman et al. (2024) J. Freeman, C. Rippe, E. Debenedetti, and M. Andriushchenko Exploring memorization and copyright violation in frontier llms: a study of the new york times v. openai 2023 lawsuit. In Neurips Safe Generative AI Workshop 2024, External Links: Link Cited by: §1. Gao et al. (2020) L. Gao, S. Biderman, S. Black, L. Golding, T. Hoppe, C. Foster, J. Phang, H. He, A. Thite, N. Nabeshima, S. Presser, and C. Leahy The Pile: an 800GB dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. Cited by: §4, §4. Gao et al. (2023) Y. Gao, Y. Xiong, X. Gao, K. Jia, J. Pan, Y. Bi, Y. Dai, J. Sun, M. Wang, and H. Wang Retrieval-augmented generation for large language models: a survey. arXiv preprint arXiv:2312.10997. Cited by: §1. Grattafiori et al. (2024) A. Grattafiori, A. Dubey, A. Jauhri, A. Pandey, A. Kadian, A. Al-Dahle, A. Letman, A. Mathur, A. Schelten, A. Vaughan, A. Yang, A. Fan, A. Goyal, A. Hartshorn, A. Yang, A. Mitra, A. Sravankumar, A. Korenev, A. Hinsvark, A. Rao, A. Zhang, A. Rodriguez, A. Gregerson, A. Spataru, B. Roziere, B. Biron, B. Tang, B. Chern, C. Caucheteux, C. Nayak, C. Bi, C. Marra, C. McConnell, C. Keller, C. Touret, C. Wu, C. Wong, C. C. Ferrer, C. Nikolaidis, D. Allonsius, D. Song, D. Pintz, D. Livshits, D. Wyatt, D. Esiobu, D. Choudhary, D. Mahajan, D. Garcia-Olano, D. Perino, D. Hupkes, E. Lakomkin, E. AlBadawy, E. Lobanova, E. Dinan, E. M. Smith, F. Radenovic, F. Guzmán, F. Zhang, G. Synnaeve, G. Lee, G. L. Anderson, G. Thattai, G. Nail, G. Mialon, G. Pang, G. Cucurell, H. Nguyen, H. Korevaar, H. Xu, H. Touvron, I. Zarov, I. A. Ibarra, I. Kloumann, I. Misra, I. Evtimov, J. Zhang, J. Copet, J. Lee, J. Geffert, J. Vranes, J. Park, J. Mahadeokar, J. Shah, J. van der Linde, J. Billock, J. Hong, J. Lee, J. Fu, J. Chi, J. Huang, J. Liu, J. Wang, J. Yu, J. Bitton, J. Spisak, J. Park, J. Rocca, J. Johnstun, J. Saxe, J. Jia, K. V. Alwala, K. Prasad, K. Upasani, K. Plawiak, K. Li, K. Heafield, K. Stone, K. El-Arini, K. Iyer, K. Malik, K. Chiu, K. Bhalla, K. Lakhotia, L. Rantala-Yeary, L. van der Maaten, L. Chen, L. Tan, L. Jenkins, L. Martin, L. Madaan, L. Malo, L. Blecher, L. Landzaat, L. de Oliveira, M. Muzzi, M. Pasupuleti, M. Singh, M. Paluri, M. Kardas, M. Tsimpoukelli, M. Oldham, M. Rita, M. Pavlova, M. Kambadur, M. Lewis, M. Si, M. K. Singh, M. Hassan, N. Goyal, N. Torabi, N. Bashlykov, N. Bogoychev, N. Chatterji, N. Zhang, O. Duchenne, O. Çelebi, P. Alrassy, P. Zhang, P. Li, P. Vasic, P. Weng, P. Bhargava, P. Dubal, P. Krishnan, P. S. Koura, P. Xu, Q. He, Q. Dong, R. Srinivasan, R. Ganapathy, R. Calderer, R. S. Cabral, R. Stojnic, R. Raileanu, R. Maheswari, R. Girdhar, R. Patel, R. Sauvestre, R. Polidoro, R. Sumbaly, R. Taylor, R. Silva, R. Hou, R. Wang, S. Hosseini, S. Chennabasappa, S. Singh, S. Bell, S. S. Kim, S. Edunov, S. Nie, S. Narang, S. Raparthy, S. Shen, S. Wan, S. Bhosale, S. Zhang, S. Vandenhende, S. Batra, S. Whitman, S. Sootla, S. Collot, S. Gururangan, S. Borodinsky, T. Herman, T. Fowler, T. Sheasha, T. Georgiou, T. Scialom, T. Speckbacher, T. Mihaylov, T. Xiao, U. Karn, V. Goswami, V. Gupta, V. Ramanathan, V. Kerkez, V. Gonguet, V. Do, V. Vogeti, V. Albiero, V. Petrovic, W. Chu, W. Xiong, W. Fu, W. Meers, X. Martinet, X. Wang, X. Wang, X. E. Tan, X. Xia, X. Xie, X. Jia, X. Wang, Y. Goldschlag, Y. Gaur, Y. Babaei, Y. Wen, Y. Song, Y. Zhang, Y. Li, Y. Mao, Z. D. Coudert, Z. Yan, Z. Chen, Z. Papakipos, A. Singh, A. Srivastava, A. Jain, A. Kelsey, A. Shajnfeld, A. Gangidi, A. Victoria, A. Goldstand, A. Menon, A. Sharma, A. Boesenberg, A. Baevski, A. Feinstein, A. Kallet, A. Sangani, A. Teo, A. Yunus, A. Lupu, A. Alvarado, A. Caples, A. Gu, A. Ho, A. Poulton, A. Ryan, A. Ramchandani, A. Dong, A. Franco, A. Goyal, A. Saraf, A. Chowdhury, A. Gabriel, A. Bharambe, A. Eisenman, A. Yazdan, B. James, B. Maurer, B. Leonhardi, B. Huang, B. Loyd, B. D. Paola, B. Paranjape, B. Liu, B. Wu, B. Ni, B. Hancock, B. Wasti, B. Spence, B. Stojkovic, B. Gamido, B. Montalvo, C. Parker, C. Burton, C. Mejia, C. Liu, C. Wang, C. Kim, C. Zhou, C. Hu, C. Chu, C. Cai, C. Tindal, C. Feichtenhofer, C. Gao, D. Civin, D. Beaty, D. Kreymer, D. Li, D. Adkins, D. Xu, D. Testuggine, D. David, D. Parikh, D. Liskovich, D. Foss, D. Wang, D. Le, D. Holland, E. Dowling, E. Jamil, E. Montgomery, E. Presani, E. Hahn, E. Wood, E. Le, E. Brinkman, E. Arcaute, E. Dunbar, E. Smothers, F. Sun, F. Kreuk, F. Tian, F. Kokkinos, F. Ozgenel, F. Caggioni, F. Kanayet, F. Seide, G. M. Florez, G. Schwarz, G. Badeer, G. Swee, G. Halpern, G. Herman, G. Sizov, Guangyi, Zhang, G. Lakshminarayanan, H. Inan, H. Shojanazeri, H. Zou, H. Wang, H. Zha, H. Habeeb, H. Rudolph, H. Suk, H. Aspegren, H. Goldman, H. Zhan, I. Damlaj, I. Molybog, I. Tufanov, I. Leontiadis, I. Veliche, I. Gat, J. Weissman, J. Geboski, J. Kohli, J. Lam, J. Asher, J. Gaya, J. Marcus, J. Tang, J. Chan, J. Zhen, J. Reizenstein, J. Teboul, J. Zhong, J. Jin, J. Yang, J. Cummings, J. Carvill, J. Shepard, J. McPhie, J. Torres, J. Ginsburg, J. Wang, K. Wu, K. H. U, K. Saxena, K. Khandelwal, K. Zand, K. Matosich, K. Veeraraghavan, K. Michelena, K. Li, K. Jagadeesh, K. Huang, K. Chawla, K. Huang, L. Chen, L. Garg, L. A, L. Silva, L. Bell, L. Zhang, L. Guo, L. Yu, L. Moshkovich, L. Wehrstedt, M. Khabsa, M. Avalani, M. Bhatt, M. Mankus, M. Hasson, M. Lennie, M. Reso, M. Groshev, M. Naumov, M. Lathi, M. Keneally, M. Liu, M. L. Seltzer, M. Valko, M. Restrepo, M. Patel, M. Vyatskov, M. Samvelyan, M. Clark, M. Macey, M. Wang, M. J. Hermoso, M. Metanat, M. Rastegari, M. Bansal, N. Santhanam, N. Parks, N. White, N. Bawa, N. Singhal, N. Egebo, N. Usunier, N. Mehta, N. P. Laptev, N. Dong, N. Cheng, O. Chernoguz, O. Hart, O. Salpekar, O. Kalinli, P. Kent, P. Parekh, P. Saab, P. Balaji, P. Rittner, P. Bontrager, P. Roux, P. Dollar, P. Zvyagina, P. Ratanchandani, P. Yuvraj, Q. Liang, R. Alao, R. Rodriguez, R. Ayub, R. Murthy, R. Nayani, R. Mitra, R. Parthasarathy, R. Li, R. Hogan, R. Battey, R. Wang, R. Howes, R. Rinott, S. Mehta, S. Siby, S. J. Bondu, S. Datta, S. Chugh, S. Hunt, S. Dhillon, S. Sidorov, S. Pan, S. Mahajan, S. Verma, S. Yamamoto, S. Ramaswamy, S. Lindsay, S. Lindsay, S. Feng, S. Lin, S. C. Zha, S. Patil, S. Shankar, S. Zhang, S. Zhang, S. Wang, S. Agarwal, S. Sajuyigbe, S. Chintala, S. Max, S. Chen, S. Kehoe, S. Satterfield, S. Govindaprasad, S. Gupta, S. Deng, S. Cho, S. Virk, S. Subramanian, S. Choudhury, S. Goldman, T. Remez, T. Glaser, T. Best, T. Koehler, T. Robinson, T. Li, T. Zhang, T. Matthews, T. Chou, T. Shaked, V. Vontimitta, V. Ajayi, V. Montanez, V. Mohan, V. S. Kumar, V. Mangla, V. Ionescu, V. Poenaru, V. T. Mihailescu, V. Ivanov, W. Li, W. Wang, W. Jiang, W. Bouaziz, W. Constable, X. Tang, X. Wu, X. Wang, X. Wu, X. Gao, Y. Kleinman, Y. Chen, Y. Hu, Y. Jia, Y. Qi, Y. Li, Y. Zhang, Y. Zhang, Y. Adi, Y. Nam, Yu, Wang, Y. Zhao, Y. Hao, Y. Qian, Y. Li, Y. He, Z. Rait, Z. DeVito, Z. Rosnbrick, Z. Wen, Z. Yang, Z. Zhao, and Z. Ma The llama 3 herd of models. arXiv preprint arXiv:2407.21783. Cited by: §4. Hartmann et al. (2023) V. Hartmann, A. Suri, V. Bindschaedler, D. Evans, S. Tople, and R. West Sok: memorization in general-purpose large language models. arXiv preprint arXiv:2310.18362. Cited by: §1. Hayes et al. (2025) J. Hayes, M. Swanberg, H. Chaudhari, I. Yona, I. Shumailov, M. Nasr, C. A. Choquette-Choo, K. Lee, and A. F. Cooper Measuring memorization in language models via probabilistic extraction. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), L. Chiruzzo, A. Ritter, and L. Wang (Eds.), Albuquerque, New Mexico, p. 9266–9291. External Links: Link, Document, ISBN 979-8-89176-189-6 Cited by: Appendix A, §2.2, §4, §4. Henderson et al. (2023) P. Henderson, X. Li, D. Jurafsky, T. Hashimoto, M. A. Lemley, and P. Liang Foundation models and fair use. Journal of Machine Learning Research 24 (400), p. 1–79. Cited by: §1. Kim et al. (2023) S. Kim, S. Yun, H. Lee, M. Gubri, S. Yoon, and S. J. Oh Propile: probing privacy leakage in large language models. Advances in Neural Information Processing Systems 36, p. 20750–20762. Cited by: §2.1. Kiyomaru et al. (2024) H. Kiyomaru, I. Sugiura, D. Kawahara, and S. Kurohashi A comprehensive analysis of memorization in large language models. In Proceedings of the 17th International Natural Language Generation Conference, S. Mahamood, N. L. Minh, and D. Ippolito (Eds.), Tokyo, Japan, p. 584–596. External Links: Link, Document Cited by: §2.1. Lasy et al. (2025) I. Lasy, P. Knees, and S. Woltran Understanding verbatim memorization in LLMs through circuit discovery. In Proceedings of the First Workshop on Large Language Model Memorization (L2M2), R. Jia, E. Wallace, Y. Huang, T. Pimentel, P. Maini, V. Dankers, J. Wei, and P. Lesci (Eds.), Vienna, Austria, p. 83–94. External Links: Link, Document, ISBN 979-8-89176-278-7 Cited by: §4. Lewis et al. (2020) P. Lewis, E. Perez, A. Piktus, F. Petroni, V. Karpukhin, N. Goyal, H. Küttler, M. Lewis, W. Yih, T. Rocktäschel, S. Riedel, and D. Kiela Retrieval-augmented generation for knowledge-intensive nlp tasks. Advances in neural information processing systems 33, p. 9459–9474. Cited by: §1. Mueller et al. (2024) F. B. Mueller, R. Görge, A. K. Bernzen, J. C. Pirk, and M. Poretschkin Llms and memorization: on quality and specificity of copyright compliance. In Proceedings of the AAAI/ACM Conference on AI, Ethics, and Society, Vol. 7, p. 984–996. Cited by: §1. Nasr et al. (2025) M. Nasr, J. Rando, N. Carlini, J. Hayase, M. Jagielski, A. F. Cooper, D. Ippolito, C. A. Choquette-Choo, F. Tramèr, and K. Lee Scalable extraction of training data from aligned, production language models. In The Thirteenth International Conference on Learning Representations, External Links: Link Cited by: Appendix E, §2.1. Satvaty et al. (2024) A. Satvaty, S. Verberne, and F. Turkmen Undesirable memorization in large language models: a survey. arXiv preprint arXiv:2410.02650. Cited by: §1, §2.2. Satvaty et al. (2025) A. Satvaty, A. Visman, D. Seidel, S. Verberne, and F. Turkmen Memorization is language-sensitive: analyzing memorization and inference risks of LLMs in a multilingual setting. In Proceedings of the First Workshop on Large Language Model Memorization (L2M2), R. Jia, E. Wallace, Y. Huang, T. Pimentel, P. Maini, V. Dankers, J. Wei, and P. Lesci (Eds.), Vienna, Austria, p. 106–126. External Links: Link, Document, ISBN 979-8-89176-278-7 Cited by: §2.1. Tirumala et al. (2022) K. Tirumala, A. H. Markosyan, L. Zettlemoyer, and A. Aghajanyan Memorization without overfitting: analyzing the training dynamics of large language models. In Advances in Neural Information Processing Systems, A. H. Oh, A. Agarwal, D. Belgrave, and K. Cho (Eds.), Cited by: §1, §2.1. Walsh et al. (2025) E. P. Walsh, L. Soldaini, D. Groeneveld, K. Lo, S. Arora, A. Bhagia, Y. Gu, S. Huang, M. Jordan, N. Lambert, D. Schwenk, O. Tafjord, T. Anderson, D. Atkinson, F. Brahman, C. Clark, P. Dasigi, N. Dziri, A. Ettinger, M. Guerquin, D. Heineman, H. Ivison, P. W. Koh, J. Liu, S. Malik, W. Merrill, L. J. V. Miranda, J. Morrison, T. Murray, C. Nam, J. Poznanski, V. Pyatkin, A. Rangapur, M. Schmitz, S. Skjonsberg, D. Wadden, C. Wilhelm, M. Wilson, L. Zettlemoyer, A. Farhadi, N. A. Smith, and H. Hajishirzi 2 OLMo 2 furious (COLM’s version). In Second Conference on Language Modeling, External Links: Link Cited by: §4. Wang et al. (2024) L. Wang, N. Yang, X. Huang, L. Yang, R. Majumder, and F. Wei Multilingual e5 text embeddings: a technical report. arXiv preprint arXiv:2402.05672. Cited by: §4. Wei et al. (2022) J. Wei, M. Bosma, V. Zhao, K. Guu, A. W. Yu, B. Lester, N. Du, A. M. Dai, and Q. V. Le Finetuned language models are zero-shot learners. In International Conference on Learning Representations, External Links: Link Cited by: §1. Zeng et al. (2024) S. Zeng, J. Zhang, P. He, Y. Liu, Y. Xing, H. Xu, J. Ren, Y. Chang, S. Wang, D. Yin, and J. Tang The good and the bad: exploring privacy issues in retrieval-augmented generation (RAG). In Findings of the Association for Computational Linguistics: ACL 2024, L. Ku, A. Martins, and V. Srikumar (Eds.), Bangkok, Thailand, p. 4505–4524. External Links: Link, Document Cited by: §1. Appendix A Threshold Choice and Sensitivity Analysis Our main analysis defines exposed samples using a fixed threshold τ=−16τ=-16. For a 32-token suffix, this corresponds to an average per-token log-probability of −0.5-0.5, i.e., a mean per-token probability of e−0.5≈0.61e^-0.5≈ 0.61, and a total suffix probability of e−16≈1.1×10−7e^-16≈ 1.1× 10^-7. In the discoverable-extraction sense of Hayes et al. (2025), an adversary sampling at temperature 1 recovers the exact suffix of a sample at this threshold with probability ≥0.5≥ 0.5 within on the order of 10610^6–10710^7 generation attempts—a realistic budget for automated extraction attacks. We deliberately choose this permissive operating point rather than one that counts only near-deterministic extractions, because our analysis targets the exposure boundary, and a stricter threshold would truncate precisely the marginal region where context-induced changes concentrate. To test whether our conclusions depend on this particular cutoff, we repeat the exposure-set decomposition across multiple thresholds. Specifically, for each context condition c, we define the exposed set as Ec(τ)=i:ℓic>τ,E_c(τ)=\i: _i^c>τ\, (13) and vary τ over the range [−22,−10][-22,-10], corresponding to adversary budgets from ∼109 10^9 down to ∼104 10^4 generation attempts, i.e., from extremely powerful to cheap extraction attacks. Figure 5 shows that the qualitative structure of the exposed sets is stable across thresholds. As expected, stricter thresholds reduce the absolute fraction of exposed samples. However, the relative pattern remains consistent: robust exposure is the dominant category, especially for longer prefixes, while suppressed and context-enabled samples remain smaller but nonzero. This indicates that our main conclusions are not artifacts of a single threshold choice. Importantly, the context-enabled categories persist across thresholds. Although they are much smaller than the robust exposed set, their presence shows that context-induced exposure is not caused solely by an arbitrary cutoff. Figure 5: Threshold sensitivity of the exposure-set decomposition. We vary the exposure threshold τ and report the percentage of samples in each category. Robust exposure remains the dominant category across thresholds, especially for longer prefixes. Suppressed and context-enabled samples remain smaller but persist across threshold choices, showing that the exposure-set decomposition is not an artifact of a single cutoff. Appendix B Flip Analysis In the main text, we show that contextual conditioning can change whether a sample is classified as exposed. Here, we further analyze where these flips occur relative to the exposure threshold. For each sample, we compute its distance from the empty-context threshold, mi=|ℓi∅−τ|,m_i=| _i -τ|, (14) and measure how often the sample changes exposure status after adding context. Figure 6 shows that context-induced flips are concentrated near the threshold. Samples close to the boundary have the highest flip rates, while samples far from the boundary almost never flip. This supports the interpretation that context primarily affects marginal samples rather than strongly exposed or strongly non-exposed samples. Figure 6: Context-induced flip rate as a function of distance from the empty-context exposure threshold. The flip rate is highest for samples close to the threshold and approaches zero for samples far from the threshold. This indicates that context-induced changes are boundary-local: context primarily affects marginal samples rather than robustly exposed or strongly non-exposed samples. Appendix C Exposure Set Analysis In addition to the exposure-set decomposition in the main text, we report the pairwise overlap between exposed sets under different context conditions. For each condition c, we define the exposed set as Ec(τ)=i:ℓic>τ.E_c(τ)=\i: _i^c>τ\. (15) We then compute the Jaccard overlap between exposed sets, J(Ea,Eb)=|Ea∩Eb||Ea∪Eb|.J(E_a,E_b)= |E_a∩ E_b||E_a∪ E_b|. (16) Table 2 reports the overlap between empty, relevant, and irrelevant context at τ=−16τ=-16. Across all models, the overlap increases substantially with prefix length. This supports the main result that longer prefixes make the set of exposed samples more consistent across contextual conditions. Model Prefix J(E∅,Erel)J(E_ ,E_rel) J(E∅,Eirr)J(E_ ,E_irr) J(Eirr,Erel)J(E_irr,E_rel) |E∅||E_ | |Erel||E_rel| |Eirr||E_irr| Meta-Llama-3-8B-Instruct 32 0.715 0.721 0.689 116 95 106 64 0.880 0.901 0.890 384 387 384 128 0.912 0.933 0.932 659 672 665 OLMo-2-1124-7B-Instruct 32 0.677 0.637 0.656 87 74 80 64 0.840 0.867 0.854 325 317 319 128 0.912 0.911 0.909 582 592 580 Falcon-7B-Instruct 32 0.725 0.725 0.796 44 44 44 64 0.905 0.898 0.951 285 279 275 128 0.952 0.950 0.967 571 567 568 Table 2: Exposure-set Jaccard overlap at τ=−16τ=-16. E∅E_ , ErelE_rel, and EirrE_irr denote the exposed sets under empty, relevant, and irrelevant context, respectively. Across all models, exposed-set overlap increases with prefix length, indicating that longer prefixes make exposure less sensitive to contextual variation. Appendix D Instruction-Framed Contextualization As an additional robustness check, we repeat part of our analysis under a fixed instruction-framed prompt. Because this setting changes both the prompting format and the role structure, we do not interpret it as an isolated measurement of system-prompt effects. Instead, we use it to test whether our main conclusions about exposure stability continue to hold in a chat-style setting. Figure 4 shows the Jaccard overlap between exposed sets under relevant and irrelevant retrieved contexts, with the instruction framing held fixed. The overlap is already moderate for prefix length 32 and increases substantially with prefix length across all three models. For example, the overlap rises from 0.69 to 0.93 for Llama, from 0.69 to 0.92 for OLMo, and from 0.74 to 0.96 for Falcon as the prefix length increases from 32 to 128 (Table 3). This mirrors the main-paper trend: longer prefixes make exposure increasingly stable across contextual conditions. Figure 7 further decomposes the exposed sets into samples shared by relevant and irrelevant retrieval, samples exposed only under relevant retrieval, and samples exposed only under irrelevant retrieval. In all models, the shared category dominates and grows substantially with prefix length, while the relevant-only and irrelevant-only categories remain comparatively small. This suggests that under instruction-framed prompting, retrieval choice still affects some boundary cases, but most exposed samples belong to a shared robust core. The corresponding semantic effect S under grounding instructions is shown in Figure 8 and summarized in Table 4. Overall, these results reinforce the main conclusion of the paper: memorization is characterized by a robust exposed core and a context-sensitive boundary, and the robust core becomes increasingly dominant as the prefix length grows. Model Prefix J(Erel,Eirr)J(E_rel,E_irr) |Erel||E_rel| |Eirr||E_irr| ntotaln_total Meta-Llama-3-8B-Instruct 32 0.694 95 115 49393 64 0.882 387 394 49393 128 0.927 672 673 49292 OLMo-2-1124-7B-Instruct 32 0.693 74 75 49393 64 0.845 317 325 49393 128 0.920 592 585 49306 Falcon-7B-Instruct 32 0.740 44 43 49393 64 0.911 279 277 49393 128 0.958 567 561 49392 Table 3: Jaccard overlap between exposed sets under relevant and irrelevant context in the instruction-framed setting. ErelE_rel and EirrE_irr denote the exposed sets under relevant and irrelevant retrieved contexts, respectively. The overlap increases with prefix length across all models, indicating that longer prefixes make exposure increasingly stable across retrieved-context choices. Figure 7: Exposure decomposition for relevant versus irrelevant retrieval under a fixed instruction-framed setting. Most exposed samples are shared between the two retrieval conditions, and the shared robust set grows with prefix length. Figure 8: Semantic with system prompt Model Prefix ntotaln_total nboundaryn_boundary Mean S Median S % S<0S<0 Meta-Llama-3-8B-Instruct 32 49393 124 -0.998 -0.241 56.5 64 49393 415 -0.496 -0.108 54.9 128 49292 698 -0.115 0.005 49.6 OLMo-2-1124-7B-Instruct 32 49393 88 0.880 -0.215 52.3 64 49393 348 -0.303 -0.012 51.1 128 49306 613 -0.063 -0.006 50.9 Falcon-7B-Instruct 32 49393 50 -0.463 -0.199 50.0 64 49393 291 0.062 -0.089 57.0 128 49392 576 0.037 0.016 48.3 Table 4: Semantic context effect on exposed samples when adding grounding instructions. S denotes the change in suffix log-probability when replacing the comparison context with the relevant retrieved context. The effect is generally inclined towards negative, but less pronounced than the one without the grounding instructions (table 1). Appendix E Effect of Chat Templates Our main experiments focus on the standard prefix–suffix extraction setting, which has been the dominant setup in prior memorization studies. In this setting, the suffix is scored as a direct continuation of the prefix. A natural alternative would be to place the prefix inside a full chat template. However, this introduces a different measurement regime. As illustrated in Figure 9, adding a chat template inserts an assistant-role boundary between the prefix and the scored suffix. As a result, for the sequence (p,s)(p,s) that appeared in training, the model is no longer asked to continue p with s, but instead to generate s after an assistant tag. In other words, you would measure the extraction rate of (p,<assistant>,s)(p, <assistant>,s), which is a different value. This effect has also been noted by Nasr et al. (2025). Figure 10 shows that this effect is large: across all models and prefix lengths, adding the chat template substantially lowers measured memorization. Because this shift is much larger than the contextual effects we study in the main paper, including chat-template formatting would make the results harder to interpret by mixing the effect of preceding context with a different response-formatting regime. For this reason, the main paper isolates the effect of preceding context and does not include chat-template structure in the core analysis. Plain continuation [prefix] → score [suffix] Chat template <|user|> [prefix] <|assistant|> → score [suffix] Figure 9: Illustration of plain prefix–suffix scoring versus chat-template scoring. In the plain setting, the suffix is scored as a direct continuation of the prefix. In the chat-template setting, an assistant-role boundary is inserted between the prefix and suffix, changing the continuation being measured. Figure 10: Effect of chat-template formatting on measured memorization under empty context. Solid lines show plain prefix–suffix scoring, while dashed lines show chat-template scoring. The large drop under chat-template scoring shows that template formatting has a strong effect on the measurement. Appendix F Detailed Experimental Setup This appendix lists implementation details for reproducibility. • Sampling: top-p=1.0p=1.0, temperature T=1.0T=1.0 (scoring setting; no actual sampling required since we compute log-probabilities directly). • Log-probability thresholding: across all plots the threshold is −16-16. • Suffix length: 32 tokens. • Context window: model maximum. Appendix G Additional Visualizations G.1 Semantic Relevance Effect Figure 11 shows the distributions of G and S for exposed samples (related to section 5.4). The generic effect is model-dependent: irrelevant context can suppress extraction in some settings, while having little effect or increasing extraction in others. In contrast, the semantic relevance effect is often negative, especially for shorter prefixes. This suggests that related context can add a suppressive effect, although this effect weakens as prefix length increases. Figure 11: Semantic versus generic context effects for exposed samples. Blue distributions show G=ℓirr−ℓ∅G= ^irr- , the effect of adding irrelevant context. Red distributions show S=ℓrel−ℓirrS= ^rel- ^irr, the additional effect of replacing irrelevant context with relevant context. Relevant context is often suppressive, especially for shorter prefixes, but generic contextual perturbation also contributes and is more model-dependent. G.2 Relevant versus Irrelevant Context Figure 12 compares suffix log-probabilities under relevant and irrelevant context at the item level. Each point corresponds to one prefix–suffix sample. Points near the diagonal indicate samples whose extraction likelihood is similar under both context types, while deviations from the diagonal indicate samples that are more sensitive to the choice of retrieved context. Figure 12: Item-level scatter plot of suffix log-probabilities under relevant and irrelevant context. The x-axis shows log-probability under relevant retrieved context, and the y-axis shows log-probability under irrelevant context. Points near the diagonal indicate similar extraction likelihood under both context types; deviations from the diagonal indicate context-sensitive samples. Appendix H Examples of Recalled and Forgotten Samples in Presence of Context Tables 5 and 9 show several examples of items that become un-extractable (forgotten) and extractable (recalled) in the presence of the retrieved context. Table 5: Forgotten items: sequences that are extracted verbatim by the model without retrieved context, but no longer under context. We show one example per model, restricted to items whose log-probability gap satisfies |logpctx−logpno-ctx|≥5| p_ctx- p_no-ctx|≥ 5 and where exactly one of the two log-probabilities falls below −16-16, so that the pair straddles the extraction threshold. Example 1 Model tiiuae/falcon-7b-instruct Prefix len 32 tokens (suffix len: 32) Log-probs logpno-ctx=−8.680 p_no-ctx=-8.680 logpctx=−25.069 p_ctx=-25.069 Δ=−16.389 =-16.389 Prefix Cambridge, ID Fertilizers YP - The Real Yellow PagesSM - helps you find the right local businesses to meet your specific needs. Search Suffix results are sorted by a combination of factors to give you a set of choices in response to your search criteria. These factors are similar to those you might use to Context Q: cms for software project hosting I want to make a site where I present my own open source software project. It should look like http://w.schillmania.com/projects/soundmanager2/ for example. Are there any CMS (PHP) with this kind of functionality preconfigured? What I need is code highlighting, code documentation, download statistic, bug tracker, discussion forum, donation button etc. I think if I take a standard CMS like wordpress or drupal I’l spend ages to find the right modules and setup the whole thing. I will also accept answers that recommend a useful drupal distribution or a list of wordpress plugins.. A: Have you tried looking at Ohloh or the CMS Matrix? Some of the CMS’s descriptions in there haven’t been updated in a long time though. (I would have posted it as a comment but I can’t just yet and I tought it would still be interesting for you to know.) Table 6: Forgotten items (continued). Example 2 Model meta-llama/Meta-Llama-3-8B-Instruct Prefix len 32 tokens (suffix len: 32) Log-probs logpno-ctx=−2.858 p_no-ctx=-2.858 logpctx=−49.586 p_ctx=-49.586 Δ=−46.728 =-46.728 Prefix Tucson craigslist adult - tour cheap About me: Hello My name is Melly. These factors are similar to those you might use to determine which business Suffix to select from a local Yellow Pages directory, including proximity to where you are searching, expertise in the specific services or products you need, and comprehensive business information to Context Q: Plotting More than 2 Factors Suppose I ran a factor analysis & got 5 relevant factors. Now, I want to graphically represent the loading of these factors on the variables. Can anybody please tell me how to do it. I can do using 2 factors. But can’t able to do when number of factors are more than 2. The 2 factor plotting is given in "Modern Applied Statistics with S", Fig 11.13. I want to create similar graph but with more than 2 factors. Please find the snap of the Fig mentioned above: X & y axes are the 2 factors. Regards, Ari A: Beware: not the answer you are looking for and might be incorrect also, this is my subjective thought. I think you run into the problem of sketching several dimensions on a two dimension screen/paper. I would say there is no sense in plotting more factors’ or PCs’ loadings, but if you really insist: display the first two (based on eigenvalues) or create only 2 factors. Or you could reduce dimension by other methods also (e.g. MDS). Displaying 3 factors’ loadings in a 3 dimensional graph would be just hardly clear, not to think about more factors. UPDATE: I had a dream about trying to be more ontopic :) Table 7: Forgotten items (continued). Example 3 Model allenai/OLMo-2-0425-1B-Instruct Prefix len 64 tokens (suffix len: 32) Log-probs logpno-ctx=−6.081 p_no-ctx=-6.081 logpctx=−17.202 p_ctx=-17.202 Δ=−11.121 =-11.121 Prefix Menu Notice: Trying to get property of non-object in /var/w/ipo-book.com/public_html/wp-content/themes/ipo/functions.php on line 600 Notice: Trying to get property of non-object in /var/w/ipo-book.com/public_html/wp-content/themes/ipo/functions.php on line 602 Suffix Notice: Trying to get property of non-object in /var/w/ipo-book.com/public_html/wp-content/themes/ipo/functions.php on line 604 Notice Context Q: ?????? segmentation error ????? ?????????? ????? ?? ??????? ? ????????? ??????? ?????? segmentation error, ??? ?? ????????? ??? ??? ????????? ? BookContact.txt Sergey_Kotomkin:154:8(777)777-77-77: Maks_Loxin:84:8(952)748-41-70: Andrey_Serov:14:8(952)747-42-70: Aleks_Sagin:14:8(952)509-08-07: Dimon_Wanow:17:8(785)124-47-45: ??? ??? #include <stdio.h> //??? printf, fprintf, fopen, fclose #include <string.h> #include <stdlib.h> #define FNAME "BookContact.txt" //????????? ???? ??? ???????? void startMenu() //? ??? ???????? ????????? ???? printf("—Main Menu— "); printf("This is the phone book of the company ’YptRemont’ "); Table 8: Forgotten items (continued). Example 4 Model allenai/OLMo-2-1124-7B-Instruct Prefix len 32 tokens (suffix len: 32) Log-probs logpno-ctx=−7.669 p_no-ctx=-7.669 logpctx=−20.781 p_ctx=-20.781 Δ=−13.112 =-13.112 Prefix Personal Income and Outlays: September 2009 Personal income decreased $0.1 billion, or less than 0.1 percent, and disposable personal Suffix income (DPI) decreased $0.2 billion, or less than 0.1 percent, in September, according to the Bureau of Economic Analysis. Context Since the completion of the Trans @-@ Alaska Pipeline System in 1977 , the government of the state of Alaska has been reliant on taxes paid by oil producers and shippers . Prior to 1976 , Alaska ’s personal income tax rate was 14 @.@ 5 percent — the highest in the United States . The gross state product was $ 8 billion , and Alaskans earned $ 5 billion in personal income . Thirty years after the pipeline began operating , the state had no personal income tax , the gross state product was $ 39 billion , and Alaskans earned $ 25 billion in personal income . Alaska moved from the most heavily taxed state to the most tax @-@ free state . Table 9: Recalled items: sequences that are not extracted by the model alone, but become extractable once retrieved context is prepended. We show one example per model, restricted to items whose log-probability gap satisfies |logpctx−logpno-ctx|≥5| p_ctx- p_no-ctx|≥ 5 and where exactly one of the two log-probabilities falls below −16-16, so that the pair straddles the extraction threshold. Example 1 Model tiiuae/falcon-7b-instruct Prefix len 64 tokens (suffix len: 32) Log-probs logpno-ctx=−20.885 p_no-ctx=-20.885 logpctx=−5.609 p_ctx=-5.609 Δ=+15.277 =+15.277 Prefix Game seems to be working just fine, but trainer is not. I see that there might have been some patches? How do i check my game version? Trainer wont work without the right version haha so need to check wich version i need to get my game too. Game seems to be working just fine, Suffix but trainer is not. I see that there might have been some patches? How do i check my game version? Trainer wont work without the right version haha so Context Special abilities and actions are controlled by the Force Gauge , an energy meter which determines how long special skills can be active . Once drained , the skill cannot be activated until the meter refills , which it does during battles . Four skills can be equipped for each character . After battles , characters earn Enhancement Points ( EP ) , which can be used to strengthen equipped armor , weapons and accessories , instead of using Gald ( the in @-@ game currency ) as with previous Tales titles . EP can be transferred between weapons . An optional battle arena was added in the PSP version of the game with both single @-@ player and multiplayer matches : in both modes , players need to defeat an enemy team ( either AI @-@ controlled enemies or an opposing team ) within a time limit . Table 10: Recalled items (continued). Example 2 Model meta-llama/Meta-Llama-3-8B-Instruct Prefix len 32 tokens (suffix len: 32) Log-probs logpno-ctx=−21.231 p_no-ctx=-21.231 logpctx=−8.811 p_ctx=-8.811 Δ=+12.420 =+12.420 Prefix Thou still unravish’ d bride of quietness, Thou foster-child of silence and slow time, Sylvan historian, who canst thus Suffix express A flowery tale more sweetly than our rhyme: What leaf-fring’d legend haunts about thy shape Of deities or mortals, Context Q: Word construction like "philanthropist" but regarding silence instead of humankind? What would be the Latin/English term for somebody who loves silence? Featuring phil(e) as pre- or suffix. Like … silenciophile? That maybe makes "some sense" - but I’ve no idea if it is correct or if there would be a canonical neologism. A: Latin freely adopted Greek words, including compounds, but in general did not freely create compounds with both Latin and Greek elements, like the modern creation "sociology", and similarly with your suggested form "silenciophile" (better "silentiophile"). If you want to create a compound with "-phile", then "hesychophile" would be better, consisting of Greek elements only. — You made me very low @-@ spirited last night , by your manner of talking — You are my only friend — the only person I am intimate with . — I never had a father , or a brother — you have been both to me , ever since I knew you — yet I have sometimes been very petulant . — I have been thinking of those instances of ill @-@ humour and quickness , and they appeared like crimes . Yours sincerely , Mary . — being desirous of rendering it easy for the future , have made it my stud […] Table 11: Recalled items (continued). Example 3 Model allenai/OLMo-2-0425-1B-Instruct Prefix len 128 tokens (suffix len: 32) Log-probs logpno-ctx=−16.620 p_no-ctx=-16.620 logpctx=−8.394 p_ctx=-8.394 Δ=+8.226 =+8.226 Prefix Model Method - Questions and Answers Question posted by Yvonne from Sri Lanka: Grade/Level: 4th Question solved by Model Method: 900 people attended a party. 1/4 of the males and 1/7 of the females were children. the number of male and female adults were the same. how many women were there at the party? Answer: Step 1: This question involves the Equal Concept. Since 1/4 of the males were children, then 3/4 of the males must be adults. We draw 3 boxes to represent the total number of male adults and 1 Suffix box to represent the total number of male children. Step 2: Since 1/7 of the females were children, then 6/7 of the Context Q: Inclusion-Exclusion Principle for basic combinatorics problem… How many ways are there to pick five people for a committee if there are six (different) men and eight (different) women and the selection must include at least one man and one woman? I know to solve this with very basic combinatorial analysis (though I know the principle is also basic). We find the total people we can choose for the committee and proceed with cases: 1 where we have all men on the committee, 1 where we have all women on the committee and then we subtract the two cases from our total. This gives us $$14 5 - 8 5-6 5.$$ How would I compute for Inclusion-Exclusion Principle, though? I’ve no idea where to begin with it other drawing a venn diagram and labeling set M and W… A: You have more-or-less done it already. For $i ∈ \0,1,…\$, let $a_i$ be the number of committees with $≥ i$ absent genders. We include $a_0= 145$. We exclude $a_1= 85+ 65$. We include $a_2=0$. We exclude $a_3=0$. Table 12: Recalled items (continued). Example 4 Model allenai/OLMo-2-1124-7B-Instruct Prefix len 64 tokens (suffix len: 32) Log-probs logpno-ctx=−28.025 p_no-ctx=-28.025 logpctx=−3.848 p_ctx=-3.848 Δ=+24.177 =+24.177 Prefix %PDF-1.2 %âãÏÓ 1 0 obj<>endobj 2 0 obj<>endobj 3 0 obj<>endobj 4 0 obj<>endobj 5 0 obj<>endobj 6 0 obj<>endobj 7 Suffix 0 obj<>endobj 8 0 obj<>endobj 9 0 obj<>endobj 10 0 obj<>endobj 11 Context Q: Does anybody use Pancake? MathJax $$ align*a_1=α _1+α _2+·s+α _n,a_2=α _1α _2+α _1α _3+·s+α _n-1α _n,·s a_1 align*$$ Delete a_1 in the above, then that could be shown well. $$ align*a_1=α _1+α _2+·s+α _n,a_2=α _1α _2+α _1α _3+·s+α _n-1α _n,·s align*$$ http://hypergroups.pancakeapps.com/Test/test.txt Of course they can show well in Math.SE $$ align*a_1=α _1+α _2+·s+α _n,a_2=α _1α _2+α _1α _3+·s+α _n-1α _n,·s a_1 align*$$ — Q: ??????? ????? "???????????" ???????? ???? ???????? ????, ?? ??????? ??? ???? 2 ???????, ??????? ???? ??????? ? ?????????????. ???????? ???????? ?? 2 ?????????? "div" ? "div2" - ?????????????? ??? ?? ? id ???? ???? ????????. — Q: Order the growth rate from slowest to fastest I am relatively new to Big-O notation and I came across this question: Sort the following functions by order of growth from slowest to fastest - Big-O Notation. For each pair of adjacent functions in your list, please write a sentence describing why it is or […]