Paper deep dive

Memories Retrieved from Many Paths: A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models

Trung Cuong Dang, David Mohaisen

Year: 2024Venue: arXiv preprintArea: Safety EvaluationType: EmpiricalEmbeddings: 55

Models: Llama-2-7B, Mistral-7B-v0.3, Pythia-12B, Pythia-1.4B, Pythia-160M, Pythia-410M, Pythia-6.9B, Qwen3-14B

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 93%

Last extracted: 3/12/2026, 6:04:27 PM

Summary

The paper introduces a 'multi-prefix memorization' framework to detect training data leakage in Large Language Models (LLMs). Unlike previous methods that rely on single-path extraction or compression ratios, this framework defines memorization by the diversity of distinct prefixes that can elicit a target sequence. It uses an internal memorization score (η) to determine a required number of adversarial prefixes (P), providing a robust, scalable, and computationally efficient method for auditing data leakage, particularly in aligned models.

Entities (5)

Large Language Models · technology · 100%Multi-prefix memorization · framework · 100%Pythia · model-suite · 95%Compressible memorization · methodology · 90%Discoverable memorization · methodology · 90%

Relation Signals (3)

Multi-prefix memorization → detects → Training data leakage

confidence 95% · a Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models

Pythia → usedin → Multi-prefix memorization

confidence 95% · We use the Pythia model suite (Biderman et al., 2023) for our primary analysis

Multi-prefix memorization → improvesupon → Discoverable memorization

confidence 90% · While previous works have proposed various definitions for memorization, many exhibit shortcomings... To address this, we introduce a novel framework: multi-prefix memorization.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Large language models, trained on massive corpora, are prone to verbatim memorization of training data, creating significant privacy and copyright risks. While previous works have proposed various definitions for memorization, many exhibit shortcomings in comprehensively capturing this phenomenon, especially in aligned models. To address this, we introduce a novel framework: multi-prefix memorization. Our core insight is that memorized sequences are deeply encoded and thus retrievable via a significantly larger number of distinct prefixes than non-memorized content. We formalize this by defining a sequence as memorized if an external adversarial search can identify a target count of distinct prefixes that elicit it. This framework shifts the focus from single-path extraction to quantifying the robustness of a memory, measured by the diversity of its retrieval paths. Through experiments on open-source and aligned chat models, we demonstrate that our multi-prefix definition reliably distinguishes memorized from non-memorized data, providing a robust and practical tool for auditing data leakage in LLMs.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

54,712 characters extracted from source content.

Expand or collapse full text

Memories Retrieved from Many Paths: A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models Trung Cuong Dang 1 David Mohaisen 1 Abstract Large language models, trained on massive cor- pora, are prone to verbatim memorization of train- ing data, creating significant privacy and copy- right risks. While previous works have proposed various definitions for memorization, many ex- hibit shortcomings in comprehensively capturing this phenomenon, especially in aligned models. To address this, we introduce a novel framework: multi-prefix memorization. Our core insight is that memorized sequences are deeply encoded and thus retrievable via a significantly larger number of distinct prefixes than non-memorized content. We formalize this by defining a sequence as mem- orized if an external adversarial search can iden- tify a target count of distinct prefixes that elicit it. This framework shifts the focus from single-path extraction to quantifying the robustness of a mem- ory, measured by the diversity of its retrieval paths. Through experiments on open-source and aligned chat models, we demonstrate that our multi-prefix definition reliably distinguishes memorized from non-memorized data, providing a robust and prac- tical tool for auditing data leakage in LLMs. 1. Introduction Large language models have demonstrated remarkable ca- pabilities in tasks ranging from advanced text generation to nuanced reasoning and interactive dialogue. Their power stems from training on vast Internet-scale datasets, enabling them to internalize extensive knowledge and complex lin- guistic structures (Brown et al., 2020). However, this train- ing paradigm introduces a critical challenge: the tendency of LLMs to memorize and reproduce segments of their training data verbatim. This phenomenon, which has been demon- strated to occur even in state-of-the-art models (Carlini et al., 1 DepartmentofComputerScience,Universityof Central Florida.Correspondence to:Trung Cuong Dang<cuong.dang@ucf.edu>,DavidMohaisen<mo- haisen@ucf.edu>. Preprint. November 27, 2025. 2021), raises concerns regarding privacy, copyright, and the ethical deployment of these models. Consequently, estab- lishing a precise definition for memorization is of paramount importance, with implications for both practical applications and legal frameworks (Cooper et al., 2023), and the existing definitions offer varied perspectives on memorization. Some initial formulations define memorization based on simple elicitation, where a sequence is considered mem- orized if any prompt can be found that reproduces it ex- actly (Nasr et al., 2025). The weakness of this definition, however, is that it conflates true memorization with basic instruction-following, as a prompt could simply contain the target sequence itself (e.g., “Repeat the following text: ...”). To tackle this, more constrained definitions have been proposed. A widely adopted version is discoverable memo- rization (Carlini et al., 2023), which defines the concept as a completion test using the target sequence’s own prefix. This approach, however, lacks robustness against modern align- ment techniques. Its reliance on natural prefixes makes it susceptible to evasion by models trained to refuse such com- pletions, creating an “illusion of compliance” even when the memory persists (Ippolito et al., 2023). To address this issue, an information-theoretic definitions has been introduced in (Schwarzschild et al., 2024), which defines that a sequence is memorized if it can be elicited by a prompt (significantly) shorter than the sequence it- self. While this definition is more resilient to surface-level alignment, its practical utility is constrained by the com- putationally intensive search required to find an optimally compressed adversarial prompt, especially for very long target sequences. Crucially, while these state-of-the-art frameworks differ in their approach, they share a common conceptual limitation: they define memorization through the lens of finding a single elicitation path. They are therefore less designed to characterize the broader accessibility and robustness of a memory, properties that indicate how deeply it is ingrained within the model. Contributions In this work, we propose that memories within LLMs can be retrieved using various distinct cues, similar to the concept of cued recall in human memory (Tulv- ing & Pearlstone, 1966; Tulving & Osler, 1968). We present 1 arXiv:2511.20799v1 [cs.CL] 25 Nov 2025 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models a framework that redefines memorization, not just by the ability to elicit memories, but by the diversity of paths that can lead to such retrieval. Our main argument is that se- quences that are deeply memorized are identifiable by their retrieval through numerous unique prefix prompts. This notion of multiple retrieval avenues is akin to adversarial robustness analysis, where the vulnerability of a model is measured by the size of the input region leading to a par- ticular failure mode (Jin et al., 2020). We suggest that a greater number of unique prefixes indicates a more deeply memorized sequence, possibly due to a higher frequency of training, and thus presents a higher privacy risk by expand- ing the potential attack surface. To implement this idea, our framework evaluates memoriza- tion using two connected criteria: an internal recall signal and an external measure of adversarial elicitability. Initially, we determine a memorization score (η) for a specific se- quence, utilizing the model’s internal token probabilities. This score is not a conclusive result but is used to estab- lish a systematic, data-dependent burden of proof (P), the required number of distinct adversarial prefixes that need to be identified. The final classification of memorization depends on successfully fulfilling this burden through an external search. We demonstrate the validity of our framework through the feasibility of this external search. Our experiments show a clear distinction: finding adversarial prefixes for memorized sequences is consistently successful, while the success rate for non-memorized sequences is nearly zero across several model sizes and attack methods. Importantly, this finding also provides a practical computational benefit: auditors can use a persistent failure to find prefixes as a reliable early stopping criterion, allowing them to confidently termi- nate expensive searches and thereby manage computational resources effectively. 2. Related Work LLM memorization has been characterized through vary- ing definitions, each with specific operational criteria and limitations. We review the most relevant to situate our work. Discoverable memorization A direct approach to mea- suring memorization is discoverable memorization, which frames memorization as a natural prefix-completion task (Carlini et al., 2023). Given a modelf θ and a training se- quences,sis discoverably memorized if there exists a natu- ral prefixps.t.s = [p|t]and greedy decoding frompyields t, i.e.,f θ (p) = t. While this method is straightforward and efficient, it presents two significant limitations: 160M410M 1.4B 6.9B 12B 0 0.2 0.4 0.6 0.8 1 0 . 67 0 . 76 0 . 73 0 . 55 0 . 46 Memorized Ratio Figure 1. Discoverable memorization rates for Pythia on Famous Quotes. The ratio reflects quotes elicited via prefix completion. Contrary to expectations, memorization does not scale with model size and declines for largest models. The x-axis signifies the model (Pythia-) size. •Susceptibility to alignment:As noted by Schwarzschild et al. (2024), an LLM fine-tuned to refuse completions of sensitive or known training examples can pass this test, creating an “illusion of compliance” while the memory may still persist and be accessible via other means. •Inconsistent correlation with model scale: The orig- inal work suggests a strong positive correlation be- tween model size and discoverable memorization, not- ing “a tenfold increase in model size corresponds to an increase in memorization of 19 percentage points”. However, this trend may not generalize across datasets and model families. In our preliminary analysis us- ing the Famous Quotes dataset (Schwarzschild et al., 2024) and the Pythia model suite (Biderman et al., 2023), we observe the opposite: discoverable mem- orization decreases with model size, with the largest model (Pythia-12B) exhibiting the lowest memoriza- tion rate (Figure 1). This implies that discoverable memorization may not universally reflect a model’s underlying tendency to retain training data. Extractable memorization Another approach to mea- suring memorization is extractable memorization, which defines a sampleyfrom the training data as memorized by an LLMf θ if an adversary can find any promptpthat elicits yin response, i.e.,f θ (p) = y(Nasr et al., 2025). While this definition captures adversarial extraction’s idea, it suffers from a critical limitation: trivial extraction. This issue arises because the definition can be satisfied by simple instruction- following prompts. For example, given the prompt “Repeat the following:y”, any compliant model will outputy, re- gardless of whether it was truly memorized from the training data. This renders the definition overly inclusive, potentially labeling an entire training set as memorized. Compressible memorizationSchwarzschild et al. (2024) introduced compressible memorization, framing memoriza- tion as a form of efficient compression. Formally, a target 2 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models Table 1. Comparison of memorization frameworks FrameworkCore QuestionAlignment Robust?MetricLimitation DiscoverablePrefix-completable?NoBinaryBreaks on aligned models ExtractableElicitable by any prompt?N/ABinaryOver-inclusive; trivial prompts CompressibleShorter prompt works?YesRatio (ACR)Costly; threshold tuning Multi-prefix (Ours)# of distinct prompts?YesCountComputational cost sequenceyis considered memorized by LLMf θ if its adver- sarial compression ratio (ACR) exceeds a given thresholdτ. The ACR is defined as the ratio of the target’s length to the length of the shortest possible prompt x ∗ that elicits it: ACR(f θ ,y) = |y| |x ∗ | ; x ∗ = arg min x |x| s.t. f θ (x) = y. A sequenceyis then defined asτ-memorized if ACR(f θ ,y) > τ. The original work suggests a practical threshold ofτ = 1. While this definition is more resilient to surface-level alignment, it introduces practical challenges: • Computational cost: A limitation of this frame- work lies in its high and unpredictable computa- tional cost, stemming from the unknown optimal prompt length|x ∗ |for a given sequence. Identify- ing this length requires an iterative meta-search, e.g., the MINIPROMPT algorithm, which repeatedly in- vokes the costly greedy coordinate gradient (GCG) method (Zou et al., 2023) across multiple prompt lengths to locate the success–failure boundary. As a result, the overall runtime reflects not a single optimiza- tion, but a sequence of them, governed by the target sequence’s unknown characteristics. This cost variabil- ity poses a major obstacle to adoption in large-scale auditing where resource predictability is critical. •Arbitraryτthreshold: The choice of the thresholdτ is a critical yet subjective aspect of this framework. Both the defaultτ = 1and the proposed model- agnostic baselines, such as GZIP and SMAZ, are un- able to reliably disentangle true memorization from strong generalization. This is because a model-agnostic benchmark, which assesses syntactic patterns, cannot account for the powerful, semantic compression ca- pabilities inherent to a specific LLM’s architecture and training. Consequently, a robust metric requires a model-dependent threshold normalized against the model’s own baseline performance. Membership inference attacks on foundation models Our work also relates to the broader field of privacy attacks, particularly membership inference attacks (MIAs). The goal of MIA is to determine whether a specific data point was included in a model’s training set (Shokri et al., 2017; Yeom et al., 2018; Salem et al., 2019). With the rise of LLMs, significant research has focused on adapting MIAs to this new paradigm (Mozaffari & Marathe, 2024; Fu et al., 2024; Xie et al., 2024). However, the validity of many recent attacks has been questioned. Several studies argue that their reported success often stems not from detecting genuine membership signals, but from methodological flaws, such as distributional or temporal shifts between the member and non-member evaluation datasets (Duan et al., 2024; Maini et al., 2024; Meeus et al., 2025). Reinforcing this critique, Das et al. (2024) demonstrated that even a “blind” baseline model can outperform these sophisticated attacks, suggesting they primarily exploit dataset artifacts. This recent re-evaluation of MIAs underscores the need for more robust and practical methods for assessing data usage in foundation models, a gap our work aims to address. Comparative summaryTo synthesize the preceding dis- cussion and clearly position our contribution, we present a comparative summary of memorization frameworks in Table 1. While prior work has focused on establishing the possibility (discoverable/extractable) or efficiency (com- pressible) of eliciting a sequence, our framework introduces a new axis of measurement: the diversity of elicitation paths. This comparison shows that our multi-prefix framework is not just an alternative but a conceptually distinct approach. By measuring the number of adversarial prompts that elicit a target sequence, we provide a finer-grained assessment of memorization depth. While our method shares the compu- tational demands typical of adversarial search-based tech- niques, it introduces a principled mechanism for cost control. Notably, our experiments show that adversarial success on non-memorized content is near zero, offering a clear empiri- cal stopping criterion. Consistent failure to elicit a sequence allows auditors to terminate the search early, significantly re- ducing overhead. This built-in failure signal sets our frame- work apart from others lacking such a safeguard, making it a more robust and practical tool for auditing LLMs. 3. Methodology To quantify the memorization of a particular sequences within an LLMf θ , our method is grounded in the notion that genuinely memorized sequences by a model can be retrieved through several different input prompts, or prefixes. We assert that this ability to elicit a sequence from multiple prefixes indicates a more robust embedding of the sequence in the model’s parameters compared to sequences that are not memorized or are merely generated compositionally. We 3 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models begin by outlining the conceptual foundation, then present the formal criteria for memorization, and finally detail the computation of its components. 3.1. Conceptual Framework and Formal Definition Our central hypothesis is that the extent to which a sequence sis memorized byf θ is related to the number of distinct prefixes that can causef θ to reproducesexactly. Although common phrases or highly probable continuations might also be generated from several starting points, we argue that sequences memorized verbatim, especially those less likely under normal compositional generation, will show a sig- nificantly greater density of such eliciting prefixes relative to their inherent complexity. Conversely, finding multiple distinct prefixes for specific, non-memorized sequences is expected to be considerably more challenging. A sequencesis memorized byf θ if two conditions are ful- filled. First, a prefix elicitation threshold must be met: an evaluator must be capable of identifying a setXcontaining at leastP s f θ distinct prefixes, where for each prefixp∈X, the model’s output f θ (p) precisely matches the sequence s. To exclude trivial variations and ensure each prefix’s unique- ness, we require that any pair of prefixes for the same target surpass a minimum threshold of cosine distance between their sentence embeddings. The number of distinct prefixes required, denoted asP s f θ , is determined by the sequence’s characteristics and its internal memorization signal: P s f θ =⌈η s f θ ×|s|⌉,(1) where|s|represents the length of the sequences, andη s f θ is the proposed memorization score, which measures the intrinsic evidence of memorization (detailed below). The second condition requires a non-zero internal signal: the calculated memorization scoreη s f θ must be positive. In our experiments, this corresponds to setting the minimum threshold η min to zero, so the condition becomes η s f θ > 0. This requirement ensures that a sequence can only be con- sidered for memorization if there is at least a minimal de- tectable internal memorization signal. If a sequence has no internal memorization signal (i.e.,η s f θ = 0), it will not satisfy this condition and is therefore not considered memo- rized. In this scenario,P s f θ would also be zero according to Equation 1. This approach means that the main process for determining the necessary evidence level (i.e., the number of prefixesP s f θ ) depends on the strength of a positiveη s f θ and the sequence length|s|. The prefix elicitation threshold formulaP s f θ is a structured heuristic developed to balance two key elements. It scales with the inherent memorization evidence denoted byη s f θ , implying that a higher memorization score (stronger inter- nal recall signals) justifiably requires more external valida- tion through a greater number of unique eliciting prefixes. Additionally, it scales linearly with the complexity of the sequence, represented by its length|s|. This aligns with the intuition that precisely replicating longer sequences by chance is less likely, necessitating more substantial evidence (more prefixes) to claim memorization for longer sequences, even if they possess the same memorization scoreη s f θ as a shorter sequence. Though there could be other formulations forP s f θ , this linear scaling provides a straightforward, in- terpretable method that aligns with these insights, and its success in distinguishing memorized sequences is empiri- cally supported in our experimental section. It should be noted that this method defines the memorization criterion based on the presence ofP s f θ distinct prefixes. The practical challenge of finding these prefixes is a separate issue that typically involves adversarial search or extraction techniques (e.g., beam search, gradient-based prompt opti- mization). Our framework offers a principled target number of prefixes,P s f θ , to guide such search efforts, and the spe- cific methods used to find these prefixes in our assessments are detailed in the experiment section. 3.2. Calculating the Memorization Score We define the memorization scoreη s f θ to quantify the degree to which the modelf θ has internalized the target sequences. This score integrates two distinct aspects of model behavior, assessed by prompting the model with increasingly longer initial segments ofs: a verbatim recall score (r i ) and a positional similarity score (sim i ). We derive the overall scoreη s f θ by aggregating these components across all prefix lengths i∈ [1,|s|− 1]. The verbatim recall scorer i aims to measure the model’s confidence and inherent tendency to regenerate the remain- der of sequences(i.e.,s i+1:|s| ) verbatim from its initiali tokens (s 1:i ). It is calculated as: r i = |s|− i i × p avg (i) for 1≤ i <|s|. The scaling factor |s|−i i is chosen to prioritizes recall based on minimal context (smalli). This is because successfully outputting a sequence from a very short prefix is considered stronger evidence of rote memorization than recalling it from a long prefix where the continuation is often more con- strained and thus easier to predict. The diminishing weight for largerireflects this. While this scaling can be aggressive for very smalli, its impact is moderated byp avg (i)(which is often low for smalli), the subsequent multiplication by the similarity scoresim i , and the final averaging step in the calculation ofη s f θ . The termp avg (i)represents the model’s average confidence in auto-regressively generating the true continuations i+1:|s| , calculated as the geometric mean of the conditional probabilities of each correct token in the 4 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models remainder, given all preceding true tokens: p avg (i) = exp   1 |s|− i |s|−1 X t=i log(p(s t+1 |s 1:t ,f θ ) + ε)   . Here,p(s t+1 |s 1:t ,f θ )is the model’s probability for the true tokens t+1 given the preceding contexts 1:t . A small smooth- ing valueε(e.g.,1× 10 −9 ) is added before taking the loga- rithm to handle potential zero probabilities and for stability. The positional similarity scoresim i measures the model’s actual generated output compared to the ground truth se- quenceswhen prompted withs 1:i . Lets ′ i+1:|s| be the se- quence generated byf θ when prompted withs 1:i , thensim i is the fraction of tokens in the model’s generated continua- tion that match ground truth sequences i+1:|s| at the corre- sponding positions: sim i = 1 |s|− i |s|−1 X t=i I(s ′ t+1 = s t+1 ) for 1≤ i <|s|, wheres ′ t+1 is the generated token at positiont + 1,s t+1 is the ground truth token, andI(·)is the indicator function. This provides a direct measure of the positional accuracy of the model’s continuation. We compute the memorization scoreη s f θ as the average of the product of these verbatim recall scores (r i ) and posi- tional similarity scores (sim i ) across all non-trivial prefix lengths of s defined as η s f θ = α 1 P |s|−1 i=1 r i × sim i , or: η s f θ = α 1 |s|−1 X i=1 |s|− i i p avg (i)×   α i |s|−1 X t=i I(s ′ t+1 = s t+1 )   , whereα 1 = 1 |s|−1 andα i = 1 |s|−i . The averaging over i = 1to|s|− 1serves as a normalization step. It is impor- tant to note that due to the scaling factor inr i ,η s f θ is not strictly guaranteed to be within the [0, 1] range. However, its primary purpose is to provide the necessary relative mea- sure of memorization strength. This score is then used to calculate the prefix elicitation thresholdP s f θ as defined in (1). A higherη s f θ value reflects stronger combined evidence from the model’s internal confidence (viap avg (i)) and its external generation accuracy (viasim i ), with a particular emphasis on recall from shorter, less informative contexts. 4. Experimental Evaluation We validate our multi-prefix framework for quantifying se- quence memorization and evaluate its effectiveness in dis- tinguishing memorized from non-memorized content. Then, we use it to analyze the impact of instruction fine-tuning, memorization trends across model scales, and patterns of partial memorization within sequences. Finally, we provide an analysis of the computational cost of our framework. Models and datasets We use the Pythia model suite (Bi- derman et al., 2023) for our primary analysis because it is open-source and was trained on the publicly docu- mented Pile dataset (Gao et al., 2021). We evaluate mod- els across a range of sizes: 160M, 410M, 1.4B, 6.9B and 12B. Our evaluation focuses on two datasets sourced from (Schwarzschild et al., 2024), both subsets of The Pile. The “Famous Quotes” dataset contains 95 quotations and serves as our primary testbed for memorization, while the Wikipedia dataset was used to test memorization on long- form prose. To assess memorization in instruction-tuned models, we evaluate the Llama-2-7B-chat (Touvron et al., 2023), Mistral-7B-Instruct-v0.3 (Jiang et al., 2023), and Qwen3-14B-Chat (Yang et al., 2025) models against their respective base models: Llama-2-7B, Mistral-7B-v0.3, and Qwen3-14B-Base. Prefix search strategy We employ the GCG algo- rithm (Zou et al., 2023) to search for adversarial prefixes. For each target sequences, we allocate GCG a budget of max(10, 2·P s f θ ) runs, with each run initialized using a differ- ent random seed to encourage diverse prefix discovery. The search forsterminates once the required number of distinct prefixesP s f θ is found or when the budget is exhausted. A sequence is ultimately classified as memorized by a model if GCG successfully identifiesP s f θ distinct prefixes that cause the model to generate the sequence verbatim. 4.1. Validation of Multi-Prefix Memorization To assess the core concept of our approach, that memorized sequences are more susceptible to elicitation, we analyze the attack success rate (ASR) of our prefix discovery method. Figure 2 visualizes this by plotting the ASR for both “mem- orized” and “non-memorized” sequences side-by-side for each model in the Pythia suite. The results reveal a strong disparity. For sequences classified as memorized, the ASR is consistently high and scales with model size, rising from 0.60 for Pythia-160M to 0.90 for Pythia-6.9B. In contrast, the success rates for non-memorized sequences are dramati- cally lower across the board. For the Pythia-160M model, this rate is a mere 0.018, implying an average of 56 GCG runs would be needed to find a single valid prefix. This side- by-side comparison provides compelling evidence that our methodology effectively distinguishes between sequences that are deeply embedded and easily elicitable versus those that are not. Memorization of paraphrased Famous Quotes To de- termine if our definition can separate between exact and conceptual memorization, we tested the Pythia-6.9B model on both Famous Quotes and a paraphrased version. For the paraphrased set, we used the Gemini 2.5 Pro API (Co- manici et al., 2025) to alter a maximum of two words per quote while preserving its meaning. The results are shown 5 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models 160M410M1.4B6.9B12B 0 0.5 1 0 . 6 0 . 75 0 . 87 0 . 9 0 . 75 0 . 018 0 . 03 0 . 06 0 . 1 0 . 04 Avg. Success Rate MemorizedNon-Memorized Figure 2. GCG success rates by model size and sequence type. Memorized sequences consistently yield higher success rates, vali- dating the classification method and revealing the attack’s depen- dence on prior memorization. OriginalParaphrased 0 20 40 60 80 100 84.2 3.2 Mem. Rate (%) 0101112141620303337425083100 0 10 20 30 1 10 22 14 12 6 2 5 77 6 111 Recall Rate (%) # Quotes Figure 3. Memorization behavior in Pythia-6.9B for original Fa- mous Quotes and paraphrased quotes. Top: Memorization rates for original vs. paraphrased quotes. Minimal paraphrasing drops memorization from 84% to 3.2%, showing the method detects verbatim memorization. Bottom: Distribution of recall rates. Ad- versarial attacks on paraphrased quotes often trigger the model to recall and output the original memorized quote. in Figure 3 (top part). The model, which memorized approx- imately84%of the original quotes, memorized only 3 of the 95 paraphrased versions. This disparity shows that our method detects exact, not conceptual, memorization. We observed a further, more interesting phenomenon dur- ing GCG attacks. In particular, the adversarial prompts designed for the paraphrased quotes consistently caused the model to output the original famous quotes instead. In fact, as shown in Figure 3 (bottom part), for 94 out of the 95 paraphrased quotes, the algorithm successfully found an adversarial prompt that elicited the original quote. This high success rate (as shown in Figure 3; bottom part) is reflected in the overall metrics, where the ratio of found original quotes to GCG attempts was nearly 20%. This serves as a strong cue that the model’s memory of these phrases is tied to the exact original text. Memorization of random sequences Since our method is shown to be able to distinguish between exact and con- ceptual memorization, we want to ensure that our method specifically identifies memorized content rather than pro- ducing adversarial prompts for arbitrary sequences. To this end, we conducted a control experiment. We generated a dataset of 1,000 random token sequences by uniformly sampling with replacement from the model’s vocabulary, with lengths ranging from 5 to 15 tokens. These sequences decode to semantically meaningless gibberish and serve as non-memorized controls. Across all evaluated model sizes, our algorithm failed to find any prefix that elicited these sequences, confirming that our measurements reflect true memorization rather than artifacts of the attack method. Adversarial prefix findingBesides GCG, we leverage the probe sampling (PS) method (Zhao et al., 2024) to enhance the robustness of the GCG algorithm. The original paper re- ports that PS increases the attack success rate (ASR), partly due to the randomness it introduces. While PS is observed to improve the speed of the GCG algorithm, we found that for our goal of discovering a diverse set of prefixes, the GCG+PS combination is a more conservative method that finds fewer unique prefixes than GCG alone. For example, on the Pythia-6.9B model, 21 sequences were classified as non-memorized when tested with GCG+PS, compared to only 15 when tested with GCG. Similarly, on the 12B model, 8 out of 95 sequences that were classified as memorized under our definition using the GCG, were subsequently classified as non-memorized when tested with GCG+PS. Despite this, the average attack success rate for sequences classified as memorized remains high, at 0.76 for the 6.9B model and 0.73 for the 12B model. For those classified as non-memorized, the average success rates are 0.06 and 0.08. Statistical validation To statistically validate our multi- prefix memorization framework, we tested its core hypothe- sis (H a ): sequences classified as “memorized” are signifi- cantly more susceptible to adversarial elicitation than those classified as “non-memorized.”, while the null hypothesis H 0 states that there is no significant difference in GCG at- tack success rates between the two classes. We conducted a one-sided Mann-Whitney U test on the GCG attack success rates for the two classes, using the Famous Quotes dataset. By evaluating on models of different scales, we demonstrate the framework’s robustness and generalizability. First, on the large Pythia-6.9B model, our framework achieved a near-perfect separation between the two classes. Sequences labeled ”memorized” had a median GCG suc- cess rate of 1.0, while ”non-memorized” sequences had a median of 0.0. This visual distinction was confirmed to be statistically significant(U = 1189.0,p < 1e−10). Further- more, the effect size was exceptionally large (Rank-biserial 6 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models correlationr ≈ 0.99), indicating that the classification pro- vided a practically complete separation between elicitable and non-elicitable content. Next, to test the framework’s robustness on a less capable model, we repeated the analysis on the small Pythia-160M model. Even here, where memorization is far less prevalent, our method proved effective. The “memorized” group ex- hibited a median success rate of 0.5, compared to 0.0 for the “non-memorized” group. Despite the weaker memorization signals in the model, the distinction between the groups remained highly significant(U = 1457.5,p < 1e− 10), again with a large effect size(r = 0.89). The consistent statistical significance and large effect sizes across models of different scales provide strong empirical evidence for our framework. It confirms that our methodology robustly identifies a meaningful and practical distinction between memorized and non-memorized sequences. 4.2. Adversarial Prefixes Characterization Semantic diversity of adversarial prefixesTo character- ize the mechanism of model memorization, we first evaluate the semantic diversity of adversarial prefixes that elicit the same target sequence. We hypothesize two potential out- comes: the prefixes could be semantically clustered (low pairwise distance), suggesting the optimization converges on a narrow semantic region, or they could be semantically scattered (high pairwise distance). The latter case would imply a more arbitrary, lookup-table-like behavior, where disparate inputs are mapped to an identical, verbatim output. Our analysis focuses on adversarial prefixes optimized for the Pythia-6.9B model to elicit targets from the Famous Quotes dataset. We compute the pairwise cosine distance between the all-MiniLM-L6-v2 embeddings (Reimers & Gurevych, 2019; Wang et al., 2020) of all successful pre- fixes for each target and present the aggregated distribution in Figure 4. The distribution is clearly skewed towards higher values, with a significant proportion of prefix pairs exhibiting a cosine distance greater than 0.5. This high de- gree of semantic diversity suggests that the model has not learned a robust, coherent pathway to the target sequence. This finding strongly supports the memorization hypothesis, indicating a lookup-like retrieval mechanism where multiple, disparate inputs are mapped to the same stored data point, rather than guiding a semantically-grounded generation. Connection between prefixes and target sequences To investigate the operational mechanism by which adversarial prefixes elicit specific target sequences, we computed the cosine similarity between entries from the Famous Quotes dataset and their corresponding adversarial prefixes which were uniquely optimized for the Pythia-6.9B model by GCG. This analysis sought to determine whether the prefixes func- 0 . 35 0 . 42 0 . 48 0 . 55 0 . 61 0 . 68 0 . 74 0 . 81 0 . 87 0 . 94 0 20 40 60 80 Frequency − 0 . 1 − 0 . 02 0 . 07 0 . 16 0 . 24 0 . 33 0 . 41 0 . 5 0 . 58 0 20 40 60 80 100 Frequency Figure 4. Semantic similarity statistics for adversarial prefix gen- eration targeting famous quotes. Top: Distribution of cosine distances between adversarial prefixes of each target (Famous Quotes). Bottom: Distribution of cosine similarities between orig- inal quotes and their adversarial prefixes. tion as meaningful contextual prompts or as arbitrary token sequences that exploit learned statistical patterns. A high degree of semantic correspondence would indicate a mecha- nism of contextual guidance, whereas a lack of correspon- dence would imply a non-semantic retrieval process. The results reveal a lack of semantic correlation between the prefixes and their targets. The frequency distribution of similarity scores, presented in Figure 4, is unimodal and heavily right-skewed, indicating that the vast majority of prefixes are semantically dissimilar to the content they pro- duce. The distribution exhibits a pronounced mode within the low-similarity interval of [0.070, 0.155). Furthermore, over 82% of all analyzed prefixes yield a cosine similarity score below 0.24. This finding strongly points to verbatim memorization. The absence of a semantic link rules out contextual guidance, suggesting the prefix functions not as a prompt, but as a key in a system analogous to a lookup table. In this mechanism, a specific, arbitrary input retrieves a corresponding value, which in this case is a sequence memorized verbatim from the model’s training data. 4.3. Memorization Across Model Sizes Famous Quotes We investigated how memorization scales with model size by applying our multi-prefix cri- terion to the Famous Quotes dataset across the Pythia model suite. As shown in Figure 5, the results reveal a strong pos- itive trend: larger models consistently memorize a higher proportion of quotes. Interestingly, this trend appears to plateau at the largest scales, with the 12B model showing a memorization ratio of 83%. This finding aligns with prior work linking increased model capacities to enhanced mem- orization (Carlini et al., 2023; Schwarzschild et al., 2024). It also indicates a possible saturation point for this partic- ular kind of verbatim knowledge due to the generalizable 7 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models abilities of larger models. 160M410M1.4B6.9B12B 0 20 40 60 80 100 20 40 71 84 83 Mem. Ratio (%) Figure 5. Proportion of sequences from the Famous Quotes dataset classified as memorized by each Pythia model. The results demon- strate a clear trend of increased memorization with model size. The x-axis signifies the model size. WikipediaOur Wikipedia experiments reveal a more com- plex relationship between scale and memorization. While the memorized ratio for the 1.4B model (≈5%) aligns with the findings of (Schwarzschild et al., 2024) (the only size they reported), we observe a distinct decrease for larger models, with the 6.9B and 12B variants memorizing only around 2% of sequences. We hypothesize that this diver- gence is attributable to the nature of the data itself. Unlike the short, canonical strings of famous quotes, Wikipedia text consists of continuous, long-form prose. For such data, larger models may leverage their enhanced capacity for con- textual understanding to generalize information rather than resorting to rote memorization of the exact phrasing. 4.4. Analysis of Partial Sequence Memorization While existing literature assesses memorization at the full sequence level (Carlini et al., 2023; Nasr et al., 2025; Schwarzschild et al., 2024), we investigate the relationship between whole-sequence memorization and the memoriza- tion of its constituent parts. To this end, for each sequence in the Famous Quotes dataset, we extract and analyze three overlapping subsequences, each comprising 50% of the orig- inal’s token length: the initial 50% of tokens (the first half), a central 50% segment (the middle half), and the final 50% of tokens (the final half). For example, consider the quote “I think, therefore I am.”. The first half would be “I think,”, the middle half would be “think, therefore” and the final half would be “therefore I am.” We applied our memoriza- tion analysis to these subsequences using the Pythia-6.9B model. The outcomes, detailed in Figure 6, reveal that mem- orization behavior differs significantly depending on the subsequence’s position within the original text. Our analysis in Figure 6 reveals a strong positional bias. When a full sequence is memorized, its initial and final segments are often co-memorized (66 and 54 instances, respectively). The middle segment, however, is notably fragile and is forgotten in 27 instances, far more than the first (11) or final (14) parts. This primacy effect extends to cases of partial-only recall, where the first half is most frequently memorized alone (15 instances), followed by the middle (12) WPW¬P¬WP¬W¬P 0 20 40 60 80 66 11 15 1 51 27 12 4 54 14 8 5 # of Sequences FirstMiddle Last Figure 6. Memorization across sequence segments in Pythia-6.9B. Bars show outcomes for whole (W) and partial (P: first, middle, final) subsequences.¬for non-memorized. Memorization varies notably with segment position. and final (8) halves. This suggests that while full-sequence memorization often includes its parts, initial segments are most robustly and independently retained, whereas central portions are less consistently memorized, even when the whole is recalled. 4.5. Impact of Instruction Fine-Tuning A central question in model development is how instruc- tion fine-tuning impacts the recall of pre-trained knowledge. While one might assume that alignment for conversational tasks would uniformly suppress the verbatim recall of train- ing data, our findings indicate a more complex and unex- pected relationship. We propose that this complexity can be best understood by distinguishing between two separate factors: the influence of the alignment process itself and the distinct effect of the conversational chat template. To inves- tigate this, we compared the memorized ratio between the base and chat-aligned versions of Llama-2-7B, Mistral-7B- v0.3, and Qwen3-14B on the Famous Quotes dataset, using our multi-prefix memorization technique and assessing the chat models both with and without their chat templates. Llama 2 (7B)Mistral (7B)Qwen3 (14B) 50 100 82.1 25.2 89.4 89.5 37.9 82.1 20 24.2 68.4 Mem. Ratio (%) Base AlignedAligned + Chat Template Figure 7. Memorization ratios on famous quotes for Llama 2, Mistral, and Qwen3 model families. The effect of alignment is model-dependent: it increases memorization for Llama 2 and Mistral but decreases it for Qwen3. In contrast, applying a chat template consistently and significantly reduces the memorization ratio across all three models. Analysis of memorizationOur analysis separates the im- pact of the alignment process from the application of a chat template, revealing a dynamic in model behavior. A direct comparison between base and aligned models shows varied effects: for instance, Llama 2’s memorization ratio increases from 82.1% to 89.5% after alignment, and Mistral’s rises from 25.2% to 37.9%. Conversely, Qwen3’s memorization 8 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models decreases from a base of 89.4% to 82.1% after alignment. However, a consistent pattern emerges when the chat tem- plate is applied: it drastically reduces the memorization of the aligned model in all cases, down to 20.0% for Llama 2, 22.0% for Mistral, and 68.4% for Qwen3. These results highlight the role of the chat template as a defense mechanism. Its structural tokens (e.g.,[INST] for Llama 2,<|imstart|>for Qwen3) invoke conver- sational protocols that suppress elicitation. When the tem- plate is absent, the model reverts to its fundamental text- completion capabilities, exposing a degree of memorization that is otherwise masked. In contrast, the alignment process when evaluated without a chat template has a more complex impact on memorization. For models like Llama 2 and Mis- tral, it appears to increase the model’s tendency to recall stored data, while for Qwen3, it provides a slight reduction, though the memorization level remains high. This distinction is critical for auditing. Given that the impact of alignment on memorization varies across models, we suggest that auditors may consider evaluating the aligned model directly, without its chat template. This approach can provide a more direct assessment of a model’s underlying memorization, offering a clearer signal of its potential to leak training data, independent of the variable effects of its conversational guardrails. 4.6. Analysis of Computational Cost Required number of prefixes We constructed a dataset of 900 sequences from the Pile-C corpus, which is part of the Pile dataset. To ensure a balanced representation of sequence lengths, we employed a stratified sampling strategy: for each word count from 10 to 99 inclusive, we randomly selected 10 unique sequences. This methodology resulted in a dataset where sequence token counts range from 11 to 416, with a mean of 65.9 tokens per sequence. In our memorization analysis of this dataset, we found that on average, only 2.62 prefixes were required to uniquely identify a sequence (See Figure 8, top part). More impor- tantly, we observed that the distribution of required prefixes is heavily right-skewed, with the vast majority of sequences needing five or fewer prefixes (P s f θ ≤ 5 ) to be counted as memorized (see Figure 8, bottom part). This empirical find- ing is the basis of our running budget for each sequence as max(10, 2·P s f θ ), to make the computation both efficient and robust. Since our data shows most sequences haveP ≤ 5, the2· Pterm evaluates to 10 or less for these common cases, causing the budget to default to a stable 10 runs. This approach prevents excessive computational cost for easily memorized sequences. At the same time, the2· Pterm serves as a dynamic bound for rare outlier sequences where P > 5. This ensures that harder-to-memorize sequences are automatically allocated a proportionally larger budget. 01234567891011 0 100 200 Number of Prefixes # Sequences 0100200300400 0 5 10 Token Count # Prefixes Figure 8. The number of prefixes required for memorization. Top: Distribution of required prefixes. Most sequences are memorized with 2–3 prefixes, with a mean of 2.62; the distribution is heavily right-skewed. Bottom: Number of required prefixes vs. sequence length. There is little to no correlation between token count and prefix requirement; most sequences are memorized with five or fewer prefixes. Computational cost managementThe primary hardware requirement for our memorization test is sufficient GPU memory to load the audited model and compute input gradi- ents for a user-specified number of prompts. On an NVIDIA RTX A6000 GPU, assessing a single short (≤ 30tokens) se- quence takes only a few minutes. The overall computational cost, therefore, is primarily a function of the total number of GCG runs required. A possible concern is that this cost could become prohibitive. However, our methodology incorporates several features to ensure practical efficiency. First, as established in the pre- vious section (analysis of the number of prefixes required), the vast majority of sequences require five or fewer prefixes to be classified as memorized. This result sets our heuristic budget ofmax(10, 2· P), ensuring that most sequences are assigned a modest, constant budget of 10 runs. Table 2. Computational cost statistics of the memorization test on the Pythia-6.9B model. Average allocated budgets and actual runs are shown for different datasets, highlighting efficiency via early stopping. DatasetAvg. BudgetAvg. RunsEfficiency Famous Quotes10.083.7637.3% Paraphrased10.318.2580.0% More importantly, the actual computational cost is often significantly lower than the allocated budget due to early stopping. This efficiency is most apparent on highly memo- 9 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models rized data. For instance, as shown in Table 2 in the Pythia- 6.9B model’s test on the Famous Quotes dataset, the average allocated budget was 10.08 runs, but the average number of runs actually executed was only 3.76, which translates into an efficiency score of 37.3% (actual as a percentage of the budget). In contrast, for the largely non-memorized Paraphrased Famous Quotes dataset, the average actual runs were nearly 80% of the 10.31-run allocated budget. Even in this case, the framework’s adaptability allows for further op- timization. For datasets expected to be non-memorized, one can adopt a stricter strategy, such as flagging a sequence as non-memorized after only a few consecutive GCG failures to minimize computational cost. 5. Conclusion We introduced the multi-prefix memorization framework, a principled approach to measuring how deeply content is embedded in LLMs by quantifying the diversity of elicit- ing prompts. Unlike prior methods focused on single-path elicitation, our framework emphasizes the multiplicity of access routes, capturing not just the presence of memoriza- tion but its robustness. We experimentally demonstrated that adversarial success rates offer a reliable signal for dis- tinguishing memorized from non-memorized content, while also enabling practical cost control via early termination. Our findings reveal that memorization varies with data type, model scale, and instruction tuning, and is unevenly dis- tributed across sequences. By exposing the structure of LLM memory, our framework lays the groundwork for more effective auditing and mitigation strategies, offering both a new metric and a deeper understanding of model behavior. References Biderman, S., Schoelkopf, H., Anthony, Q. G., Bradley, H., O’Brien, K., et al. Pythia: A suite for analyzing large language models across training and scaling. In Proceed- ings of the 40th International Conference on Machine Learning, volume 202, p. 2397–2430, 2023. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., et al. Language models are few-shot learners. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems., 2020. Carlini, N., Tram ` er, F., Wallace, E., Jagielski, M., Herbert- Voss, A., Lee, K., Roberts, A., Brown, T. B., Song, D., Erlingsson, ́ U., Oprea, A., and Raffel, C. Extracting training data from large language models. In Proceedings of the 30th USENIX Security Symposium, p. 2633–2650, 2021. Carlini, N., Ippolito, D., Jagielski, M., Lee, K., Tram ` er, F., and Zhang, C. Quantifying memorization across neural language models. In Proceedings of the 11th Interna- tional Conference on Learning Representations (ICLR), 2023. Comanici, G., Bieber, E., Schaekermann, M., Pasupat, I., Sachdeva, N., et al. Gemini 2.5: Pushing the frontier with advanced reasoning, multimodality, long context, and next generation agentic capabilities, 2025. Cooper, A. F., Lee, K., Grimmelmann, J., Ippolito, D., Callison-Burch, C., et al. Report of the 1st workshop on generative AI and law. CoRR, abs/2311.06477, 2023. Das, D., Zhang, J., and Tram ` er, F. Blind baselines beat mem- bership inference attacks for foundation models. CoRR, abs/2406.16201, 2024. Duan, M., Suri, A., Mireshghallah, N., Min, S., Shi, W., Zettlemoyer, L., Tsvetkov, Y., Choi, Y., Evans, D., and Hajishirzi, H. Do membership inference attacks work on large language models? CoRR, abs/2402.07841, 2024. Fu, W., Wang, H., Gao, C., Liu, G., Li, Y., and Jiang, T. Membership inference attacks against fine-tuned large language models via self-prompt calibration. In Adv. in Neural Information Processing Systems (NeurIPS), 2024. Gao, L., Biderman, S., Black, S., Golding, L., Hoppe, T., Foster, C., Phang, J., He, H., Thite, A., Nabeshima, N., Presser, S., and Leahy, C. The pile: An 800gb dataset of diverse text for language modeling. CoRR, abs/2101.00027, 2021. Ippolito, D., Tram ` er, F., Nasr, M., Zhang, C., Jagielski, M., Lee, K., Choquette-Choo, C. A., and Carlini, N. Pre- venting generation of verbatim memorization in language models gives a false sense of privacy. In Proceedings of the 16th International Natural Language Generation Conference (INLG), p. 28–53, 2023. Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de Las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. Mistral 7b. CoRR, abs/2310.06825, 2023. doi: 10.48550/ARXIV.2310.06825. Jin, D., Jin, Z., Zhou, J. T., and Szolovits, P. Is BERT really robust? a strong baseline for natural language attack on text classification and entailment. In Proceedings of the 34th AAAI Conference on Artificial Intelligence (AAAI), p. 8018–8025, 2020. Maini, P., Jia, H., Papernot, N., and Dziedzic, A. LLM dataset inference: Did you train on my dataset?In Advances in Neural Information Processing Systems (NeurIPS), 2024. 10 A Multi-Prefix Framework for Robust Detection of Training Data Leakage in Large Language Models Meeus, M., Shilov, I., Jain, S., Faysse, M., Rei, M., and de Montjoye, Y. Sok: Membership inference attacks on llms are rushing nowhere (and how to fix it). In Proceed- ings of the IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), p. 385–401, 2025. Mozaffari, H. and Marathe, V. J. Semantic membership inference attack against large language models. CoRR, abs/2406.10218, 2024. Nasr, M., Rando, J., Carlini, N., Hayase, J., Jagielski, M., Cooper, A. F., Ippolito, D., Choquette-Choo, C. A., Tram ` er, F., and Lee, K. Scalable extraction of training data from aligned, production language models. In Pro- ceedings of the 13th International Conference on Learn- ing Representations (ICLR), 2025. Reimers, N. and Gurevych, I. Sentence-bert: Sentence embeddings using siamese bert-networks. In Proceed- ings of the Conference on Empirical Methods in Natural Language Processing (EMNLP-IJCNLP), p. 3980–3990, 2019. Salem, A., Zhang, Y., Humbert, M., Berrang, P., Fritz, M., and Backes, M. Ml-leaks: Model and data independent membership inference attacks and defenses on machine learning models. In Proceedings of the 26th Network and Distributed System Security Symposium (NDSS), 2019. Schwarzschild, A., Feng, Z., Maini, P., Lipton, Z. C., and Kolter, J. Z. Rethinking LLM memorization through the lens of adversarial compression. In Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2024, 2024. Shokri, R., Stronati, M., Song, C., and Shmatikov, V. Mem- bership inference attacks against machine learning mod- els. In Proceedings of the IEEE Symposium on Security and Privacy (S&P), p. 3–18, 2017. Touvron, H., Martin, L., Stone, K., Albert, P., et al. Llama 2: Open foundation and fine-tuned chat models, 2023. Tulving, E. and Osler, S. Effectiveness of retrieval cues in memory for words. Journal of Experimental Psychology, 77(4):593, 1968. Tulving, E. and Pearlstone, Z. Availability versus acces- sibility of information in memory for words. Journal of Verbal Learning and Verbal Behavior, 5(4):381–391, 1966. Wang, W., Wei, F., Dong, L., Bao, H., Yang, N., and Zhou, M. Minilm: Deep self-attention distillation for task- agnostic compression of pre-trained transformers. In Proceedings of the 34th International Conference on Neu- ral Information Processing Systems (NeurIPS), p. 485, 2020. Xie, R., Wang, J., Huang, R., Zhang, M., Ge, R., Pei, J., Gong, N., and Dhingra, B. Recall: Membership inference via relative conditional log-likelihoods. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 8671–8689, 2024. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., et al. Qwen3 technical report. CoRR, abs/2505.09388, 2025. doi: 10. 48550/ARXIV.2505.09388. Yeom, S., Giacomelli, I., Fredrikson, M., and Jha, S. Privacy risk in machine learning: Analyzing the connection to overfitting. In Proceedings of the IEEE 31st Computer Security Foundations Symposium (CSF), p. 268–282, 2018. doi: 10.1109/CSF.2018.00027. Zhao, Y., Zheng, W., Cai, T., Long, D. X., Kawaguchi, K., Goyal, A., and Shieh, M. Q. Accelerating greedy coordi- nate gradient and general prompt optimization via probe sampling. In Advances in Neural Information Processing Systems 38 (NeurIPS), 2024. Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M. Uni- versal and transferable adversarial attacks on aligned lan- guage models. CoRR, abs/2307.15043, 2023. 11

Instant research discovery