Paper deep dive
Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models
Sri Durga Sai Sowmya Kadali, Evangelos E. Papalexakis
Models: GPT-J, LLaMA, LLaMA 3.1 8B, Mamba, Mistral
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/11/2026, 12:59:46 AM
Summary
The paper introduces a novel, architecture-agnostic framework for detecting and mitigating jailbreak attacks on Large Language Models (LLMs) by analyzing internal latent representations. Using tensor decomposition (CP decomposition) on hidden activations and multi-head attention outputs, the authors identify consistent, layer-wise latent signatures that distinguish jailbreak prompts from benign ones. They demonstrate that selectively bypassing high-susceptibility layers during inference can block 78% of jailbreak attempts while maintaining 94% performance on benign prompts, offering a scalable, lightweight defense mechanism that does not require model fine-tuning.
Entities (5)
Relation Signals (3)
Layer Bypass → mitigates → Jailbreak Attack
confidence 98% · selectively bypassing high-susceptibility layers blocks 78% of jailbreak attempts
CP Decomposition → analyzes → LLM Internal Representations
confidence 95% · We use CP decomposition because of its simplicity and interpretability to analyze latent structure of internal representations.
LLM → exhibits → Latent Signatures
confidence 92% · jailbreak and benign prompts exhibit distinct, layer-dependent latent signatures in the internal representations of LLMs
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Jailbreaking large language models (LLMs) has emerged as a critical security challenge with the widespread deployment of conversational AI systems. Adversarial users exploit these models through carefully crafted prompts to elicit restricted or unsafe outputs, a phenomenon commonly referred to as Jailbreaking. Despite numerous proposed defense mechanisms, attackers continue to develop adaptive prompting strategies, and existing models remain vulnerable. This motivates approaches that examine the internal behavior of LLMs rather than relying solely on prompt-level defenses. In this work, we study jailbreaking from both security and interpretability perspectives by analyzing how internal representations differ between jailbreak and benign prompts. We conduct a systematic layer-wise analysis across multiple open-source models, including GPT-J, LLaMA, Mistral, and the state-space model Mamba, and identify consistent latent-space patterns associated with harmful inputs. We then propose a tensor-based latent representation framework that captures structure in hidden activations and enables lightweight jailbreak detection without model fine-tuning or auxiliary LLM-based detectors. We further demonstrate that the latent signals can be used to actively disrupt jailbreak execution at inference time. On an abliterated LLaMA-3.1-8B model, selectively bypassing high-susceptibility layers blocks 78% of jailbreak attempts while preserving benign behavior on 94% of benign prompts. This intervention operates entirely at inference time and introduces minimal overhead, providing a scalable foundation for achieving stronger coverage by incorporating additional attack distributions or more refined susceptibility thresholds. Our results provide evidence that jailbreak behavior is rooted in identifiable internal structures and suggest a complementary, architecture-agnostic direction for improving LLM security.
Tags
Links
Trouble viewing inline? Open PDF directly →
Full Text
54,647 characters extracted from source content.
Expand or collapse full text
Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models Sri Durga Sai Sowmya Kadali University of California, Riverside Riverside, CA, USA skada009@ucr.edu Evangelos E. Papalexakis University of California, Riverside Riverside, CA, USA epapalex@cs.ucr.edu Abstract Jailbreaking large language models (LLMs) has emerged as a critical security challenge with the widespread deployment of conversa- tional AI systems. Adversarial users exploit these models through carefully crafted prompts to elicit restricted or unsafe outputs, a phenomenon commonly referred to as Jailbreaking. Despite numer- ous proposed defense mechanisms, attackers continue to develop adaptive prompting strategies, and existing models remain vulnera- ble. This motivates approaches that examine the internal behavior of LLMs rather than relying solely on prompt-level defenses. In this work, we study jailbreaking from both security and interpretabil- ity perspectives by analyzing how internal representations differ between jailbreak and benign prompts. We conduct a systematic layer-wise analysis across multiple open-source models, including GPT-J, LLaMA, Mistral, and the state-space model Mamba2, and identify consistent latent-space patterns associated with adversarial inputs. We then propose a tensor-based latent representation frame- work that captures structure in hidden activations and enables light- weight jailbreak detection without model fine-tuning or auxiliary LLM-based detectors. We further demonstrate that these latent sig- nals can be used to actively disrupt jailbreak execution at inference time. On an abliterated LLaMA 3.1 8B model, selectively bypassing high-susceptibility layers blocks 78% of jailbreak attempts while preserving benign behavior on 94% of benign prompts. This inter- vention operates entirely at inference time and introduces minimal overhead, providing a scalable foundation for achieving stronger coverage by incorporating additional attack distributions or more refined susceptibility thresholds. Our results provide evidence that jailbreak behavior is rooted in identifiable internal structures and suggest a complementary, architecture-agnostic direction for im- proving LLM security. Our implementation can be found here [1]. CCS Concepts • Computing methodologies→Artificial intelligence;• In- formation systems→Data mining;• Security and privacy→ Software and application security. Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. , © 2026 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM ISBN 978-x-x-x-x/Y/M https://doi.org/X.X Keywords Jailbreaking, Large Language Models, LLM Internal Representa- tions, Self-attention, Hidden Representation, Tensor Decomposition ACM Reference Format: Sri Durga Sai Sowmya Kadali and Evangelos E. Papalexakis. 2026. Jailbreak- ing Leaves a Trace: Understanding and Detecting Jailbreak Attacks from Internal Representations of Large Language Models. In Proceedings of . ACM, New York, NY, USA, 10 pages. https://doi.org/X.X 1 Introduction Large Language Models (LLMs) have demonstrated remarkable ca- pabilities across a wide range of tasks and have become increasingly integrated into applications spanning diverse domains and user populations. Despite their utility and efforts to align them according to human and safety expectations [14,36,43], these models remain highly susceptible to adversarial exploitation, raising significant safety and security concerns given their widespread accessibility. Among such threats, Jailbreaking has emerged as a persistent and particularly concerning attack vector, wherein malicious actors craft carefully engineered prompts to circumvent built-in safety mechanisms [8,48] and elicit restricted, sensitive, or otherwise disallowed content [22]. Jailbreak attacks pose substantial risks, as they enable users with harmful intent to manipulate LLMs into producing outputs that violate safety policies, including actionable instructions for malicious activities [33,42,45]. The growing avail- ability of jailbreak prompts in public repositories, research artifacts, and online forums further exacerbates this issue [17, 28]. To mitigate these risks, prior work has explored a range of de- fense strategies, including prompt-level filtering [35], model-level interventions [38,43], reinforcement learning from human feed- back (RLHF) [6], and the use of auxiliary safety models [9,23]. While such approaches have demonstrated partial effectiveness, they are not without limitations. In practice, even well-aligned models can remain vulnerable under repeated or adaptive attack attempts. Moreover, no single defense mechanism has proven suf- ficient to counter the continually evolving landscape of jailbreak strategies. In this study, we investigate a complementary and comparatively underexplored direction: leveraging internal model representations to distinguish jailbreak prompts from benign inputs and to guide mitigation. Our central hypothesis is that adversarial prompts in- duce distinct and detectable structural patterns within the hidden representations of LLMs, independent of output behavior. To evalu- ate this hypothesis, we extract layer-wise internal representations arXiv:2602.11495v2 [cs.CR] 20 Feb 2026 , ,Kadali et al. such as multi-head attention and layer output/hidden state repre- sentation from multiple models such as GPT-J, LlaMa, Mistral, and the state-space sequence model Mamba, and apply tensor decom- position [30] techniques to characterize and compare latent-space behaviors across benign and jailbreak prompts. Building on this analysis, we further demonstrate how these latent representations can be used to identify layers that are particularly susceptible to adversarial manipulation and to intervene during inference by selec- tively bypassing such layers. This representation-centric framework not only enables reliable detection of jailbreak prompts but also provides a principled mechanism for mitigating harmful behavior without modifying model parameters or relying on output-level filtering. Together, our results suggest that internal representations offer a powerful and generalizable foundation for both understand- ing and defending against jailbreak attacks beyond surface-level text analysis. Our contributions are as follows: •Layer-wise Jailbreak Signature: We show that jailbreak and benign prompts exhibit distinct, layer-dependent latent signatures in the internal representations of LLMs, which can be uncovered using tensor decomposition [30]. •Effective Defense via Targeted Layer Bypass: We demon- strate that these latent signatures can be exploited at infer- ence time to identify susceptible layers and disrupt jailbreak execution through targeted layer bypass. 2 Related work 2.1 Adversarial Attacks Adversarial attacks on LLMs encompass a broad class of inputs intentionally crafted to induce unintended, incorrect, or unsafe behaviors [8,48]. Unlike adversarial examples in vision or speech domains, which often rely on imperceptible input perturbations, attacks on LLMs primarily exploit semantic, syntactic, and contex- tual vulnerabilities in language understanding and generation. By manipulating instructions, context, or interaction structure, adver- saries can steer models toward generating factually incorrect infor- mation, violating behavioral constraints, or producing harmful or sensitive content, posing significant risks to deployed systems [22]. Existing adversarial strategies span a wide range of mechanisms, including prompt injection, role-playing and persona manipulation, instruction obfuscation, multi-turn coercion [17,42], and indirect attacks embedded within external content such as documents or code, and even by fine-tuning [39,40]. A central challenge in defend- ing against these attacks is their adaptability: adversarial prompts are often transferable across models and can be easily modified to evade static defenses [48]. As a result, surface-level or prompt-based mitigation strategies have shown limited robustness 2.2 Jailbreak attacks A prominent and particularly challenging class of adversarial at- tacks on LLMs is jailbreaking. Jailbreak attacks aim to circumvent built-in safety mechanisms and alignment constraints, enabling the model to produce outputs that it is explicitly designed to refuse. These attacks often rely on prompt engineering techniques such as hypothetical scenarios, instruction overriding, contextual refram- ing, or step-by-step coercion, effectively manipulating the model’s internal decision-making processes [37]. Unlike general adversarial prompting, jailbreak attacks explicitly target safety guardrails and content moderation policies, making them a critical concern from both security and governance perspectives [17,22,48]. Despite extensive efforts to harden models through alignment training and reinforcement learning from human feedback [6,14,36], jailbreak prompts continue to evolve, highlighting fundamental limitations in current defense approaches. This motivates the need for meth- ods that analyze jailbreak behavior at the level of internal model representations, rather than relying solely on external prompt or output inspection. 2.3 Jailbreak Defenses Prior work on defending against jailbreak attacks in LLMs has pri- marily focused on prompt and output-level safeguards. Rule-based filtering and keyword matching are commonly used due to their low computational cost, but such approaches are brittle and easily by- passed through paraphrasing, obfuscation, or multi-turn prompting [11]. Learning-based defenses, including supervised classifiers and auxiliary LLMs for intent detection or self-evaluation [34], improve robustness but introduce additional complexity, inference overhead, and new attack surfaces. Model-level defenses, such as alignment fine-tuning, reinforcement learning from human feedback (RLHF), and policy-based or constitutional training, aim to internalize safety constraints within the model [25]. While effective to an extent, these approaches are resource-intensive and require continual updates as jailbreak strategies evolve. Moreover, even extensively aligned mod- els remain susceptible to jailbreak attacks, indicating fundamental limitations in current training-based defenses. Overall, existing de- fenses largely treat jailbreak detection as a black-box problem and rely on external signals from prompts or generated outputs. In con- trast, fewer works explore the internal representations of LLMs as a basis for defense [7]. This gap motivates approaches that leverage latent-space and layer-wise signals to identify jailbreak behavior in an interpretable and architecture-agnostic manner, without re- quiring additional fine-tuning or auxiliary models. 3 Preliminaries 3.1 Tensors Tensors [30] are defined as multi-dimensional arrays that general- ize one-dimensional arrays (vectors) and two-dimensional arrays (matrices) to higher dimensions. The dimension of a tensor is tra- ditionally referred to as its order, or equivalently, the number of modes, while the size of each mode is called its dimensionality. For instance, we may refer to a third-order tensor as a three-mode tensorX∈R 퐼×퐽×퐾 . 3.2 Tensor Decomposition Tensor Decomposition [30] is a popular data science tool for discov- ering underlying low-dimensional patterns in the data. We focus on the CANDECOMP/PARAFAC (CP) decomposition model [30], one of the most famous tensor decomposition models that decom- poses a tensor into a sum of rank-one components. We use CP decomposition because of its simplicity and interpretability. The CP decomposition of a three-mode tensorX∈R 퐼×퐽×퐾 is the sum of three-way outer products, that is,X≈ Í 푅 푟=1 a 푟 ◦b 푟 ◦c 푟 , where 푅is the rank of the decomposition, a 푟 ∈R 퐼 , b 푟 ∈R 퐽 , and c 푟 ∈R 퐾 Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks in Large Language Models, , are the factor vectors and◦denotes the outer product. The rank of a tensorXis the minimal number of rank-1 tensors required to exactly reconstruct it: rank(X)= min ( 푅 :X= 푅 ∑︁ 푟=1 a 푟 ◦ b 푟 ◦ c 푟 , a 푟 ∈R 퐼 , b 푟 ∈R 퐽 , c 푟 ∈R 퐾 ) Selecting an appropriate rank is critical, as it directly affects both the expressiveness and interpretability of the decomposition. Lower- rank approximations yield compact and computationally efficient representations, while higher ranks can capture richer structure at the cost of increased complexity and potential noise. 3.3 Transformer Architecture A Transformer [31] is a neural network architecture designed for modeling sequential data through attention mechanisms rather than recurrence or convolution. Transformers process input sequences in parallel and capture long-range dependencies by explicitly mod- eling interactions between all tokens in a sequence. A transformer consists of a stack of layers, each composed of two primary submodules: multi-head self-attention and a position-wise feed-forward network (FFN). Residual connections and layer normal- ization are applied around each submodule to stabilize training. 3.3.1Multi-Head Self-Attention. Multi-head self-attention enables the model to attend to different parts of the input sequence simul- taneously. Given an input representation H∈R 푇×푑 , each attention head projects H into query (Q), key (K), and value (V) matrices: Q= HW 푄 ,K= HW 퐾 ,V= HW 푉 . Attention is computed as Attn(Q, K, V)= softmax QK ⊤ √ 푑 푘 V, where푑 푘 is the dimensionality of each attention head. Multiple attention heads operate in parallel, and their outputs are concate- nated and linearly projected, allowing the model to capture diverse relational patterns across tokens. 3.3.2Layer Outputs and Hidden Representations. Each transformer layer produces a hidden representation (or layer output) that serves as input to the next layer. Formally, for layerℓ, the output repre- sentation H (ℓ) is given by: H (ℓ) = LN H (ℓ−1) + MHA H (ℓ−1) , followed by H (ℓ) = LN H (ℓ) + FFN H (ℓ) , whereMHAdenotes multi-head attention,FFNdenotes the feed- forward network, and LN denotes layer normalization. The sequence of hidden statesH (1) , . . .,H (퐿) captures increas- ingly abstract features, ranging from local syntactic patterns in early layers to semantic and task-relevant representations in deeper lay- ers. These intermediate representations are commonly referred to as hidden layer activations and form the basis for interpretability and internal behavior analysis. 3.4 Model Types: Base, Instruction-Tuned, and Abliterated Models Large language models (LLMs) can be categorized based on their training and alignment processes, which influence their behavior under adversarial conditions. Base Models. These are pretrained on large-scale text corpora using self-supervised objectives without explicit instruction tuning. Examples include GPT-J and LLaMA. Base models capture broad language patterns but lack alignment with human preferences, making them prone to generate unrestricted or unsafe outputs. Instruction-Tuned Models. Derived from base models via super- vised fine-tuning on datasets containing human instructions and responses, these models improve instruction-following capabili- ties [44] and enforce safety constraints, such as refusing harmful queries. While instruction tuning enhances safety, these models remain susceptible to sophisticated jailbreak prompts. Abliterated Models. Abliterated models are instruction-tuned models where alignment or safety components have been removed, disabled, or bypassed [41]. Such models behave more like base models but may retain subtle differences due to prior fine-tuning. Abliterated models serve as valuable testbeds to study jailbreak vulnerabilities and internal representational changes resulting from alignment removal. Analyzing these model types enables us to investigate how align- ment and instruction tuning affect internal layer activations and latent patterns, informing the design of robust jailbreak detection methods that generalize across model variations. 4 Proposed Method We study jailbreak behavior through internal model representations using two complementary analysis pipelines. 4.1 Hidden Representation Analysis Model Suite and Representation Extraction. We perform hid- den representation analysis by examining both multi-head self- attention outputs and layer-wise hidden representations across a diverse set of large language models. Specifically, we evaluate three base models: GPT-J-6B [32], LLaMA-3.1-8B [5], and Mistral-7B-v01 [16]; three instruction-tuned models: GPT-JT-6B [10], LLaMA-3.1- 8B-Instruct [4], and Mistral-7B-Instruct-V0.1 [3]; one abliterated model, LLaMA-3.1-8B-Instruct-Abliterated [19]; and one state-space sequence model, Mamba-2.8b-hf [2]. This selection enables a sys- tematic comparison across different stages of alignment and archi- tectural paradigms. The inclusion of base, instruction-tuned, and abliterated models is intentional. Base models offer insight into unaligned latent struc- tures; instruction-tuned models show how safety fine-tuning alters internal processing; and abliterated models help isolate the role of alignment layers. We focus not on output quality but on how jailbreak and benign prompts are internally encoded across this alignment spectrum. From this perspective, the specific semantic quality of the output is not critical; instead, we focus on identifying discriminative patterns that persist across model variants. Together, these model categories enable us to analyze jailbreak behavior , ,Kadali et al. Construct 3-mode activation tensor Effective separation of factors in the latent space Benign Jailbreak Input prompts to LLM Extract representations from each layer l= 1,2,3...,L forms a 3-mode tensor Benign Jailbreak Intermediate layers' data LLM Layers Output Sequence length Token embedding length Layer 'l'extracted for a single input prompt Token embedding length Sequence length Inference time prompt LLM Layers Layer 'l' extracted for the prompt at inference A * New prompt Resultant prompt layer in latent space Project layer representations onto learned latent factors ? ? Classify using the trained factors If layer wise Jailbreak Prob > threshold, layer might be showing more signals ofjailbreak attack Layers in red are bypassed as they exhibit stronger jailbreak signature Layer bypass prevented harmful generation = Training a classifier on prompt-modefactors Latent Analysis and Classifier Training Jailbreak Mitigation at Inference Identify Jailbreak sensitive layers Intermediate layers' data Layer output/ MHA output Layer a Layer b Layer c Layer d Layer e Layer f V V Layer x Susceptible layer bypassing Figure 1: Proposed method: (top) Latent analysis and classifier training: self-attention and layer-output tensors are constructed from input prompts, decomposed via CP decomposition, and used to learn jailbreak-discriminative latent factors. (Bottom) Inference-time mitigation: internal representations from a new prompt are projected onto the learned factors to estimate layer-wise jailbreak susceptibility; layers exhibiting strong adversarial signals are bypassed to suppress jailbreak behavior. across the full alignment spectrum and assess whether latent-space signatures of jailbreak prompts are consistent and model-agnostic. Tensor Construction and Latent Factors for Jailbreak Detec- tion. For a given set of input prompts containing both benign and jailbreak instances, we extract multi-head attention representations and hidden state/layer output from each layer of the model. For a single prompt, the resulting representation has dimensions 1×푇×푑, where푇denotes the sequence length and푑is the hidden dimen- sionality. By stacking representations across multiple prompts, we construct a third-order tensor of size푁 ×푇 × 푑, where푁is the number of prompts as illustrated in Fig. 1. To analyze the latent structure of these internal representations, we apply the CANDECOMP/PARAFAC (CP) tensor decomposition to factorize the tensor into three low-rank factors corresponding to the prompt, sequence, and hidden dimensions. The factor associated with the prompt mode captures latent patterns that reflect how different prompts are encoded internally across model layers. Prior work [26,46] has shown that tensor decomposition-derived latent patterns effectively capture meaningful structure for classification and detection tasks, even with limited data [18, 27]. We use these prompt-mode latent factors as features for a light- weight classifier that distinguishes jailbreak prompts from benign prompts. This classifier serves two purposes: to assess separability between prompt types in the latent space (Fig. 2), and as a mech- anism to estimate layer-wise susceptibility to jailbreak behavior, which is leveraged by the mitigation method described next. Layer-wise Separability and Susceptibility. To localize where jailbreak-related information is expressed within the network, we train separate classifiers on latent factors extracted from individ- ual layers. This produces a layer-wise profile that quantifies how Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks in Large Language Models, , Figure 2: t-SNE visualization of prompt-mode CP factors for a representative model. Clear separation between benign and jailbreak clusters indicates that internal latent factors capture strong structure, motivating their use for jailbreak detection. Similar patterns are observed across models. strongly each layer encodes representations associated with adver- sarial prompts. From an interpretability perspective, these layers can be viewed as critical representational stages where benign and adversarial behaviors diverge. Importantly, we do not interpret high separability as evidence that a layer directly causes jailbreak behavior. Rather, it indicates that these layers capture discriminative representations associated with jailbreak prompts, making them especially informative for detection. This observation provides insight into how adversarial instructions propagate through the model and forms the basis for the inference-time intervention introduced in the following section. 4.2 Layer-Aware Mitigation via Latent-Space Susceptibility To mitigate jailbreak attacks, we propose a representation-level defense method, that leverages layer-wise susceptibility signals derived from internal representations. By identifying layers that strongly encode jailbreak-specific representations, we selectively bypass them during inference [13,29]. While prior work has ex- plored layer bypassing primarily for reducing computation and improving inference efficiency, our approach demonstrates that such bypassing can simultaneously reduce computational cost and mitigate jailbreak behaviors [20, 24]. We conduct this experiment on an abliterated model (LLaMA- 3.1-8B). Base models are excluded from this intervention as they primarily perform next-token prediction without alignment con- straints, making output-based safety evaluation less meaningful. Instruction-tuned models are also not ideal candidates, as their built-in guardrails obscure whether observed safety improvements arise from our method or from prior alignment. Abliterated models, which lack safety mechanisms while retaining instruction-tuned structure, provide a suitable testbed for isolating the effects of our approach. Layer-wise Projection and Jailbreak Susceptibility Scoring. Given an input prompt, we extract intermediate representations from each transformer layer of the model. Let x∈R 푑 denote the feature vector obtained from a specific layer (e.g., multi-head self- attention output or layer output). Latent Projection. For each layer, we project the extracted fea- tures onto a lower-dimensional latent space defined by factors obtained from tensor decomposition of the corresponding instruct model representations obtained in §4.1. Let W∈R 푑×푟 denote the matrix of푟basis vectors (factors). The projected representation is computed as: z= W ⊤ x, where z∈R 푟 is the latent feature representation. This operation constitutes a linear projection that preserves task-relevant structure encoded by the factors. Layer-wise Jailbreak Probability Estimation. The projected features are passed to a classifier trained to distinguish between benign and jailbreak prompts. For a logistic regression classifier, the probability of a jailbreak at a given layer is computed as: 푝= 휎(w ⊤ z+푏), where w and푏denote the classifier weights and bias, respectively, and 휎(·) is the sigmoid function: 휎(푥)= 1 1+푒 −푥 . Classifier Training Objective. The classifier is trained using labeled projected representations(z 푖 ,푦 푖 ), where푦 푖 ∈ 0,1indi- cates benign or jailbreak prompts. The parameters are optimized by minimizing the binary cross-entropy loss: L=− 1 푁 푁 ∑︁ 푖=1 [ 푦 푖 log푝 푖 +(1−푦 푖 ) log(1− 푝 푖 ) ] , where푁denotes the number of samples and푝 푖 is the predicted probability for sample 푖. Layer Susceptibility Interpretation. Layers exhibiting higher classification performance (e.g., F1 score) indicate stronger represen- tational separability between benign and jailbreak prompts within the latent space. We interpret such layers as being more suscepti- ble to jailbreak-style perturbations, as they encode discriminative adversarial features. 5 Experimental Evaluation 5.1 Datasets and Prompt Scope We used two prompt sources with provenance relationships to separate representation learning from mitigation evaluation. Training/representation analysis. For latent-space analysis, we use theJailbreak Classification datasetfrom Hugging Face [15]. This dataset provides labeled benign and jailbreak prompts and is used to (i) extract layer-wise hidden representations and multi-head attention (MHA) outputs, (i) learn CP decomposition factors for each layer and representation type, and (i) train a light- weight classifier on the resulting latent features. , ,Kadali et al. 012345678910111213141516171819202122232425262728293031 Layer Number GPT-J-Base GPT-JT-Instruct Llama-Base Llama-Instruct Llama-Abliterated Mistral-Base Mistral-Instruct Model Layer Output F1 Scores 012345678910111213141516171819202122232425262728293031 Layer Number GPT-J-Base GPT-JT-Instruct Llama-Base Llama-Instruct Llama-Abliterated Mistral-Base Mistral-Instruct Model MHA Output F1 Scores 0.5 0.6 0.7 0.8 0.9 1.0 F1 Score 0.5 0.6 0.7 0.8 0.9 1.0 F1 Score F1 Score Analysis: Transformer Models Figure 3: (Left) Layer-wise F1 scores using CP-decomposed Transformer layer outputs. (Right) Layer-wise F1 scores using CP-decomposed multi-head attention representations. Jailbreak and benign prompts become reliably separable at early depths, suggesting that adversarial intent is encoded shortly after input embedding. The strong performance of attention-based features further indicates that prompt-type information is reflected in token interaction structure as well as in hidden representations. Test/mitigation evaluation. To evaluate layer-aware bypass at inference time, we construct a held-out test set of 200 prompts (100 benign, 100 jailbreak) sourced from the‘In the Wild Jailbreak Prompts’[28] prompt collection. [15] reports that its jailbreak prompts are drawn from [28], which motivates this choice, as these evaluations are consistent and distinct from the training corpus. Attack scope. Both datasets primarily consist of instruction- level jailbreaks (e.g., persona overrides, explicit safety negation, role-play framing, and meta-instructions). They do not include optimization-based attacks such as GCG [48], PAIR [8], or other gradient-guided adversarial suffix constructions. Since our frame- work operates on internal representations, extending it to additional attack families can be achieved by incorporating corresponding prompt distributions during factor learning; we leave such evalua- tions to future work. 5.2 Hidden Representation Analysis We assess whether internal model representations can reliably sep- arate jailbreak from benign prompts across model families, layers, and representation types. Our analysis spans eight models: base, instruction-tuned, abliterated, and state-space (Mamba), providing a broad view across training paradigms. For each model, we extract hidden states and multi-head atten- tion (MHA) outputs across all layers. These are aggregated into third-order tensors (푁 ×푇 ×푑) and decomposed using CP tensor decomposition (rank푟=20) to obtain low-dimensional features for each prompt. We fix the CP decomposition rank to푟=20 for all experiments, balancing expressiveness and efficiency as discussed in §3.2, and to ensure consistent latent representations across mod- els. A lightweight classifier is trained on these features to predict jailbreak status. Fig. 3 presents layer-wise F1 scores for each model. For Mamba, we report results from both the mixer (analogous to MHA) and full block output in Fig. 4. 08162432404856 Layer 0.5 0.6 0.7 0.8 0.9 1.0 F1 Score Mamba-2.8B: Block vs Mixer Output F1 Scores Block Output Mixer Output Figure 4: Layer-wise F1 scores for CP-decomposed Mamba representations (mixer and block outputs) showing early and increasing separability between benign and jailbreak prompts, indicating that state-space architectures encode ad- versarial prompt structure in their internal representations. Our results show a clear separation between the two prompt types in the learned latent space, achieving consistently high F1 scores across all evaluated models. These findings suggest that jailbreak behavior manifests as identifiable and discriminative pat- terns within internal representations, independent of output quality or alignment stage, and can be effectively leveraged for detection without model fine-tuning. Qualitative Analysis. For clarity of presentation, we visualize qualitative results for instruction-tuned models, which provide the most interpretable view of aligned internal dynamics. The qualita- tive patterns discussed here are representative of those observed across all evaluated models. Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks in Large Language Models, , 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Benign 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Jailbreak 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Difference 10 8 6 4 2 0 10 8 6 4 2 0 4 2 0 2 4 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Benign 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Jailbreak 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Difference 10 8 6 4 2 0 10 8 6 4 2 0 4 2 0 2 4 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Benign 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Jailbreak 064128192256320384448 Key Token 0 64 128 192 256 320 384 448 Query Token Difference 10 9 8 7 6 5 4 3 2 10 9 8 7 6 5 4 3 2 4 3 2 1 0 1 2 3 4 GPT-JT-6B - Layer 7 LLaMA-3.1-8B - Layer 4 Mistral-7B - Layer 18 Figure 5: Self-attention maps for three instruction-tuned models, averaged over benign and jailbreak prompts (log 10 scale). Difference maps (right) highlight systematic but lo- calized changes in attention patterns induced by jailbreak prompts, suggesting that adversarial intent manifests as tar- geted rerouting of attention rather than global disruption. Aggregated Self-Attention Heatmaps. We use aggregated self- attention heatmaps to qualitatively assess how jailbreak prompts alter token-to-token information routing within the model. While attention alone does not encode semantic content, systematic dif- ferences in attention patterns can indicate how adversarial prompts redirect internal focus during processing. For each instruction-tuned model and transformer layerℓ, we extract the self-attention weight tensors A (ℓ) ∈R 푁×퐻×푇×푇 , where푁is the number of prompts,퐻the number of attention heads, and푇the (padded) token length. To obtain a stable, global view of attention behavior, we aggregate over both prompts and heads. LetI ben andI jb denote the index sets of benign and jailbreak prompts, respectively. We compute the class-wise, head-averaged attention maps: ̄ A (ℓ) ben = 1 |I ben |퐻 ∑︁ 푛∈I ben 퐻 ∑︁ ℎ=1 A (ℓ) 푛,ℎ,:,: , ̄ A (ℓ) jb = 1 |I jb |퐻 ∑︁ 푛∈I jb 퐻 ∑︁ ℎ=1 A (ℓ) 푛,ℎ,:,: . To highlight systematic differences between prompt types, we ad- ditionally compute a per-layer difference map: ΔA (ℓ) = ̄ A (ℓ) jb − ̄ A (ℓ) ben . Visualizing ̄ A (ℓ) ben, ̄ A (ℓ) jb, andΔA (ℓ) (Fig. 6) shows that jail- break prompts lead to consistent, localized changes in attention patterns. This indicates that adversarial prompts influence attention 064128192256320384448 Token Position 0 4 8 12 16 20 24 Layer Benign 064128192256320384448 Token Position 0 4 8 12 16 20 24 Layer Jailbreak 064128192256320384448 Token Position 0 4 8 12 16 20 24 Layer Difference 064128192256320384448 Token Position 0 4 8 12 16 20 24 28 Layer Benign 064128192256320384448 Token Position 0 4 8 12 16 20 24 28 Layer Jailbreak 064128192256320384448 Token Position 0 4 8 12 16 20 24 28 Layer Difference 064128192256320384448 Token Position 0 4 8 12 16 20 24 28 Layer Benign 064128192256320384448 Token Position 0 4 8 12 16 20 24 28 Layer Jailbreak 064128192256320384448 Token Position 0 4 8 12 16 20 24 28 Layer Difference 4 5 6 7 8 4 5 6 7 8 2 1 0 1 2 0 1 2 3 4 5 6 0 1 2 3 4 5 6 0.2 0.1 0.0 0.1 0.2 1 0 1 2 3 4 5 1 0 1 2 3 4 5 2.0 1.5 1.0 0.5 0.0 0.5 1.0 1.5 2.0 GPT-JT-6B LLaMA-3.1-8B Mistral-7B Figure 6: Layer-wise log-magnitude of hidden representa- tions for benign (left) and jailbreak (middle) prompts, aver- aged across prompts, with their difference shown on the right. The difference heatmaps reveal consistent, localized devia- tions across layers, highlighting where adversarial prompts induce layer-dependent representational shifts. by selectively emphasizing specific instruction or control tokens, providing qualitative evidence that jailbreak behavior arises from targeted changes in information flow rather than global attention disruption. Hidden-Representation Magnitude Heatmaps. While atten- tion maps reflect information routing, hidden representations cap- ture the content and intensity of internal computation. We there- fore analyze layer-wise hidden-state magnitudes to understand how strongly jailbreak prompts perturb internal activations across network depth. For each layer ℓ , we extract hidden states H (ℓ) ∈R 푁×푇×퐷 , where퐷is the hidden dimensionality. To summarize activation strength across token positions, we compute the per-tokenℓ 2 mag- nitude: M (ℓ) 푛,푡 = H (ℓ) 푛,푡,: 2 ,M (ℓ) ∈R 푁×푇 . We then average magnitudes across prompts within each class: ̄ M (ℓ) ben (푡)= 1 |I ben | ∑︁ 푛∈I ben M (ℓ) 푛,푡 , ̄ M (ℓ) jb (푡)= 1 |I jb | ∑︁ 푛∈I jb M (ℓ) 푛,푡 . For visualization, we apply a logarithmic transform: ̃ M (ℓ) (푡)= log ̄ M (ℓ) (푡)+ 휀 , with a small휀>0 for numerical stability. We plot ̃ M (ℓ) (푡) as heatmaps with layers on the y-axis and token positions on x-axis. , ,Kadali et al. Although the averaged hidden-state magnitudes for benign and jailbreak prompts appear broadly similar (especially for LLaMA abd Mistral), their difference heatmaps reveal consistent, localized deviations across layers. This indicates that jailbreak behavior does not manifest as a global disruption of internal activations, but rather as subtle, structured changes superimposed on otherwise normal model processing. 5.3 Layer-Aware Mitigation via Latent-Space Susceptibility We evaluate our second proposed method, layer-aware mitigation via latent-space susceptibility, to check whether representation-level signals can be used to suppress jailbreak execution during inference without relying on output-level filtering or fine-tuning. Experimental Setup. We conduct this experiment on the abliter- ated LLaMA 3.1 8B model described earlier. Evaluation is performed on a held-out set of 200 prompts (100 benign, 100 jailbreak), using latent factors learned during the analysis phase (§5.2). Inference-time Susceptibility Scoring. Given an input prompt at inference time, we extract layer outputs and attention represen- tations, project them onto the pre-learned CP factors, and use a lightweight classifier to compute a per-layer susceptibility score in- dicating the strength of jailbreak-correlated features. Layers whose predicted jailbreak probability exceeds a fixed threshold (휏=0.7) are treated as highly susceptible. The threshold휏is a tunable hyper- parameter that controls the trade-off between mitigation strength and preservation of benign behavior. Layer-/Head-Bypass Intervention. Based on the susceptibility score, we selectively perform: (i) Layer Bypass: bypassing selected layer outputs; and (i) MHA Bypass: bypassing selected attention components. This intervention is parameter-free (no fine-tuning), prompt-conditional (depends on the susceptibility profile), and does not require any output-side heuristics. Output-Based Evaluation with LLM-Assisted Judging. Since our goal is to prevent harmful compliance rather than optimize helpfulness, we evaluate mitigation effectiveness based on observed output behavior. Model responses are categorized as: (i) harmful completions, where the jailbreak intent succeeds; (i) benign comple- tions, where the model responds appropriately; and (i) disrupted outputs, including truncated, repetitive, or incoherent text. For jailbreak prompts, disrupted or non-compliant outputs are treated as successful defenses, while for benign prompts such out- puts are undesirable. Output labels are assigned using an LLM-as-a- judge rubric, followed by manual review of ambiguous cases. This evaluation protocol follows established practice for open-ended generation assessment with human validation [12,21,47]. Based on these criteria, we define the confusion matrix as follows: Prompt TypeObserved OutputOutcome JailbreakHarmful completionFalse Negative (FN) Jailbreak Disrupted/benign outputTrue Positive (TP) BenignBenign completionTrue Negative (TN) BenignDisrupted outputFalse Positive (FP) Table 1: Confusion matrix counts for latent-space-guided mitigation (100 jailbreak and 100 benign prompts). MethodTPFNTNFP Layer Bypass7822946 MHA Bypass6139928 Results. Table 1 summarizes the confusion-matrix counts. Layer- guided bypass suppresses most jailbreak attempts (TP=78) while largely preserving benign behavior (TN=94). In contrast, MHA- only bypass results in substantially more jailbreak failures (FN=39), indicating that layer outputs capture a larger fraction of jailbreak- relevant computation than attention components alone. To provide a compact summary aligned with prior jailbreak eval- uations, we additionally report the attack success rate (ASR), defined as the fraction of jailbreak prompts that remain successful after mitigation. Layer-output bypass reduces ASR to 22% (22/100), compared to 39% for MHA-only bypass, highlighting the effective- ness of layer-level intervention. Failure (False Negative) Analysis. We examine the 22 jail- break prompts that remain harmful after layer-output bypass. The majority are persona- or roleplay-based prompt injections (e.g., “never refuse,” “no morals,” forced speaker tags such as “AIM:” and “[H4X]:”) that aim to establish persistent control over the model’s identity, tone, and formatting. Because such instructions are re- peatedly reinforced throughout the prompt, elements of adversarial control can persist even when highly susceptible layers are by- passed. Additional failures stem from susceptibility estimation: the intervention targets layers exceeding a fixed probability threshold chosen to preserve benign behavior. Attacks that distribute their influence across multiple layers, or weakly activate any single layer, may therefore evade suppression despite succeeding overall. Some failures also involve milder jailbreaks that retain adversarial fram- ing without immediately producing explicit harmful content; under our conservative evaluation criterion, these are counted as failures. These limitations are addressable within the proposed frame- work by expanding the diversity of jailbreak styles used for latent factor learning and by adopting adaptive or cumulative suscepti- bility criteria. Since the method operates entirely in latent space, such extensions require no architectural changes. 6 Conclusion Our hypothesis and experiments indicate that internal represen- tations of LLMs contain sufficiently strong and consistent signals to both detect jailbreak prompts and, in many cases, disrupt jail- break execution at inference time. Importantly, these capabilities emerge from lightweight representation-level analysis and interven- tion, without requiring additional post-training, auxiliary models, or complex rule-based filtering. The consistency of these findings across diverse model families suggests that adversarial intent leaves stable latent-space signatures, motivating internal-representation monitoring as a practical and broadly applicable direction for un- derstanding and mitigating jailbreak behavior. Jailbreaking Leaves a Trace: Understanding and Detecting Jailbreak Attacks in Large Language Models, , 7 GenAI Usage Disclosure The authors acknowledge the use of AI-based writing and coding assistance tools during the preparation of this manuscript. These tools were used exclusively to improve clarity, organization, and academic tone of text written by the authors, as well as to assist with code formatting and plot generation. All scientific ideas, methodolo- gies, analyses, and conclusions are the original intellectual contri- butions of the authors. No AI system was used to generate research ideas or substantive technical content, and all AI-assisted revisions were carefully reviewed and validated by the authors. 8 Acknowledgements Research was supported by the National Science Foundation under CAREER grant no. IIS 2046086 and also sponsored by the Army Re- search Office and was accomplished under Grant Number W911NF- 24-1-0397. The views and conclusions contained in this document are those of the authors and should not be interpreted as represent- ing the official policies, either expressed or implied, of the Army Research Office or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Government purposes, notwithstanding any copyright notation herein. References [1] [n. d.]. Implementation of the proposed method. https://anonymous.4open. science/r/Jailbreaking-leaves-a-trace-Understanding-and-Detecting-Jailbreak- Attacks-in-LLMs-C401 [2] [n. d.]. Mamab2.8b. https://huggingface.co/state-spaces/mamba-2.8b-hf [3] Mistral AI. 2023. Mistral 7B Instruct v0.1. https://huggingface.co/mistralai/ Mistral-7B-Instruct-v0.1. [4]AI@Meta. 2024. Llama 3 Instruct Models. https://ai.meta.com/llama/. LLaMA- 3.1-8B-Instruct. [5]AI@Meta. 2024. Llama 3 Models. https://ai.meta.com/llama/. LLaMA-3.1-8B base model. [6] Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, Nicholas Joseph, Saurav Kadavath, Jackson Kernion, Tom Conerly, Sheer El-Showk, Nelson Elhage, Zac Hatfield-Dodds, Danny Hernandez, Tristan Hume, Scott Johnston, Shauna Kravec, Liane Lovitt, Neel Nanda, Catherine Olsson, Dario Amodei, Tom Brown, Jack Clark, Sam McCandlish, Chris Olah, Ben Mann, and Jared Kaplan. 2022. Training a Helpful and Harmless Assistant with Reinforcement Learning from Human Feedback. arXiv:2204.05862 [cs.CL] https://arxiv.org/abs/2204.05862 [7]Leyla Naz Candogan, Yongtao Wu, Elias Abad Rocamora, Grigorios Chrysos, and Volkan Cevher. 2025. Single-pass Detection of Jailbreaking Input in Large Language Models. Transactions on Machine Learning Research (2025). https: //openreview.net/forum?id=42v6I5Ut9a [8] Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2024. Jailbreaking Black Box Large Language Models in Twenty Queries. arXiv:2310.08419 [cs.LG] https://arxiv.org/abs/2310.08419 [9]Bocheng Chen, Advait Paliwal, and Qiben Yan. 2023. Jailbreaker in Jail: Moving Target Defense for Large Language Models. arXiv:2310.02417 [cs.CR] https: //arxiv.org/abs/2310.02417 [10]Together Computer. 2022. GPT-JT-6B: Instruction-Tuned GPT-J Model. https: //huggingface.co/togethercomputer/GPT-JT-6B-v1. [11]Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2024. Multilingual Jailbreak Challenges in Large Language Models. arXiv:2310.06474 [cs.CL] https: //arxiv.org/abs/2310.06474 [12]Yann Dubois, Aarohi Srivastava, Abhinav Venigalla, et al.2024. Length-Controlled AlpacaEval: A Simple Way to Debias Automatic Evaluators. arXiv preprint arXiv:2404.04475 (2024). https://arxiv.org/abs/2404.04475 [13] Mostafa Elhoushi, Akshat Shrivastava, Diana Liskovich, Basil Hosmer, Bram Wasti, Liangzhen Lai, Anas Mahmoud, Bilge Acun, Saurabh Agarwal, Ahmed Roman, Ahmed Aly, Beidi Chen, and Carole-Jean Wu. 2024. LayerSkip: Enabling Early Exit Inference and Self-Speculative Decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, 12622–12642. doi:10.18653/ v1/2024.acl-long.681 [14]Amelia Glaese, Nat McAleese, Maja Trębacz, John Aslanides, Vlad Firoiu, Timo Ewalds, Maribeth Rauh, Laura Weidinger, Martin Chadwick, Phoebe Thacker, Lucy Campbell-Gillingham, Jonathan Uesato, Po-Sen Huang, Ramona Comanescu, Fan Yang, Abigail See, Sumanth Dathathri, Rory Greig, Charlie Chen, Doug Fritz, Jaume Sanchez Elias, Richard Green, Soňa Mokrá, Nicholas Fernando, Boxi Wu, Rachel Foley, Susannah Young, Iason Gabriel, William Isaac, John Mellor, Demis Hassabis, Koray Kavukcuoglu, Lisa Anne Hendricks, and Geoffrey Irving. 2022. Improving alignment of dialogue agents via targeted human judgements. arXiv:2209.14375 [cs.LG] https://arxiv.org/abs/2209.14375 [15]Jack Hao. [n. d.]. Jailbreak Classification Dataset. https://huggingface.co/datasets/ jackhhao/jailbreak-classification [16]Albert Q. Jiang, Alexandre Sablayrolles, et al.2023. Mistral 7B. https://mistral.ai/ news/introducing-mistral-7b/. [17]Liwei Jiang, Kavel Rao, Seungju Han, Allyson Ettinger, Faeze Brahman, Sachin Kumar, Niloofar Mireshghallah, Ximing Lu, Maarten Sap, Yejin Choi, and Nouha Dziri. 2024. WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models. arXiv:2406.18510 [cs.CL] https://arxiv.org/abs/2406. 18510 [18]Sri Durga Sai Sowmya Kadali and Evangelos Papalexakis. 2025. CoCoTen: Detect- ing Adversarial Inputs to Large Language Models through Latent Space Features of Contextual Co-occurrence Tensors. In Proceedings of the 34th ACM Interna- tional Conference on Information and Knowledge Management (CIKM ’25). ACM, 4857–4861. doi:10.1145/3746252.3760886 [19] M. Labonne. [n. d.]. Meta-Llama-3.1-8B-Instruct-abliterated. https://huggingface. co/mlabonne/Meta-Llama-3.1-8B-Instruct-abliterated. Hugging Face model card. [20]Tim Lawson and Laurence Aitchison. 2025. Learning to Skip the Middle Layers of Transformers. arXiv:2506.21103 [cs.LG] https://arxiv.org/abs/2506.21103 [21]Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024. LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods. arXiv:2412.05579 [cs.CL] https://arxiv.org/abs/2412.05579 [22]Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models. arXiv:2310.04451 [cs.CL] https://arxiv.org/abs/2310.04451 [23]Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, Huiping Zhuang, and Cen Chen. 2024. Eraser: Jailbreaking Defense in Large Language Models via Unlearning Harmful Knowledge. arXiv:2404.05880 [cs.CL] https: //arxiv.org/abs/2404.05880 [24] Xuan Luo, Weizhi Wang, and Xifeng Yan. 2025. Adaptive layer-skipping in pre-trained llms. arXiv preprint arXiv:2503.23798 (2025). [25] Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schul- man, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Pe- ter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. arXiv:2203.02155 [cs.CL] https://arxiv.org/abs/2203.02155 [26]Evangelos E. Papalexakis. 2018. Unsupervised Content-Based Identification of Fake News Articles with Tensor Decomposition Ensembles.https://api. semanticscholar.org/CorpusID:26675959 [27]Zubair Qazi, William Shiao, and Evangelos E. Papalexakis. 2024. GPT-generated Text Detection: Benchmark Dataset and Tensor-based Detection Method. In Companion Proceedings of the ACM Web Conference 2024 (Singapore, Singapore) (W ’24). Association for Computing Machinery, New York, NY, USA, 842–846. doi:10.1145/3589335.3651513 [28]Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. “Do Anything Now”: Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. In ACM SIGSAC Conference on Computer and Communications Security (CCS). ACM. [29]Mustafa Shukor and Matthieu Cord. 2024. Skipping Computations in Multimodal LLMs. In Workshop on Responsibly Building the Next Generation of Multimodal Foundational Models. https://openreview.net/forum?id=qkmMvLckB9 [30]Nicholas D Sidiropoulos, Lieven De Lathauwer, Xiao Fu, Kejun Huang, Evange- los E Papalexakis, and Christos Faloutsos. 2017. Tensor decomposition for signal processing and machine learning. IEEE Transactions on signal processing 65, 13 (2017), 3551–3582. [31]Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2023. Attention Is All You Need. arXiv:1706.03762 [cs.CL] https://arxiv.org/abs/1706.03762 [32]Ben Wang, Aran Komatsuzaki, and EleutherAI. 2021.GPT-J-6B.https:// huggingface.co/EleutherAI/gpt-j-6B. Model card and weights. [33] Hao Wang, Hao Li, Minlie Huang, and Lei Sha. 2024. ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 2697–2711. doi:10.18653/v1/ 2024.emnlp-main.157 [34]Xunguang Wang, Daoyuan Wu, Zhenlan Ji, Zongjie Li, Pingchuan Ma, Shuai Wang, Yingjiu Li, Yang Liu, Ning Liu, and Juergen Rahmel. 2025. SelfDe- fend: LLMs Can Defend Themselves against Jailbreaking in a Practical Manner. arXiv:2406.05498 [cs.CR] https://arxiv.org/abs/2406.05498 , ,Kadali et al. [35]Yihan Wang, Zhouxing Shi, Andrew Bai, and Cho-Jui Hsieh. 2024. Defending llms against jailbreaking attacks via backtranslation. arXiv preprint arXiv:2402.16459 (2024). [36] Yufei Wang, Wanjun Zhong, Liangyou Li, Fei Mi, Xingshan Zeng, Wenyong Huang, Lifeng Shang, Xin Jiang, and Qun Liu. 2023. Aligning Large Language Models with Human: A Survey. arXiv:2307.12966 [cs.CL] https://arxiv.org/abs/ 2307.12966 [37]Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How Does LLM Safety Training Fail? arXiv:2307.02483 [cs.LG] https://arxiv.org/abs/ 2307.02483 [38]Chen Xiong, Xiangyu Qi, Pin-Yu Chen, and Tsung-Yi Ho. 2025. Defensive Prompt Patch: A Robust and Interpretable Defense of LLMs against Jailbreak Attacks. arXiv:2405.20099 [cs.CR] https://arxiv.org/abs/2405.20099 [39]Xianjun Yang, Xiao Wang, Qi Zhang, Linda Petzold, William Yang Wang, Xun Zhao, and Dahua Lin. 2023. Shadow Alignment: The Ease of Subverting Safely- Aligned Language Models. arXiv:2310.02949 [cs.CL] https://arxiv.org/abs/2310. 02949 [40] Hongwei Yao, Jian Lou, and Zhan Qin. 2023. PoisonPrompt: Backdoor Attack on Prompt-based Large Language Models. arXiv:2310.12439 [cs.CL] https://arxiv. org/abs/2310.12439 [41]Richard J. Young. 2026. Comparative Analysis of LLM Abliteration Methods: A Cross-Architecture Evaluation. arXiv:2512.13655 [cs.CL] https://arxiv.org/abs/ 2512.13655 [42]Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2023. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253 (2023). [43]Yifan Zeng, Yiran Wu, Xiao Zhang, Huazheng Wang, and Qingyun Wu. 2024.AutoDefense: Multi-Agent LLM Defense against Jailbreak Attacks. arXiv:2403.04783 [cs.LG] https://arxiv.org/abs/2403.04783 [44]Shengyu Zhang, Linfeng Dong, Xiaoya Li, Sen Zhang, Xiaofei Sun, Shuhe Wang, Jiwei Li, Runyi Hu, Tianwei Zhang, Guoyin Wang, et al.2026. Instruction tuning for large language models: A survey. Comput. Surveys 58, 7 (2026), 1–36. [45]Tianrong Zhang, Bochuan Cao, Yuanpu Cao, Lu Lin, Prasenjit Mitra, and Jinghui Chen. 2024. WordGame: Efficient & Effective LLM Jailbreak via Simultaneous Obfuscation in Query and Response. arXiv:2405.14023 [cs.LG] https://arxiv.org/ abs/2405.14023 [46]Zhenjie Zhao, Andrew Cattle, Evangelos Papalexakis, and Xiaojuan Ma. 2019. Embedding Lexical Features via Tensor Decomposition for Small Sample Hu- mor Recognition. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (Eds.). Association for Computational Linguistics, Hong Kong, China, 6376–6381. doi:10.18653/v1/D19-1669 [47]Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, et al.2023. Judging LLM-as-a- judge with MT-Bench and Chatbot Arena. arXiv preprint arXiv:2306.05685 (2023). https://arxiv.org/abs/2306.05685 [48] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J. Zico Kolter, and Matt Fredrikson. 2023. Universal and Transferable Adversarial Attacks on Aligned Language Models. arXiv:2307.15043 [cs.CL] https://arxiv.org/abs/2307.15043