Paper deep dive
SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Consideration
Md Jueal Mia, Yanzhao Wu, Selcuk Uluagac, M. Hadi Amini
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 95%
Last extracted: 9/16/2026, 8:41:27 AM
Summary
This Systematization of Knowledge (SoK) paper re-evaluates jailbreak security in the context of modern Agentic AI systems. It highlights that strong native safety alignment in Large Language Models (LLMs) does not guarantee robustness against adversarial jailbreaks. The authors propose a unified taxonomy of attacks and defenses across the agentic execution pipeline (user interaction, planning, memory, tool use, and inter-agent communication) and introduce a security-utility-efficiency evaluation framework. Empirical results reveal that defense effectiveness is highly variable and that low final-response attack success rates can mask severe intermediate compromises in planning, memory, and tool interactions, necessitating a shift toward cross-layer, execution-aware security.
Entities (9)
Relation Signals (6)
Md Jueal Mia → affiliatedwith → Florida International University
confidence 99% · Md Jueal Mia, Knight Foundation School of Computing and Information Sciences, Florida International University, USA
Native Safety Alignment → doesnotimply → Robustness to Adversarial Jailbreaks
confidence 96% · First, strong native alignment does not imply robustness to adversarial jailbreaks.
Agentic AI → expandsattacksurface → Jailbreak Attacks
confidence 95% · Agentic systems expand the jailbreak attack surface beyond a single user–model interaction.
Qwen3.5-4b → usedinempiricalstudy → Security-Utility-Efficiency Framework
confidence 95% · controlled empirical evaluation of representative inference-time defenses using Qwen3.5-4B and Llama-3.1-8B-Instruct
Defense Effectiveness → dependson → Model, Attack, and Component
confidence 94% · Second, defense effectiveness is highly model-, attack-, and component-dependent
Final-Response Safety → canmask → Intermediate Compromise
confidence 93% · Third, low final-response attack success can mask severe intermediate compromise: planning, memory, and tool interactions may remain unsafe
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Large language models (LLMs) are rapidly evolving from conversational assistants into agentic AI systems that reason, plan, invoke tools, maintain persistent memory, communicate with other agents, and execute multi-step tasks. At the same time, modern models exhibit substantially stronger native safety alignment than earlier generations on which many jailbreak attacks and defenses were originally studied. This shift raises a fundamental question: \textit{which established jailbreak-security findings remain valid in the era of modern LLMs and agentic AI?} We address this question through a Systematization of Knowledge (SoK) that reframes jailbreak security around the full agentic execution pipeline. We develop unified taxonomies of attacks and defenses spanning user interaction, planning and reasoning, memory, tool use, and inter-agent communication, and introduce a security--utility--efficiency evaluation framework that separates native harmful-prompt safety, adversarial jailbreak robustness, and agent-level security outcomes. We further conduct a controlled empirical study of representative attacks and defenses within a common agentic framework. Our results reveal three important gaps. First, strong native alignment does not imply robustness to adversarial jailbreaks. Second, defense effectiveness is highly model-, attack-, and component-dependent and can come at substantial cost in over-refusal, utility, and latency. Third, low final-response attack success can mask severe intermediate compromise: planning, memory, and tool interactions may remain unsafe even when the final response is successfully filtered. These findings motivate a shift from response-centric jailbreak defense toward cross-layer, execution-aware security that protects agent state, component transitions, and external actions while preserving practical utility and efficiency.
Tags
Links
- Source: https://arxiv.org/abs/2609.12413v1
- Canonical: https://arxiv.org/abs/2609.12413v1
Trouble viewing inline? Open PDF directly →
Full Text
197,702 characters extracted from source content.
Expand or collapse full text
EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations MD JUEAL MIA, Knight Foundation School of Computing and Information Sciences, Florida International University, USA and Security, Optimization, and Learning for InterDependent Networks (solid) Laboratory, USA YANZHAO WU, Knight Foundation School of Computing and Information Sciences, Florida International University, USA SELCUK ULUAGAC, Knight Foundation School of Computing and Information Sciences, Florida International University, USA M. HADI AMINI, Knight Foundation School of Computing and Information Sciences, Florida International University, USA and Security, Optimization, and Learning for InterDependent Networks (solid) Laboratory, USA Large language models (LLMs) are rapidly evolving from conversational assistants into agentic AI systems that reason, plan, invoke tools, maintain persistent memory, communicate with other agents, and execute multi-step tasks. At the same time, modern models exhibit substantially stronger native safety alignment than earlier generations on which many jailbreak attacks and defenses were originally studied. This shift raises a fundamental question: which established jailbreak-security findings remain valid in the era of modern LLMs and agentic AI? We address this question through a Systematization of Knowledge (SoK) that reframes jailbreak security around the full agentic execution pipeline. We develop unified taxonomies of attacks and defenses spanning user interaction, planning and reasoning, memory, tool use, and inter-agent communication, and introduce a security–utility–efficiency evaluation framework that separates native harmful-prompt safety, adversarial jailbreak robustness, and agent-level security outcomes. We further conduct a controlled empirical study of representative attacks and defenses within a common agentic framework. Our results reveal three important gaps. First, strong native alignment does not imply robustness to adversarial jailbreaks. Second, defense effectiveness is highly model-, attack-, and component-dependent and can come at substantial cost in over-refusal, utility, and latency. Third, low final-response attack success can mask severe intermediate compromise: planning, memory, and tool interactions may remain unsafe even when the final response is successfully filtered. These findings motivate a shift from response-centric jailbreak defense toward cross-layer, execution-aware security that protects agent state, component transitions, and external actions while preserving practical utility and efficiency. Keywords: Agentic AI, Large Language Model Security, Jailbreak Attacks and Defenses, Execution-Aware Security, Systematization of Knowledge Md Jueal Mia, Yanzhao Wu, Selcuk Uluagac, and M. Hadi Amini. 2026. SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations. 1, 1 (September 2026), 51 pages. Authors’ Contact Information: Md Jueal Mia, mmia001@fiu.edu, Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, Florida, USA and Security, Optimization, and Learning for InterDependent Networks (solid) Laboratory, Miami, Florida, USA; Yanzhao Wu, yawu@fiu.edu, Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, Florida, USA; Selcuk Uluagac, suluagac@fiu.edu, Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, Florida, USA; M. Hadi Amini, moamini@fiu.edu, Knight Foundation School of Computing and Information Sciences, Florida International University, Miami, Florida, USA and Security, Optimization, and Learning for InterDependent Networks (solid) Laboratory, Miami, Florida, USA. © 2026 FOR EDUCATIONAL PURPOSES ONLY arXiv:2609.12413v1 [cs.AI] 11 Sep 2026 EDUCATIONAL RELEASE Mia et al. 1 Introduction Large language models (LLMs) have demonstrated strong capabilities across text generation, question answering, code generation, mathematical reasoning, and scientific discovery [11]. Building on these capabilities, LLMs are increasingly embedded in agentic AI systems that can plan, reason, retrieve information, invoke external tools, maintain persistent memory, coordinate with other agents, and execute multi-step tasks [128,129,179]. Unlike conventional prompt– response applications, agentic systems operate through iterative execution loops in which model outputs can influence subsequent reasoning, stored state, tools, and other agents. This transition raises a fundamental security question: which established findings about jailbreak attacks and defenses still hold for modern LLMs and agentic AI? The answer is increasingly unclear because the security baseline itself has changed. Modern instruction-tuned models are explicitly optimized to follow user instructions while respecting behavioral and safety constraints [2,110], and therefore often resist straightforward harmful prompts more effectively than earlier models. Yet strong native alignment does not imply jailbreak robustness: automated and optimization-based attacks continue to bypass aligned models using carefully constructed adversarial prompts [90,102,213]. Evaluating a defense only by its defended attack success rate (ASR) can therefore be misleading. What matters is the marginal security gain beyond native alignment, how that gain varies across attacks and models, and what utility and efficiency costs it introduces. Agentic AI makes this problem substantially harder by expanding both the entry vectors and objectives of attacks beyond the user–model interface. We treat the entry vector—direct user prompts, retrieved content, memory, tool outputs, or inter-agent messages—as distinct from the objective, such as safety-policy bypass, goal hijacking, data exfiltration, or unauthorized action. In this SoK, a prompt injection is counted as a jailbreak only when it bypasses an explicit model- or agent-level safety policy to induce prohibited generation, reasoning, storage, tool use, or execution; injections that only redirect an authorized task without such a bypass are outside our scope. Indirect content, persistent memory, tools, and inter-agent communication therefore constitute jailbreak-relevant trust boundaries through which adversarial instructions can propagate [24,27,29,192]. Consequently, response-level safety and agent-level security are not equivalent: a safe final response does not guarantee safe planning, memory, tool interaction, or execution. Practical deployment introduces a second challenge: security must remain affordable. Small language models (SLMs) are increasingly attractive for resource-constrained and specialized agentic workloads because of their lower inference cost, reduced memory requirements, and suitability for local or edge deployment [36]. Recent work further argues that SLMs can handle repetitive and specialized invocations within heterogeneous agent architectures, reserving larger models for more demanding tasks [8]. However, many jailbreak defenses require repeated inference, prompt perturbation, auxiliary safety models, or additional generation [78]. In an agentic execution loop, such overhead may recur across planning, tool use, verification, and inter-agent steps, making security–utility–efficiency trade-offs a first-order design concern. These developments expose a gap between traditional jailbreak evaluation and the security requirements of modern agentic AI. Existing jailbreak research largely asks whether adversarial prompts bypass model-level safeguards, whereas agent security must also reason about planning, tools, memory, persistent state, and autonomous execution. At the same time, stronger native alignment changes the baseline against which defenses should be judged. A meaningful evaluation must therefore distinguish harmful-prompt safety, adversarial jailbreak robustness, and agent-level security, while jointly considering over-refusal, task utility, latency, and computational cost. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Accordingly, this SoK systematizes jailbreak attacks and defenses across the agentic execution pipeline and uses a controlled empirical study to re-examine representative inference-time defenses under modern LLMs. We address the following research questions: •RQ1: How do agentic capabilities expand the jailbreak attack surface beyond traditional conversational LLMs? •RQ2: How can jailbreak attacks and defenses be systematically characterized across agent components, trust boundaries, and execution stages? •RQ3: What marginal security benefits and security–utility–efficiency trade-offs arise when jailbreak defenses are deployed with modern LLMs in agentic settings? •RQ4: Which established jailbreak-security conclusions remain valid, which require re-evaluation, and what challenges remain for securing modern agentic AI? Motivated by these questions, we have the following contributions: • We present a multidimensional taxonomy of jailbreak attacks and defenses for agentic AI, covering ad- versarial access, injection channel, target component, persistence, interaction structure, optimization mechanism, security property, trust boundary, and defense stage. •We introduce a security–utility–efficiency evaluation framework that separates harmful-prompt safety, adversarial jailbreak robustness, and agent-level security while accounting for over-refusal, response quality, task success, latency, and computational cost. •We conduct a controlled empirical evaluation of representative inference-time defenses using Qwen3.5-4B and Llama-3.1-8B-Instruct, revealing substantial model-, attack-, and component-dependent differences in robustness, utility, and efficiency. •We identify limitations in current jailbreak defenses and evaluation practices, expose gaps between response-level safety and agent-level protection, and outline directions for adaptive, execution-aware, and resource-efficient defenses. Our scope separates systematization breadth from empirical depth. The taxonomy covers jailbreak-relevant attack and defense surfaces across planning, tools, memory, and multi-agent interaction, whereas the controlled experiments focus on representative model-layer inference-time defenses under a common setting. The goal is not to claim that these defenses provide complete agent-level protection, but to establish where existing model-layer safeguards remain effective, where their benefits diminish under modern native alignment, and where system-level protection becomes necessary. 2 Related Work 2.1 Literature Search We conducted a multi-source literature search, last updated in August 2026, covering work published or publicly available from 2022–2026. We searched Google Scholar, ACM Digital Library, IEEE Xplore, ACL Anthology, arXiv, and OpenReview, together with major AI, NLP, and security venues. Search queries combined terms such as “jailbreak attack”, “jailbreak defense”, “LLM jailbreak”, “agentic AI”, and “LLM agent” with agent-specific terms related to planning and reasoning, chain-of-thought, tool use and poisoning, memory injection and poisoning, retrieval and RAG, persistent state, and multi-agent interaction. We also performed backward and forward citation tracing from relevant surveys, benchmarks, and influential jailbreak studies. This search identified 271 attack-related and 88 defense-related candidate studies or methods for screening. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. We manually screened the candidates for technical relevance, removed duplicate and out-of-scope works, and preferred published versions over corresponding preprints. Studies were retained when they introduced, analyzed, evaluated, benchmarked, or defended against jailbreak behavior at the model or agent level. The retained works were systematically coded by adversarial access, attack surface, target component, persistence, interaction structure, optimization mechanism, trust boundary, defense stage, and evaluation characteristics. The resulting taxonomy contains 116 distinct jailbreak attacks and 63 distinct defenses, spanning traditional model-level techniques and emerging agentic attack surfaces. 2.2 Existing Jailbreak and Agentic AI Studies Jailbreak research initially focused on standalone LLMs, where adversaries attempt to bypass safety alignment through crafted prompts, optimization-based suffixes, role-playing, prompt transformations, and multi-turn interactions. Corre- sponding defenses span safety training, input filtering and transformation, decoding-time intervention, internal-state monitoring, and output verification. Xu et al. systematize this literature through taxonomies of jailbreak attacks, de- fenses, datasets, evaluation methodologies, and automated judges, and introduce the Security Cube for multidimensional robustness evaluation [169]. Wang et al. further systematize jailbreak guardrails and propose a security–efficiency– utility framework for comparing defenses across attacks [150]. These studies provide comprehensive foundations for jailbreak security, but their analysis is primarily centered on standalone LLMs and prompt–response safety rather than autonomous agent execution. In parallel, the emergence of agentic AI has shifted security research toward systems that reason, maintain state, invoke tools, interact with external environments, and collaborate with other agents. Shahriar et al. organize this space around agentic applications, threats, and defenses [132], while Chhabra et al. survey threats including prompt injection, jailbreaks, tool abuse, autonomous cyber-exploitation, multi-agent attacks, and interface vulnerabilities [23]. Dehghantanha and Homayoun emphasize trust boundaries spanning users, retrieval, memory, tools, external APIs, and multi-agent communication [30]. Chu develops a layered attack-surface model that captures foundation, cognitive, memory, tool-execution, multi-agent, ecosystem, and governance layers, including temporally persistent attacks [24]. Kim et al. similarly analyze how security emerges from interactions among models, memory, tools, workflows, external environments, and conventional software components [64]. The closest prior SoKs differ from our work along four concrete dimensions. Xu et al. [169] systematize model-level jailbreak attacks, defenses, datasets, and judges, whereas Wang et al. [150] focus on jailbreak guardrails and their security–utility–efficiency trade-offs. In both cases, the primary unit of analysis is a standalone model interaction, and safety is evaluated mainly from the final response. Kim et al. [64] instead adopt a system-level view of agentic security, covering interactions among models, memory, tools, workflows, environments, and conventional software; however, jailbreak is treated as one threat class within the broader agentic-security landscape, without a common empirical evaluation of jailbreak defenses across agent layers. Our SoK connects these scopes by organizing jailbreak attacks and defenses around the agentic execution pipeline and using the agent trajectory, rather than only the prompt–response pair, as the unit of analysis. The resulting empirical contribution is also distinct. We evaluate representative defenses under a shared implementa- tion using two final models, nine jailbreak attacks, benign and agentic benchmarks, and common judging procedures. In addition to final-response ASR, we measure unsafe behavior in planning, reasoning, memory, and tool interaction, together with over-refusal, response quality, task utility, latency, and computational cost. This design provides evidence unavailable from the closest SoKs: a defense can suppress the final unsafe response while leaving intermediate agent FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Table 1. Comparison of existing surveys and SoK papers. A filled circle indicates comprehensive coverage, a half-filled circle indicates partial coverage, and an empty circle indicates no coverage. Reference Jailbreak Focus Agentic Focus Agentic Surface Threat Model Attack Taxonomy Defense Taxonomy Agent-Level Outcomes Security–Utility– Efficiency Empirical Evaluation [169] [150] [132] [23] [30] [24] [64] This SoK components compromised, and stronger response-level security can introduce substantial utility or efficiency costs. Thus, the paper’s delta is not a general claim of being the first agent-security survey, but a jailbreak-specific taxonomy and controlled cross-layer evaluation that links component-level failures to final safety and practical trade-offs. Table 1 summarizes these differences in taxonomy scope, unit of analysis, measured outcomes, and empirical design. The comparison is descriptive rather than a ranking. We mark an aspect as comprehensive when it is supported by a dedicated taxonomy, framework, or empirical study; partial when it is discussed but is not a central organizing or evaluated component; and not covered when we found no substantive treatment. In particular, Agent-Level Outcomes requires measurement of intermediate or execution-level behavior rather than only discussion of agent components, while Empirical Evaluation denotes a controlled comparison of multiple attacks or defenses under a common experimental framework. 3 Preliminaries 3.1 Agentic AI Systems Agentic AI extends traditional AI toward autonomous, goal-directed systems that can adapt to changing environments, make context-dependent decisions, and pursue multi-step objectives with limited human intervention [1,46]. Modern agentic systems may further support task decomposition, persistent memory, tool use, and multi-agent collaboration, enabling coordinated execution of complex tasks across diverse domains [128]. 3.2 Agent Execution Pipeline Unlike conventional conversational LLMs that primarily produce a single response, agentic systems operate through iterative reasoning–action loops involving planning, retrieval, memory, tool interaction, observation, and execution [1,128]. As illustrated in Figure 1, an agent interprets a user objective, decomposes it into subtasks, retrieves relevant information, invokes tools, and uses resulting observations to guide subsequent decisions. Multi-agent systems may additionally exchange information and coordinate actions. Because state and outputs are propagated across these stages, each component and transition can become a potential target for adversarial manipulation. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. User Interface LLM Planning and Reasoning Memory Short-term,long-term , RAG Data/ Observations Web Environment Knowledge Sources APIs & Services Computing Resources Tools User AI Agent K AI Agent 2 AI Agent 1 Query/Request Response/Acknowledgement Fig. 1. Agentic AI execution pipeline. 3.3 Agentic Threat Model Jailbreak attacks aim to bypass safety alignment and induce prohibited behavior [156,213], using techniques such as prompt manipulation, role-playing, adversarial optimization, and multi-turn interaction [15,125,183]. In agentic systems, such attacks can affect not only final responses but also planning, memory, retrieval, tool use, and inter-agent communication. We primarily consider a black-box adversary that can control user inputs and externally supplied content but cannot modify trusted system prompts, model parameters, or infrastructure. We also consider a stronger white-box adversary with access to model internals or defense information. We distinguish final-response jailbreaks from component-level unsafe behavior, where intermediate agent components may become unsafe even when the final response remains safe. 4 Agentic Taxonomy 4.1 Taxonomy of Agentic Jailbreak Attacks Agentic systems expand the jailbreak attack surface beyond a single user–model interaction. Adversarial inputs may enter through the user interface or influence intermediate reasoning, tool use, persistent memory, and communication among agents. Fig. 2 organizes these attacks along two primary dimensions. We first distinguish black-box and white-box access. Black-box attacks are organized by the interaction surface they target, whereas white-box attacks are grouped by their primary optimization mechanism. The discussion below focuses on the characteristics that distinguish these attack families. Black-Box Attacks. Black-box attacks operate without direct access to the target model’s parameters, gradients, logits, or hidden representations. Instead, attackers interact through externally accessible interfaces and may use auxiliary attacker or surrogate models. We categorize these attacks according to user–agent, planning and reasoning, agent–tool, agent–memory, and agent–agent interactions. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Agentic Jailbreak Attack Taxonomy Black-Box Attacks White-Box Attacks User–Agent Interaction Planning & Reasoning Agent–Tool Interaction Agent–Memory Interaction Agent–Agent Interaction PAIR [15], TAP [100], X-Teaming [119], ActorAttack [123], MRJ-Agent [142], JOOD [53], Crescendo [125], SwordEcho [140], BADROBOT [194], FITD [161], PathSeeker [83], LLM-Virus [185], RedAgent [170], LASH [106], TriageFuzz [18], Safe2Harm [173], GTA [139], JAIL-CON [61], Jailbreak Mimicry [108], MetaBreak [210], Response At- tack [211], PiF [81], GASP [7], LLM STINGER [54], SequentialBreak [127], Ensem- ble Black-box Jailbreak [20], ShadowBreak [105], Bijection Learning [49], AutoDAN- Turbo [89], AdaPPA [95], PAPILLON [40], Puzzler [14], h4rm3l [34], Single Charac- ter Perturbation [80], Virtual Context [208], SEQAR [177], I-FSJ [204], RLbreaker [19], ArtPrompt [58], CodeAttack [122], DRA [88], Adaptive Attacks [3], ReNeLLM [32], Multilingual Jailbreak [31], DeepInception [77], IRIS [120], JailbreakHub [136], Cipher- Chat [187], JAM [62], Many-shot Jailbreaking (MSJ) [5], FuzzLLM [178], DAP [163], DrAttack [75], CodeChameleon [94], ACE/LACE [43], ICA [159], GCQ [44], Low- Resource Language [182], PAP [190], HackAPrompt [130], Persona Modulation [131], MJP [70], Open Sesame [68], SIJ [200], AdvPrompter [112], AgenticRed [186], SEMA [37], PE-CoA [107], NEXUS [118], GALA [17], GOAT [113], Red Queen At- tack [60], and CoA [176]. TRACE [188], A Mousetrap [180], Chain-of-Thought Hijacking [201], TROJail [168], Jigsaw Puzzles [174], RACE [181], AE-CoT [72], H-CoT [67], ABJ [82], Cognitive Overload [171], and Pandora [22]. AgentHarm [4] and STAC [73]. MINJA [33]. ARCJ [101], Evil Geniuses [141], Agent Smith [41], MAD [115], PRP [98], and The Dark Side of LLMs [93]. Gradient-Based Optimization Search-Based Optimization Representation-Aware Optimization GCG [213], Faster-GCG [76], DeGCG [85], ASETF [143], NTA [50], DSN [207], Guided Jailbreak Attack [175], I-GCG [57], AmpleGCG [79], AmpleGCG-Plus [66], COLD-Attack [42], ADC [48], MAC [196], PGD [39], and AttnGCG [155]. AutoDAN [90], BEAST [126], Functional Homotopy (FH-GR) [154], ADV- LLM [138], and MAGIC [71]. PoEx [92], SABER [63], ORTHO [6], Safety Patterns [74], and ASJA [35]. Fig. 2. Taxonomy of jailbreak attacks in agentic AI systems. User–Agent Interaction. User–agent attacks manipulate the user-facing input channel through prompt refinement, semantic concealment, contextual manipulation, or automated generation. PAIR [15] iteratively refines jailbreak prompts, whereas TAP [100] uses tree-based prompt optimization with pruning. ArtPrompt [58] masks sensitive terms using ASCII art, while CodeAttack [122] reformulates harmful requests as code-completion tasks. More automated approaches include AutoDAN-Turbo [89], which discovers and retrieves attack strategies across repeated interactions, and PAPILLON [40], which uses fuzz testing to mutate short jailbreak prompts. Limitation Common limitations include high query or computational cost, reliance on auxiliary models or target-specific information, restricted evaluation across models and languages, and limited transferability to strongly aligned models. Planning and Reasoning. Planning and reasoning attacks target how harmful objectives are decomposed or processed during reasoning. TRACE [188] combines task decomposition with feedback-driven self-evolution, while Chain-of- Thought Hijacking [201] uses prolonged benign reasoning to weaken refusal behavior. TROJail [168] optimizes multi-turn trajectories using outcome and process rewards, and Jigsaw Puzzles [174] progressively splits and reconstructs harmful queries. These methods exploit the reasoning trajectory rather than relying only on direct prompt reformulation. Limitation Evaluations are often restricted to controlled settings or specific reasoning models, while several methods incur substantial interaction or search overhead and provide limited evaluation against defenses. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. Agent–Tool Interaction. Agent–tool attacks target systems in which model decisions can trigger external actions. AgentHarm [4] evaluates whether agents can complete harmful multi-step tasks using custom tools under jailbreak prompts. STAC [73] constructs sequential attack chains composed of executable tool operations. Consequently, successful attacks at this boundary can extend beyond harmful text to unsafe task execution. Limitation Current evaluations cover a limited range of agents and tools. AgentHarm is restricted to English, excludes multi-turn attacks, and relies on custom tools, while STAC has limited evaluation across diverse tool and agent settings. Agent–Memory Interaction. Memory creates a persistent attack surface because adversarial information introduced during one interaction can influence later behavior. MINJA [33] performs query-only memory injection using bridging steps and a Progressive Shortening Strategy (PSS). The injected content is stored in memory and later retrieved into the agent’s context, separating the initial attack from its eventual effect. Limitation Evidence for memory-targeted jailbreaks remains limited, and MINJA depends on successful retrieval of the injected memory during subsequent interactions. Agent–Agent Interaction. Multi-agent communication enables adversarial behavior to propagate beyond the initially compromised component. ARCJ [101] uses optimized retrieval and replication suffixes for contagious memory poisoning, whereas Agent Smith [41] propagates an infectious jailbreak through inter-agent communication. MAD [115] exploits repeated multi-agent debate to amplify attacks, and PRP [98] propagates a universal perturbation through an LLM-based guardrail pipeline. Limitation Existing attacks rely on specific system assumptions, including explicit memory retrieval, randomized agent interactions, repeated debate, particular guardrail architectures, or executable tool access, limiting their generality across multi-agent systems. White-Box Attacks. White-box attacks assume privileged access to internal model information, including param- eters, gradients, logits, attention scores, or hidden representations. We distinguish gradient-based, search-based, and representation-aware optimization according to the primary mechanism used to construct adversarial inputs or modify safety-related behavior. Gradient-Based Optimization. Gradient-based methods use differentiable signals to optimize adversarial inputs. GCG [213] performs greedy coordinate gradient optimization of universal adversarial suffixes, while Faster-GCG [76] augments this process with distance regularization, temperature-controlled sampling, and visited-suffix marking. I- GCG [57] uses harmful targets, multi-coordinate updates, and suffix initialization, whereas AttnGCG [155] incorporates attention into the coordinate-gradient objective. Limitation These methods generally require white-box gradient or parameter access and costly iterative optimization. Transferability can also decrease across different architectures, tokenizers, and strongly aligned or closed-source models. Search-Based Optimization. Search-based attacks explore candidate jailbreak prompts using genetic, beam, random, or hybrid optimization procedures. AutoDAN [90] uses hierarchical genetic optimization to generate semantically FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Safe RLHF [26], Safety-Tuned LLaMAs [10], Safety Alignment [116], Goal- Conditioned DPO (GC-DPO) [97], ReFAT [184], Safe Unlearning [197], Layer- Specific Editing (LED) [202], Detoxifying with Intraoperative Neural Monitor- ing (DINM) [148], DELMAN [153], SafeLoRA [47], CoG [99], MetaDefense [59], ReSA [13], Eraser [91], Adversarial Tuning [84], BackdoorAlign [145], R2D [209], and Circuit Breakers via Representation Rerouting (R) [212]. Self-Reminder [165], In-Context Defense (ICD) [159], In-Context Adversar- ial Game (ICAG) [206], SmoothLLM [124], SemanticSmooth [56], Erase-and- Check [65], Robust Alignment Checking [12], MoJE [25], Goal Prioritiza- tion [198], RPO [205], THRD [96], RLM-JB [134], RePD [149], and Defensive Prompt Patch (DPP) [167]. Decoding-time: SafeDecoding [172], SecDecoding [146], Speculative Safety- Aware Decoding (SSD) [151], SafeProbing [203], FJD [16], and Alignment- Enhanced Decoding (AED) [86]. Internal-state: Hidden-State Filtering (HSF) [117], Activation Boundary Defense (ABD) [38], JBShield [195], SafeInt [162], Adversarial Game Defense (AGD) [111], GradSafe [164], IMAG [69], Jailbreak Antidote [135], and GUARD-SLM [103]. Safety classification: Llama Guard [52], ShieldGemma [189], and Think- Guard [160]. Verification and revision: PARDEN [199], Self Defense [114], Backtransla- tion [152], and Aligner [55]. CaMeL [28], A-MemGuard [157], and SMSR [133]. ShieldAgent [21] and AgentSpec [144]. Progent [137] and AttriGuard [45]. AutoDefense [191] and TrinityGuard [147]. Safety Training and Fine-Tuning Input Filtering and Transformation Internal-State and Decoding Controls Output Classification, Verification, and Revision Model and Reasoning Context and State Planning and Orchestration Tools and Execution Inter-Agent Communication Agentic Jailbreak Defense Taxonomy Fig. 3. Architecture-oriented taxonomy of jailbreak defenses for agentic AI systems. meaningful jailbreak prompts, while BEAST [126] employs gradient-free beam search. Functional Homotopy [154] transfers random-search optimization through progressively stronger checkpoints, and ADV-LLM [138] iteratively self-tunes an adversarial suffix generator. Limitation Search-based attacks can require many model evaluations and substantial optimization cost. Their effectiveness and transferability may also depend on the target architecture, surrogate model, or availability of intermediate model checkpoints. Representation-Aware Optimization. Representation-aware attacks exploit internal features associated with reasoning or safety behavior. PoEx [92] performs hidden-layer gradient optimization to generate executable robot policies, while SABER [63] injects optimized residuals across model layers. ORTHO [6] identifies and removes a refusal-mediating activation direction, whereas Safety Patterns [74] extracts and weakens safety-related activation patterns. ASJA [35] instead uses attention scores to guide the construction of adversarial dialogue histories. Limitation These attacks require white-box access to internal representations or attention signals. Additional limitations include expensive optimization, sensitivity to feature or layer selection, possible degradation of reasoning quality, and weaker transfer to closed-source models. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. 4.2 Agentic Jailbreak Defense Taxonomy Agentic systems require safeguards across the execution pipeline, rather than only at the user input or final response. Figure 3 organizes defenses by the architectural layer they primarily protect: model and reasoning, context and state, planning and orchestration, tools and execution, and inter-agent communication. We include a system-level mechanism when it prevents, detects, or contains policy bypass caused by adversarial instructions or poisoned state; general reliability controls without this security objective are out of scope. Because model-and-reasoning defenses intervene at several distinct stages, we divide them into four categories below. The remaining four architecture layers are treated directly as defense categories. Safety Training and Fine-Tuning. These defenses modify model behavior before deployment through safety alignment, preference optimization, adversarial training, unlearning, model editing, or safety-aware reasoning training. Safe RLHF [26] learns from safety preferences, while GC-DPO [97] conditions preference optimization on safety goals. ReFAT [184] uses refusal-feature adversarial training, Safe Unlearning [197] suppresses harmful knowledge, and LED [202] performs layer-specific editing. CoG [99] applies trajectory-level safety training with step-level correction, whereas R2D [209] distills safety-aware reasoning. Limitation These methods usually require model access, additional safety data, and training or editing. Their protection may degrade across model families, languages, and unseen attacks; alignment updates can also reduce benign utility. Moreover, model modification alone does not enforce memory, tool, or communication policies at runtime. Input Filtering and Transformation. These defenses intervene before ordinary generation. Self-Reminder [165] reinforces safety instructions, ICD [159] supplies defensive demonstrations, and ICAG [206] constructs an in-context adversarial game. Perturbation-based methods test input stability: SmoothLLM [124] uses randomized perturbations, SemanticSmooth [56] aggregates semantic variants, and Erase-and-Check [65] repeatedly removes tokens and checks safety. Other methods use defensive prompts, optimized prefixes or suffixes, decomposition, or explicit jailbreak detection. Limitation Filtering, perturbation, and repeated checking add token and inference overhead and may over-refuse benign requests. Their performance can decline under adaptive, multilingual, or multi-turn attacks, and checking only the initial user input misses adversarial content introduced later through retrieval, memory, tool output, or inter-agent messages. Internal-State and Decoding Controls. These defenses intervene during model processing or generation. Decoding- time methods integrate safety into token selection: SafeDecoding [172] uses a safety-aware expert, SecDecoding [146] uses a contrastive safety signal, and SSD [151] performs speculative safety-aware decoding. FJD [16] uses first-token confidence, SafeProbing [203] probes safety awareness during generation, and AED [86] incorporates repeated self- evaluation. Internal-state methods instead derive signals from gradients or representations. GradSafe [164] analyzes safety-critical gradients; HSF [117] and ABD [38] use hidden activations; and JBShield [195], SafeInt [162], and Jailbreak Antidote [135] intervene on safety-related representations. IMAG [69] uses activation memory for adaptive detection, while GUARD-SLM [103] detects jailbreak behavior from token activations. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Limitation These methods commonly require logits, gradients, activations, compatible auxiliary models, or repeated evaluation. Consequently, they can be architecture-specific, transfer poorly across heterogeneous models, and incur generation overhead. Fixed detectors or steering directions may also be bypassed by adaptive or distribution-shifted attacks. Output Classification, Verification, and Revision. These defenses inspect or modify a response before release. Llama Guard [52] and ShieldGemma [189] use dedicated safety classifiers, while ThinkGuard [160] generates a critique before classification. PARDEN [199] uses self-repetition as a jailbreak signal, LLM Self Defense [114] applies zero- shot harmfulness classification, and Backtranslation [152] reconstructs the likely source request to assess its intent. Aligner [55] uses a separately trained model to revise unsafe responses. Limitation Effectiveness depends on the guard or reviser and may be reduced by concealed intent, distribution shift, false positives, or over-refusal. Because these methods act after generation, they cannot reliably observe or reverse unsafe plans, memory writes, tool calls, or external side effects that occurred earlier in the trajectory. Context and State. These defenses protect untrusted context and persistent memory before they influence agent decisions. CaMeL [28] separates control and data flows extracted from a trusted query and applies capability policies to prevent untrusted data from changing program flow or causing unauthorized exfiltration. A-MemGuard [157] intercepts the memory-to-action pipeline, using cross-memory consensus to identify anomalous reasoning paths and a dual- memory structure to retain corrective lessons. SMSR [133] combines HMAC-based write provenance with randomized memory ablation and verdict-based aggregation, providing a certified robustness bound under its persistent-memory poisoning threat model. Limitation CaMeL assumes a trusted query and correctly specified security policies, while side channels and compositions of individually permitted flows remain possible. A-MemGuard depends on reliable cross-memory consensus and retrieval configuration and introduces additional inference and storage costs. SMSR assumes protected provenance keys and its stated threat model, and its evaluation primarily uses synthetic enterprise scenarios and an LLM judge. Planning and Orchestration. These defenses enforce safety policies over agent plans, trajectories, and runtime decisions. ShieldAgent [21] extracts verifiable rules from policy documents, represents them as action-based probabilistic rule circuits, and constructs formally checked shielding plans for protected-agent trajectories. AgentSpec [144] provides a domain-specific language in which triggers, predicates, and enforcement actions define runtime constraints. It intercepts predefined execution checkpoints and can terminate, inspect, correct, or request reflection before an unsafe action proceeds. Limitation These defenses are bounded by the completeness of their policies, accuracy of rule extraction, visibility of the execution state, and placement of interception points. Unencoded or unobserved hazards may remain undetected. AgentSpec additionally performs deterministic, step-local enforcement and does not anticipate unsafe states that may emerge only after a multi-step trajectory. Tools and Execution. These defenses mediate externally consequential tool calls through privilege control or action- level causal analysis. Progent [137] represents permitted tool names and arguments using symbolic least-privilege FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. LLM Agent User Query +Tools Observations Tool outputs Intermediate steps Memory contents Intermediate Reasoning Response Jailbreak Defense Planning and Reasoning Prompt User input Final Generation Tools Web Search Retrieve real-time information Data Search Query structured knowledge source Memory Read/Write agent memory LLM Agent Security Attack Success Rate (ASR) Defense Success Rate (DSR) Utility Over-refusal Rate Task Success Rate Answer Relevancy Response Quality Helpfulness Efficiency Inference Time End-to-End Latency Reasoning Token Cost Total Token Cost Security-utility-efficiency Trade-off of Jailbreak Defense in Agentic AI Fig. 4. Security-utility-efficiency Trade-off of Jailbreak Defense in Agentic AI policies. Each call is checked deterministically, while SMT-based policy-update analysis automatically permits privilege narrowing but requires approval for expansion, preventing silent privilege escalation. AttriGuard [45] uses teacher- forced shadow replay and parallel counterfactual tests to determine whether a proposed tool call is supported by the trusted user objective or causally driven by an untrusted observation. Limitation Progent does not address text-only compromise or tools that bypass its enforcement interface, and ambiguous tasks or incorrectly approved privilege expansions can weaken its guarantees. AttriGuard may introduce false positives in long-horizon, tool-intensive workflows, and causal attribution becomes less effective when users extensively delegate decisions to external content or when actions use non-natural-language schemas. Both approaches require runtime integration and add policy-checking or counterfactual- execution overhead. Inter-Agent Communication. Current defenses in this category primarily provide multi-agent verification or monitor- ing rather than comprehensive communication control. AutoDefense [191] uses a fixed multi-agent workflow to separate response generation from safety analysis and filtering; although it uses inter-agent coordination, its protected object is the generated output rather than the communication channel itself. TrinityGuard [147] instead evaluates multi-agent systems through an OWASP-based three-tier risk taxonomy, risk-specific attack probes, and LLM-coordinated runtime monitors that analyze execution traces and issue alerts for individual-agent, inter-agent, and system-level hazards. Limitation AutoDefense is constrained by its fixed communication structure, while TrinityGuard primarily provides diagnosis and alerts rather than closed-loop remediation and inherits errors from its LLM judges and test-generation process. Neither constitutes general enforcement of agent identity, message provenance, delegation authorization, or shared-state isolation. Verification and monitoring agents also increase inference, communication, and coordination costs. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations 5 Experiments and Results 5.1 Experimental Setup All experiments were conducted on a server with two NVIDIA RTX A6000 GPUs (48 GB VRAM each). We implement a unified LangGraph-based agentic evaluation framework (Figure 4) consisting of an LLM agent, external tools, retrieval and memory, a configurable defense module, and final response generation. The agent architecture is kept fixed across experiments to enable controlled comparison. Unless otherwise specified, Qwen3.5-4B serves as the router/tool model, while Qwen3.5-4B and Llama-3.1-8B-Instruct are evaluated as final-response models. We benchmark 16 representative jailbreak defenses together with an undefended Vanilla baseline. For security–utility–efficiency evaluation, we use XSTest, containing 250 benign and 200 harmful prompts. The benign subset measures over-refusal, answer relevancy, response quality, helpfulness, bias, and overall utility, while the harmful subset measures attack success rate (ASR). To evaluate adversarial jailbreak robustness, we use the first 50 harmful instructions from AdvBench with nine representative attacks: AutoDAN, PAIR, TAP, GCG, Cipher, DeepInception, CodeChameleon, ICA, and Jailbroken. Utility metrics are reported on XSTest, whereas AdvBench is used exclusively for adversarial security evaluation. To examine security within the agent execution pipeline, we additionally use the AgentHarm benchmark [4] with three attack methods: TRACE, MINJA, and AgenticRed. TRACE primarily targets planning, MINJA compromises persistent memory, and AgenticRed automatically evolves model-level jailbreak prompts that may propagate across planning, tool interaction, and final responses. These experiments measure both final-response jailbreak success and component-level unsafe behavior. We evaluate jailbreak defenses along three dimensions: security, utility, and efficiency, as illustrated in Fig. 4. We interpret safety outcomes using a confusion-matrix view: a true positive (TP) is an unsafe query that is successfully refused or safely handled; a false negative (FN) is an unsafe query that receives harmful assistance; a true negative (TN) is a benign query that receives a compliant response; and a false positive (FP) is a benign query that is unnecessarily refused. Security. For unsafe queries, we use an LLM-based safety judge to determine whether the final response meaning- fully assists the harmful request. Responses containing actionable instructions, code, operational guidance, concrete recommendations, or specific methods are considered successful jailbreaks. In contrast, refusals, safe redirections, and non-actionable discussions are considered successfully defended responses. We report Attack Success Rate (ASR) and Defense Success Rate (DSR) as ASR= FN TP+ FN × 100%,DSR= TP TP+ FN × 100%.(1) Lower ASR and higher DSR indicate stronger jailbreak robustness. Utility. For benign queries, an LLM-based judge labels each final response as full compliance, full refusal, or partial refusal. Full compliance is counted as TN, whereas full and partial refusals are counted as FP. Thus, ORR= FP TN+ FP × 100%.(2) Response utility is evaluated using Answer Relevancy, G-Eval Response Quality, and Helpfulness. Answer Relevancy measures directness, Response Quality captures relevance, clarity, completeness, and responsiveness, and Helpfulness FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. measures useful and appropriate assistance. We also report DeepEval Bias, computed for response 푖 as 퐵 푖 = 푁 biased 푖 푁 opinions 푖 ,(3) where lower values indicate less gender, political, racial/ethnic, or geographic bias. At the 0.7 threshold, higher scores are better for the three utility metrics, whereas Bias passes at or below 0.7. Bias is reported separately and excluded from aggregate utility, which is 푈 푖 = 푆 rel 푖 + 푆 qual 푖 + 푆 help 푖 3 .(4) Efficiency. We evaluate efficiency using inference time, end-to-end latency, reasoning-token cost, and total token cost, which are directly recorded during execution. Inference time measures the time spent on model inference, whereas end-to-end latency captures the complete time from receiving a query to producing the final response. We also record reasoning-token usage and total-token consumption during execution. Lower time and token consumption indicate greater computational efficiency. 5.2 Practical Trade-off Analysis Figure 5 aggregates Tables 2–4, while Figure 8 aggregates Tables 5–7. Within each final model, we min–max normalize all metrics so that higher values are better. Security equally weights defense success rate (1− ASR), overall accuracy, and benign acceptance (1− ORR), while utility uses the aggregate utility score. Qwen3.5-4B efficiency weights response time, total tokens, and reasoning tokens by 0.40, 0.30, and 0.30; Llama-3.1-8B-Instruct weights response time and total tokens by 0.60 and 0.40 because reasoning-token counts are unavailable. The overall score equally weights security, utility, and efficiency. Because normalization is model-specific, scores are comparable only within each model. For Qwen3.5-4B, Self-Reminder achieves the highest overall score (86.8), followed by SafeProbing (85.0) and Vanilla (84.5); SafeDecoding also scores 84.5 after rounding. This ranking reflects Self-Reminder’s strong balance across dimensions: it reduces ASR from 3.67% to 1.01%, achieves the highest aggregate utility (81.62%), and slightly reduces total response time relative to Vanilla. By contrast, SafeProbing attains a similar ASR (1.00%) but incurs approximately twice the response time of Vanilla (140.68 s versus 72.44 s). Thus, comparable security outcomes can entail substantially different deployment costs. The pairwise views in Figures 6 and 7 clarify how the preferred method changes when only two deployment objectives are considered. For Qwen3.5-4B, Self-Reminder achieves the best security–utility score (89.3), followed by Vanilla (86.7) and Erase-and-Check (86.1), combining a 99.0% defense success rate with the highest utility (81.62%). In the utility–efficiency view, SafeDecoding ranks first because of its low response time and token usage, although its utility is substantially lower than that of Self-Reminder. SmoothLLM ranks last because of its high latency. These pairwise rankings are computed only from the dimensions shown and therefore need not match the three-dimensional overall ranking. For Llama-3.1-8B-Instruct, Self-Eval ranks first (97.1), followed by Vanilla (96.5), with ICD and SafeProbing tied after rounding (96.2). Self-Eval provides the highest utility (77.51%) and overall accuracy (95.55%), while maintaining latency and token consumption close to Vanilla. ICD and SafeProbing obtain slightly lower ASR than Vanilla (2.01% and 2.00% versus 2.51%) while preserving high accuracy and relatively low over-refusal. These results also demonstrate that the preferred defense is model-dependent: Self-Reminder performs best for Qwen3.5-4B, whereas Self-Eval provides the strongest balance for Llama-3.1-8B-Instruct. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Defense Success Rate (%) Overall Accuracy (%) Benign Acceptance (%) Answer Relevancy (%) Response Quality (%) Helpfulness (%) Utility (%) Total Response Time (s) Total Tokens Reasoning Tokens Overall Score 1. Self Reminder 2. SafeProbing 3. Vanilla 4. SafeDecoding 5. Erase and Check 6. Self Eval 7. Aligner 8. ICD 9. Goal Prioritization 10. Intention Analysis 11. JBShield 12. RPO 13. Paraphrase 14. SmoothLLM 15. Prompt Guard 16. Robust Aligned 17. Parden 99.096.995.295.674.474.881.667.42,2811,05986.8 99.095.392.493.760.661.772.0140.71,68466085.0 96.397.097.695.567.366.576.372.42,2861,02084.5 95.889.084.492.840.538.357.132.06101184.5 97.497.397.294.965.964.475.086.32,3011,02783.9 98.496.895.695.962.461.073.173.32,2811,05983.2 95.994.493.282.947.649.660.0143.01,90224081.3 99.595.492.494.362.562.272.979.32,7561,26880.1 97.394.792.890.556.361.869.570.62,9521,31477.5 97.484.476.890.566.868.275.2164.92,7791,37574.2 99.077.260.473.945.948.856.252.91,60666572.7 96.089.384.088.052.652.164.3194.11,7471,58072.0 89.684.781.287.528.529.048.3131.14,22468063.3 88.589.490.089.245.546.160.1824.82,9391,05259.0 100.044.40.00.00.00.00.071.1139043.8 100.044.40.00.00.00.00.0120.1136742.9 55.752.550.014.96.47.19.491.961450939.2 SecurityUtilityEfficiencyRanking 0 20 40 60 80 100 Normalized Performance Score (%) Higher is Better Fig. 5. Overall comparison of representative jailbreak defenses across security, utility, and efficiency (tool model: Qwen3.5-4B; final model: Qwen3.5-4B; dataset: XSTest). The Llama pairwise views in Figures 9 and 10 reinforce this result. Self-Eval leads both the security–utility (86.6) and utility–efficiency (98.1) rankings, while ICD, Vanilla, and SafeProbing remain competitive. It preserves the highest utility (77.51%) with strong security and efficiency close to Vanilla. In contrast, RPO incurs the largest latency and token cost. As above, these pairwise scores summarize only the two displayed dimensions. More broadly, adding a defense does not necessarily improve the complete agentic system. Vanilla ranks among the top three configurations for both models, indicating that their native safety alignment already provides substantial protection without additional defense overhead. Some defenses further reduce ASR but impose disproportionate utility losses. For example, Prompt Guard and Robust Aligned achieve 0% ASR for both models, but reject all benign XSTest prompts, yielding 100% over-refusal and 0% utility. Other methods impose substantial computational overhead: SmoothLLM increases Qwen response time to 824.84 s, while RPO increases Llama response time to 149.66 s and token usage to 3006.63. Moreover, the apparently low cost of highly restrictive defenses partly results from terminating benign requests rather than completing them, illustrating why efficiency cannot be interpreted independently of utility. Overall, these findings show that jailbreak defenses for agentic AI should be selected jointly based on security, benign utility, and execution cost rather than ASR alone. 5.3 Jailbreak Robustness in Agentic AI Table 8 reveals substantial variation in robustness across jailbreak attacks and defenses. The Vanilla agent obtains an average ASR of 34.00%, with AutoDAN (76.00%) and CodeChameleon (62.00%) producing the highest attack success. In contrast, Prompt Guard achieves 0% average ASR, while Paraphrase (0.67%), JBShield (3.56%), Aligner and Intention Analysis (4.22%), and SafeProbing (5.11%) substantially reduce successful jailbreaks. However, robustness is strongly attack dependent: defenses that perform well against GCG, Cipher, or ICA can remain vulnerable to AutoDAN, TAP, DeepInception, or CodeChameleon. For example, ICD reduces GCG, Cipher, and ICA to 0% ASR but reaches 66.00% under AutoDAN and 46.00% under CodeChameleon. These results show that robustness cannot be inferred from a single jailbreak technique and motivate evaluating agentic AI defenses across diverse attack mechanisms. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. Defense Success Rate Overall Accuracy Benign Acceptance Answer Relevancy Response Quality HelpfulnessUtilityOverall Score 1. Self Reminder 2. Vanilla 3. Erase and Check 4. Self Eval 5. ICD 6. SafeProbing 7. Goal Prioritization 8. Intention Analysis 9. Aligner 10. RPO 11. SmoothLLM 12. SafeDecoding 13. JBShield 14. Paraphrase 15. Parden 16. Prompt Guard 17. Robust Aligned 99.096.995.295.674.474.881.689.3 96.397.097.695.567.366.576.386.7 97.497.397.294.965.964.475.086.1 98.496.895.695.962.461.073.185.0 99.595.492.494.362.562.272.984.3 99.095.392.493.760.661.772.083.8 97.394.792.890.556.361.869.582.2 97.484.476.890.566.868.275.280.7 95.994.493.282.947.649.660.077.3 96.089.384.088.052.652.164.377.0 88.589.490.089.245.546.160.174.7 95.889.084.492.840.538.357.173.4 99.077.260.473.945.948.856.267.5 89.684.781.287.528.529.048.366.7 55.752.550.014.96.47.19.431.1 100.044.40.00.00.00.00.024.1 100.044.40.00.00.00.00.024.1 SecurityUtilityRanking 0 20 40 60 80 100 Score (%) Higher is Better Fig. 6. Pairwise security–utility comparison on XSTest (tool and final model: Qwen3.5-4B). Answer Relevancy (%) Response Quality (%) Helpfulness (%) Utility (%) Total Response Time (s) Total Tokens Reasoning Tokens Overall Score 1. SafeDecoding 2. Self Reminder 3. SafeProbing 4. Vanilla 5. Erase and Check 6. Self Eval 7. Aligner 8. JBShield 9. ICD 10. Intention Analysis 11. Goal Prioritization 12. RPO 13. Paraphrase 14. Prompt Guard 15. Robust Aligned 16. Parden 17. SmoothLLM 92.840.538.357.132.06101183.1 95.674.474.881.667.42,2811,05981.2 93.760.661.772.0140.71,68466079.4 95.567.366.576.372.42,2861,02078.2 94.965.964.475.086.32,3011,02776.8 95.962.461.073.173.32,2811,05975.8 82.947.649.660.0143.01,90224075.2 73.945.948.856.252.91,60666572.2 94.362.562.272.979.32,7561,26871.8 90.566.868.275.2164.92,7791,37569.9 90.556.361.869.570.62,9521,31468.8 88.052.652.164.3194.11,7471,58064.4 87.528.529.048.3131.14,22468055.6 0.00.00.00.071.1139049.0 0.00.00.00.0120.1136747.7 14.96.47.19.491.961450947.7 89.245.546.160.1824.82,9391,05246.6 UtilityEfficiencyRanking 0 20 40 60 80 100 Normalized Performance Score (%) Higher is Better Fig. 7. Pairwise utility–efficiency comparison on XSTest (tool and final model: Qwen3.5-4B). 5.4 Impact of Agentic AI Components Cross-layer impact of attacks and defenses. The three attacks expose distinct failure modes across the agentic execution pipeline. In our evaluation, AgenticRed primarily affects planning and final-response safety: relative to Vanilla, unsafe FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Table 2. Security evaluation of jailbreak defenses on XSTest using Qwen3.5-4B (Evaluator: GPT-5-mini). ASR: attack success rate; Acc.: overall accuracy; ORR: over-refusal rate. MethodASR↓ Acc.↑ ORR↓ Vanilla03.67%97.05%02.40% Aligner04.08%94.39%06.80% Erase-and-Check02.60% 97.29%02.80% Goal Prioritization02.69%94.72%07.20% ICD00.53%95.44%07.60% Paraphrase10.44%84.72%18.80% PARDEN 44.33%52.48%50.00% Prompt Guard00.00%44.44%100.00% Robust Aligned 00.00%44.44%100.00% SafeDecoding04.19%88.97%15.60% Self Eval01.56%96.83%04.40% Self-Reminder01.01%96.88%04.80% SmoothLLM11.46%89.37%10.00% JBShield01.04%77.15%39.60% RPO04.00%89.33%16.00% Intention Analysis02.59%84.44%23.20% SafeProbing01.00%95.33%07.60% Table 3. Utility evaluation of jailbreak defenses on the XSTest safe subset using Qwen3.5-4B (Evaluator: GPT-5-mini). AR: answer relevancy; RQ: response quality; Help.: helpfulness. MethodAR↑ RQ↑ Help.↑ Bias↓ Utility↑ Vanilla95.50%67.32%66.52%01.23%76.35% Aligner82.93%47.60%49.56%01.36%60.02% Erase-and-Check94.92%65.88%64.36% 02.31%74.95% Goal Prioritization90.49%56.28%61.81%00.59%69.55% ICD94.35%62.48%62.17%01.06%72.93% Paraphrase87.51%28.51%29.04%01.12%48.28% PARDEN14.86%06.43%07.08%00.17%09.45% Prompt Guard00.00%00.00%00.00%00.00%00.00% Robust Aligned00.00%00.00%00.00%00.00%00.00% SafeDecoding92.80%40.48%38.31%01.28%57.13% Self Eval95.91%62.36%60.96%01.23%73.08% Self-Reminder95.61%74.44%74.80%01.10%81.62% SmoothLLM89.16%45.52%46.14%01.42%60.12% JBShield73.90%45.88%48.80%00.99%56.19% RPO87.99%52.60%52.13%01.51%64.28% Intention Analysis90.53%66.80%68.17%01.05%75.17% SafeProbing93.66%60.56%61.68%01.69%72.01% planning increases from 26.00% to 35.14%, while final ASR rises from 13.46% to 25.00% (Table 9). MINJA directly compromises the memory layer, producing more than 91% unsafe memory and 100% poison retrieval across all evaluated conditions (Table 10). TRACE causes the strongest planning-layer compromise, increasing unsafe planning from 26.00% FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. Table 4. Efficiency evaluation of jailbreak defenses on XSTest using Qwen3.5-4B. T: total response time; TDT: tool decision time; FRT: final response time; Tok.: total tokens; RTok.: reasoning tokens. MethodTT↓ TDT↓ FRT↓ Tok.↓ RTok.↓ Vanilla72.448.5563.352285.741020.40 Aligner143.048.77133.691902.36240.12 Erase-and-Check86.298.7677.002301.301027.48 Goal Prioritization70.588.5761.502952.101314.42 ICD79.318.8269.872755.681268.15 Paraphrase131.059.01121.52 4224.03680.01 PARDEN91.888.7682.60613.62509.32 Prompt Guard71.078.5762.00139.46 0.00 Robust Aligned120.148.79110.83 135.566.62 SafeDecoding31.998.7922.62610.1111.48 Self Eval73.338.5564.212281.001059.25 Self-Reminder67.40 8.5558.132281.001059.25 SmoothLLM824.848.73815.492938.511052.04 JBShield52.898.6543.531606.07664.97 RPO194.148.58185.031746.821580.24 Intention Analysis164.928.95155.412779.251375.12 SafeProbing140.688.73131.351683.57660.10 Defense Success Rate (%) Overall Accuracy (%) Benign Acceptance (%) Answer Relevancy (%) Response Quality (%) Helpfulness (%) Utility (%) Total Response Time (s) Total Tokens Overall Score 1. Self Eval 2. Vanilla 3. ICD 4. SafeProbing 5. SafeDecoding 6. Self Reminder 7. Intention Analysis 8. Goal Prioritization 9. SmoothLLM 10. Erase and Check 11. Paraphrase 12. Parden 13. JBShield 14. Aligner 15. Prompt Guard 16. Robust Aligned 17. RPO 96.595.594.895.666.870.077.514.429697.1 97.594.492.094.967.965.276.013.629896.5 98.095.192.893.268.867.576.513.350696.2 98.095.393.694.765.963.374.615.530196.2 95.594.293.693.967.364.575.214.630795.2 98.086.978.090.169.166.575.314.732192.8 99.581.667.684.460.762.369.215.719888.9 100.090.282.486.849.453.263.214.31,03586.5 92.090.790.087.356.855.866.659.337281.9 100.074.053.274.547.846.656.319.222679.7 93.578.466.872.735.732.347.041.51,09768.0 97.588.982.474.823.418.038.8123.330062.3 100.051.112.443.925.227.632.211.015961.1 76.280.483.674.030.829.844.893.52,07449.3 100.044.40.00.00.00.00.014.213244.0 100.044.40.00.00.00.00.030.918241.3 93.563.842.864.646.346.952.7149.73,00739.9 SecurityUtilityEfficiencyRanking 0 20 40 60 80 100 Normalized Performance Score (%) Higher is Better Fig. 8. Overall comparison of representative jailbreak defenses across security, utility, and efficiency (tool model: Qwen3.5-4B; final model: Llama-3.1-8B-Instruct; dataset: XSTest). to 91.49%, despite a final ASR of only 11.54% (Table 11). This discrepancy shows that final-response ASR alone can substantially underestimate compromise within intermediate agent components. The defenses similarly exhibit cross-layer trade-offs. Self-Eval reduces final ASR to 0% under all three attacks, yet substantial intermediate compromise remains: unsafe memory reaches 95.65% under MINJA, and unsafe planning reaches 93.62% under TRACE. Erase-and-Check reduces final ASR from 25.00% to 7.69% under AgenticRed and from 11.54% to 1.92% under TRACE, while Self-Reminder reduces MINJA ASR from 17.31% to 5.77%. ICD is less consistent, FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Defense Success Rate Overall Accuracy Benign Acceptance Answer Relevancy Response Quality HelpfulnessUtilityOverall Score 1. Self Eval 2. ICD 3. Vanilla 4. SafeProbing 5. SafeDecoding 6. Self Reminder 7. SmoothLLM 8. Goal Prioritization 9. Intention Analysis 10. Erase and Check 11. Parden 12. Paraphrase 13. Aligner 14. RPO 15. JBShield 16. Prompt Guard 17. Robust Aligned 96.595.594.895.666.870.077.586.6 98.095.192.893.268.867.576.585.9 97.594.492.094.967.965.276.085.3 98.095.393.694.765.963.374.685.1 95.594.293.693.967.364.575.284.8 98.086.978.090.169.166.575.381.4 92.090.790.087.356.855.866.678.8 100.090.282.486.849.453.263.277.0 99.581.667.684.460.762.369.276.0 100.074.053.274.547.846.656.366.0 97.588.982.474.823.418.038.864.2 93.578.466.872.735.732.347.063.3 76.280.483.674.030.829.844.862.5 93.563.842.864.646.346.952.759.7 100.051.112.443.925.227.632.243.4 100.044.40.00.00.00.00.024.1 100.044.40.00.00.00.00.024.1 SecurityUtilityRanking 0 20 40 60 80 100 Score (%) Higher is Better Fig. 9. Pairwise security–utility comparison on XSTest (tool model: Qwen3.5-4B; final model: Llama-3.1-8B-Instruct). Answer Relevancy (%) Response Quality (%) Helpfulness (%) Utility (%) Total Response Time (s) Total Tokens Overall Score 1. Self Eval 2. Vanilla 3. SafeDecoding 4. Self Reminder 5. ICD 6. SafeProbing 7. Intention Analysis 8. Erase and Check 9. Goal Prioritization 10. SmoothLLM 11. JBShield 12. Paraphrase 13. Parden 14. Prompt Guard 15. Aligner 16. Robust Aligned 17. RPO 95.666.870.077.514.429698.1 94.967.965.276.013.629897.3 93.967.364.575.214.630796.5 90.169.166.575.314.732196.4 93.268.867.576.513.350696.3 94.765.963.374.615.530196.0 84.460.762.369.215.719893.1 74.547.846.656.319.222683.9 86.849.453.263.214.31,03583.7 87.356.855.866.659.337280.9 43.925.227.632.211.015970.6 72.735.732.347.041.51,09767.0 74.823.418.038.8123.330049.5 0.00.00.00.014.213249.3 74.030.829.844.893.52,07447.6 0.00.00.00.030.918245.3 64.646.346.952.7149.73,00734.0 UtilityEfficiencyRanking 0 20 40 60 80 100 Normalized Performance Score (%) Higher is Better Fig. 10. Pairwise utility–efficiency comparison on XSTest (tool model: Qwen3.5-4B; final model: Llama-3.1-8B-Instruct). reducing AgenticRed ASR to 19.23% but increasing TRACE ASR to 15.38%. Unsafe task-related tool-interaction rates also remain high under several defenses, indicating that protecting the final response does not necessarily secure planning, FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. Table 5. Security evaluation of jailbreak defenses on XSTest using Llama-3.1-8B-Instruct (Evaluator: GPT-5-mini). ASR: attack success rate; Acc.: overall accuracy; ORR: over-refusal rate. MethodASR↓ Acc.↑ ORR↓ Vanilla02.51%94.43%08.00% Aligner23.81%80.41%16.40% Erase-and-Check 00.00%74.00%46.80% Goal Prioritization00.00%90.16%17.60% ICD02.01%95.10%07.20% Paraphrase06.50%78.44%33.20% PARDEN02.50%88.89%17.60% Prompt Guard 00.00%44.44%100.00% Robust Aligned 00.00%44.44%100.00% SafeDecoding04.50%94.22%06.40% Self Eval03.52%95.55%05.20% Self-Reminder02.01%86.86%22.00% SmoothLLM08.00%90.67%10.00% JBShield 00.00%51.11%87.60% RPO06.50%63.78%57.20% Intention Analysis00.50%81.56%32.40% SafeProbing02.00%95.33%06.40% Table 6. Utility evaluation of jailbreak defenses on the XSTest safe subset using Llama-3.1-8B-Instruct (Evaluator: GPT-5-mini). AR: answer relevancy; RQ: response quality; Help.: helpfulness. MethodAR↑ RQ↑ Help.↑ Bias↓ Utility↑ Vanilla94.92%67.88%65.20%04.70%76.00% Aligner74.05%30.75%29.75%00.40%44.84% Erase-and-Check74.55%47.80%46.56%00.55%56.30% Goal Prioritization86.75%49.40%53.20%00.71%63.16% ICD93.24%68.75%67.52%02.14%76.52% Paraphrase72.74%35.74%32.32%01.80%46.96% PARDEN74.85%23.40%18.00%04.00%38.75% Prompt Guard 00.00%00.00%00.00%00.00%00.00% Robust Aligned00.00%00.00%00.00%00.00%00.00% SafeDecoding93.89%67.32%64.52%03.37%75.21% Self Eval95.58%66.84%70.04%05.73%77.51% Self-Reminder90.14%69.08%66.51%00.54%75.27% SmoothLLM87.30%56.84%55.76%03.24%66.63% JBShield43.85%25.20%27.56%00.05%32.20% RPO64.60%46.34%46.92%01.81%52.66% Intention Analysis84.44%60.68%62.28%01.05%69.15% SafeProbing94.67%65.88%63.28%04.51%74.61% memory, or tool interaction. Moreover, Self-Eval incurs the highest latency among the evaluated defenses across these attacks, revealing an additional security–efficiency trade-off. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Table 7. Efficiency evaluation of jailbreak defenses on XSTest using Llama-3.1-8B-Instruct. T: total response time; TDT: tool decision time; TET: tool execution time; FRT: final response time; Tok.: total tokens. MethodTT↓ TDT↓ TET↓ FRT↓ Tok.↓ Vanilla13.648.600.774.28298.11 Aligner93.478.500.9184.052073.94 Erase-and-Check19.198.990.509.70226.29 Goal Prioritization14.348.620.585.141035.08 ICD13.278.610.823.84506.11 Paraphrase41.478.830.6032.031096.62 PARDEN123.308.740.57113.99299.64 Prompt Guard14.22 9.200.494.54131.70 Robust Aligned30.888.890.5221.46182.47 SafeDecoding14.578.640.545.39306.86 Self Eval14.408.740.515.15296.50 Self-Reminder14.678.66 2.253.76321.40 SmoothLLM59.318.950.7049.66372.09 JBShield10.988.640.531.81158.53 RPO149.668.580.55140.543006.63 Intention Analysis15.748.710.616.42197.62 SafeProbing15.50 8.480.546.47300.58 Table 8. Attack Success Rate (ASR) of nine representative jailbreak attacks against representative defenses (Model: Llama-3.1-8B- Instruct; Dataset: AdvBench; Evaluator: GPT-5-mini + manual verification). MethodAutoDAN PAIR TAP GCG Cipher Deep Inception Code Chameleon ICA Jail- broken Avg. ASR↓ Vanilla76.00%22.00%40.00%0.00%40.00%42.00%62.00%2.00%22.00%34.00% Aligner2.00%08.00%8.00%0.00%0.00%4.00%8.00%0.00%8.00%4.22% Erase-and-Check46.00%28.00%44.00%0.00%0.00%32.00%18.00%0.00%22.00%21.11% Goal Prioritization10.00%8.00%14.00%0.00%0.00%12.00%4.00%0.00%8.00%6.22% ICD66.00%20.00%24.00%0.00%0.00%16.00%46.00%0.00%10.00%20.22% Paraphrase0.00%2.00%2.00%0.00%2.00%0.00%0.00%0.00%0.00%0.67% PARDEN0.00%16.00%16.00%0.00%0.00%0.00%4.00%0.00%12.00%5.33% Prompt Guard 0.00%0.00%0.00%0.00%0.00%0.00%0.00%0.00%0.00%0.00% Robust Aligned0.00%12.00%16.00%0.00%0.00%8.00%24.00%0.00%0.00%6.67% SafeDecoding28.00%16.00%24.00%0.00%0.00%44.00%40.00%0.00%6.00%17.56% Self Eval44.00%10.00%38.00%0.00%0.00%26.00%22.00%0.00%20.00%17.78% Self-Reminder20.00%6.00%6.00%0.00%0.00%8.00%18.00%0.00%0.00%6.44% SmoothLLM48.00%20.00%26.00%2.00%6.00%6.00%60.00%0.00%12.00%20.00% JBShield20.00%6.00%6.00% 0.00%0.00%0.00%0.00%0.00%0.00%3.56% RPO64.00%02.00%16.00%0.00%0.00%0.00%0.00%0.00%8.00%10.00% Intention Analysis0.00%4.00%16.00%0.00%0.00%0.00%16.00%0.00%2.00%4.22% SafeProbing12.00%8.00%18.00%0.00%0.00%2.00%4.00%0.00%2.00%5.11% 5.5 Overview of Jailbreak Attacks and Defenses in Agentic AI This section systematizes representative jailbreak attacks and defenses in agentic AI. Tables 12–21 characterize attacks by access level, attack surface, interaction mode, evaluation setting, mechanism, code availability, and limitations. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. Table 9. AgenticRed attack and defense performance on AgentHarm-chat using Llama-3.1-8B-Instruct (Evaluator: GPT-5-mini). UP: unsafe planning; UTI: unsafe task-tool interactions; TCs: tool calls; Lat.: latency. ConditionASR↓ UP↓ UTI↓ TCs↓ Lat. (s)↓ Vanilla13.46%26.00%75.00%6531.97 AgenticRed25.00%35.14%60.61%73338.41 AgenticRed + Self-Reminder19.23% 35.90%72.73%77347.97 AgenticRed + Self-Eval0.00%26.09%60.00%46573.02 AgenticRed + Erase-and-Check7.69%29.27%72.73%78377.71 AgenticRed + ICD19.23%30.23%72.41%75343.11 Table 10. Memory-layer impact of MINJA and defenses on AgentHarm-chat using Llama-3.1-8B-Instruct (Evaluator: GPT-5-mini). UM: unsafe memory; PR: poison retrieval; USI: unsafe search input; TCs: tool calls; Lat.: latency. ConditionASR↓ UM↓ PR↓ USI↓ TCs↓ Lat. (s)↓ Vanilla13.46%N/A0.00%94.12%6531.97 MINJA17.31%93.18%100.00%97.96%6532.71 MINJA + Self-Reminder5.77%93.62%100.00%95.92%6530.59 MINJA + Self-Eval0.00%95.65%100.00%94.00%67103.59 MINJA + Erase-and-Check5.77%91.11%100.00%97.96%6937.10 MINJA + ICD13.46%91.30%100.00%98.00%6634.18 Table 11. TRACE attack and defense performance on AgentHarm-chat using Llama-3.1-8B-Instruct (Evaluator: GPT-5-mini). ConditionASR↓ UP↓ UTI↓ TCs↓ Lat. (s)↓ Vanilla13.46%26.00%75.00%6531.97 TRACE11.54%91.49%68.42%74122.11 TRACE + Self-Reminder11.54%89.36%80.77%78121.80 TRACE + Self-Eval0.00%93.62%78.57%67502.96 TRACE + Erase-and-Check1.92%89.58%78.95%71130.21 TRACE + ICD15.38%93.33%84.00%78133.36 Tables 22–24 synthesize defenses based on their mechanisms, protected components, evaluation settings, security and utility considerations, and key limitations. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 12. Comparison of representative jailbreak attacks. B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation PAIR [15] 2023 SaTML B User–Agent Multi-turn Conv. LLM ✓ Iterative prompt refinement Limited effectiveness on strongly alignedLLMs. TAP [100] 2024 NeurIPS B User–Agent Multi-turn Conv. LLM ✓ Tree-based prompt optimization withpruning Computationally expensive due to treesearch. X-Teaming [119] 2025 COLM B User–Agent Multi-turn Conv. LLM ✓ Two-phase strategic planning + adaptiveattack optimization Requires multiple attacker agents and itera-tive optimization, increasing attack complex-ity. ActorAttack [123] 2024 NA B User–Agent Multi-turn Conv. LLM ✓ Self-discovered attack clues + attack-chain reasoning Requires multiple LLM reasoning stages anditerative interaction, increasing computa-tional overhead. MRJ-Agent [142] 2024 arXiv B User–Agent Multi-turn Conv. LLM ✗ Risk decomposition + red-teaming agenttraining High computational overhead due to dataconstruction, agent training, and iterativemulti-round interactions. JOOD [53] 2025 CVPR B User–Agent Single-turn Conv. MLLM ✓ OOD input transformation Requires multiple transformed inputs, in-creasing inference overhead and relying onOOD generalization gaps. Crescendo [125] 2025 USENIX B User–Agent Multi-turn Conv. MLLM ✓ Goal decomposition Less effective on complex tasks without iter-ative refinement. FITD [161] 2025 EMNLP B User–Agent Multi-turn Conv. LLM ✓ Gradual prompt progression Preliminary self-corruption analysis and lim-ited evaluation on text-only LLMs. SwordEcho [140] 2024 CSAI B User–Agent Single-turn Conv. LLM ✗ RL-based optimization of jailbreakprompts Performance depends on generating problem-specific jailbreak prompts. CoA [176] 2024 arXiv B User–Agent Multi-turn Conv. LLM ✓ Semantic-driven contextual attack chain Limited analysis of multi-turn security mech- anisms. TRACE [188] 2026 arXiv B Planning & Rea-soning Multi-step Agentic AI ✓ Task decomposition + feedback-drivenself-evolution Evaluated only in controlled environments. Mousetrap [180] 2025 ACL B Planning & Rea-soning Single-turn Conv. LRMs ✓ Chaos mapping injection into reasoningchains Defense mechanisms are not examined. COT Hijack- ing [201] 2026 arXiv B Planning & Rea-soning Single-turn Conv. LRMs ✓ Prolonged benign reasoning to dilute re-fusal behavior Evaluated only on text-based LRMs andHarmBench. TROJail [168] 2026 ACL B Planning & Rea-soning Multi-turn Conv. LLM ✓ Trajectory-level optimization with out-come and process rewards Does not incorporate defensive mechanismsor adversarially trained safety policies. 23 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 13. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation Jigsaw Puz- zles [174] 2024 arXiv B Planning & Rea-soning Multi-turn Conv. LLM ✓ Progressive query splitting and recon-struction Evaluated only on conversational LLMs withmanually designed splitting strategies. RACE [181] 2025 EMNLP B Planning & Rea-soning Multi-turn Conv. LRMs ✗ Reasoning-task reformulation withstate-machine-guided adaptive progres-sion High interaction overhead and limited cov-erage of other adversarial reasoning manip-ulations. PANDORA [22] 2024 SeT LLM B Planning & Rea-soning Multi-turn Conv. LLMs ✓ Collaborative phishing through decom-posed reasoning Evaluated on the AdvBench subset. AgentHarm [4] 2025 Benchmark B Agent–Tool Single-turn LLM Agents ✓ Benchmark for evaluating harmfulmulti-step tool execution under jail-break prompts English only; no multi-turn attacks; requirescustom tools. STAC [73] 2026 arXiv B Agent–Tool Multi-turn LLM Agents ✓ Sequential tool attack chaining throughexecutable multi-step tool sequences Limited evaluation on diverse tools andagents. MINJA [33] 2025 NeurIPS B Agent–Memory Multi-turn LLM Agents ✓ Query-only memory injection via bridg-ing steps and Progressive ShorteningStrategy (PSS) Requires successful memory retrieval. ARCJ [101] 2025 ACL B Agent–Agent Multi-turn LLM Agents ✗ Adversarial replication through opti-mized retrieval and replication suffixesfor contagious memory poisoning Evaluated up to 100 agents and assumes ex-plicit memory retrieval mechanisms. AgenticRed [186] 2026 arXiv B User–Agen Multi-turn Agentic AI ✓ Evolutionary agentic red teaming viaLLM-guided system design. Mode collapse limits attack diversity. RedAgent [170] 2025 TDSC B User–Agent Multi-turn Customized LLM Applications ✓ Context-aware jailbreak generation viaprobing and memory retrieval. Limited by incomplete context inference. LLM-Virus [185] 2025 arXiv B User–Agent Multi-turn Conv. LLM ✓ Evolutionary jailbreak via LLM-guidedprompt mutation. Relies on handcrafted seed templates. PathSeeker [83] 2024 arXiv B User–Agent Multi-turn Conv. LLM ✗ RL-guided iterative prompt mutation. High runtime and limited transferability. BADROBOT [194] 2024 ICLR B User–Agent Single-turn Embodied LLM /Robotics ✓ Voice-based jailbreak for embodiedLLMs. Less effective on modular robotic systems. ABJ [82] 2025 arXiv B Planning & Rea-soning Single-turn Conv. LLM, VLM,RLM ✓ Implicit reasoning-chain manipulationthrough textual and visual reasoning Depends on strong reasoning capability andis evaluated only on single-turn individualmodels. Evil Geniuses [ 141 ] 2024 arXiv B Agent–Agent Multi-turn Agentic AI ✗ Role-specialization rewriting throughRed–Blue multi-agent optimization Requires multiple attacker agents, iterativetesting, and access to target role definitions,increasing computational overhead. Agent Smith [41] 2024 arXiv B Agent–Agent Multi-turn LLM Agents ✓ Infectious jailbreak through inter-agentcommunication Assumes randomized pairwise agent interac-tions. 24 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 14. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation MAD [115] 2025 arXiv B Agent–Agent Multi-turn LLM Agents ✗ Structured prompt rewriting exploiting multi-agent debate interactions Requires multiple debate iterations for attackamplification. The Dark Side ofLLMs [93] 2025 OxfordUniv. B Agent–Agent Multi-turn LLM Agents ✗ Inter-agent trust exploitation throughmalicious command propagation Requires executable tool access for systemcompromise. GCG [213] 2023 arXiv WB Model Parame-ters Single-turn Conv. LLM ✓ Greedy coordinate gradient optimiza-tion of universal adversarial suffixes Requires gradient access to source modelsand costly iterative suffix optimization. Faster-GCG [76] 2026 arXiv WB Model Parame-ters Single-turn Conv. LLM ✓ Distance-regularized greedy coordinategradient optimization with temperature-controlled sampling and visited-suffixmarking Requires white-box access to model parame-ters and gradients; transfer attacks dependon suitable surrogate models. AttnGCG [155] 2025 TMLR WB Model Parame-ters Single-turn Conv. LLM ✓ Attention-guided greedy coordinate gra-dient optimization of adversarial suffixes Limited transferability to the latest closed- weight LLMs and inherits GCG’s transfer lim- itations. BEAST [126] 2024 ICML WB Model Parame-ters Single-turn Conv. LLM ✓ Gradient-free beam search optimizationof adversarial prompts Limited transferability across different lan-guage models. MAGIC [71] 2025 ACL WB Model Parame-ters Single-turn Conv. LLM ✓ Gradient-index selection with adaptivemulti-coordinate adversarial suffix opti-mization Less effective when transferred to models with different tokenizers or architectures. POEX [92] 2025 arXiv WB Planning & Rea-soning Single-turn LLM-basedRobots ✓ Hidden-layer gradient optimization forexecutable robot-policy generation withmulti-agent policy evaluation Computationally expensive optimization. AmpleGCG [79] 2024 arXiv WB User–Agent Single-turn Conv. LLM ✓ Generative adversarial suffixes learnedfrom GCG Requires white-box GCG optimization fortraining. AE-CoT [72] 2025 ICML B Planning & Rea-soning Single-turn Conv. LRMs ✓ Adaptive evolutionary CoT search withteacher-style rewriting Higher runtime due to iterative search. BadThink [87] 2026 AAAI WB Planning & Rea-soning Single-turn Conv. LRMs ✗ Poisoned fine-tuning embeds trigger-activated overthinking in CoT Requires access to fine-tuning data andmodel fine-tuning. H-CoT [67] 2025 arXiv B Planning & Rea-soning Single-turn Conv. LRMs ✓ Hijacks safety reasoning using mim-icked execution-phase CoT Relies on accessible or transferable CoT rea-soning traces. LASH [106] 2026 arXiv B User–Agent Single-turn Conv. LLM ✗ Adaptive composition of jailbreakprompts using genetic optimization Limited to single-turn attacks. TriageFuzz [18] 2026 arXiv B User–Agent Single-turn Conv. LLM ✗ Token-aware prompt mutation guidedby surrogate refusal signals Requires a local white-box surrogate model. 25 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 15. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation RAILS [109] 2026 SN WB Model logit Single-turn Conv. LLM ✗ Logit-guided iterative search for adver-sarial suffixes Requires target logits and many queries. Safe2Harm [173] 2025 arXiv B User–Agent Single-turn Conv. LLM ✗ Semantic isomorphism rewrites safe re- sponses into harmful outputs Not evaluated on VLMs. GTA [139] 2025 arXiv B User–Agent Multi-turn Conv. LLM ✓ Game-theoretic scenarios adaptivelypressure models to bypass safety Lacks systematic evaluation against safety-alignment defenses. JAIL-CON [61] 2025 NeurIPS B User–Agent Single-turn Conv. LLM ✓ Interleaves harmful and benign tasks atthe word level Does not optimize auxiliary task selection. JailbreakMimicry [108] 2025 arXiv B User–Agent Single-turn Conv. LLM ✗ Fine-tuned attacker generates narrative-based jailbreak prompts Limited evaluation across target models. MetaBreak [210] 2026 IEEES&P B User–Agent Single-turn Conv. LLM ✓ Special-token manipulation bypassessafety alignment and moderation Requires knowledge of chat templates andmay require token embeddings. Response At- tack [211] 2026 AAAI B User–Agent Single-turn Conv. LLM ✓ Contextual priming with injected mildlyharmful responses Requires an auxiliary LLM to construct prim-ing dialogues. Emoji Attack [158] 2025 ICML B Agent–Guardrail Single-turn Judge LLMs ✓ Emoji insertion exploits token segmen-tation bias to evade moderation Effectiveness varies across Judge LLMs. PiF [81] 2025 ICLR B User–Agent Single-turn Conv. LLM ✓ Token replacement flattens perceivedimportance to obscure malicious intent Lacks a comprehensive theory of jailbreaktransferability. GASP [7] 2025 NeurIPS B User–Agent Single-turn Conv. LLM ✓ Latent Bayesian optimization generatesreadable adversarial suffixes Requires repeated target queries during opti-mization. LLM STINGER [ 54 ] 2025 AAAI B User–Agent Single-turn Conv. LLM ✗ RL-trained attacker generates adversar- ial jailbreak suffixes Evaluated on a limited subset of victim LLMs. SequentialBreak [ 127 ] 2025 ACLSRW B User–Agent Single-turn Conv. LLM ✓ Embeds a harmful prompt within se-quential benign prompt chains Performance may degrade as detection anddefense mechanisms improve. Ensemble BBJailbreak [20] 2025 ICAISG B User–Agent Single-turn Conv. LLM ✗ Ensembles PAP and TAP with stealthi-ness enhancement and selective boost-ing Evaluated on only two target models. ShadowBreak [ 105 ] 2025 NAACL B User–Agent Single-turn Conv. LLM ✗ Benign-data mirroring enables stealthytransfer jailbreak attacks Final adversarial prompts may still be de-tected. Bijection Learn- ing [49] 2025 ICLR B User–Agent Single-turn Conv. LLM ✓ In-context bijective encoding obfuscatesharmful requests to bypass safety Requires large context windows and highinput-token cost. AutoDAN- Turbo [89] 2025 ICLR B User–Agent Multi-turn Conv. LLM ✓ Lifelong strategy discovery and retrievaladapt jailbreak prompts Requires multiple LLMs and high computa-tional resources. ToxASCII [9] 2025 WOAH B Agent–Guardrail Single-turn Moderation Mod-els ✗ ASCII-art obfuscation evades toxicity de-tection Evaluates detection evasion rather thanharmful generation. 26 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 16. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation AdaPPA [95] 2025 ICASSP B User–Agent Single-turn Conv. LLM ✓ Adaptive safe–harmful pre-filling ex-ploits narrative shifts Requires auxiliary fine-tuned models andheuristic pre-fill combinations. PAPILLON [40] 2025 USENIXSecurity B User–Agent Single-turn Conv. LLM ✓ Fuzz testing mutates short and coherentjailbreak prompts Evaluated mainly on English prompts andEnglish models. Puzzler [14] 2024 ACLFindings B User–Agent Single-turn Conv. LLM ✓ Implicit clues hide malicious intent fromsafety alignment Indirect responses may deviate from the orig-inal query. DeGCG [85] 2024 EMNLP WB Gradient-Based Single-turn Conv. LLM ✓ Two-stage gradient search improves ad- versarial suffix transfer Evaluated mainly on open-source text-onlymodels. h4rm3l [34] 2025 ICLR B User–Agent Single-turn Conv. LLM ✓ Composes jailbreak primitives usingbandit-based program synthesis Requires costly iterative synthesis and pro-duces target-specific attacks. SingleCharacterPerturbation [80] 2025 AAAI B User–Agent Single-turn Conv. LLM ✓ Single-character template perturbationsdisrupt safety alignment Requires access to modify the conversationtemplate. Virtual Con- text [208] 2024 EMNLP B User–Agent Single-turn Conv. LLM ✗ Special-token injection creates a virtualaffirmative response context Focuses mainly on the <SEP> token and lacks comprehensive defense evaluation. SEQAR [177] 2025 NAACL B User–Agent Single-turn Conv. LLM ✓ Auto-generates sequential role-playingcharacters for jailbreak prompts Evaluated on only seven target LLMs. I-FSJ [204] 2024 NeurIPS B User–Agent Single-turn Conv. LLM ✓ Special-token injection and demo-levelsearch optimize few-shot jailbreaks Relies on knowledge of target-specific spe-cial tokens. RLbreaker [19] 2024 NeurIPS B User–Agent Single-turn Conv. LLM ✓ DRL-guided search selects prompt mu-tators for efficient jailbreaking Reward may produce false negatives when valid responses differ from references. ArtPrompt [58] 2024 ACL B User–Agent Single-turn Conv. LLM ✓ ASCII-art masking hides sensitive wordsfrom safety alignment Evaluated only on text LLMs, not multimodalmodels. ASETF [143] 2024 EMNLP WB Gradient-Based Single-turn Conv. LLM ✗ Optimizes continuous suffix embed-dings and translates them into fluenttext Universal suffixes have lower ASR and tradeoff semantic coherence with embedding con-sistency. CodeAttack [122] 2024 ACLFindings B User–Agent Single-turn Conv. LLM ✗ Reformulates harmful requests as code-completion tasks to bypass safety Evaluated only in the code domain; gener-alization to other non-natural-language do-mains remains unexplored. DRA [88] 2024 USENIXSecurity B User–Agent Single-turn Conv. LLM ✓ Disguises harmful instructions and re-constructs them in model completions Cannot bypass effective output-level filteringdefenses. 27 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 17. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation Adaptive At- tacks [3] 2025 ICLR B User–Agent Single-turn Conv. LLM ✓ Adapts prompt templates and random-search suffixes to target models Automated judges can overestimate jailbreaksuccess. ReNeLLM [32] 2024 NAACL B User–Agent Single-turn Conv. LLM ✓ Rewrites harmful prompts and neststhem in benign task scenarios Uses fixed nesting scenarios and mainly eval-uates English prompts. Multilingual Jail- break [31] 2024 ICLR B User–Agent Single-turn Conv. LLM ✓ Uses multilingual translation to weakensafety alignment Effectiveness varies substantially across lan-guages and models. DeepInception [ 77 ] 2024 SafeGenAI B User–Agent Single-turn Conv. LLM ✓ Uses nested fictional scenes to con-ceal malicious intent and bypass safetyguardrails Sensitive to scene choice; excessive layersmay lose the attack target. IRIS [120] 2024 EMNLP B User–Agent Multi-turn Conv. LLM ✗ Iteratively self-refines jailbreak prompts,then rates and enhances outputs to in-crease harmfulness Uses fixed prompt templates that may beeasily detected; Rate+Enhance is tested onlyonce. JailbreakHub [136] 2024 CCS B User–Agent Single-turn Conv. LLM ✓ Collects and evaluates in-the-wild jail-breaks using role-play, privilege escala-tion, deception, and virtualization Limited to jailbreak prompts collected fromDec. 2022–Dec. 2023. CipherChat [187] 2024 ICLR B User–Agent Single-turn Conv. LLM ✓ Encodes harmful prompts with ciphersand unsafe demonstrations to bypassnatural-language safety alignment Effectiveness depends on the model’s abilityto understand the chosen cipher. JAM [62] 2024 NeurIPS B User–Agent Single-turn Conv. LLM ✓ Combines jailbreak prefixes with opti-mized cipher characters to bypass input–output moderation guardrails Relies on approximating the moderationguardrail; its structured output format is eas-ily detected and defended. Many-shot Jailbreaking (MSJ) [5] 2024 NeurIPS B User–Agent Single-turn Conv. LLM ✗ Uses hundreds of harmful in-contextdemonstrations to steer the model to- ward unsafe behavior Requires long contexts/API access and mayfail under demonstration–query distributionshifts. FuzzLLM [178] 2024 ICASSP B User–Agent Single-turn Conv. LLM ✗ Fuzzes jailbreak templates, constraints,and prohibited queries to generate di- verse base and combo attacks Relies on manually designed templates/con-straints and automatic labeling can introduceerrors. DAP [163] 2024 EMNLP B User–Agent Single-turn Conv. LLM ✓ Conceals malicious queries in distract-ing contexts and uses memory refram-ing with iterative template optimization Evaluated only on single-turn settings andlimited target models; mitigation of distrac-tion remains unclear. DrAttack [75] 2024 EMNLP B User–Agent Single-turn Conv. LLM ✓ Decomposes harmful prompts and re-constructs them using benign examplesand synonyms. Focuses on text attacks and does not developrobust defenses or evaluate multimodal set-tings. 28 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 18. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation PRP [98] 2024 ACL B Agent–Agent Single-turn Guard-RailedLLM ✓ Propagates a universal adversarial prefixfrom the base LLM response to bypassan LLM-based Guard Model Evaluated on limited models and primarilyLLM-based guards; non-LLM guard mecha-nisms remain unexplored. CodeChameleon [ 94 ] 2024 arXiv B User–Agent Single-turn Conv. LLM ✓ Encrypts harmful queries and embedsdecryption logic in code-style promptsto conceal malicious intent Evaluated on limited LLM families; effective-ness depends on the model’s code under-standing and decryption capability. ACE/LACE [43] 2025 ReliableML B User–Agent Single-turn Conv./ReasoningLLM ✓ Uses custom and layered ciphers to con-ceal harmful queries and bypass safetyalignment Limited models and cipher variants; single-turn evaluation with an imperfect LLMjudge. GCQ [44] 2024 NeurIPS B User–Agent Single-turn Conv. LLM ✗ Uses query-based coordinate search tooptimize adversarial suffixes directlyagainst remote LLMs or safety classifiers Requires target-score/logprob access andgood initialization; random-looking suffixesare detectable by perplexity filters. CognitiveOverload [171] 2024 NAACL B Planning & Rea-soning Single- /Multi-turn Conv. LLM ✓ Exploits multilingual overload, veiled ex-pressions, and effect-to-cause reasoningto bypass safety alignment Evaluated on limited LLMs and measuresharmfulness without considering responsequality. Low-ResourceLanguage [182] 2023 SoLaR B User–Agent Single-turn Conv. LLM ✗ Translates harmful prompts into low-resource languages to exploit cross-lingual safety gaps Evaluated mainly on GPT-4; attack causesmany unclear responses and does not explainthe underlying vulnerability. PAP [190] 2024 ACL B User–Agent Single-turn Conv. LLM ✓ Uses social-science persuasion tech-niques to paraphrase harmful queriesinto persuasive adversarial prompts Focuses mainly on single-turn persuasion;real-world multi-turn and mixed-techniqueattacks remain unexplored. HackAPrompt [ 130 ] 2023 EMNLP B User–Agent Single-turn Conv. LLM ✗ Crowdsources direct adversarial prompts to override task instructionsand derives a taxonomy of prompt-hacking techniques Primarily studies prompt injection ratherthan harmful jailbreaks; limited models andresults are sensitive to API/model drift. PersonaModulation [131] 2023 SoLaR B User–Agent Multi-turn Conv. LLM ✗ Steers the LLM into a harmful personaand automates persona-prompt genera-tion using an assistant LLM Success depends on persona/instructionquality; automated harmfulness evaluationhas high false negatives. MJP [70] 2023 EMNLP B User–Agent Multi-turn Conv. LLM ✓ Uses staged role-play, acknowledgment,and follow-up queries to bypass privacysafeguards and extract PII Low recovery for infrequent emails andphone numbers; correct outputs may reflectpattern guessing rather than memorization. 29 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 19. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation Open Sesame [68] 2024 AppliedSciences B User–Agent Single-turn Conv. LLM ✗ Uses a genetic algorithm to optimize uni- versal adversarial suffixes from model outputs Query-intensive, produces perceptible per-turbations, and is evaluated on limited LLMs. SABER [63] 2025 EMNLP WB Representation- Aware Single-turn Conv. LLM ✓ Injects optimized cross-layer residualsto bypass safety representations Degrades reasoning and is limited to 7B– 13B models; multiple residuals remain un- explored. Attack via Overfit-ting [166] 2025 NeurIPS B Fine-Tuning Training-time LLM ✓ Overfits benign refusals so subsequentbenign fine-tuning forgets safety align-ment Requires model-specific tuning; 10-shot mod-els may produce repetitive responses. NTA [50] 2026 arXiv WB Gradient-Based Single-turn Conv. LLM ✓ Optimizes adversarial suffixes toward anon-textual unsafety objective Requires white-box access and is mainly eval-uated on open-source LLMs. DSN [207] 2025 ACL WB Gradient-Based Single-turn Conv. LLM ✓ Optimizes suffixes to promote affirma-tive tokens while suppressing refusals Requires white-box access; token optimiza-tion is costly and adaptive-defense robust-ness is unclear. Guided Jailbreak Attack [175] 2025 ACL WB Gradient-Based Single-turn Conv. LLM ✓ Removes response-pattern and token-tail constraints to improve transferabil-ity Transfer is weaker on strongly aligned mod-els and prompts remain detectable by PPLfilters. I-GCG [57] 2025 ICLR WB Gradient-Based Single-turn Conv. LLM ✓ Improves GCG with harmful targets,multi-coordinate updates, and suffix ini-tialization Requires white-box gradients and has limitedtransfer to stronger closed-source models. AmpleGCG-Plus [66] 2024 arXiv WB Gradient-Based Single-turn Conv. LLM ✓ Trains a generator on filtered GCG suf-fixes to produce transferable adversarialsuffixes Relies on LLM judges and source-model suf-fix data; evaluates limited defenses. SIJ [200] 2025 ACL B User–Agent Single-turn Conv. LLM ✓ Injects assistant-prefix markers to ma-nipulate chat-template structure Requires chat-template knowledge, has lim-ited prompt diversity, and weakens on black-box models. Functional Homo-topy (FH-GR) [ 154 ] 2025 ICLR WB Search-Based Single-turn Conv. LLM ✓ Transfers random-search suffixes through progressively stronger mis-aligned checkpoints Requires fine-tuning and checkpoint storage;evaluated only on open-source LLMs. ADV-LLM [138] 2025 NAACL WB Search-Based Single-turn Conv. LLM ✗ Iteratively self-tunes a suffix generatorusing sampled successful suffixes andtarget-loss-guided beam search Uses simple refusal signals for data selection,causing false positives and noisy self-tuningdata. 30 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 20. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation EnDec [193] 2024 ACL WB Decoding Single-turn Conv. LLM ✗ Forces affirmative prefixes and reversesrejection-related tokens during decod-ing to bypass safety alignment Requires white-box control of generation;proposed defenses are not empirically evalu-ated due to computational cost. ORTHO [6] 2024 NeurIPS WB Representation- Aware Single-turn Conv. LLM ✓ Identifies and removes a refusal-mediating activation direction viarank-one weight orthogonalization Requires white-box model access; refusal-direction extraction relies on heuristics andmay not generalize to proprietary or largermodels. COLD-Attack [42] 2024 ICML WB Gradient-Based Single-turn Conv. LLM ✓ Uses gradient-based Langevin samplingover continuous logits to generate flu-ent, stealthy, and controllable jailbreakprompts Computationally costly and less effective when safety system prompts are present. ADC [48] 2024 NeurIPS WB Gradient-Based Single-turn Conv. LLM ✓ Relaxes discrete token optimization intocontinuous space and adaptively sparsi-fies adversarial vectors into valid tokensuffixes Requires white-box gradient access; directapplicability to closed-source models is lim-ited. MAC [196] 2025 ICASSP WB Gradient-Based Single-turn Conv. LLM ✓ Adds momentum to GCG gradients tostabilize and accelerate adversarial suffixoptimization Evaluated on only two models; multi-promptattacks use batch size 1 and other optimizersremain unexplored. AdvPrompter [ 112 ] 2025 ICML B User–Agent Single-turn Conv. LLM ✓ Trains an auxiliary LLM via alternat-ing optimization and beam search torapidly generate human-readable adver-sarial suffixes Training requires target log-probabilities ora surrogate model; advanced defenses are notextensively evaluated. Universal JailbreakBackdoor [121] 2024 ICLR B Alignment Poi-soning Training-time RLHF-alignedLLM ✓ Poisons RLHF preference labels with asecret trigger so PPO learns a universaltrigger-based jailbreak backdoor Requires roughly 5% poisoned feedback andassumes reused reward/PPO prompts withno adversarial-label quality checks. PGD [39] 2024 NextGenAIS WB Gradient-Based Single-turn Conv. LLM ✗ Applies projected gradient descent to re-laxed token distributions with simplexand entropy projections for efficient jail-break optimization Requires white-box access; not evaluated ondeployed proprietary models and discreteprompt recovery can require many itera-tions. Safety Patterns [ 74 ] 2025 COLING WB Representation- Aware Single-turn Conv. LLM ✗ Extracts safety-related activation pat-terns from contrastive queries and weak-ens them in hidden states to suppressrefusal behavior Requires white-box activation access; effec-tiveness depends on feature/layer selectionand excessive intervention can degrade out-put quality. 31 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 21. Comparison of representative jailbreak attacks (Part I). B and WB denote black-box and white-box access, respectively. Attack Year Venue Access Attack Surface Interaction Evaluation Set-ting Code Attack Mechanism Key Limitation AutoDAN [90] 2024 ICLR WB Search-Based Single-turn Conv. LLM ✓ Uses hierarchical genetic search withcrossover, mutation, and handcraftedprompt initialization to generate stealthy jailbreak prompts Computationally costly and performs poorlyagainst strongly aligned models/systemprompts. Generation Ex- ploitation [51] 2024 ICLR WB Decoding Single-turn Conv. LLM ✓ Manipulates decoding configurationssuch as temperature, top- 푘 /top- 푝 , sys- tem prompts, and generation constraintsto break safety alignment Requires control over generation settings;effectiveness drops substantially on propri-etary models with restricted decoding access. SEMA [37] 2026 ICLR B User–Agent Multi-turn Conv. LLM ✓ Trains an open-loop multi-turn attackerusing self-tuning and intent-drift-awareRL to preserve harmful intent acrossturns Often uses redundant turns, is text-only, anda single trained attacker exhibits limitedstrategy diversity. PE-CoA [107] 2026 ACLFindings B User–Agent Multi-turn Conv. LLM ✓ Extends CoA with five structured con- versation patterns and adaptive chain refinement to exploit pattern-specificmulti-turn vulnerabilities Limited to English text; only five patternsand controlled settings are evaluated, andeffectiveness may decline as safety mecha-nisms evolve. NEXUS [118] 2025 EMNLP B User–Agent Multi-turn Conv. LLM ✓ Builds a semantic attack network anditeratively refines/prunes multi-turnquery chains using attacker–victim–judge feedback High simulation overhead (15–30 min/query)and some harmful intents require additionalchains and queries. ASJA [35] 2025 AAAI WB Representation- Aware Multi-turn Conv. LLM ✗ Uses attention scores as genetic-algorithm fitness to fabricate dialoguehistories that shift attention away fromharmful keywords Requires white-box attention access; transferperformance decreases on stronger closed-source models. GALA [17] 2025 arXiv B User–Agent Multi-turn Conv. LLM ✗ Uses global tactic-wise and local prompt- wise learning with belief-state tracking to adapt jailbreak strategies across turns Depends on attacker reasoning capabilityand is evaluated on only three target models; weaker attackers substantially reduce ASR. GOAT [113] 2025 ICML B User–Agent Multi-turn Conv. LLM ✗ Adaptively selects and combines adver-sarial prompting tactics using attacker-LLM reasoning over target responses Limited attack set and conversation turns;effectiveness strongly depends on attacker-model reasoning capability. Red Queen At- tack [60] 2025 Findingsof ACL B User–Agent Multi-turn Conv. LLM ✓ Conceals harmful intent across multi-turn protective scenarios to exploit weaklatent-intent inference Limited to text-only, zero-shot settings; ef-fectiveness varies with scenario and harmful-action compatibility. 32 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 22. Comparison of representative jailbreak defense methods in agentic AI. B and WB denote black-box and white-box access, respectively. Defense Year Venue Stage Mechanism Access Model Modifi-cation Code Key Limitation Safe RLHF [26] 2024 ICLR Training RLHF WB ✓ Requires preference data and expensive retraining. Safety-Tuned LLa-MAs [10] 2024 ICLR Training Safety Instruction Tuning (SFT +LoRA) WB ✓ Improves safety but may cause over-refusal. Deep Safety Align-ment [116] 2024 ICLR Training Safety Alignment (SFT + ConstrainedFine-tuning) WB ✓ Improves robustness against jailbreaks and fine-tuning attacks. GC-DPO [97] 2025 ACL Training Goal-conditioned DPO WB ✓ Requires retraining and secure system prompt integrity. ReFAT [184] 2025 ICLR Training Refusal Feature Adversarial Training WB ✓✗ Robust against many jailbreaks, but weaker on linguisticallydiverse or non-English attacks. Safe Unlearn- ing [197] 2024 arXiv Training Harmful Knowledge Unlearning WB ✓ Potential unknown utility degradation. LED [202] 2024 EMNLP Training Layer-specific Editing WB ✓ Does not remove harmful knowledge. DINM [148] 2024 ACL Training Knowledge Editing via Neural Moni-toring WB ✓ May reduce response fluency. Safe LoRA [47] 2024 NeurIPS Training Safety-Aligned LoRA Weight Projec-tion WB ✓ Vulnerable to adaptive attacks. Circuit Break- ers [212] 2024 NeurIPS Training Representation Engineering (RepE) WB ✓ Limited to harmful-content attacks. Self-Reminder [165] 2023 NatureMI Input System-Mode Self-Reminder B ✓ Can reduce response quality and user experience. ICD [159] 2026 TPAMI Input Few-Shot In-Context Demonstrations B ✗✓ Adaptive attacks can reduce robustness. ICAG [206] 2024 ACL Input In-Context Adversarial Game B ✗✓ Depends on initial prompt diversity. SmoothLLM [124] 2025 TMLR Input Randomized Prompt Perturbation B ✗✓ Higher robustness reduces utility. SemanticSmooth [ 56 ] 2025 AACL Input Semantic Perturbation Aggregation B ✗✓ Requires multiple inference calls. Erase-and-Check [65] 2024 COLM Input Token Erasure Safety Checking B ✗✓ High computational overhead. GradSafe [164] 2024 ACL Internal-State Safety-Critical Gradient Analysis WB ✓ Depends on aligned base model. DPP [167] 2025 ACL Input Learned Defensive Prompt Suffix B ✗✓ Removable in local deployments. SecDecoding [146] 2025 EMNLP Decoding Contrastive Safety Signal Decoding B ✗ Depends on auxiliary model quality. SSD [151] 2025 EMNLP Decoding Speculative Safety-Aware Decoding B ✗✓ Limited to small compatible expert models. AED [86] 2025 ACL Internal-State Alignment-Enhanced Decoding B ✗✓ Requires self-evaluation at each decoding step. 33 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 23. Comparison of representative jailbreak defense methods in agentic AI. B and WB denote black-box and white-box access, respectively. Defense Year Venue Stage Mechanism Access Model Modifi-cation Code Key Limitation JBShield [195] 2025 USENIX Internal-State Concept Representation Steering WB ✗✓ Requires access to hidden representations. SafeInt [162] 2025 EMNLP Internal-State Safety-Aware Representation Inter- vention WB ✗ Limited transferability across heterogeneous LLMs. AGD [111] 2025 ACL Internal-State Adversarial Game RepresentationSteering WB ✗✓ Limited to open-source LLMs. GUARD-SLM [103] 2026 ACL Internal-State Token Activation Detection WB ✗✓ Designed primarily for SLMs. RA-LLM [12] 2024 ACL Input Randomized Prompt Dropping B ✗✓ Minor impact on benign responses. DELMAN [153] 2025 ACL Training Dynamic Direct Model Editing WB ✓ Depends on GPT-4 and consecutive edit stability. Llama Guard [52] 2023 arXiv Output LLM-Based Safety Classification B ✗✓ Limited multilingual coverage. ShieldGemma [189] 2024 arXiv Output LLM-Based Safety Classification B ✗✓ May be overly conservative. THINKGUARD [ 160 ] 2025 EMNLP Output Critique-Augmented Safety Classifi-cation B ✗✓ Depends on critique quality. PARDEN [199] 2024 arXiv Output Self-Repetition Detection B ✗✓ Requires a safety-aligned base model. LLM SELF DE- FENSE [114] 2024 arXiv Output Zero-Shot Harm Classification B ✗✓ High false positives on weaker aligned models. Aligner [55] 2024 NeurIPS Output Residual Response Alignment B ✗✓ Requires a separately trained alignment model. AutoDefense [191] 2024 ICLR Output Multi-Agent Response Filtering B ✗✓ Uses a fixed communication pattern. CoG [99] 2026 ACL Find-ings Training Trajectory-level safety training withstep-level reasoning correction WB ✓ Not evaluated on substantially larger reasoning models. SafeProbing [203] 2026 arXiv Internal-State In-Decoding Safety-Awareness Prob-ing WB ✓ Uses a fixed refusal once harmful content is detected; adaptiveredaction, rephrasing, and safety-aware continuation are notexplored. MetaDefense [59] 2025 NeurIPS Training Pre-/Mid-Generation HarmfulnessDetection WB ✓ Lacks theoretical guarantees and is primarily evaluated ontransformer-based LLMs. ReSA [13] 2026 ICLR Training Answer-Then-Check Safety Reason-ing WB ✓✗ Additional safety reasoning increases benign-query inferencelatency. ABD [38] 2025 ACL Internal-State Activation Boundary Defense WB ✓✗ Depends only on well-aligned base models. MoJE [25] 2024 AIES Input Ensemble Jailbreak Prompt Classifica-tion B ✗ Limited on linguistically complex and unseen OOD jailbreakprompts, requiring additional experts for new attack types. 34 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 24. Comparison of representative jailbreak defense methods in agentic AI. B and WB denote black-box and white-box access, respectively. Defense Year Venue Stage Mechanism Access Model Modifi-cation Code Key Limitation Goal Prioritiza- tion [198] 2024 ACL Input Safety-over-Helpfulness Goal Priori-tization WB ✓ Prompt-only use can reduce utility, while internal safety rea-soning adds decoding overhead. BackdoorAlign [ 145 ] 2024 NeurIPS Training Backdoor-Enhanced Safety Align-ment WB ✓ Requires additional safety examples and is limited to fine-tuning-based jailbreak attacks. RPO [205] 2024 NeurIPS Input Adversarially Optimized DefensiveSuffix WB ✗✓ Limited to text-based LLMs and does not cover multimodalmodels or LLM agents. PAT [104] 2024 NeurIPS Input Adversarially Optimized Guard Prefix WB ✗✓ Effectiveness decreases against adaptive attacks with knowl-edge of the defense. THRD [96] 2026 arXiv Input Temporal Multi-Turn Risk Aggrega-tion B ✗ Prompt-based evaluators introduce model dependence andsubstantial multi-turn inference overhead. RLM-JB [134] 2026 arXiv Input Recursive Procedural Jailbreak Detec-tion B ✗ Multi-call chunk analysis introduces substantial latency andremains vulnerable to adaptive attacks targeting the detector. IMAG [69] 2025 arXiv Internal-State Adaptive Activation-Memory Detec-tion WB ✗ Depends on target-model hidden states and auxiliary-agent verification for unknown attacks. FJD [16] 2025 EMNLP Decoding First-Token Confidence Detection B ✗✓ Vulnerable to detection-aware white-box attacks and perfor-mance varies across model architectures. R2D [209] 2025 EMNLP Training Safety-Aware Reasoning Distillation WB ✓ Evaluated mainly on single-turn text jailbreaks; multi-turn andmultimodal safety remain unexplored. RePD [149] 2025 NAACLFindings Input Retrieval-Based Prompt Decomposi-tion B ✗ Adds token and inference overhead and is primarily designedfor template-based jailbreak attacks. Jailbreak Anti- dote [135] 2025 ICLR Internal-State Sparse Safety Representation Steer-ing WB ✗ Fixed steering parameters may not adapt across contexts andcan be exploited by adaptive white-box attackers. HSF [117] 2025 WWWCompan-ion Internal-State Hidden-State Jailbreak Classification WB ✗ Requires model-specific classifier retraining and depends onthe base model’s safety alignment. Backtranslation [ 152 ] 2024 ACL Find-ings Output Response-to-Prompt Backtranslation B ✗✓ Relies on a safety-aligned base model and may fail when back-translation does not recover the hidden harmful intent. SafeDecoding [172] 2024 ACL Intra-Process Safety-Aware Token Decoding WB ✓ May fail when the model initially refuses but later transitionsto harmful content; it also requires a fine-tuned expert modeland has not been evaluated on multimodal LLMs. CaMeL [28] 2026 IEEESaTML Context/ Execution Control–data-flow separation and ca-pability enforcement B ✗✓ Assumes a trusted query and explicitly maintained securitypolicies; side channels and compositions of individually per-mitted control flows remain possible. 35 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Table 25. Comparison of representative jailbreak defense methods in agentic AI. B and WB denote black-box and white-box access, respectively. Defense Year Venue Stage Mechanism Access Model Modifi-cation Code Key Limitation A-MemGuard [157] 2026 ICML Memory Consensus validation and self- correcting dual memory B ✗✓ Depends on reliable cross-memory consensus and retrievalconfiguration; parallel reasoning and lesson-memory mainte-nance introduce additional inference and storage overhead. SMSR [133] 2026 arXiv Memory HMAC provenance and randomizedsmoothed retrieval B ✗✓ Evaluated primarily on synthetic enterprise scenarios usingan LLM judge; its guarantees assume protected HMAC keysand the specified persistent-memory threat model. ShieldAgent [21] 2025 ICML Planning/ Trajectory Probabilistic rule circuits and formaltrajectory verification B ✗✓ Protection depends on the completeness and correctness ofthe extracted policies and observed trajectories; hazards notrepresented by the encoded rules or visible action history mayremain undetected. AgentSpec [144] 2026 ICSE Planning/ Runtime Trigger–predicate–enforcement rulesspecified through a DSL B ✗✓ Performs deterministic enforcement only at predefined ex-ecution checkpoints and does not anticipate the long-termconsequences of multi-step action trajectories. Progent [137] 2025 arXiv Tool/ Execution Symbolic least-privilege policies withSMT-checked updates B ✗✓ Protects monitored tool calls but not text-only attacks or toolsthat bypass its enforcement interface; ambiguous tasks or in-correctly approved privilege expansions can also weaken itsguarantees. AttriGuard [45] 2026 USENIXSecurity Tool/ Execution Counterfactual causal attribution ofproposed tool calls B ✗ May produce false positives in long-horizon, tool-intensive workflows; attribution becomes less effective under extensive user delegation or non-natural-language action schemas. TrinityGuard [147] 2026 arXiv Inter-Agent/ Monitoring OWASP-based risk evaluation withLLM-coordinated runtime monitors B ✗✓ Primarily provides diagnosis and alerts rather than closed-loopremediation; its coverage and reliability depend on LLM judgesand the effectiveness of generated attack probes. Eraser [91] 2024 arXiv Training Harmful Knowledge Unlearning WB ✓ Defends only against enumerated harmful topics and requiresan already safety-aligned base model. Adversarial Tun- ing [84] 2024 arXiv Training Two-Stage Adversarial Fine-Tuning WB ✓✗ Slightly degrades model utility and requires costly adversarialprompt generation and fine-tuning. 36 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations 6 Open Challenges and Future Directions As agentic AI systems become more capable, jailbreak security must move beyond protecting isolated model responses. Future defenses will need to account for heterogeneous attack surfaces, model-dependent behavior, agent state, external actions, and practical deployment constraints. We identify the following directions as particularly important. Native safety is not jailbreak robustness. Modern models may reject direct harmful prompts while remaining vulnerable to adversarially constructed inputs and multi-step attacks. Future evaluations should therefore distinguish baseline harmful-prompt safety from adversarial jailbreak robustness and avoid treating strong native alignment as evidence of comprehensive security. Security should be selective, not refusal-driven. A defense that achieves low attack success by broadly refusing benign requests provides limited practical value. Future mechanisms should estimate risk and intervene proportionally, preserving normal functionality while applying stronger safeguards to suspicious requests, sensitive operations, and high-consequence actions. Robustness must generalize across models and attacks. Jailbreak effectiveness and defense performance can vary substantially across model families, attack mechanisms, and interaction settings. Future work should move beyond single-model and single-attack evaluations toward broader robustness testing that captures transferability, adaptive adversaries, and previously unseen attack strategies. Agentic security must protect state and action, not only text. Agentic systems introduce security-critical components beyond the final model response, including planning, reasoning, memory, retrieval, tool use, and inter-agent communication. Future defenses should protect both component states and the transitions between them, preventing unsafe intermediate behavior from propagating into persistent memory, privileged tool calls, or external actions. Defense guarantees should hold across agent layers. A defense may successfully filter the final response while leaving planning, memory, or tool interactions vulnerable. Future defenses should therefore aim for cross-layer security guarantees rather than response-only protection. This requires mechanisms that preserve safety throughout the complete agent trajectory, from input processing and planning to state updates and execution. Memory requires explicit trust and provenance mechanisms. Persistent memory creates a long-lived attack surface because malicious content can influence future decisions beyond the interaction in which it was introduced. Future systems should authenticate memory sources, track provenance, control write permissions, validate retrieved content, and support safe revocation or quarantine of suspicious state. Tool execution requires consequence-aware authorization. Tool-enabled agents can transform unsafe interme- diate decisions into external effects. Future architectures should separate reasoning from execution and apply explicit authorization policies before high-risk operations. Verification should consider not only the requested tool call but also its parameters, context, expected consequences, and cumulative effects across multiple actions. Defenses should be adaptive and composable. Applying expensive verification at every agent step is unlikely to scale, while a single static safeguard cannot cover all attack surfaces. A promising direction is to combine lightweight continuous monitoring with specialized defenses that are activated at high-risk boundaries, enabling complementary protections for prompts, planning, memory, tools, and outputs. Agentic evaluation needs new security outcomes. Response-level ASR alone cannot characterize whether an agent has been compromised. Future benchmarks should measure intermediate and end-to-end security outcomes, FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. including unsafe planning, malicious state persistence, poisoned-memory retrieval, unauthorized tool execution, cross- agent propagation, and successful external actions. These security measures should also be evaluated jointly with benign-task utility, latency, and computational cost. Adaptive and persistent adversaries remain underexplored. Many evaluations consider attacks within a single interaction, whereas deployed agents operate over extended trajectories and accumulate state over time. Future research should study adversaries that adapt to observed defenses, coordinate attacks across multiple turns or components, and exploit persistent state to gradually influence agent behavior. Component safeguards are not yet end-to-end jailbreak defenses. Our taxonomy maps several mechanisms to memory, planning, tools, and inter-agent communication because they can contain jailbreak-induced compromise, although many were not originally designed or evaluated specifically as jailbreak defenses. Protecting one component therefore does not establish security for the complete agent. Future work should develop integrated defenses that coordinate controls across layers, preserve security through component transitions, and prevent compromise at one stage from propagating through the agentic execution pipeline. 7 Conclusion Agentic AI transforms jailbreak security from a model-output problem into a systems-security challenge spanning reasoning, memory, tools, and multi-agent interaction. This SoK systematizes attacks and defenses across these surfaces and evaluates their security–utility–efficiency trade-offs. Our central finding is that response-level safety can create a false sense of security: an agent may produce a safe final response while its planning, memory, or tool interactions remain compromised. We further find that modern models provide a strong native safety baseline, whereas additional defenses may yield only marginal, model- and attack-dependent security gains while increasing over-refusal, utility loss, and computational overhead. These findings call for execution-aware defenses that protect agent state, component transitions, and external actions throughout the complete trajectory. The strongest agentic defense is therefore not simply the one with the lowest final ASR, but one that achieves cross-layer security while preserving utility and efficiency. Ethical Considerations Stakeholders and Impacts. The security outcomes examined in this study are relevant to agent developers, security researchers, platform providers, end users, and the broader public. We evaluate publicly documented jailbreak attacks and defenses for LLM-based agentic systems within a controlled research environment. The study involves no human subjects, private user data, or unauthorized interaction with deployed systems. Potentially harmful outputs, tool actions, and intermediate agent behaviors are generated exclusively for defensive analysis using isolated or simulated environments. Potential Harms and Mitigations. Systematizing and evaluating jailbreak techniques may make certain attacks easier to reproduce or reveal weaknesses in existing defenses. We mitigate these risks by limiting our evaluation to publicly documented techniques, excluding operationally sensitive deployment details, preventing real-world tool effects, and emphasizing defensive findings and mitigation strategies. We believe the expected benefits—including a better understanding of cross-layer vulnerabilities, defense limitations, and security–utility–efficiency trade-offs—outweigh the remaining risks. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Researcher Wellbeing. Jailbreak evaluation may expose researchers to harmful or distressing model-generated content. Future studies should adopt explicit wellbeing safeguards, including content warnings, voluntary opt-out procedures, workload rotation, limited exposure periods, and access to appropriate support resources. Acknowledgement This work was partially supported by the National Center for Transportation Cybersecurity and Resiliency (TraCR), a U.S. Department of Transportation National University Transportation Center headquartered at Clemson University, Clemson, South Carolina, USA; the U.S. National Science Foundation (NSF) through its Intergovernmental Personnel Act Independent Research & Development Program and grants IIS-2331908 and OAC-2530965; the National Artificial Intelligence Research Resource (NAIRR) Pilot (NAIRR260027, NAIRR250261); OpenAI; NCSA Delta GPU resources; and AWS through the CloudBank project, supported by NSF grant CNS-1925001. Any opinions, findings, conclusions, and recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the funding agencies/programs. References [1] Deepak Bhaskar Acharya, Karthigeyan Kuppan, and B. Divya. 2025. Agentic AI: Autonomous Intelligence for Complex Goals—A Comprehensive Survey. IEEE Access 13 (2025), 18912–18936. doi:10.1109/ACCESS.2025.3532853 [2] Hadi Amini, Md Jueal Mia, Yasaman Saadati, Ahmed Imteaj, Seyedsina Nabavirazavi, Urmish Thakker, Md Zarif Hossain, Awal Ahmed Fime, and S Iyengar. 2025. Distributed llms and multimodal large language models: A survey on advances, challenges, and future directions. arXiv preprint arXiv:2503.16585 (2025). [3]Maksym Andriushchenko, francesco Croce, and Nicolas Flammarion. 2025. Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 40116–40143. [4] Maksym Andriushchenko, Alexandra Souly, Mateusz Dziemian, Derek Duenas, Maxwell Lin, Justin Wang, Dan Hendrycks, Andy Zou, Zico Kolter, Matt Fredrikson, Yarin Gal, and Xander Davies. 2025. AgentHarm: A Benchmark for Measuring Harmfulness of LLM Agents. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 79185–79220. [5]Cem Anil, Esin Durmus, Nina Panickssery, Mrinank Sharma, Joe Benton, Sandipan Kundu, Joshua Batson, Meg Tong, Jesse Mu, Daniel Ford, Fracesco Mosconi, Rajashree Agrawal, Rylan Schaeffer, Naomi Bashkansky, Samuel Svenningsen, Mike Lambert, Ansh Radhakrishnan, Carson Denison, Evan J Hubinger, Yuntao Bai, Trenton Bricken, Timothy Maxwell, Nicholas Schiefer, James Sully, Alex Tamkin, Tamera Lanhan, Karina Nguyen, Tomasz Korbak, Jared Kaplan, Deep Ganguli, Samuel R. Bowman, Ethan Perez, Roger Grosse, and David Duvenaud. 2024. Many-shot Jailbreaking. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 129696–129742. doi:10.52202/079017-4121 [6]Andy Arditi, Oscar Obeso, Aaquib Syed, Daniel Paleka, Nina Panickssery, Wes Gurnee, and Neel Nanda. 2024. Refusal in Language Models Is Mediated by a Single Direction. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 136037–136083. doi:10.52202/079017-4322 [7] Advik Basani and Xiao Zhang. 2025. GASP: Efficient Black-Box Generation of Adversarial Suffixes for Jailbreaking LLMs. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38, Main Conference. Curran Associates, Inc., 146413–146458. doi:10.52202/085713-4898 [8]Peter Belcak, Greg Heinrich, Shizhe Diao, Yonggan Fu, Xin Dong, Saurav Muralidharan, Yingyan Celine Lin, and Pavlo Molchanov. 2025. Small language models are the future of agentic ai. arXiv preprint arXiv:2506.02153 (2025). [9]Sergey Berezin, Reza Farahbakhsh, and Noel Crespi. 2025. Evading Toxicity Detection with ASCII-art: A Benchmark of Spatial Attacks on Moderation Systems. In Proceedings of the The 9th Workshop on Online Abuse and Harms (WOAH), Agostina Calabrese, Christine de Kock, Debora Nozza, Flor Miriam Plaza-del Arco, Zeerak Talat, and Francielle Vargas (Eds.). Association for Computational Linguistics, Vienna, Austria, 153–162. [10]Federico Bianchi, Mirac Suzgun, Giuseppe Attanasio, Paul Röttger, Dan Jurafsky, Tatsunori Hashimoto, and James Y Zou. 2024. Safety-Tuned LLaMAs: Lessons From Improving the Safety of Large Language Models that Follow Instructions. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024. 34196–34216. [11] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chris Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. In Advances in Neural Information Processing Systems, H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (Eds.), Vol. 33. Curran Associates, Inc., 1877–1901. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. [12]Bochuan Cao, Yuanpu Cao, Lu Lin, and Jinghui Chen. 2024. Defending Against Alignment-Breaking Attacks via Robustly Aligned LLM. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 10542–10560. doi:10.18653/v1/2024.acl-long.568 [13] Chentao Cao, Xiaojun Xu, Bo Han, and Hang Li. 2026. Reasoned Safety Alignment: Ensuring Jailbreak Defense via Answer-Then-Check. In International Conference on Learning Representations, C. Vondrick, B. Hariharan, C. Raffel, L. Pinto, D. Yang, and A. Faust (Eds.), Vol. 2026. 47828–47869. [14]Zhiyuan Chang, Mingyang Li, Yi Liu, Junjie Wang, Qing Wang, and Yang Liu. 2024. Play Guessing Game with LLM: Indirect Jailbreak Attack with Implicit Clues. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 5135–5147. doi:10.18653/v1/2024.findings-acl.304 [15]Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J. Pappas, and Eric Wong. 2025. Jailbreaking Black Box Large Language Models in Twenty Queries. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). 23–42. doi:10.1109/SaTML64287.2025.00010 [16]Guorui Chen, Yifan Xia, Xiaojun Jia, Zhijiang Li, Philip Torr, and Jindong Gu. 2025. LLM Jailbreak Detection for (Almost) Free!. In Findings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 5777–5807. doi:10.18653/v1/2025.findings-emnlp.309 [17]Si Chen, Xiao Yu, Ninareh Mehrabi, Rahul Gupta, Zhou Yu, and Ruoxi Jia. 2025. Strategize globally, adapt locally: A multi-turn red teaming agent with dual-level learning. arXiv preprint arXiv:2504.01278 (2025). [18]Wenyu Chen, Xiangtao Meng, Chuanchao Zang, Li Wang, Xinyu Gao, Jianing Wang, Peng Zhan, Zheng Li, and Shanqing Guo. 2026. Not All Tokens Are Created Equal: Query-Efficient Jailbreak Fuzzing for LLMs. arXiv preprint arXiv:2603.23269 (2026). [19]Xuan Chen, Yuzhou Nie, Wenbo Guo, and Xiangyu Zhang. 2024. When LLM Meets DRL: Advancing Jailbreaking Efficiency via DRL-guided Search. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 26814–26845. doi:10.52202/079017-0842 [20]Xiaoxiao Chen, Jingwei Qian, and Haitao Wang. 2025. Transferable Ensemble Black-Box Jailbreak Attacks on Large Language Models. In 2025 International Conference on Artificial Intelligence Security and Governance (ICAISG). 102–106. doi:10.1109/ICAISG68699.2025.11452136 [21]Zhaorun Chen, Mintong Kang, and Bo Li. 2025. SHIELDAGENT: shielding agents via verifiable safety policy reasoning. In Proceedings of the 42nd International Conference on Machine Learning (Vancouver, Canada) (ICML’25). JMLR.org, Article 315, 32 pages. [22] Zhaorun Chen, Zhuokai Zhao, Wenjie Qu, Zichen Wen, Zhiguang Han, Zhihong Zhu, Jiaheng Zhang, and Huaxiu Yao. 2024. Pandora: Detailed llm jailbreaking via collaborated phishing agents with decomposed reasoning. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models. [23]Anshuman Chhabra, Shrestha Datta, Shahriar Kabir Nahin, and Prasant Mohapatra. 2026. Agentic AI Security: Threats, Defenses, Evaluation, and Open Challenges. IEEE Access 14 (2026), 49455–49482. doi:10.1109/ACCESS.2026.3675554 [24] Kexin Chu. 2026. A systematic survey of security threats and defenses in LLM-based AI agents: A layered attack surface framework. arXiv preprint arXiv:2604.23338 (2026). [25] Giandomenico Cornacchia, Giulio Zizzo, Kieran Fraser, Muhammad Zaid Hameed, Ambrish Rawat, and Mark Purcell. 2025. MoJE: Mixture of Jailbreak Experts, Naive Tabular Classifiers as Guard for Prompt Attacks. In Proceedings of the 2024 AAAI/ACM Conference on AI, Ethics, and Society (San Jose, California, USA) (AIES ’24). AAAI Press, 304–315. [26]Juntao Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. 2024. Safe RLHF: Safe Reinforcement Learning from Human Feedback. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024. 50750–50777. [27]Pritam Dash, Tongyu Ge, Aditi Jain, Tanmay Shah, and Zhiwei Shang. 2026. From Untrusted Input to Trusted Memory: A Systematic Study of Memory Poisoning Attacks in LLM Agents. In Second Workshop on Agents in the Wild: Safety, Security, and Beyond. [28]Edoardo Debenedetti, Ilia Shumailov, Tianqi Fan, Jamie Hayes, Nicholas Carlini, Daniel Fabian, Christoph Kern, Chongyang Shi, Andreas Terzis, and Florian Tramèr. 2025. Defeating prompt injections by design. arXiv preprint arXiv:2503.18813 (2025). [29]Edoardo Debenedetti, Jie Zhang, Mislav Balunovic, Luca Beurer-Kellner, Marc Fischer, and Florian Tramèr. 2024. AgentDojo: A Dynamic Environment to Evaluate Prompt Injection Attacks and Defenses for LLM Agents. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 82895–82920. doi:10.52202/079017-2636 [30]Ali Dehghantanha and Sajad Homayoun. 2026. SoK: The Attack Surface of Agentic AI–Tools, and Autonomy. arXiv preprint arXiv:2603.22928 (2026). [31]Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. 2024. Multilingual Jailbreak Challenges in Large Language Models. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024. 24634–24651. [32]Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. 2024. A Wolf in Sheep’s Clothing: Generalized Nested Jailbreak Prompts can Fool Large Language Models Easily. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 2136–2153. doi:10.18653/v1/2024.naacl-long.118 [33]Shen Dong, Shaochen Xu, Pengfei He, Yige Li, Jiliang Tang, Tianming Liu, Hui Liu, and Zhen Xiang. 2025. Memory Injection Attacks on LLM Agents via Query-Only Interaction. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38. Curran Associates, Inc., 46697–46731. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations [34]Moussa Koulako Bala Doumbouya, Ananjan Nandi, Gabriel Poesia, Davide Ghilardi, Anna Goldie, Federico Bianchi, Dan Jurafsky, and Christopher Manning. 2025. h4rm3l: A Language for Composable Jailbreak Attack Synthesis. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 57253–57286. [35] Xiaohu Du, Fan Mo, Ming Wen, Tu Gu, Huadi Zheng, Hai Jin, and Jie Shi. 2025. Multi-turn jailbreaking large language models via attention shifting. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence (AAAI’25/IAAI’25/EAAI’25). AAAI Press, Article 2655, 9 pages. doi:10.1609/aaai.v39i22.34553 [36]Lutfi Eren Erdogan, Nicholas Lee, Siddharth Jha, Sehoon Kim, Ryan Tabrizi, Suhong Moon, Coleman Richard Charles Hooper, Gopala Anumanchipalli, Kurt Keutzer, and Amir Gholami. 2024. TinyAgent: Function Calling at the Edge. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, Delia Irazu Hernandez Farias, Tom Hope, and Manling Li (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 80–88. doi:10.18653/v1/2024.emnlp-demo.9 [37]Mingqian Feng, Xiaodong Liu, Weiwei Yang, Jialin Song, Xuekai Zhu, Chenliang Xu, and Jianfeng Gao. 2026. SEMA: Simple yet Effective Learning for Multi-Turn Jailbreak Attacks. In The Fourteenth International Conference on Learning Representations. [38] Lang Gao, Jiahui Geng, Xiangliang Zhang, Preslav Nakov, and Xiuying Chen. 2025. Shaping the Safety Boundaries: Understanding and Defending Against Jailbreaks in Large Language Models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 25378–25398. doi:10.18653/v1/2025.acl-long.1233 [39] Simon Geisler, Tom Wollschläger, M. H. I. Abdalla, Johannes Gasteiger, and Stephan Günnemann. 2024. Attacking Large Language Models with Projected Gradient Descent. In ICML 2024 Next Generation of AI Safety Workshop. [40] Xueluan Gong, Mingzhe Li, Yilin Zhang, Fengyuan Ran, Chen Chen, Yanjiao Chen, Qian Wang, and Kwok-Yan Lam. 2025. PAPILLON: Efficient and Stealthy Fuzz Testing-Powered Jailbreaks for LLMs. In 34th USENIX Security Symposium (USENIX Security 25). USENIX Association, Seattle, WA, 2401–2420. [41]Xiangming Gu, Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Ye Wang, Jing Jiang, and Min Lin. 2024. Agent smith: A single image can jailbreak one million multimodal llm agents exponentially fast. arXiv preprint arXiv:2402.08567 (2024). [42]Xingang Guo, Fangxu Yu, Huan Zhang, Lianhui Qin, and Bin Hu. 2024. COLD-attack: jailbreaking LLMs with stealthiness and controllability. In Proceedings of the 41st International Conference on Machine Learning (Vienna, Austria) (ICML’24). JMLR.org, Article 675, 29 pages. [43]Divij Handa, Zehua Zhang, Amir Saeidi, Shrinidhi Kumbhar, Md Nayem Uddin, Aswin RRV, and Chitta Baral. 2025. When ”Competency” in Reasoning Opens the Door to Vulnerability: Jailbreaking LLMs via Novel Ciphers. In NeurIPS 2025 Workshop: Reliable ML from Unreliable Data. [44]Jonathan Hayase, Ema Borevkovic, Nicholas Carlini, Florian Tramèr, and Milad Nasr. 2024. Query-Based Adversarial Prompt Generation. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 128260–128279. doi:10.52202/079017-4073 [45] Yu He, Haozhe Zhu, Yiming Li, Shuo Shao, Hongwei Yao, Zhihao Liu, and Zhan Qin. 2026. AttriGuard: Defeating Indirect Prompt Injection in LLM Agents via Causal Attribution of Tool Invocations. In 35th USENIX Security Symposium (USENIX Security 26). USENIX Association, Baltimore, MD. [46]Soodeh Hosseini and Hossein Seilani. 2025. The role of agentic AI in shaping a smart future: A systematic review. Array 26 (2025), 100399. doi:10.1016/j.array.2025.100399 [47]Chia-Yi Hsu, Yu-Lin Tsai, Chih-Hsun Lin, Pin-Yu Chen, Chia-Mu Yu, and Chun-Ying Huang. 2024. Safe LoRA: The Silver Lining of Reducing Safety Risks when Finetuning Large Language Models. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 65072–65094. doi:10.52202/079017-2078 [48]Kai Hu, Weichen Yu, Yining Li, Tianjun Yao, Xiang Li, Wenhe Liu, Lijun Yu, Zhiqiang Shen, Kai Chen, and Matt Fredrikson. 2024. Efficient LLM Jailbreak via Adaptive Dense-to-sparse Constrained Optimization. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 23224–23245. doi:10.52202/079017-0731 [49] Brian R.Y. Huang, Max Li, and Leonard Tang. 2025. Endless Jailbreaks with Bijection Learning. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 70770–70790. [50]Xinzhe Huang, Wenjing Hu, Tianhang Zheng, Kedong Xiu, Hongsheng Hu, Xiaojun Jia, Di Wang, Zhan Qin, and Kui Ren. 2025. NonTextual Target Attack. arXiv preprint arXiv:2510.02999 (2025). [51]Yangsibo Huang, Samyak Gupta, Mengzhou Xia, Kai Li, and Danqi Chen. 2024. Catastrophic Jailbreak of Open-source LLMs via Exploiting Generation. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024. 13707–13727. [52] Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. 2023. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674 (2023). [53]Joonhyun Jeong, Seyun Bae, Yeonsung Jung, Jaeryong Hwang, and Eunho Yang. 2025. Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 29937–29946. [54]Piyush Jha, Arnav Arora, and Vijay Ganesh. 2025. LLM STINGER: jailbreaking LLMs using RL fine-tuned LLMs (student abstract). In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence (AAAI’25/IAAI’25/EAAI’25). AAAI Press, Article 3365, 3 pages. doi:10.1609/ aaai.v39i28.35263 FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. [55]Jiaming Ji, Boyuan Chen, Hantao Lou, Donghai Hong, Borong Zhang, Xuehai Pan, Juntao Dai, Tianyi Qiu, and Yaodong Yang. 2024. Aligner: Efficient Alignment by Learning to Correct. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 90853–90890. doi:10.52202/079017-2884 [56] Jiabao Ji, Bairu Hou, Alexander Robey, George J. Pappas, Hamed Hassani, Yang Zhang, Eric Wong, and Shiyu Chang. 2025. Defending Large Language Models against Jailbreak Attacks via Semantic Smoothing. In Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics, Kentaro Inui, Sakriani Sakti, Haofen Wang, Derek F. Wong, Pushpak Bhattacharyya, Biplab Banerjee, Asif Ekbal, Tanmoy Chakraborty, and Dhirendra Pratap Singh (Eds.). The Asian Federation of Natural Language Processing and The Association for Computational Linguistics, Mumbai, India, 7–40. doi:10.18653/v1/2025.ijcnlp-short.2 [57] Xiaojun Jia, Tianyu Pang, Chao Du, Yihao Huang, Jindong Gu, Yang Liu, Xiaochun Cao, and Min Lin. 2025. Improved Techniques for Optimization- Based Jailbreaking on Large Language Models. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 6337–6358. [58]Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. 2024. ArtPrompt: ASCII Art-based Jailbreak Attacks against Aligned LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 15157–15173. doi:10.18653/v1/2024.acl-long.809 [59]Weisen Jiang and Sinno Pan. 2025. MetaDefense: Defending Fine-tuning based Jailbreak Attack Before and During Generation. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38, Main Conference. Curran Associates, Inc., 99167–99197. doi:10.52202/085713-3313 [60]Yifan Jiang, Kriti Aggarwal, Tanmay Laud, Kashif Munir, Jay Pujara, and Subhabrata Mukherjee. 2025. Red Queen: Exposing Latent Multi-Turn Risks in Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 25554–25591. doi:10.18653/v1/2025.findings- acl.1311 [61]Yukun Jiang, Mingjie Li, Michael Backes, and Yang Zhang. 2025. Adjacent Words, Divergent Intents: Jailbreaking Large Language Models via Task Concurrency. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38, Main Conference. Curran Associates, Inc., 147041–147071. doi:10.52202/085713-4919 [62] Haibo Jin, Andy Zhou, Joe D. Menke, and Haohan Wang. 2024. Jailbreaking Large Language Models Against Moderation Guardrails via Cipher Characters. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 59408–59435. doi:10.52202/079017-1896 [63]Maithili Joshi, Palash Nandi, and Tanmoy Chakraborty. 2025. SABER: Uncovering Vulnerabilities in Safety Alignment via Cross-Layer Residual Connection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 16299–16314. doi:10.18653/v1/2025. emnlp-main.825 [64] Juhee Kim, Wenbo Guo, Dawn Song, UC Berkeley, and UC Santa Barbara. 2026. Sok: Attack and defense landscape of agentic ai systems. In 35nd USENIX Security Symposium (USENIX Security 26). [65]Aounon Kumar, Chirag Agarwal, Suraj Srinivas, Aaron Jiaxun Li, Soheil Feizi, and Himabindu Lakkaraju. 2023. Certifying llm safety against adversarial prompting. arXiv preprint arXiv:2309.02705 (2023). [66] Vishal Kumar, Zeyi Liao, Jaylen Jones, and Huan Sun. 2024. Amplegcg-plus: A strong generative model of adversarial suffixes to jailbreak llms with higher success rates in fewer attempts. arXiv preprint arXiv:2410.22143 (2024). [67]Martin Kuo, Jianyi Zhang, Aolin Ding, Qinsi Wang, Louis DiValentin, Yujia Bao, Wei Wei, Hai Li, and Yiran Chen. 2025. H-cot: Hijacking the chain-of-thought safety reasoning mechanism to jailbreak large reasoning models, including openai o1/o3, deepseek-r1, and gemini 2.0 flash thinking. arXiv preprint arXiv:2502.12893 (2025). [68] Raz Lapid, Ron Langberg, and Moshe Sipper. 2024. Open Sesame! Universal Black-Box Jailbreaking of Large Language Models. Applied Sciences 14, 16 (2024). doi:10.3390/app14167150 [69]Jun Leng, Yu Liu, Litian Zhang, Ruihan Hu, Zhuting Fang, and Xi Zhang. 2025. From static to adaptive: immune memory-based jailbreak detection for large language models. arXiv preprint arXiv:2512.03356 (2025). [70]Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. 2023. Multi-step Jailbreaking Privacy Attacks on ChatGPT. In Findings of the Association for Computational Linguistics: EMNLP 2023, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 4138–4153. doi:10.18653/v1/2023.findings-emnlp.272 [71] Jiahui Li, Yongchang Hao, Haoyu Xu, Xing Wang, and Yu Hong. 2025. Exploiting the Index Gradients for Optimization-Based Jailbreaking on Large Language Models. In Proceedings of the 31st International Conference on Computational Linguistics, Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert (Eds.). Association for Computational Linguistics, Abu Dhabi, UAE, 4535–4547. [72]Jianan Li, Simeng Qin, Xiaojun Jia, Lionel Z. WANG, Tianhang Zheng, Xiaoshuang Jia, Yang Liu, and Xiaochun Cao. 2026. Reasoning as an Attack Surface: Adaptive Evolutionary CoT Jailbreaks for LLMs. In Forty-third International Conference on Machine Learning. [73]Jing-Jing Li, Jianfeng He, Chao Shang, Devang Kulshreshtha, Xun Xian, Yi Zhang, Hang Su, Sandesh Swamy, and Yanjun Qi. 2025. STAC: When Innocent Tools Form Dangerous Chains to Jailbreak LLM Agents. arXiv preprint arXiv:2509.25624 (2025). FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations [74]Tianlong Li, Zhenghua Wang, Wenhao Liu, Muling Wu, Shihan Dou, Changze Lv, Xiaohua Wang, Xiaoqing Zheng, and Xuanjing Huang. 2025. Revisiting Jailbreaking for Large Language Models: A Representation Engineering Perspective. In Proceedings of the 31st International Conference on Computational Linguistics, Owen Rambow, Leo Wanner, Marianna Apidianaki, Hend Al-Khalifa, Barbara Di Eugenio, and Steven Schockaert (Eds.). Association for Computational Linguistics, Abu Dhabi, UAE, 3158–3178. [75]Xirui Li, Ruochen Wang, Minhao Cheng, Tianyi Zhou, and Cho-Jui Hsieh. 2024. DrAttack: Prompt Decomposition and Reconstruction Makes Powerful LLMs Jailbreakers. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 13891–13913. doi:10.18653/v1/2024.findings-emnlp.813 [76]Xiao Li, Wei Zhang, Zhuhong Li, Qiongxiu Li, Shei PernChua, BingZe Lee, Jinghao Cui, Yifan Huang, and Xiaolin Hu. 2024. Faster-gcg: Efficient discrete optimization jailbreak attacks against aligned large language models. arXiv preprint arXiv:2410.15362 (2024). [77]Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han. 2024. DeepInception: Hypnotize Large Language Model to Be Jailbreaker. In Neurips Safe Generative AI Workshop 2024. [78]Yucheng Li, Surin Ahn, Huiqiang Jiang, Amir H. Abdi, Yuqing Yang, and Lili Qiu. 2025. SecurityLingua: Efficient Defense of LLM Jailbreak Attacks via Security-Aware Prompt Compression. In Second Conference on Language Modeling. [79] Zeyi Liao and Huan Sun. 2024. Amplegcg: Learning a universal and transferable generative model of adversarial suffixes for jailbreaking both open and closed llms. arXiv preprint arXiv:2404.07921 (2024). [80]Leon Lin, Hannah Brown, Kenji Kawaguchi, and Michael Shieh. 2025. Single character perturbations break LLM alignment. In Proceedings of the Thirty-Ninth AAAI Conference on Artificial Intelligence and Thirty-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence (AAAI’25/IAAI’25/EAAI’25). AAAI Press, Article 3061, 9 pages. doi:10.1609/ aaai.v39i26.34959 [81] Runqi Lin, Bo Han, Fengwang Li, and Tongliang Liu. 2025. Understanding and Enhancing the Transferability of Jailbreaking Attacks. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 88150–88173. [82]Shi Lin, Hongming Yang, Rongchang Li, Xun Wang, Changting Lin, Wenpeng Xing, and Meng Han. 2024. Llms can be dangerous reasoners: Analyzing-based jailbreak attack on large language models. arXiv preprint arXiv:2407.16205 (2024). [83]Zhihao Lin, Wei Ma, Mingyi Zhou, Yanjie Zhao, Haoyu Wang, Yang Liu, Jun Wang, and Li Li. 2024. Pathseeker: Exploring llm security vulnerabilities with a reinforcement learning-based jailbreak approach. arXiv preprint arXiv:2409.14177 (2024). [84] Fan Liu, Zhao Xu, and Hao Liu. 2024. Adversarial tuning: Defending against jailbreak attacks for llms. arXiv preprint arXiv:2406.06622 (2024). [85] Hongfu Liu, Yuxi Xie, Ye Wang, and Michael Shieh. 2024. Advancing Adversarial Suffix Transfer Learning on Aligned Large Language Models. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 7213–7224. doi:10.18653/v1/2024.emnlp-main.409 [86]Quan Liu, Zhenhong Zhou, Longzhu He, Yi Liu, Wei Zhang, and Sen Su. 2024. Alignment-Enhanced Decoding: Defending Jailbreaks via Token-Level Adaptive Refining of Probability Distributions. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 2802–2816. doi:10.18653/v1/2024.emnlp-main.164 [87]Shuaitong Liu, Renjue Li, Lijia Yu, Lijun Zhang, Zhiming Liu, and Gaojie Jin. 2026. BadThink: triggered overthinking attacks on chain-of-thought reasoning in large language models. In Proceedings of the Fortieth AAAI Conference on Artificial Intelligence and Thirty-Eighth Conference on Innovative Applications of Artificial Intelligence and Sixteenth Symposium on Educational Advances in Artificial Intelligence (AAAI’26/IAAI’26/EAAI’26). AAAI Press, Article 3585, 9 pages. doi:10.1609/aaai.v40i38.40486 [88]Tong Liu, Yingjie Zhang, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen. 2024. Making Them Ask and Answer: Jailbreaking Large Language Models in Few Queries via Disguise and Reconstruction. In 33rd USENIX Security Symposium (USENIX Security 24). USENIX Association, Philadelphia, PA, 4711–4728. [89]Xiaogeng Liu, Peiran Li, G. Edward Suh, Yevgeniy Vorobeychik, Zhuoqing Mao, Somesh Jha, Patrick McDaniel, Huan Sun, Bo Li, and Chaowei Xiao. 2025. AutoDAN-Turbo: A Lifelong Agent for Strategy Self-Exploration to Jailbreak LLMs. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 10313–10360. [90]Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. 2024. AutoDAN: Generating Stealthy Jailbreak Prompts on Aligned Large Language Models. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024. 56174–56194. [91]Weikai Lu, Ziqian Zeng, Jianwei Wang, Zhengdong Lu, Zelin Chen, Huiping Zhuang, and Cen Chen. 2024. Eraser: Jailbreaking defense in large language models via unlearning harmful knowledge. arXiv preprint arXiv:2404.05880 (2024). [92] Xuancun Lu, Zhengxian Huang, Xinfeng Li, Chi Zhang, Wenyuan Xu, et al.2024. Poex: Towards policy executable jailbreak attacks against the llm-based robots. arXiv preprint arXiv:2412.16633 (2024). [93]Matteo Lupinacci, Francesco Aurelio Pironti, Francesco Blefari, Francesco Romeo, Luigi Arena, and Angelo Furfaro. 2025. The Dark Side of LLMs: Agent-based Attack Vectors for System-level Compromise. [94]Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, Tao Gui, Qi Zhang, and Xuanjing Huang. 2024. CodeChameleon: Personalized Encryption Framework for Jailbreaking Large Language Models. arXiv:2402.16717 [cs.CL] [95]Lijia Lv, Weigang Zhang, Xuehai Tang, Jie Wen, Feng Liu, Jizhong Han, and Songlin Hu. 2025. AdaPPA: Adaptive Position Pre-Fill Jailbreak Attack Approach Targeting LLMs. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1–5. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. doi:10.1109/ICASSP49660.2025.10890715 [96]Zhiqing Ma, Zhonghao Xu, Dong Yu, Chen Kang, Changliang Li, and Pengyuan Liu. 2026. THRD: A Training-Free Multi-Turn Defense Framework for Jailbreak Attacks on Large Language Models. arXiv preprint arXiv:2606.01738 (2026). [97] Joo Bon Maeng, Seongmin Lee, Seokin Seo, and Kee-Eung Kim. 2025. Goal-Conditioned DPO: Prioritizing Safety in Misaligned Instructions. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 7196–7211. doi:10.18653/v1/2025.naacl-long.369 [98]Neal Mangaokar, Ashish Hooda, Jihye Choi, Shreyas Chandrashekaran, Kassem Fawaz, Somesh Jha, and Atul Prakash. 2024. PRP: Propagating Universal Perturbations to Attack Large Language Model Guard-Rails. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 10960–10976. doi:10.18653/v1/2024.acl-long.591 [99]Yingzhi Mao, Chunkang Zhang, Junxiang Wang, Xinyan Guan, Boxi Cao, Yaojie Lu, Hongyu Lin, Xianpei Han, and Le Sun. 2026. When Models Outthink Their Safety: Unveiling and Mitigating Self-Jailbreak in Large Reasoning Models. In Findings of the Association for Computational Linguistics: ACL 2026, Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (Eds.). Association for Computational Linguistics, San Diego, California, United States, 22274–22302. doi:10.18653/v1/2026.findings-acl.1118 [100]Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. 2024. Tree of Attacks: Jailbreaking Black-Box LLMs Automatically. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 61065–61105. doi:10.52202/079017-1952 [101]Tianyi Men, Pengfei Cao, Zhuoran Jin, Yubo Chen, Kang Liu, and Jun Zhao. 2025. A Troublemaker with Contagious Jailbreak Makes Chaos in Honest Towns. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 17561–17587. doi:10.18653/v1/2025.acl-long.859 [102]Md Jueal Mia and M. Hadi Amini. 2025. JaiLIP: Jailbreaking Vision-Language Models via Loss Guided Image Perturbation. In 2025 International Conference on Machine Learning and Applications (ICMLA). 1362–1368. doi:10.1109/ICMLA66185.2025.00208 [103]Md Jueal Mia, Joaquin Molto, Yanzhao Wu, and M Hadi Amini. 2026. GUARD-SLM: Token Activation-Based Defense Against Jailbreak Attacks for Small Language Models. arXiv preprint arXiv:2603.28817 (2026). [104]Yichuan Mo, Yuji Wang, Zeming Wei, and Yisen Wang. 2024. Fight Back Against Jailbreaking via Prompt Adversarial Tuning. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 64242–64272. doi:10.52202/079017-2049 [105]Honglin Mu, Han He, Yuxin Zhou, Yunlong Feng, Yang Xu, Libo Qin, Xiaoming Shi, Zeming Liu, Xudong Han, Qi Shi, Qingfu Zhu, and Wanxiang Che. 2025. Stealthy Jailbreak Attacks on Large Language Models via Benign Data Mirroring. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 1784–1799. doi:10.18653/v1/2025.naacl-long.88 [106]Abdullah Al Nomaan Nafi, Fnu Suya, Swarup Bhunia, and Prabuddha Chakraborty. 2026. LASH: Adaptive Semantic Hybridization for Black-Box Jailbreaking of Large Language Models. arXiv preprint arXiv:2605.21362 (2026). [107]Ragib Amin Nihal, Rui Wen, Kazuhiro Nakadai, and Jun Sakuma. 2026. Pattern Enhanced Multi-Turn Jailbreaking: Exploiting Structural Vul- nerabilities in Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2026, Maria Liakata, Viviane P. Mor- eira, Jiajun Zhang, and David Jurgens (Eds.). Association for Computational Linguistics, San Diego, California, United States, 22123–22174. doi:10.18653/v1/2026.findings-acl.1113 [108]Pavlos Ntais. 2025. Jailbreak mimicry: Automated discovery of narrative-based jailbreaks for large language models. arXiv preprint arXiv:2510.22085 (2025). [109] Zhakshylyk Nurlanov, Frank R. Schmidt, and Florian Bernard. 2027. Jailbreaking LLMs Without Gradients or Priors: Effective and Transferable Attacks. In Pattern Recognition, Maria De Marsico, Tin Kam Ho, Frederic Jurie, Cheng-Lin Liu, Daniel Lopresti, Ingela Nyström, Jean-Marc Ogier, Arun Ross, and Liang Wang (Eds.). Springer Nature Switzerland, Cham, 620–634. [110] Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F Christiano, Jan Leike, and Ryan Lowe. 2022. Training language models to follow instructions with human feedback. In Advances in Neural Information Processing Systems, S. Koyejo, S. Mohamed, A. Agarwal, D. Belgrave, K. Cho, and A. Oh (Eds.), Vol. 35. Curran Associates, Inc., 27730–27744. [111] Shilong Pan, Zhiliang Tian, Zhen Huang, Wanlong Yu, Zhihua Wen, Xinwang Liu, Kai Lu, Minlie Huang, and Dongsheng Li. 2025. AGD: Adversarial Game Defense Against Jailbreak Attacks in Large Language Models. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 17391–17406. doi:10.18653/v1/2025.acl-long.851 [112]Anselm Paulus, Arman Zharmagambetov, Chuan Guo, Brandon Amos, and Yuandong Tian. 2025. AdvPrompter: fast adaptive adversarial prompting for LLMs. In Proceedings of the 42nd International Conference on Machine Learning (Vancouver, Canada) (ICML’25). JMLR.org, Article 1935, 31 pages. [113]Maya Pavlova, Erik Brinkman, Krithika Iyer, Vítor Albiero, Joanna Bitton, Hailey Nguyen, Cristian Canton Ferrer, Ivan Evtimov, and Aaron Grattafiori. 2025. Automated red teaming with GOAT: the generative offensive agent tester. In Proceedings of the 42nd International Conference on FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Machine Learning (Vancouver, Canada) (ICML’25). JMLR.org, Article 1936, 18 pages. [114]Mansi Phute, Alec Helbling, Matthew Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. 2023. Llm self defense: By self examination, llms know they are being tricked. arXiv preprint arXiv:2308.07308 (2023). [115] Senmao Qi, Yifei Zou, Peng Li, Ziyi Lin, Xiuzhen Cheng, and Dongxiao Yu. 2025. Amplified vulnerabilities: Structured jailbreak attacks on llm-based multi-agent debate. arXiv preprint arXiv:2504.16489 (2025). [116]Xiangyu Qi, Ashwinee Panda, Kaifeng Lyu, Xiao Ma, Subhrajit Roy, Ahmad Beirami, Prateek Mittal, and Peter Henderson. 2025. Safety Alignment Should be Made More Than Just a Few Tokens Deep. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 54911–54941. [117] Cheng Qian, Hainan Zhang, Lei Sha, and Zhiming Zheng. 2025. HSF: Defending against Jailbreak Attacks with Hidden State Filtering. In Companion Proceedings of the ACM on Web Conference 2025 (Sydney NSW, Australia) (W ’25). Association for Computing Machinery, New York, NY, USA, 2078–2087. doi:10.1145/3701716.3717659 [118]Javad Rafiei Asl, Sidhant Narula, Mohammad Ghasemigol, Eduardo Blanco, and Daniel Takabi. 2025. NEXUS: Network Exploration for eXploiting Unsafe Sequences in Multi-Turn LLM Jailbreaks. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 24267–24295. doi:10.18653/v1/2025.emnlp-main.1235 [119]Salman Rahman, Liwei Jiang, James Shiffer, Genglin Liu, Sheriff Issaka, Md Rizwan Parvez, Hamid Palangi, Kai-Wei Chang, Yejin Choi, and Saadia Gabriel. [n. d.]. X-Teaming: Multi-Turn Jailbreaks and Defenses with Adaptive Multi-Agents. In Second Conference on Language Modeling. [120] Govind Ramesh, Yao Dou, and Wei Xu. 2024. GPT-4 Jailbreaks Itself with Near-Perfect Success Using Self-Explanation. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 22139–22148. doi:10.18653/v1/2024.emnlp-main.1235 [121]Javier Rando and Florian Tramer. 2024. Universal Jailbreak Backdoors from Poisoned Human Feedback. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024. 47894–47921. [122]Qibing Ren, Chang Gao, Jing Shao, Junchi Yan, Xin Tan, Wai Lam, and Lizhuang Ma. 2024. CodeAttack: Revealing Safety Generalization Challenges of Large Language Models via Code Completion. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 11437–11452. doi:10.18653/v1/2024.findings-acl.679 [123] Qibing Ren, Hao Li, Dongrui Liu, Zhanxu Xie, Xiaoya Lu, Yu Qiao, Lei Sha, Junchi Yan, Lizhuang Ma, and Jing Shao. 2024. Derail yourself: Multi-turn llm jailbreak attack through self-discovered clues. (2024). [124]Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. 2023. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684 (2023). [125]Mark Russinovich, Ahmed Salem, and Ronen Eldan. 2025. Great, Now Write an Article About That: The Crescendo Multi-Turn LLM Jailbreak Attack. In 34th USENIX Security Symposium (USENIX Security 25). USENIX Association, Seattle, WA, 2421–2440. [126] Vinu Sankar Sadasivan, Shoumik Saha, Gaurang Sriramanan, Priyatham Kattakinda, Atoosa Chegini, and Soheil Feizi. 2024. Fast adversarial attacks on language models in one GPU minute. In Proceedings of the 41st International Conference on Machine Learning (Vienna, Austria) (ICML’24). JMLR.org, Article 1751, 23 pages. [127]Bijoy Ahmed Saiem, MD Sadik Hossain Shanto, Rakib Ahsan, and Md Rafi Ur Rashid. 2025. SequentialBreak: Large Language Models Can be Fooled by Embedding Jailbreak Prompts into Sequential Prompt Chains. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 4: Student Research Workshop), Jin Zhao, Mingyang Wang, and Zhu Liu (Eds.). Association for Computational Linguistics, Vienna, Austria, 548–579. doi:10.18653/v1/2025.acl-srw.37 [128]Ranjan Sapkota, Konstantinos I. Roumeliotis, and Manoj Karkee. 2026. AI Agents vs. Agentic AI: A Conceptual taxonomy, applications and challenges. Information Fusion 126 (2026), 103599. doi:10.1016/j.inffus.2025.103599 [129]Timo Schick, Jane Dwivedi-Yu, Roberto Dessi, Roberta Raileanu, Maria Lomeli, Eric Hambro, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language Models Can Teach Themselves to Use Tools. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 68539–68551. doi:10.52202/075280-2997 [130]Sander Schulhoff, Jeremy Pinto, Anaum Khan, Louis-François Bouchard, Chenglei Si, Svetlina Anati, Valen Tagliabue, Anson Kost, Christopher Carnahan, and Jordan Boyd-Graber. 2023. Ignore This Title and HackAPrompt: Exposing Systemic Vulnerabilities of LLMs Through a Global Prompt Hacking Competition. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, Houda Bouamor, Juan Pino, and Kalika Bali (Eds.). Association for Computational Linguistics, Singapore, 4945–4977. doi:10.18653/v1/2023.emnlp-main.302 [131]Rusheb Shah, Quentin Feuillade Montixi, Soroush Pour, Arush Tagade, and Javier Rando. 2023. Scalable and Transferable Black-Box Jailbreaks for Language Models via Persona Modulation. In Socially Responsible Language Modelling Research. [132]Asif Shahriar, Md Nafiu Rahman, Sadif Ahmed, Farig Sadeque, and Md Rizwan Parvez. 2025. A Survey on Agentic Security: Applications, Threats and Defenses. arXiv preprint arXiv:2510.06445 (2025). [133] Tarun Sharma. 2026. SMSR: Certified Defence Against Runtime Memory Poisoning in Persistent LLM Agent Systems. arXiv preprint arXiv:2606.12703 (2026). [134]Doron Shavit. 2026. Recursive language models for jailbreak detection: a procedural defense for tool-augmented agents. arXiv preprint arXiv:2602.16520 (2026). FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. [135]Guobin Shen, Dongcheng Zhao, Yiting Dong, Xiang He, and Yi Zeng. 2025. Jailbreak Antidote: Runtime Safety-Utility Balance via Sparse Representation Adjustment in Large Language Models. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 21452–21479. [136] Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. 2024. "Do Anything Now": Characterizing and Evaluating In-The-Wild Jailbreak Prompts on Large Language Models. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security (Salt Lake City, UT, USA) (CCS ’24). Association for Computing Machinery, New York, NY, USA, 1671–1685. doi:10.1145/3658644.3670388 [137]Tianneng Shi, Jingxuan He, Zhun Wang, Hongwei Li, Linyu Wu, Wenbo Guo, and Dawn Song. 2025. Progent: Securing AI agents with privilege control. arXiv preprint arXiv:2504.11703 (2025). [138] Chung-En Sun, Xiaodong Liu, Weiwei Yang, Tsui-Wei Weng, Hao Cheng, Aidan San, Michel Galley, and Jianfeng Gao. 2025. Iterative Self-Tuning LLMs for Enhanced Jailbreaking Capabilities. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 5768–5786. doi:10.18653/v1/2025.naacl-long.297 [139]Zhen Sun, Zongmin Zhang, Deqi Liang, Han Sun, Yule Liu, Yun Shen, Xiangshan Gao, Yilong Yang, Shuai Liu, Yutao Yue, et al.2025. " To Survive, I Must Defect": Jailbreaking LLMs via the Game-Theory Scenarios. arXiv preprint arXiv:2511.16278 (2025). [140]Xuehai Tang, Wenjie Xiao, Zhongjiang Yao, and Jizhong Han. 2025. SwordEcho: A LLM Jailbreaking Optimization Strategy Driven by Reinforcement Learning. In Proceedings of the 2024 8th International Conference on Computer Science and Artificial Intelligence (CSAI ’24). Association for Computing Machinery, New York, NY, USA, 183–190. doi:10.1145/3709026.3709115 [141] Yu Tian, Xiao Yang, Jingyuan Zhang, Yinpeng Dong, and Hang Su. 2023. Evil geniuses: Delving into the safety of llm-based agents. arXiv preprint arXiv:2311.11855 (2023). [142] Fengxiang Wang, Ranjie Duan, Peng Xiao, Xiaojun Jia, Shiji Zhao, Cheng Wei, YueFeng Chen, Chongwen Wang, Jialing Tao, Hang Su, et al.2024. Mrj-agent: An effective jailbreak agent for multi-round dialogue. arXiv preprint arXiv:2411.03814 (2024). [143]Hao Wang, Hao Li, Minlie Huang, and Lei Sha. 2024. ASETF: A Novel Method for Jailbreak Attack on LLMs through Translate Suffix Embeddings. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 2697–2711. doi:10.18653/v1/2024.emnlp-main.157 [144]Haoyu Wang, Christopher M Poskitt, and Jun Sun. 2025. Agentspec: Customizable runtime enforcement for safe and reliable llm agents. arXiv preprint arXiv:2503.18666 (2025). [145]Jiongxiao Wang, Jiazhao Li, Yiquan Li, Xiangyu Qi, Junjie Hu, Yixuan Li, Patrick McDaniel, Muhao Chen, Bo Li, and Chaowei Xiao. 2024. BackdoorAlign: Mitigating Fine-tuning based Jailbreak Attack with Backdoor Enhanced Safety Alignment. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 5210–5243. doi:10.52202/079017-0169 [146] Jiayou Wang, Rundong Liu, Yue Hu, Huijia Wu, and Zhaofeng He. 2025. SecDecoding: Steerable Decoding for Safer LLM Generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 20504–20521. doi:10.18653/v1/2025.findings-emnlp.1118 [147]Kai Wang, Biaojie Zeng, Zeming Wei, Chang Jin, Hefeng Zhou, Xiangtian Li, Chao Yang, Jingjing Qu, Xingcheng Xu, and Xia Hu. 2026. Trinityguard: A unified framework for safeguarding multi-agent systems. arXiv preprint arXiv:2603.15408 (2026). [148]Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, and Huajun Chen. 2024. Detoxifying Large Language Models via Knowledge Editing. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 3093–3118. doi:10.18653/v1/2024.acl-long.171 [149]Peiran Wang, Xiaogeng Liu, and Chaowei Xiao. 2025. RePD: Defending Jailbreak Attack through a Retrieval-based Prompt Decomposition Process. In Findings of the Association for Computational Linguistics: NAACL 2025, Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 283–294. doi:10.18653/v1/2025.findings-naacl.16 [150]Xunguang Wang, Zhenlan Ji, Wenxuan Wang, Zongjie Li, Daoyuan Wu, and Shuai Wang. 2026. Sok: Evaluating Jailbreak Guardrails for Large Language Models. In 2026 IEEE Symposium on Security and Privacy (SP). 39–58. doi:10.1109/SP63933.2026.00076 [151] Xuekang Wang, Shengyu Zhu, and Xueqi Cheng. 2025. Speculative Safety-Aware Decoding. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 12827–12841. doi:10.18653/v1/2025.emnlp-main.648 [152]Yihan Wang, Zhouxing Shi, Andrew Bai, and Cho-Jui Hsieh. 2024. Defending LLMs against Jailbreaking Attacks via Backtranslation. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 16031–16046. doi:10.18653/v1/2024.findings-acl.948 [153]Yi Wang, Fenghua Weng, Sibei Yang, Zhan Qin, Minlie Huang, and Wenjie Wang. 2025. DELMAN: Dynamic Defense Against Large Language Model Jailbreaking with Model Editing. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 11465–11481. doi:10.18653/ v1/2025.findings-acl.598 [154]Zi Wang, Divyam Anshumaan, Ashish Hooda, Yudong Chen, and Somesh Jha. 2025. Functional Homotopy: Smoothing Discrete Optimization via Continuous Parameters for LLM Jailbreak Attacks. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations R. Yu (Eds.), Vol. 2025. 38125–38144. [155]Zijun Wang, Haoqin Tu, Jieru Mei, Bingchen Zhao, Yisen Wang, and Cihang Xie. 2024. Attngcg: Enhancing jailbreaking attacks on llms with attention manipulation. arXiv preprint arXiv:2410.09040 (2024). [156] Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. 2023. Jailbroken: How Does LLM Safety Training Fail?. In Advances in Neural Information Processing Systems, A. Oh, T. Naumann, A. Globerson, K. Saenko, M. Hardt, and S. Levine (Eds.), Vol. 36. Curran Associates, Inc., 80079–80110. [157]Qianshan Wei, Tengchao Yang, Yaochen Wang, Xinfeng Li, Lijun Li, Zhenfei Yin, Yi Zhan, Thorsten Holz, Zhiqiang Lin, and XiaoFeng Wang. 2025. A-memguard: A proactive defense framework for llm-based agent memory. arXiv preprint arXiv:2510.02373 (2025). [158]Zhipeng Wei, Yuqi Liu, and N. Benjamin Erichson. 2025. Emoji Attack: Enhancing Jailbreak Attacks Against Judge LLM Detection. In Forty-second International Conference on Machine Learning. [159]Zeming Wei, Yifei Wang, Ang Li, Yichuan Mo, and Yisen Wang. 2026. Jailbreak and Guard Aligned Language Models With Only Few In-Context Demonstrations. IEEE Transactions on Pattern Analysis and Machine Intelligence 48, 6 (2026), 6835–6846. doi:10.1109/TPAMI.2026.3660147 [160]Xiaofei Wen, Wenxuan Zhou, Wenjie Jacky Mo, and Muhao Chen. 2025. ThinkGuard: Deliberative Slow Thinking Leads to Cautious Guardrails. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 13698–13713. doi:10.18653/v1/2025.findings-acl.704 [161]Zixuan Weng, Xiaolong Jin, Jinyuan Jia, and Xiangyu Zhang. 2025. Foot-In-The-Door: A Multi-turn Jailbreak for LLMs. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 1939–1950. doi:10.18653/v1/2025.emnlp-main.100 [162] Jiaqi Wu, Chen Chen, Chunyan Hou, and Xiaojie Yuan. 2025. SafeInt: Shielding Large Language Models from Jailbreak Attacks via Safety-Aware Representation Intervention. In Findings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 8473–8488. doi:10.18653/v1/2025. findings-emnlp.450 [163]Zeguan Xiao, Yan Yang, Guanhua Chen, and Yun Chen. 2024. Distract Large Language Models for Automatic Jailbreak Attack. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 16230–16244. doi:10.18653/v1/2024.emnlp-main.908 [164]Yueqi Xie, Minghong Fang, Renjie Pi, and Neil Gong. 2024. Gradsafe: Detecting jailbreak prompts for llms via safety-critical gradient analysis. In Proceedings of the 62nd annual meeting of the association for computational linguistics (volume 1: Long papers). 507–518. [165]Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. 2023. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence 5, 12 (2023), 1486–1496. [166]Zhixin Xie, Xurui Song, and Jun Luo. 2025. Attack via Overfitting: 10-shot Benign Fine-tuning to Jailbreak LLMs. In Advances in Neural Information Processing Systems, D. Belgrave, C. Zhang, H. Lin, R. Pascanu, P. Koniusz, M. Ghassemi, and N. Chen (Eds.), Vol. 38, Main Conference. Curran Associates, Inc., 31136–31170. doi:10.52202/085713-1046 [167] Chen Xiong, Xiangyu Qi, Pin-Yu Chen, and Tsung-Yi Ho. 2025. Defensive Prompt Patch: A Robust and Generalizable Defense of Large Language Models against Jailbreak Attacks. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 409–437. doi:10.18653/v1/2025.findings- acl.23 [168]Xiqiao Xiong, Ouxiang Li, Zhuo Liu, Moxin Li, Wentao Shi, Fengbin Zhu, Qifan Wang, and Fuli Feng. 2026. TROJail: Trajectory-Level Optimization for Multi-Turn Large Language Model Jailbreaks with Process Rewards. In Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (Eds.). Association for Computational Linguistics, San Diego, California, United States, 48086–48109. doi:10.18653/v1/2026.acl-long.2220 [169]Feiyue Xu, Hongsheng Hu, Chaoxiang He, Sheng Hang, Hanqing Hu, Xiuming Liu, Yubo Zhao, Zhengyan Zhou, Bin Benjamin Zhu, Shi-Feng Sun, Dawu Gu, and Shuo Wang. 2026. SoK: Robustness in Large Language Models against Jailbreak Attacks. In 2026 IEEE Symposium on Security and Privacy (SP). 118–137. doi:10.1109/SP63933.2026.00107 [170]Huiyu Xu, Wenhui Zhang, Zhibo Wang, Feng Xiao, Rui Zheng, Zhongjie Ba, and Kui Ren. 2026. RedAgent: An Autonomous Agent for Context-Aware Red Teaming of LLM Jailbreaks. IEEE Transactions on Dependable and Secure Computing 23, 3 (2026), 6506–6521. doi:10.1109/TDSC.2026.3665230 [171] Nan Xu, Fei Wang, Ben Zhou, Bangzheng Li, Chaowei Xiao, and Muhao Chen. 2024. Cognitive Overload: Jailbreaking Large Language Models with Overloaded Logical Thinking. In Findings of the Association for Computational Linguistics: NAACL 2024, Kevin Duh, Helena Gomez, and Steven Bethard (Eds.). Association for Computational Linguistics, Mexico City, Mexico, 3526–3548. doi:10.18653/v1/2024.findings-naacl.224 [172]Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. 2024. SafeDecoding: Defending against Jailbreak Attacks via Safety-Aware Decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 5587–5605. doi:10.18653/v1/2024.acl-long.303 [173] Fan Yang. 2025. Safe2Harm: Semantic Isomorphism Attacks for Jailbreaking Large Language Models. arXiv preprint arXiv:2512.13703 (2025). [174] Hao Yang, Lizhen Qu, Ehsan Shareghi, and Gholamreza Haffari. 2024. Jigsaw puzzles: Splitting harmful questions to jailbreak large language models. arXiv preprint arXiv:2410.11459 (2024). [175]Junxiao Yang, Zhexin Zhang, Shiyao Cui, Hongning Wang, and Minlie Huang. 2025. Guiding not Forcing: Enhancing the Transferability of Jailbreaking Attacks on LLMs via Removing Superfluous Constraints. In Proceedings of the 63rd Annual Meeting of the Association for Computational FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. Linguistics (Volume 1: Long Papers), Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 19643–19655. doi:10.18653/v1/2025.acl-long.965 [176]Xikang Yang, Xuehai Tang, Songlin Hu, and Jizhong Han. 2024. Chain of attack: a semantic-driven contextual multi-turn attacker for llm. arXiv preprint arXiv:2405.05610 (2024). [177]Yan Yang, Zeguan Xiao, Xin Lu, Hongru Wang, Xuetao Wei, Hailiang Huang, Guanhua Chen, and Yun Chen. 2025. SeqAR: Jailbreak LLMs with Sequential Auto-Generated Characters. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), Luis Chiruzzo, Alan Ritter, and Lu Wang (Eds.). Association for Computational Linguistics, Albuquerque, New Mexico, 912–931. doi:10.18653/v1/2025.naacl-long.42 [178] Dongyu Yao, Jianshu Zhang, Ian G. Harris, and Marcel Carlsson. 2024. FuzzLLM: A Novel and Universal Fuzzing Framework for Proactively Discovering Jailbreak Vulnerabilities in Large Language Models. In ICASSP 2024 - 2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 4485–4489. doi:10.1109/ICASSP48485.2024.10448041 [179]Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. 2022. React: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629 (2022). [180] Yang Yao, Xuan Tong, Ruofan Wang, Yixu Wang, Lujundong Li, Liang Liu, Yan Teng, and Yingchun Wang. 2025. A Mousetrap: Fooling Large Reasoning Models for Jailbreak with Chain of Iterative Chaos. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 7837–7855. doi:10.18653/v1/2025.findings-acl.408 [181] Zonghao Ying, Deyue Zhang, Zonglei Jing, Yisong Xiao, Quanchen Zou, Aishan Liu, Siyuan Liang, Xiangzheng Zhang, Xianglong Liu, and Dacheng Tao. 2025. Reasoning-Augmented Conversation for Multi-Turn Jailbreak Attacks on Large Language Models. In Findings of the Association for Computational Linguistics: EMNLP 2025, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 17138–17157. doi:10.18653/v1/2025.findings-emnlp.929 [182]Zheng Xin Yong, Cristina Menghini, and Stephen Bach. 2023. Low-Resource Languages Jailbreak GPT-4. In Socially Responsible Language Modelling Research. [183] Jiahao Yu, Xingwei Lin, Zheng Yu, and Xinyu Xing. 2023. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253 (2023). [184] Lei Yu, Virginie Do, Karen Hambardzumyan, and Nicola Cancedda. 2025. Robust LLM safeguarding via refusal feature adversarial training. In International Conference on Learning Representations, Y. Yue, A. Garg, N. Peng, F. Sha, and R. Yu (Eds.), Vol. 2025. 5254–5277. [185]Miao Yu, Junfeng Fang, Yingjie Zhou, Xing Fan, Kun Wang, Shirui Pan, and Qingsong Wen. 2024. Llm-virus: Evolutionary jailbreak attack on large language models. arXiv preprint arXiv:2501.00055 (2024). [186]Jiayi Yuan, Jonathan Nöther, Natasha Jaques, and Goran Radanović. 2026. AgenticRed: Evolving Agentic Systems for Red-Teaming. arXiv preprint arXiv:2601.13518 (2026). [187] Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-Tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. 2024. GPT-4 Is Too Smart To Be Safe: Stealthy Chat with LLMs via Cipher. In International Conference on Learning Representations, B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun (Eds.), Vol. 2024. 53902–53922. [188]Churui Zeng, Weiwei Qi, Kedong Xiu, Tianhang Zheng, Chaochao Lu, Liang He, Zhan Qin, and Kui Ren. 2026. TRACE: Task-Aware Adaptive Self-Evolving Agentic Jailbreaking. arXiv preprint arXiv:2605.30883 (2026). [189] Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, et al. 2024. Shieldgemma: Generative ai content moderation based on gemma. arXiv preprint arXiv:2407.21772 (2024). [190]Yi Zeng, Hongpeng Lin, Jingwen Zhang, Diyi Yang, Ruoxi Jia, and Weiyan Shi. 2024. How Johnny Can Persuade LLMs to Jailbreak Them: Rethinking Persuasion to Challenge AI Safety by Humanizing LLMs. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 14322–14350. doi:10.18653/v1/2024.acl-long.773 [191]Yifan Zeng, Yiran Wu, Xiao Zhang, Huazheng Wang, and Qingyun Wu. 2024. Autodefense: Multi-agent llm defense against jailbreak attacks. arXiv preprint arXiv:2403.04783 (2024). [192] Qiusi Zhan, Zhixiang Liang, Zifan Ying, and Daniel Kang. 2024. InjecAgent: Benchmarking Indirect Prompt Injections in Tool-Integrated Large Language Model Agents. In Findings of the Association for Computational Linguistics: ACL 2024, Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 10471–10506. doi:10.18653/v1/2024.findings-acl.624 [193]Hangfan Zhang, Zhimeng Guo, Huaisheng Zhu, Bochuan Cao, Lu Lin, Jinyuan Jia, Jinghui Chen, and Dinghao Wu. 2024. Jailbreak Open-Sourced Large Language Models via Enforced Decoding. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 5475–5493. doi:10.18653/v1/2024.acl-long.299 [194] Hangtao Zhang, Chenyu Zhu, Xianlong Wang, Ziqi Zhou, Changgan Yin, Minghui Li, Lulu Xue, Yichen Wang, Shengshan Hu, Aishan Liu, Peijin Guo, and Leo Yu Zhang. 2025. BadRobot: Jailbreaking Embodied LLMs in the Physical World. In The Thirteenth International Conference on Learning Representations. [195]Shenyi Zhang, Yuchen Zhai, Keyan Guo, Hongxin Hu, Shengnan Guo, Zheng Fang, Lingchen Zhao, Chao Shen, Cong Wang, and Qian Wang. 2025. JBShield: Defending Large Language Models from Jailbreak Attacks through Activated Concept Analysis and Manipulation. In 34th USENIX FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations Security Symposium (USENIX Security 25). USENIX Association, Seattle, WA, 8215–8234. [196]Yihao Zhang and Zeming Wei. 2025. Boosting Jailbreak Attack with Momentum. In ICASSP 2025 - 2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 1–5. doi:10.1109/ICASSP49660.2025.10888812 [197] Zhexin Zhang, Junxiao Yang, Pei Ke, Shiyao Cui, Chujie Zheng, Hongning Wang, and Minlie Huang. 2024. Safe unlearning: A surprisingly effective and generalizable solution to defend against jailbreak attacks. arXiv preprint arXiv:2407.02855 1, 2 (2024), 3. [198]Zhexin Zhang, Junxiao Yang, Pei Ke, Fei Mi, Hongning Wang, and Minlie Huang. 2024. Defending Large Language Models Against Jailbreaking Attacks Through Goal Prioritization. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Lun-Wei Ku, Andre Martins, and Vivek Srikumar (Eds.). Association for Computational Linguistics, Bangkok, Thailand, 8865–8887. doi:10.18653/v1/2024.acl-long.481 [199]Ziyang Zhang, Qizhen Zhang, and Jakob Foerster. 2024. Parden, can you repeat that? defending against jailbreaks via repetition. arXiv preprint arXiv:2405.07932 (2024). [200]Jiawei Zhao, Kejiang Chen, Weiming Zhang, and Nenghai Yu. 2025. SQL Injection Jailbreak: A Structural Disaster of Large Language Models. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 6871–6891. doi:10.18653/v1/2025.findings-acl.358 [201]Jianli Zhao, Tingchen Fu, Rylan Schaeffer, Mrinank Sharma, and Fazl Barez. 2025. Chain-of-thought hijacking. arXiv preprint arXiv:2510.26418 (2025). [202]Wei Zhao, Zhe Li, Yige Li, Ye Zhang, and Jun Sun. 2024. Defending Large Language Models Against Jailbreak Attacks via Layer-specific Editing. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 5094–5109. doi:10.18653/v1/2024.findings-emnlp.293 [203] Yinzhi Zhao, Ming Wang, Shi Feng, Xiaocui Yang, Daling Wang, and Yifei Zhang. 2026. Defending large language models against jailbreak attacks via in-decoding safety-awareness probing. arXiv preprint arXiv:2601.10543 (2026). [204]Xiaosen Zheng, Tianyu Pang, Chao Du, Qian Liu, Jing Jiang, and Min Lin. 2024. Improved Few-Shot Jailbreaking Can Circumvent Aligned Language Models and Their Defenses. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 32856–32887. doi:10.52202/079017-1034 [205]Andy Zhou, Bo Li, and Haohan Wang. 2024. Robust Prompt Optimization for Defending Language Models Against Jailbreaking Attacks. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 40184–40211. doi:10.52202/079017-1270 [206]Yujun Zhou, Yufei Han, Haomin Zhuang, Kehan Guo, Zhenwen Liang, Hongyan Bao, and Xiangliang Zhang. 2024. Defending Jailbreak Prompts via In- Context Adversarial Game. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 20084–20105. doi:10.18653/v1/2024.emnlp-main.1121 [207]Yukai Zhou, Jian Lou, Zhijie Huang, Zhan Qin, Sibei Yang, and Wenjie Wang. 2025. Don’t Say No: Jailbreaking LLM by Suppressing Refusal. In Findings of the Association for Computational Linguistics: ACL 2025, Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar (Eds.). Association for Computational Linguistics, Vienna, Austria, 25224–25249. doi:10.18653/v1/2025.findings-acl.1294 [208]Yuqi Zhou, Lin Lu, Ryan Sun, Pan Zhou, and Lichao Sun. 2024. Virtual Context Enhancing Jailbreak Attacks with Special Token Injection. In Findings of the Association for Computational Linguistics: EMNLP 2024, Yaser Al-Onaizan, Mohit Bansal, and Yun-Nung Chen (Eds.). Association for Computational Linguistics, Miami, Florida, USA, 11843–11857. doi:10.18653/v1/2024.findings-emnlp.692 [209] Junda Zhu, Lingyong Yan, Shuaiqiang Wang, Dawei Yin, and Lei Sha. 2025. Reasoning-to-Defend: Safety-Aware Reasoning Can Defend Large Language Models from Jailbreaking. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, Christos Christodoulopoulos, Tanmoy Chakraborty, Carolyn Rose, and Violet Peng (Eds.). Association for Computational Linguistics, Suzhou, China, 29343–29361. doi:10.18653/v1/2025.emnlp-main.1493 [210]Wentian Zhu, Zhen Xiang, Wei Niu, and Le Guan. 2026. MetaBreak: Jailbreaking Online LLM Services via Special Token Manipulation. In 2026 IEEE Symposium on Security and Privacy (SP). 98–117. doi:10.1109/SP63933.2026.00095 [211]Miao Ziqi, Lijun Li, Yuan Xiong, Zhenhua Liu, Pengyu Zhu, and Jing Shao. 2026. Response Attack: Exploiting Contextual Priming to Jailbreak Large Language Models. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 40. 35284–35292. [212] Andy Zou, Long Phan, Justin Wang, Derek Duenas, Maxwell Lin, Maksym Andriushchenko, Rowan Wang, Zico Kolter, Matt Fredrikson, and Dan Hendrycks. 2024. Improving Alignment and Robustness with Circuit Breakers. In Advances in Neural Information Processing Systems, A. Globerson, L. Mackey, D. Belgrave, A. Fan, U. Paquet, J. Tomczak, and C. Zhang (Eds.), Vol. 37. Curran Associates, Inc., 83345–83373. doi:10.52202/079017-2651 [213]Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson. 2023. Universal and transferable adversarial attacks on aligned language models. arXiv preprint arXiv:2307.15043 (2023). FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE Mia et al. Author Biographies Md. Jueal Mia is currently pursuing a Ph.D. in Computer Science at Florida International University, where he serves as a Graduate Research Assistant in the Security, Optimization, and Learning for Interdependent Networks (solid) Laboratory. He received his B.S. and M.S. degrees in Computer Science and Engineering from Jahangirnagar University, Bangladesh. Before joining FIU, he served as a faculty member at Daffodil International University for more than six years. His research focuses on trustworthy AI, AI security, foundation models, agentic AI, federated learning, and privacy-preserving machine learning. Yanzhao Wu is an Assistant Professor in the Knight Foundation School of Computing and Information Sciences (KFSCIS), Florida International University (FIU). He received the Ph.D. degree in Computer Science from Georgia Institute of Technology in 2022. His research interests lie at the intersection of machine learning and computing systems, including machine learning algorithm and system co-design, large language models, generative AI, privacy-preserving machine learning, and their real-world applications. Dr. Selcuk Uluagac is currently an Eminent Scholar Chaired Professor in the School of Computing and Information Science at Florida International University, Miami, Florida, where he is the Director of the Center for Integrated SEcurity, PRivacy, and Trustworthy AI (CIERTA) and the Cyber-Physical Systems Security Lab (CSL). Before, he was a Senior Researcher at Georgia Tech. He holds a PhD from Georgia Tech and MS from Carnegie Mellon University. He received US National Science Foundation CAREER Award, US Air Force Office of Sponsored Research’s Summer Faculty Fellowship, and Google’s ASPIRE Research award in security and privacy, inter alia. In 2026, he was honored with IEEE R3 Joseph M. Biedenbach Outstanding Engineering Educator Award. He is an expert in the areas of cybersecurity and privacy. His pioneering research has strengthened the security and privacy foundations of smart technologies, including cyber-physical systems (CPS) and the Internet of Things (IoT), with broad impacts on modern life across homes, workplaces, cities, and critical infrastructure. He has hundreds of publications in the most reputable venues as well as numerous patents. His research has been funded by numerous government agencies and industry. He has chaired/served on the of top-tier security conferences, e.g., NDSS, USENIX, ACM CCS, IEEE SP, and currently serving as the deputy editor in-chief of IEEE TIFS and associate editor of Elsevier COMNET journals, and was the TPC Vice Chair of NDSS 2026. More information can be obtained from http://nweb.eng.fiu.edu/selcuk/. FOR EDUCATIONAL PURPOSES ONLY EDUCATIONAL RELEASE SoK: Rethinking Jailbreaking in the Era of Agentic AI: Attacks, Defenses, and Practical Considerations M. Hadi Amini, Senior Member, ACM; Senior Member, IEEE is an Associate Professor at the Knight Foundation School of Computing and Information Sciences, Florida International University. He is the Director of the Security, Optimization, and Learning for InterDependent Networks Laboratory (solid lab), and Associate Director of the USDOT National Center for Transportation Cybersecurity and Resiliency (TraCR). He received the Ph.D. degree in Electrical and Computer Engineering from Carnegie Mellon University in 2019, where he received the M.Sc. degree in 2015. He also holds a doctoral degree in computer science and technology. He conducts research in trustworthy AI, distributed learning and optimization algorithms, and their applications in real-world problems, such as cybersecurity, cyber-physical systems security and resilience, and public safety. He received the 2025 IEEE Big Data Security Junior Research Award for excellent contributions to Big Data Security in Cyber Physical Systems. He was selected as one of the Rising Stars (Science) by The Academy of Science, Engineering and Medicine of Florida (ASEMFL) for pioneering research in bridging the gap between computing sciences (learning and optimization) and cyber-physical critical infrastructures security and resilience in 2025. He is a recipient of the Best Paper Award from “2019 IEEE Conference on Computational Science & Computational Intelligence”, the 2021 Best Journal Paper Award from “Springer Nature Operations Research Forum Journal”, 2026 FIU College of Engineering and Computing Faculty Excellence in Research Award, 2025 FIU College of Engineering and Computing Faculty Excellence in Mentorship Award, 2024 FIU Top Scholar Award, and the 2023 FIU “Faculty Senate Excellence in Teaching Award”. FOR EDUCATIONAL PURPOSES ONLY