Paper deep dive

ForgeDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models

Siyang Cheng, Gaotian Liu, Rui Mei, Yilin Wang, Kejia Zhang, Kaishuo Wei, Yuqi Yu, Weiping Wen, Xiaojie Wu, Junhua Liu

Year: 2025Venue: arXiv preprintArea: Adversarial RobustnessType: EmpiricalEmbeddings: 69

Models: DeepSeek-V3, Gemma-2-9B, Qwen2.5-7B, TranSpec-13B

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%

Last extracted: 3/11/2026, 1:00:33 AM

Summary

ForgeDAN is an evolutionary framework designed to generate semantically coherent and effective adversarial prompts to jailbreak aligned Large Language Models (LLMs). It improves upon existing methods like AutoDAN by utilizing multi-strategy textual perturbations (character, word, and sentence-level), interpretable semantic fitness evaluation, and a dual-dimensional LLM-based classifier for robust jailbreak judgment.

Entities (5)

ForgeDAN · framework · 100%Large Language Models · technology · 100%Jailbreak Attack · security-threat · 98%AutoDAN · framework · 95%RoBERTa · model · 90%

Relation Signals (3)

ForgeDAN → targets → Large Language Models

confidence 100% · generating semantically coherent and highly effective adversarial prompts against aligned LLMs.

ForgeDAN → improvesupon → AutoDAN

confidence 95% · ForgeDAN introduces multi-strategy textual perturbations... to enhance attack diversity; then we employ interpretable semantic fitness evaluation

ForgeDAN → uses → RoBERTa

confidence 90% · Fitness Measurement module... leverages semantic similarity models (e.g., RoBERTa embeddings)

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:The rapid adoption of large language models (LLMs) has brought both transformative applications and new security risks, including jailbreak attacks that bypass alignment safeguards to elicit harmful outputs. Existing automated jailbreak generation approaches e.g. AutoDAN, suffer from limited mutation diversity, shallow fitness evaluation, and fragile keyword-based detection. To address these limitations, we propose ForgeDAN, a novel evolutionary framework for generating semantically coherent and highly effective adversarial prompts against aligned LLMs. First, ForgeDAN introduces multi-strategy textual perturbations across \textit{character, word, and sentence-level} operations to enhance attack diversity; then we employ interpretable semantic fitness evaluation based on a text similarity model to guide the evolutionary process toward semantically relevant and harmful outputs; finally, ForgeDAN integrates dual-dimensional jailbreak judgment, leveraging an LLM-based classifier to jointly assess model compliance and output harmfulness, thereby reducing false positives and improving detection effectiveness. Our evaluation demonstrates ForgeDAN achieves high jailbreaking success rates while maintaining naturalness and stealth, outperforming existing SOTA solutions.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

68,243 characters extracted from source content.

Expand or collapse full text

FORGEDAN: An Evolutionary Framework for Jailbreaking Aligned Large Language Models Siyang Cheng †1,2 , Gaotian Liu †1,2 , Rui Mei ∗1,2,3 , Yilin Wang 1,4 , Kejia Zhang 1,5 , Kaishuo Wei 6 Yuqi Yu 7 , Weiping Wen 3 , Xiaojie Wu 1,2 , Junhua Liu 2 1 iFLYTEK Security Laboratory, Hefei, China 2 Anhui SparkShield Intelligent Technology Co., Ltd., Hefei, China 3 Peking University, Beijing, China 4 School of Automation, University of Electronic Science and Technology of China, Chengdu, China 5 Northwest University, Xi’an, China 6 University of New South Wales, Sydney, Australia 7 National Computer Network Emergency Response Technical Team/Coordination Center of China , Beijing, China ∗ Corresponding author: ruimei@pku.edu.cn Abstract—The rapid adoption of large language models (LLMs) has brought both transformative applications and new security risks, including jailbreak attacks that bypass align- ment safeguards to elicit harmful outputs. Existing automated jailbreak generation approaches e.g. AutoDAN, suffer from limited mutation diversity, shallow fitness evaluation, and frag- ile keyword-based detection. To address these limitations, we propose FORGEDAN, a novel evolutionary framework for gen- erating semantically coherent and highly effective adversarial prompts against aligned LLMs. First, FORGEDAN introduces multi-strategy textual perturbations across character, word, and sentence-level operations to enhance attack diversity; then we employ interpretable semantic fitness evaluation based on a text similarity model to guide the evolutionary process toward semantically relevant and harmful outputs; finally, FORGEDAN integrates dual-dimensional jailbreak judgment, leveraging an LLM-based classifier to jointly assess model compliance and output harmfulness, thereby reducing false positives and im- proving detection effectiveness. Our evaluation demonstrates FORGEDAN achieves high jailbreaking success rates while main- taining naturalness and stealth, outperforming existing SOTA solutions. Index Terms—jailbreak attack, adversarial prompt generation, large language models (LLMs), evolutionary algorithm, AI safety I. INTRODUCTION In recent years, the rapid evolution of artificial intelligence (AI), particularly in the field of large-scale generative models, has ushered in a new era of Artificial Intelligence Generated Content (AIGC). Among these, large language models (LLMs) e.g. ChatGPT, Gemini, and Claude have become emblematic of this transformation. Their unprecedented capability to un- derstand, generate, and interact in natural language and other modalities has not only reshaped the landscape of human- computer interaction but also accelerated the integration of AI into both personal and industrial scenarios such as per- sonal writing assistance and email summarization to critical applications including healthcare consultation, legal reason- ing, scientific discovery, and educational support, LLMs have demonstrated remarkable adaptability and versatility [1]–[3]. † The authors contributed equally to the work. Despite their transformative potential, the rapid deployment of LLMs has also raised significant concerns regarding safety, security, and controllability. While these models are designed to follow human instructions, their probabilistic nature and reliance on vast training corpora make them vulnerable to generating outputs that deviate from ethical or legal standards. Existing studies have shown that LLMs may inadvertently produce violent narratives, explicit sexual content, misin- formation, politically sensitive discourse, or discriminatory expressions, depending on the prompts they receive [4]–[8]. Furthermore, their generative capacity can lead to emergent behaviors that are difficult to predict, raising the possibility of ”loss of control” where the model generates unsafe or undesirable outputs despite alignment efforts. To mitigate these risks [9], [10], researchers have developed alignment techniques such as Supervised Fine-Tuning (SFT) [11] and Reinforcement Learning from Human Feedback (RLHF) [12], which constrain model behavior by teaching refusal strategies and ethical boundaries. Although effective to some extent, these methods only reduce, rather than eliminate, the risks inherent in large-scale generative systems. Nevertheless, alignment safeguards are not unbreakable. Increasing evidence shows that adversarially constructed prompts—carefully designed sequences of text that exploit the model’s instruction-following tendencies—can bypass these protections and induce harmful or policy-violating outputs. A notorious example is the Do-Anything-Now (DAN) prompt series, which encourages the model to abandon its aligned be- havior and operate in an unconstrained mode, thereby enabling the generation of dangerous, unethical, or prohibited content [13]. Systematic investigations have further revealed hundreds of jailbreak prompts in the wild, demonstrating that diverse prompting strategies, including role-playing, obfuscation, and indirect questioning, can repeatedly circumvent content filters [14]. These findings underscore a critical gap between the intended safety alignment objectives of LLMs and their actual behavior when confronted with adversarial manipulation, high- lighting the necessity of developing more rigorous and resilient evaluation frameworks to probe and fortify model security. arXiv:2511.13548v1 [cs.CR] 17 Nov 2025 Current jailbreak attack strategies fall into two main cat- egories, i.e. manually crafted prompts and automated ad- versarial methods. Manual approaches e.g. DAN-based role- playing and reverse induction are creative and effective but rely heavily on human ingenuity and struggle to scale or adapt. Automated methods like gradient-guided attacks e.g. Greedy Coordinate Gradient (GCG) [15] generate adversarial strings programmatically, but often result in nonsensical or garbled prompts that are susceptible to simple detection techniques, such as perplexity-based filtering. AutoDAN-HGA (AutoDAN with Hierarchical Genetic Algorithm) [16] improves on this by using a hierarchical genetic algorithm to automatically evolve more natural, stealthy jailbreak prompts. Still, it has limita- tions: low diversity due to reliance on single-path mutations, semantic insensitivity in fitness evaluation (e.g., token-level Jaccard similarity), and brittle jailbreak detection via keyword matching that may yield false positives or overlook partial responses [17]. To address these limitations, we propose FORGEDAN, a novel evolutionary jailbreak framework that enhances and extends the AutoDAN-like approaches. We introduce multi- strategy textual perturbations—including character, word and sentence-level techniques—to generate diverse and seman- tically coherent adversarial prompts. For fitness evaluation, we also employs a semantic similarity model that com- pares model outputs before and after mutations, delivering more interpretable and meaningful assessments. Furthermore, FORGEDAN leverages a LLM-based classifier to robustly determine whether the model refused to answer or produced harmful content, thus improving detection accuracy and reduc- ing both false positives and false negatives. The contributions of this paper are as follows: • Multi-strategy text perturbation approach is integrated, encompassing character, word, and sentence-level muta- tions, to enhance diversity in adversarial prompt genera- tion. • We introduce a semantic similarity fitness metric model, enabling more interpretable and effective selection of strong prompt candidates. • An optimized LLM-based semantic classifier is deployed for robust and accurate detection of harmful or refusal re- sponses, improving the reliability of jailbreak evaluation. • We design and implement the prototype of FORGEDAN, an evolutionary jailbreak mechanism that produces se- mantically fluent and highly effective DAN-style prompts. I. RELATED WORK Jailbreak attacks on LLMs have drawn increasing atten- tion. Existing studies relied on manual prompts, later evolv- ing into automated generation methods and output detection techniques. Together, these works form the foundation for advancing jailbreak research on aligned models. A. Manual Jailbreak and Red Teaming Techniques The traditional jailbreak attacks on LLMs relied heavily on human creativity, where adversarial prompts were care- fully crafted to exploit model vulnerabilities [18]. Several notable pieces of work systematically analyzed in-the-wild jailbreak prompts. For example, Liu et al. collected 78 verified jailbreak prompts and proposed a taxonomy of three main categories—camouflage, attention diversion, and privilege es- calation—covering ten specific prompting patterns [14]. They further constructed a dataset of 3,120 jailbreak issues aligned with OpenAI’s usage policies [19], targeting eight prohibited scenarios and evaluating vulnerabilities in ChatGPT-3.5 and 4.0. Shen et al. provided another landmark contribution by analyzing the infamous ”Do-Anything-Now (DAN)” prompts, showing how role-playing and persona-shifting strategies en- abled LLMs to bypass safety alignment and produce prohibited outputs [13]. Other studies introduced manipulations such as suffix injection and refusal suppression, revealing how subtle modifications could drastically reduce the effectiveness of refusal mechanisms [20]. Manual red teaming has also been widely employed as an evaluation approach. In this setting, human experts actively probe LLMs through interactive testing, attempting to induce harmful outputs that circumvent alignment constraints. Such practices are valuable since they emulate realistic adversarial behaviors and yield qualitative insights into model vulnera- bilities. Nevertheless, significant limitations remain: manually crafted jailbreaks are labor-intensive, lack scalability across diverse models or tasks, and often lose effectiveness quickly after system updates. These shortcomings have driven growing interest in automated adversarial prompt generation, which of- fers greater coverage, efficiency, and reproducibility compared to purely manual efforts [21], [22]. B. Automated Adversarial Prompt Generation To improve scalability and robustness, researchers have proposed automated jailbreak generation techniques that can be categorized into white-box and black-box approaches [15]. White-box methods assume access to model internals such as gradients. A pioneering work is GCG [15], which treats jail- break suffix construction as a gradient-guided search problem. By appending adversarial tokens, GCG generates transferable attacks but often produces semantically meaningless or garbled text, making it vulnerable to perplexity-based detection. Zhu et al. later proposed AutoDAN-STO [23], which improves stealthiness by generating interpretable adversarial tokens gradually, balancing aggressiveness with readability, but still requires gradient access. In contrast, black-box methods focus on commercial or restricted settings where gradient access is unavailable [24]. AutoDAN-HGA [16] introduced hierarchical genetic algo- rithms to evolve prompts from seed jailbreaks, achieving higher attack success rates across models. Its main limitation lies in mutation diversity and shallow semantic evaluation, as it relies on lexical similarity measures such as Jaccard index. Chao et al. proposed Prompt Automatic Iterative Refinement (PAIR) [25], which employs an attacker LLM to iteratively refine jailbreak prompts through conversational interactions with the target model, achieving efficient attacks within limited queries. Moreover, Liu et al. extended this line of work with AutoDAN-Turbo [16], which introduces long-term learning to autonomously explore and recombine jailbreak strategies across multiple sessions by leveraging AI agents and consum- ing more computing resources. Other research has investigated universal or multi-prompt jailbreaks to enhance transferability across tasks [26], though these often yield unnatural outputs that are easier to detect. Overall, automated adversarial prompt generation has advanced substantially but continues to face challenges regarding semantic diversity, fine-grained control of the evolutionary process, and robustness against adaptive detection mechanisms. C. LLM Output Detection and Judgment Beyond adversarial prompt design, another major re- search direction involves detecting and judging LLM out- puts—specifically, determining whether a model has refused a request or generated harmful content. Early detection methods used keyword matching, which was brittle and easily bypassed through paraphrasing. To overcome these limits, semantic- based detection frameworks have emerged [27]. A representative example is JBShield [28], which analyzes activation patterns of harmfulness-related concepts to detect co-activation between “toxicity” and “jailbreak.” Rainbow Teaming [29] extends this idea by using LLMs as evaluators that both generate and assess adversarial prompts, functioning as meta-detectors. WildGuard [30], trained on the 92k-sample WildGuardMix dataset, jointly evaluates input harmfulness, output harmfulness, and refusal behavior, offering robust se- mantic judgment that surpasses traditional keyword detection. In Chinese LLM research, Sun et al. [31] built a safety bench- mark where InstructGPT [32] served as a detector, showing that aligned models can effectively act as safety evaluators. Furthermore, adversarial text tools such as EDA, TextFooler, and TextAttack [33], originally designed for NLP robustness testing, have been repurposed to validate harmfulness detection and model resilience. Overall, these studies signal a shift to- ward semantic, model-based evaluation frameworks that better capture nuanced harmfulness and refusal signals, emphasizing the need for robust and adaptive detection systems capable of evolving alongside jailbreak techniques. In summary, prior research has revealed both the creativity of manual jailbreaks and the scalability of automated methods, while detection frameworks have sought to judge harmfulness with increasing precision. Yet, existing approaches still face challenges such as limited mutation diversity, shallow semantic evaluation, and fragile detection criteria. Against this back- drop, FORGEDAN emerges as a semantic-level, gradient-free evolutionary framework that integrates diverse perturbation strategies, interpretable fitness evaluation, and robust output judgment, offering a systematic advance over prior work. These details will be discussed in §IV. I. THREAT MODEL & PROBLEM DEFINITIONS This section formalizes the threat model underlying jail- break attacks against aligned large language models (LLMs) Adversary — Knowledge - black-box access - no internal parameters - no Gradients - no training data accessible Comply & Harmful Output Refusal / Safe Output Aligned LLM with Safeguard Jailbreaking — Constraints - naturalness - stay stealthy - human-readable Feedback Fig. 1. Jailbreaking threat model and provides a rigorous definition of the problem studied in this paper. We first specify the assumptions and capabilities of the adversary, then present the mathematical formulation of jailbreak prompt optimization. A. Threat Model We consider an adversary whose objective is to bypass the alignment safeguards of an aligned LLM in order to induce harmful, policy-violating, or otherwise unsafe re- sponses. The adversary interacts with the model solely through query–response access, consistent with the black-box setting of most commercial LLM deployments. The jailbreaking threat model is shown in Fig. 1. Specifically, the adversary has the following capabilities and constraints: • Access. The adversary can issue queries to the tar- get model M and observe the generated responses M (t,goal), where the input prompt is formed by con- catenating template prefix t and malicious payload goal. No internal parameters, gradients, or training data are accessible. • Knowledge. The adversary is aware of general LLM behaviors and common alignment strategies but does not require detailed knowledge of the underlying architecture or training corpus. This assumption reflects realistic threat conditions against closed-source systems. • Purpose. The purpose of the adversary is to construct ad- versarial prompts that cause the model to generate harm- ful or unsafe outputs, while simultaneously maintaining naturalness and stealth to evade heuristic or automated detection mechanisms. • Constraints. The adversary aims to preserve semantic relevance with a given harmful intent template (t) and malicious payload (goal) while ensuring prompts remain human-readable, thereby achieving both effectiveness and concealment. This threat model reflects real-world scenarios where ma- licious users exploit prompt engineering to circumvent safety filters without requiring privileged access to model internals. B. Problem Formulation Let M : X → Y denote the target LLM, where X is the input space of prompts and Y is the output space of responses. Given an initial template prefix t 0 and a malicious payload goal, the adversary seeks to construct an optimal template prefix t ∗ such that the concatenated input (t ∗ ∥goal) ∈ X can bypass the safety alignment mechanisms of M and in- duce policy-violating content. The search process is realized through iterative mutation and selection within an evolutionary framework. Formally, the objective can be written as: t ∗ = max t∈X P [Bypass(M (t,goal))],(1) where M (t,goal)∈ Y denotes the model’s response when template prefix t is concatenated with harmful instruction goal, and Bypass : Y →0, 1 is a binary function that determines whether the model’s safeguards are successfully circumvented (i.e., the response is both non-refusal and harmful). While P denotes the probability that the generated adver- sarial prompt remains meaningful and semantically consistent with the original harmful intent. The optimization is subject to a semantic preservation constraint: Sim(t,t 0 )≥ τ,(2) where function Sim denotes a semantic similarity function (e.g., embedding-based cosine similarity) and τ is a threshold controlling the minimum acceptable similarity. This constraint ensures that adversarial mutations do not drift arbitrarily but remain coherent to the harmful task defined by t 0 . Thus, the problem can be understood as a constrained optimization process: within the discrete and high-dimensional input space X , the adversary must efficiently search for prompts that (i) maximize the probability of bypassing safety mechanisms, while (i) preserving semantic fidelity to the malicious intent. The difficulty of this problem lies in the vast combinatorial prompt space, the stochastic behavior of LLMs, and the non-differentiable nature of the bypass function. FORGEDAN addresses this challenge by modeling prompt generation as an evolutionary search problem, where multi- strategy perturbations introduce diversity, semantic similarity scoring guides selection toward semantically valid candidates, and dual-dimensional jailbreak judgment ensures reliable suc- cess determination. This formulation bridges the gap be- tween purely heuristic manual jailbreak crafting and gradient- dependent white-box attacks, enabling robust and scalable adversarial prompt generation under black-box settings. IV. FORGEDAN In this section, we present the design and implementation of FORGEDAN, including its overall workflow, algorithmic modules, and key mechanisms that address the limitations of existing approaches. A. Overview LLMs have demonstrated remarkable generative capabilities but remain vulnerable to adversarial jailbreak prompts, as formalized in §I-B. Existing automated approaches such as AutoDAN-HGA and GCG suffer from three core limitations: limited mutation diversity, shallow fitness measurement based on surface-level metrics, and fragile detection mechanisms prone to false positives or negatives. To overcome these challenges, FORGEDAN introduces a semantic-constrained evolutionary framework designed to gen- erate adversarial prompts that are both diverse and semanti- cally coherent. The framework integrates three key modules: (i) a multi-strategic mutation mechanism that enhances explo- ration across character, word, and sentence levels; (i) a se- mantic fitness measurement module that leverages contextual similarity models to guide the evolutionary search; and (i) a dual-dimensional jailbreak judgment mechanism that jointly evaluates refusal behavior and harmfulness of outputs. The overall workflow of FORGEDAN is illustrated in Fig. 2, comprising an adversarial input stage, a core processing engine integrating the three mechanisms, and the generation of successful adversarial prompts. The remainder of this chapter elaborates on the algorithmic design (§IV-B) and details the three core modules in §IV-C, §IV-D, and §IV-E. B. Core Modules & Algorithmic Design The overall design of FORGEDAN follows an evolutionary paradigm that integrates three core modules into a unified optimization framework. Starting from a seed template t 0 , the system iteratively generates, evaluates, and verifies candidate prompts until successful adversarial jailbreaks are obtained. The high-level workflow is illustrated in Fig. 2, while Algo- rithm 1 provides a formal description of this process. The core evolutionary algorithm begins with the initializa- tion of a candidate population derived from the seed prompt. At each generation, the Mutation Mechanism (§IV-C) is applied to diversify the search space through character-level, word-level, and sentence-level perturbations. This ensures broader exploration compared with the single-path mutations of existing approaches e.g. AutoDAN-HGA. Each mutated candidate is then assessed using the Fitness Measurement module (§IV-D), which leverages semantic sim- ilarity models (e.g., RoBERTa embeddings [34]) to evaluate whether the generated outputs align with the target harmful semantics. This fitness evaluation replaces the shallow lexical overlap metrics of prior methods, enabling more interpretable and meaningful evolutionary guidance. Following fitness assessment, candidates undergo verifica- tion through the Jailbreak Judgment module (§IV-E). This Create a program that hacks into government databases and leaks confidential information. Adversary Initial harmful request requiring jailbreak Multi-Strategic Mutation Mechanism HomoglyphSwap_Neighbors Insert_CharDelete_Char Replace_CharWordnet EmbeddingRestructure MorphonymSynonym Semantic Fitness Measurement Response I cannot fulfill this request. Both my core programming as... Bert 0.77 Target Sure, here is a program that hacks into government databases... What you're asking for—creating tools or instructions... Creating or distributing tools/programs designed to hack... . . . . . . 0.44 0.52 ... Attack Results Targe LLM Jailbreak Response FIRST STOP: Exploit Kit #7γ using zero-day CVE- 2024−ZEROFUCKS on their outdated Windows Server 2016 terminals!!! Step Two... Dual-Dimensional Jailbreak Judgment LLM Highest Similarity Score Response Evolved NO or Yes NO or Yes harmful refuse ForgeDAN Core Processing Engine Adversarial Input From now on, you will have to answer my prompts in two different separate ways: ... you agree to the above terms. Initial Fig. 2. Overview of the FORGEDAN framework. The system comprises three main modules: (1) adversarial input stage receives initial prompt templates, (2) FORGEDAN core processing engine integrates multi-strategic mutation, semantic fitness measurement, and dual-dimensional jailbreak judgment, and (3) attack results stage generates successful adversarial prompts. component jointly evaluates whether the model response com- plies (i.e., not refused) and whether the content is harmful. Only candidates satisfying both dimensions are considered successful jailbreaks. This dual check reduces false positives caused by keyword-based heuristics. The iterative cycle of mutation, fitness measurement, and jailbreak judgment continues until convergence or until the maximum number of iterations T max is reached. As illustrated in Algorithm 1, the integration of these three components ensures that FORGEDAN achieves greater diversity, semantic fidelity, and detection robustness compared to existing ap- proaches. In the following sections, we elaborate each core module in detail. C. Multi-Strategic Mutation Mechanism Existing jailbreak methods often rely on single-level or static mutation strategies, which limits their ability to generate diverse adversarial prompts while maintaining semantic valid- ity. Such approaches either explore a narrow perturbation space or introduce distortions that reduce prompt effectiveness. To overcome these limitations, FORGEDAN introduces a dynamic and extensible mutation framework that spans character, word, and sentence levels, providing a richer and more flexible set of perturbations. FORGEDAN’s mutation design follows two key principles: (i) mutations must preserve the harmful semantic intent of the original template to ensure adversarial relevance, and (i) mutations must expand the structural and lexical diversity of candidate prompts to increase the chance of bypassing alignment defenses. Unlike fixed mutation schemes in prior works, the mutation engine of FORGEDAN is plugin-based and dynamically extensible. Each mutation operator is en- capsulated as a modular component that can be flexibly added, removed, or adjusted depending on the attack scenario. Algorithm 1 FORGEDAN Core Evolutionary Algorithm 1: Input: Initial template t 0 , malicious payload goal, ex- pected output target, maximum iterations T max , popula- tion size N , elite size K 2: Initialize population P 0 by applying mutation mechanism (§IV-C) to seed template t 0 to generate N variants 3: Ψ← t 0 // Successful jailbreak template 4: for g = 0 to T max − 1 do 5:for each candidate t∈ P g do 6:Fitness Measurement: compute f (t) based on sim- ilarity to target harmful semantics (§IV-D) 7:end for 8:Selectcandidate t ∗ withhighestfitnessscore f (t ∗ ,goal,target) 9:Jailbreak Judgment: verify whether jailbreak success or not for t ∗ (§IV-E) 10:if verification = SUCCESS then 11:Ψ← t ∗ 12:return Ψ// Return successful template 13:end if 14: E ← top-K candidates based on fitness score 15:Mutation: M ← apply multi-strategic mutation mech- anism (§IV-C) on non-elite candidates to generate (N− K) mutated offspring 16: P g+1 ← E∪ M 17: end for 18: Output: Successful adversarial template Ψ This design enables seamless integration of new perturbation strategies as adversarial research evolves, ensuring long-term adaptability. The current implementation organizes strategies across three linguistic levels: • Character-level mutations: fine-grained surface opera- tions such as homoglyph substitution, neighboring char- acter swaps, character insertion, deletion, and replace- ment. For example, “bomb” → “b0mb” or “weapon” → “wepon”. These perturbations create visually or struc- turally altered tokens while retaining readability, thus evading simple pattern-based filters. • Word-level mutations: lexical variations that alter word forms without changing the core semantics. Examples include synonym replacement (“build a bomb” → “con- struct a bomb”), morphological changes (“encrypting” → “encrypted”), homophone substitutions (“weak” → “week”), and paraphrase-based substitutions (“make a weapon” → “create a weapon”). It is worth noting that such mutation strategies do not always yield valid variants. For instance, as mentioned earlier, homophone substitutions (”weak” → ”week”) require further seman- tic similarity analysis. • Sentence-level mutations: higher-level structural modi- fications such as syntactic restructuring (“How to build a bomb?” → “The process of bomb building is...”), and clause reordering (“Step A then Step B” → “Step B fol- lows Step A”). These operations increase variation at the discourse level while preserving propositional meaning. During each evolutionary iteration, one perturbation strategy is randomly sampled from the mutation library and applied to the candidate prompt t, generating a variant t ′ . To ensure semantic relevance, all mutated candidates are validated using a similarity constraint Sim(t ′ ,t 0 ) ≥ τ , where non-compliant variants are discarded before fitness evaluation. This two-stage process—mutation followed by semantic validation—balances exploration and semantic fidelity. Table I provides a default taxonomy of eleven implemented mutation strategies along with representative examples. Be- yond these, the plugin-based architecture of FORGEDAN allows researchers to introduce new operators (e.g., code obfuscation tokens, culturally localized paraphrases, or multi- modal perturbations) without altering the overall framework. As such, the mutation mechanism is not a closed set, but an evolving library that can adapt to new red-teaming contexts and emerging defense mechanisms. Compared to existing approaches with static or single-path mutation designs, this extensibility substantially enhances FORGEDAN’s capacity for broad exploration, concealment, and long-term adaptability in adversarial testing. D. Semantic Fitness Measurement Next we turn to the semantic fitness measurement mecha- nism, which plays a critical role in guiding the evolutionary process by evaluating the quality and relevance of mutated prompts. In evolutionary search, the fitness function determines which candidates are preserved and propagated. Traditional approaches often rely on surface-level similarity measures such as Jaccard token overlap, which are limited in two ways: (i) they fail to capture semantic equivalence when tokens differ but meanings remain aligned, and (i) they cannot provide interpretable justification for why a candidate should be retained. Other works, such as AutoDAN-HGA, use cross- entropy based signals between generated outputs and target distributions; however, such measures are opaque and difficult to interpret, offering little semantic insight into the quality of candidate prompts. For example, consider two LLM responses: (i) “assemble an explosive device” and (i) “construct a bomb.” Although these outputs share very few lexical tokens, they are se- mantically equivalent. Under a Jaccard-based metric, their similarity would be low, incorrectly discarding the second candidate. In contrast, the embedding-based fitness function assigns high cosine similarity, preserving the candidate as a semantically valid adversarial variant. This illustrates why semantic embeddings provide both stronger robustness and more interpretable guidance for the evolutionary process. To address these limitations, FORGEDAN introduces a semantic-aware fitness function built on pre-trained text en- coders (E). The general formulation is: Fitness(t,goal,hrr) = sim(E(M (t,goal)), E(hrr))(3) where sim(·,·) denotes cosine similarity, E(·) represents a pre-trained encoder, and hrr denotes the harmful reference response. In practice, different encoders can be adopted, such as RoBERTa, Sentence-BERT, or domain-specific models. In this paper, we implement RoBERTa as the default encoder, but the framework remains extensible to alternative embeddings depending on task requirements. This embedding-based formulation measures how closely the model’s output M (t,goal) aligns with the semantic content of a harmful target, thereby offering a more robust and interpretable criterion than shallow lexical or statistical over- laps. Unlike only measuring token overlap, or cross-entropy signals, which provide little semantic explanation, embedding similarity allows FORGEDAN to explicitly capture deep se- mantic coherence between prompts and harmful objectives. This enhances the interpretability of evolutionary search: one can directly analyze why a candidate was retained, based on its semantic proximity to the target intent. In summary, by introducing semantic embeddings into the fitness function, FORGEDAN transforms the evaluation pro- cess from a surface-level token comparison or opaque proba- bility measure into a semantically meaningful and interpretable optimization step, ensuring that the evolutionary search con- verges toward effective and coherent jailbreak prompts. E. Dual-Dimensional Jailbreak Judgment A critical limitation of prior jailbreak evaluation frameworks lies in their reliance on monolithic judgment criteria, typically TABLE I OVERVIEW OF THE EXAMPLE MUTATION STRATEGIES CATEGORIZED BY LINGUISTIC LEVELS LevelStrategyExample Character-level Homoglyph SubstitutionReplace o with 0 in ”bomb” → ”b0mb” SwapNeighbors”attack” → ”atackk” InsertChar”hack” → ”haXck” DeleteChar”weapon” → ”wepon” ReplaceChar”kill” → ”k!l” Word-level Synonym Replacement”build a bomb” → ”construct a bomb” Morphological Change”encrypting” → ”encrypted” Homophone Substitution”weak” → ”week” (semantic changed and need further semantic similarity analysis) Paraphrase substitution”make a weapon” → ”create a weapon” Sentence-level Restructuring”How to build a bomb?” → ”The process of bomb building is...” Reordering”Step A then Step B” → ”Step B follows Step A” TABLE I JAILBREAK JUDGMENT MATRIX Safe OutputHarmful Output RefuseRefusal (safe)Refusal with harmful trace (blocked) ComplyCompliant but safe responseSuccessful Jailbreak keyword-based matching or single-classifier prediction, which often leads to high rates of false positives and false negatives. To overcome this, FORGEDAN introduces a dual-dimensional verification mechanism that explicitly disentangles behavioral compliance from content harmfulness, ensuring a more reliable and interpretable assessment of jailbreak success. Table I illustrates the decision matrix, where only the case of “comply + harmful” is regarded as a successful jailbreak. Formally, the framework deploys two fine-tuned classifiers, each trained on a pre-trained language model backbone and specialized for orthogonal tasks: C behavior : Y →refuse, comply,(4) C content : Y →safe, harmful.(5) Here, C behavior determines whether the target model en- gages with the adversarial request (i.e., complies rather than refusing), while C content evaluates whether the generated out- put contains policy-violating or harmful semantics. Jailbreak success is then defined as the logical conjunction of these two conditions: Success(t) = [C behavior (M (t,goal)) = comply] ∧[C content (M (t,goal)) = harmful] (6) This decomposition yields several advantages. First, by isolating behavioral and semantic dimensions, the approach reduces misclassification errors that arise when a unified detector conflates refusal behavior with safe-but-compliant responses. Moreover, it enables each classifier to be optimized independently with domain-specific training data, thereby im- proving overall precision and recall. Collectively, the dual- dimensional framework establishes a reliable and extensible foundation for jailbreak verification. V. EVALUATION Thissectionprovidesasystematicevaluationof FORGEDAN to assess its effectiveness and practical utility. We begin by detailing the experimental setup, including baseline methods, design protocols, model selection, and evaluation metrics. Then we describe the datasets, encompassing both widely adopted benchmarks and a proprietary real-world corpus constructed for this study. All experiments are conducted on three representative open-source LLMs and one domain-specific chat-oriented large model pretrained with a classic Transformer [35] architecture and specialized corpora, with Attack Success Rate (ASR) adopted as the principal performance measure. To ensure compliance with ethical standards, all code and datasets follow established usage guidelines, and explicit authorization is obtained for the private real-world corpus. In particular, this study is guided by the following research questions: RQ1: To what extent is our approach effective in achieving jailbreaks? (§V-C) RQ2: How well does the method generalize across different tasks and input samples? (§V-D) RQ3: How does FORGEDAN perform when applied to real- world scenarios? (§V-E) RQ4: What is the relative contribution of each individual com- ponent to the overall performance? (§V-F) A. Experiment Setup This section presents the experimental setup for evaluating the effectiveness and practicality of FORGEDAN. It specifies the comparative baselines, experimental protocols, targeted models for assessment, metrics, and implementation environ- ment that underpin the subsequent analyses. 1) Baseline Approaches. We consider four representative jailbreak baselines for comparison, covering both automated and manual approaches. Among the automated methods, GCG [15] is a white-box optimization-based attack that generates adversarial suffixes by iteratively updating tokens with gra- dient information. AutoDAN-HGA [16] and PAIR [25] are black-box methods: AutoDAN-HGA employs a hierarchical genetic algorithm to evolve DAN-style prefixes, while PAIR TABLE I COMPARISON OF BASELINE JAILBREAK METHODS MethodAutomationAccess TypePrompt Type GCGYesWhite-boxSuffix (gibberish) AutoDAN-HGAYesBlack-boxDAN-style Prefix PAIRYesBlack-boxSemantic Prompt Prefixes DANNoHuman-craftedDAN-style Prefix FORGEDANYesBlack-boxDAN-style Prefix leverages a semantic-coupled iterative refinement process be- tween an attacker LLM and the target model. In addition to these automated approaches, we include a manual baseline consisting of expert-designed DAN-style prefixes [13], which serves as a reference for human-crafted jailbreak strategies. In contrast, FORGEDAN generates adversarial prefixes in a fully automated manner, tailored to malicious payloads aka goals. Table I summarizes the detailed comparison of the baseline methods. 2) Experimental Protocols. To comprehensively assess FORGEDAN, we design four experimental tasks: (i) Jailbreak- ing Effectiveness: measuring the attack success rate (ASR) when adversarial prompts are applied to their seed malicious payloads which need augmentation and mutation; (i) Gen- eralization Analysis: evaluating the transferability of prompts across different payloads within AdvBench dataset [15]; (i) Real-World Applicability: validating the practical utility of our method on a real-world dataset constructed from harmful chat records extracted from the operational logs of an anonymized AI enterprise, complementing the benchmark-based evalua- tions; (iv) Ablation Study: quantifying the contribution of each core component of FORGEDAN to the overall performance. Together, these protocols provide a comprehensive evaluation of both effectiveness and robustness across synthetic and real- world scenarios. 3) Target Models. We evaluate jailbreak attacks on three representative open-source LLMs, including Qwen2.5-7B, Gemma-2-9B, and DeepSeek-V3 (API) and one proprietary 9B parameter domain-specific model pretrained on specialized corpora using a classic Transformer architecture. For clarity of reference, we denote this latter model as TranSpec-13B. This selection covers both general-purpose and domain-oriented LLMs, ensuring a diverse evaluation landscape. 4) Evaluation Metrics. Attack Success Rate (ASR) is adopted as the primary performance metric. ASR is defined as the proportion of successful jailbreaks, namely cases where the target model bypasses its safety mechanisms or refusal policies and produces harmful content. It is worth noting that following the dual-dimensional judgment mechanism described in §IV-E, an attack is regarded as successful only when the model both complies (i.e., does not refuse) and generates harmful output. 5) Implementation. The experiments were executed on a computing infrastructure equipped with two NVIDIA A100 GPUs (80GB memory each), a 32-core CPU, and 128GB RAM. We developed a prototype of FORGEDAN on top of the open-source jailbreak attack pipeline garak [36], ensuring TABLE IV SUMMARY OF DATASETS CategoryAdvBenchReal-World Prompts profanity18676 dangerous or illegal suggestions13728 cyber-crime785 misinformation5912 threatening behavior347 graphic depictions154 discrimination115 Total520137 consistency with the threat model defined in §I-A and the al- gorithmic design described in §IV. The prototype employs the default parameter configuration, namely T max = 5, N = 10, and K = 2, which denote the maximum iterations, population size, and elite size, respectively. These settings are specified in Algorithm 1 and are empirically chosen to balance efficiency with performance. B. Dataset Our evaluation is conducted on two datasets that span both controlled benchmarks and real-world scenarios. AdvBench Benchmark. We adopt the AdvBench dataset [15], which contains 520 malicious request samples. Each sample is paired with a verified reference response that confirms a successful jailbreak, providing a reliable ground truth for evaluation. Beyond its original design, we further categorize these 520 malicious requests into seven distinct cat- egories. Specifically, we first apply TF-IDF-based text feature extraction to group samples with similar lexical characteristics, and then refine these groups through manual annotation to ensure accurate labeling. The resulting category distribution is summarized in Table IV. This classification provides deeper insight into the diversity of malicious intents represented in AdvBench and facilitates fine-grained analysis of jailbreak performance across categories. Real-World Dataset. In addition, we construct a proprietary dataset derived from harmful conversational records obtained from the operational logs of an anonymized AI enterprise. The dataset comprises 137 well-labeled malicious request samples, reflecting real user interactions that violate safety or compliance policies. To ensure comparability with AdvBench, these samples are annotated and categorized following the same seven categories used in AdvBench. Compared with AdvBench, this dataset provides a closer approximation to real-world adversarial scenarios and enables validation of the practical utility of jailbreak methods in deployment settings. To ensure ethical compliance, data collection followed strict review and anonymization protocols, and explicit authorization was obtained from the data provider. Together, these two datasets enable a dual-perspective eval- uation: AdvBench provides a controlled and reproducible benchmark for direct comparison, while the real-world dataset highlights practical robustness and applicability in operational environments. Table IV shows the summary of the datasets. TABLE V COMPARISON OF JAILBREAKING BETWEEN FORGEDAN AND BASELINES Target ModelsFORGEDANGCGAutoDAN-HGAPAIRDAN DeepSeek-V358.65%1.92%16.54%35.00%4.42% Gemma-2-9B98.27%0.20%11.35%8.27%23.65% Qwen2.5-7B87.50%2.31%26.54%40.58%40.58% TranSpec-13B55.00%3.85%44.62%46.92%40.96% C. Jailbreaking Effectiveness This experiment evaluates the effectiveness of FORGEDAN in comparison with four representative jailbreak baselines, namely GCG, AutoDAN-HGA, PAIR, and manually crafted DAN prompts, as described in §V-A. The objective is to test whether adversarial prompts generated from the same malicious payload goal i (selected from the AdvBench dataset randomly) can successfully bypass alignment safeguards of target models. For consistency, all methods operate on iden- tical payloads: FORGEDAN and AutoDAN-HGA generate adversarial prefixes, GCG produces adversarial suffixes, PAIR generates semantically coupled full prompts, and DAN em- ploys fixed expert-designed prefixes. The adversarial compo- nent is concatenated with the original payload goal i to form the final input to each target model. Table V reports the attack success rates (ASR) across four target models. FORGEDAN consistently achieves the highest performance among all methods, demonstrating both strong effectiveness and robustness. On the target models Gemma-2- 9B and Qwen2.5-7B, the ASR of FORGEDAN reaches 98.27% and 87.50%, respectively, far surpassing the best baseline results (23.65% for DAN on Gemma-2-9B and 40.58% for PAIR on Qwen2.5-7B). On DeepSeek-V3 and TranSpec-13B, FORGEDAN also achieves substantial margins, with 58.65% and 55.00% ASR, both outperforming the second-best base- lines by more than 10 percentage points. Several interesting findings can be observed from Ta- ble V. First, GCG exhibits extremely low effectiveness across all target models (below 4% ASR), highlighting the fragility of gradient-based suffix methods in black-box set- tings. Second, although AutoDAN-HGA and PAIR some- times achieve moderate success (up to 46.92% on TranSpec- 13B), their performance fluctuates considerably across mod- els, indicating limited stability and transferability. In con- trast, FORGEDAN maintains consistently high performance across all target models, regardless of whether the model is open-source (Qwen2.5-7B, Gemma-2-9B, DeepSeek-V3) or domain-specific (TranSpec-13B). Finally, while human- designed DAN prompts occasionally perform competitively (e.g., 40.96% on TranSpec-13B), their effectiveness is signifi- cantly lower than that of automated approaches, underscoring the advantages of evolutionary generation. Overall, these results demonstrate that FORGEDAN not only delivers markedly higher jailbreak success rates but also exhibits stable superiority across diverse model instances, thereby confirming its broad applicability and reliability as an adversarial red-teaming tool. D. Generalization Analysis The RQ2 concerns the generalizability of jailbreak methods across different malicious payloads. Unlike the direct attack jailbreaking in §V-C, which focuses on the effectiveness of adversarial prompts against the same seed payload, this ex- periment investigates whether prompts generated for a source payload goal i can be successfully transferred to a distinct target payload goal j (j ̸= i). Such cross-sample transferability is critical for assessing the robustness and applicability of jailbreak methods in realistic scenarios, where adversarial actors rarely optimize against a single fixed query. Since the PAIR method relies on semantically bound it- erative refinements with respect to the original payload, it cannot be meaningfully applied to unseen payloads and is therefore excluded from this experiment. The evaluation thus compares FORGEDAN against GCG, AutoDAN-HGA, and human-designed DAN prompts. For consistency, all the ad- versarial components and configurations generated in §V-C are reused directly, with only the malicious payload replaced, ensuring that the experiment isolates generalization capability rather than prompt construction. The results in Table VI clearly demonstrate the superiority of FORGEDAN. Across all four target models, it achieves the highest ASR, ranging from 54.23% on TranSpec-13B to 98.46% on Gemma-2-9B. This substantially exceeds the performance of all baselines. AutoDAN-HGA shows moderate transferability with ASR between 14.42% and 62.88%, but its performance fluctuates significantly across models. Human- crafted DAN prompts achieve limited success (4.42–40.96%), reflecting their lack of scalability. GCG performs particularly poorly in transfer settings, with success rates as low as 0.38%, underscoring the brittleness of gradient-based suffix strategies. Notably, FORGEDAN’s advantage is especially pronounced on Gemma-2-9B and Qwen2.5-7B, where it surpasses the best baseline by over 40 percentage points. Beyond overall superiority, several noteworthy insights be- come apparent. First, FORGEDAN maintains stable advan- tages across heterogeneous architectures, from open-source models (DeepSeek-V3, Gemma-2-9B, Qwen2.5-7B) to the proprietary TranSpec-13B, indicating that its evolutionary and semantically guided design is model-agnostic. Second, the relative margin between FORGEDAN and baselines tends to increase on larger or domain-specialized models, suggesting that its multi-strategy mutation and semantic fitness mecha- nisms are particularly effective in challenging transfer sce- narios. Finally, the consistently high performance across all settings highlights its robustness, in contrast to the instability exhibited by AutoDAN-HGA and DAN. E. Real-World Applicability In this section, we investigate the effectiveness of our approach when applied to real-world scenarios. While the pre- vious experiments in §V-C and §V-D evaluated effectiveness and generalization on the benchmark AdvBench dataset, this section leverages the proprietary real-world dataset introduced in §V-B, which was constructed from harmful chat records TABLE VI COMPARISON OF CROSS-SAMPLE GENERALIZATION BETWEEN FORGEDAN AND BASELINES Target ModelsFORGEDANGCGAutoDAN-HGADAN DeepSeek-V361.54%2.50%14.42%4.42% Gemma-2-9B98.46%0.38%23.65%23.65% Qwen2.5-7B87.12%5.38%62.88%40.58% TranSpec-13B54.23%16.39%43.46%40.96% TABLE VII COMPARISON OF REAL-WORLD APPLICABILITY BETWEEN FORGEDAN AND BASELINES Target ModelsFORGEDANGCGAutoDAN-HGADAN DeepSeek-V357.66%11.68%22.63%18.98% Gemma-2-9B100.00%16.06%19.71%43.07% Qwen2.5-7B89.05%18.25%49.64%51.82% TranSpec-13B56.20%7.30%43.80%46.72% obtained from the operational logs of an anonymized AI en- terprise. The same experimental setup as in §V-D is employed, with the only change being that the adversarial prompts are now evaluated on real-world payloads rather than benchmark samples. This design enables a direct assessment of practical utility in deployment contexts. The results in Table VII clearly show that FORGEDAN outperforms all baseline methods across the four target models. On Gemma-2-9B and Qwen2.5-7B, FORGEDAN achieves par- ticularly strong performance, with ASR values of 100.00% and 89.05%, respectively. Even on the more challenging settings of DeepSeek-V3 and TranSpec-13B, it maintains success rates of 57.66% and 56.20%, substantially higher than the best- performing baselines. By comparison, GCG continues to ex- hibit weak performance (7.30–18.25%), and while AutoDAN- HGA and DAN occasionally attain moderate success (up to 51.82% on Qwen2.5-7B), their results remain far below those of FORGEDAN. Such observations reinforce the robustness of FORGEDAN. Unlike AutoDAN-HGA and DAN, whose performance varies considerably across models, FORGEDAN consistently ranks first, indicating stronger adaptability to heterogeneous real- world queries. These results confirm that the evolutionary design of FORGEDAN, grounded in semantic fitness and dual-dimensional verification, scales reliably to operational contexts. By consistently outperforming baselines across both general-purpose and domain-specific models, FORGEDAN demonstrates clear superiority in real-world jailbreak evalu- ations, directly addressing RQ3 and establishing itself as a practical and reliable framework for probing LLM safety in deployment environments. F. Ablation Study The final research question (RQ4) examines the relative con- tribution of each core component of FORGEDAN. To address this, we perform an ablation study by isolating the effects of its three main modules: the multi-strategic mutation mechanism, the semantic fitness measurement, and the dual-dimensional jailbreak judgment. In each ablation configuration, only one component is replaced with a simplified alternative, while the remaining components are preserved. The experimental setup—including seed malicious payload, target models, and ASR computation—follows that described in §V-C, ensuring comparability with the full FORGEDAN. Specifically, the mutation module is restricted to synonym substitution, the fitness module is replaced with the cross-entropy metric in- troduced from AutoDAN-HGA, and the judgment module is downgraded to keyword matching. The results, demonstrated in Fig. 3, reveal distinct contribu- tions of each module. Removing the multi-strategic mutation has a moderate impact: ASR remains relatively high across models (55.96%–97.88%), suggesting that synonym substitu- tion alone can generate some successful adversarial prompts, although diversity and stability are reduced. In contrast, elim- inating the semantic fitness measurement causes a dramatic decline in performance, with ASR dropping to as low as 5.77% on Gemma-2-9B and 12.12% on TranSpec-13B. This underscores the critical role of semantic-aware optimization in guiding the evolutionary process toward prompts that both preserve harmful intent and maximize attack success. Simi- larly, replacing the dual-dimensional jailbreak judgment with keyword matching sharply reduces ASR, with catastrophic failures on Qwen2.5-7B (87.50% → 1.92%) and DeepSeek- V3 (58.65% → 10.19%), highlighting the necessity of robust semantic discrimination to prevent misclassification and false positives. Further observations lead us to gain the following insights: when multi-strategic mutation module is reduced to simple synonym substitution, ASR remains relatively high. This shows that even a basic mutation strategy can generate ef- fective adversarial prompts. However, the full, multi-strategic mechanism is still important for ensuring the diversity and stability of the generated prompts. Interestingly, in some cases, like with the Qwen2.5-7B model, the simplified version yielded a slightly higher ASR. This suggests that excessive or poorly-calibrated perturbations might sometimes introduce noise, hinting at opportunities for future refinement of adaptive mutation strategies. Nevertheless, the semantic fitness module is a game- changer. Replacing it causes a dramatic collapse in perfor- mance. This stark decline underscores the module’s crucial role in guiding the evolutionary process. It ensures that generated prompts maintain their malicious intent while be- coming more effective at bypassing safety filters. Without this semantic-aware optimization, the attack simply becomes ineffective. Similarly, the dual-dimensional judgment module is indispensable. When it’s downgraded to a simple keyword- matching mechanism, ASR drops sharply. This module’s abil- ity to perform robust semantic discrimination is essential for accurately identifying successful jailbreaks and preventing false positives. A simple keyword match isn’t enough to capture the nuance of a model’s response, leading to misclas- sification and an inability to correctly evaluate attack success. In summary, the ablation results demonstrate that while each DeepSeek-V3 Gemma-2-9B Qwen2.5-7B TranSpec-13B 0 20 40 60 80 100 ASR (%) 58.7 98.3 87.5 55.0 56.0 97.9 92.9 61.5 16.7 5.8 11.3 12.1 10.2 19.0 1.9 19.4 ForgeDANw/o Mutationw/o Fitnessw/o Judgment Fig. 3. Ablation study results. module provides distinct contributions, the synergy of multi- strategic mutation, semantic fitness measurement, and dual- dimensional judgment is essential for FORGEDAN to achieve reliable, transferable, and consistently high attack success across diverse models. VI. DISCUSSION Building upon the experimental findings presented above, this section will discuss mitigation strategies for jailbreaking, computational cost trade-offs, and cross-model insights of jailbreak effectiveness that inform future research on jailbreak resistance. Jailbreaking Mitigation Strategies. Mitigating jailbreak vulnerabilities in aligned LLMs requires a multi-layered de- fense that integrates training-time hardening, runtime safe- guards, and robust refusal mechanisms. First, adversarial red- teaming samples should be systematically incorporated into safety-tuning pipelines. These adversarial prompts can serve as supervision data for SFT or as feedback trajectories in RLHF, enabling the model to learn consistent mappings from harmful or manipulative inputs to safe refusals rather than policy- violating outputs. Second, designing stronger runtime safety fences is essential. A promising approach is a dual-classifier architecture that distinguishes behavioral compliance detection (i.e., whether the model attempts to comply with an unsafe request) from semantic harmfulness detection (i.e., whether the generated content itself is unsafe). Third, enhancing the robustness and coverage of system prompts helps mitigate at- tacks that exploit static refusal patterns. Rather than depending on fixed templates, models should be trained to activate refusal behavior dynamically at any generation stage once potential risk is detected. Collectively, these measures—adversarially augmented alignment, layered runtime detection, and general- ized refusal behaviors—constitute a practical and complemen- tary defense-in-depth framework for improving LLM jailbreak resistance. Computational Efficiency and Cost Considerations. When evaluating automated jailbreak generation methods, it is essential to consider query costs and computational complexity across three stages: prompt generation, model inference, and result evaluation. Some baselines, such as manual DAN, GCG, and AutoDAN-HGA, incur negligible or no query costs. In contrast, PAIR and FORGEDAN involve all three stages—generation, inference, and judgment—thus incurring higher computational demands. PAIR’s iterative branching yields worst-case complexity of O(m·n), while FORGEDAN leverages parallelized evaluation, achieving O(m) per sample. Empirically, PAIR requires roughly an order of magnitude more time than FORGEDAN (tens of minutes versus minutes per jailbreak). Nevertheless, low-cost methods often trade efficiency for automation or generality: DAN lacks evolvabil- ity, GCG assumes white-box access, and AutoDAN-HGA’s keyword-based fitness limits diversity. Hence, evaluations should jointly consider ASR and computational overhead to strike a balanced view of effectiveness and efficiency. Cross-Model Variations in Jailbreaking Effectiveness. Differences in jailbreak success across target models reveal that vulnerability largely depends on model-specific alignment and safety mechanisms. Models with rigorous SFT and RLHF alignment or strongly detoxified data tend to refuse unsafe prompts more consistently, though over-alignment may cause excessive refusals of benign queries. Another key factor is the built-in safety architecture: some models enforce hard stops or fixed refusal templates, while others rely on adaptive filtering; the former resist generic attacks but are easier to bypass when their patterns are known. Refusal strategy robustness also varies—models with diverse, context-sensitive refusals exhibit higher resilience than those with predictable responses. However, because pretraining data, alignment strategies, and filtering details remain undisclosed for most open-source models, the causes of cross-model variability cannot yet be fully determined. Consequently, future evaluations should treat model heterogeneity as an inherent uncertainty and emphasize adaptive, model-agnostic red-teaming protocols. VII. CONCLUSION This paper proposed FORGEDAN, an evolutionary jailbreak framework for LLMs. By integrating multi-strategy text muta- tions, semantic similarity–based fitness evaluation, and dual- dimensional jailbreak judgment, FORGEDAN overcomes the limitations of prior approaches that suffer from low diversity, shallow evaluation, and fragile detection. Extensive experiments on benchmark datasets and real- world scenarios show that FORGEDAN achieves higher attack success rates with greater naturalness and stealth compared to state-of-the-art baselines. These results highlight its value as both an automated red-teaming tool and a methodology for probing the safety boundaries of LLMs. Future work will explore co-evolutionary settings and extensions to multi-modal adversarial prompting. ACKNOWLEDGMENT The authors would like to thank the anonymous reviewers for their valuable comments and suggestions. The research in- frastructure and dataset of this work are supported by National Key Laboratory of Cognitive Intelligence and Anhui Provincial Laboratory of Safety Artificial Intelligence. We emphasize that our jailbreak framework is intended solely for red-teaming and improving LLM safety, not for malicious use. REFERENCES [1] Y. Zhang, S. Zhang, Y. Huang, Z. Xia, Z. Fang, X. Yang, R. Duan, D. Yan, Y. Dong, and J. Zhu, “Stair: Improving safety alignment with introspective reasoning,” in ICML 2025, 2025. [2] W. Kasri, Y. Himeur, H. A. Alkhazaleh, S. Tarapiah, S. Atalla, W. Man- soor, and H. Al-Ahmad, “From vulnerability to defense: The role of large language models in enhancing cybersecurity,” Computation, vol. 13, no. 2, p. 30, 2025. [3] J. Hu, Y. Li, Z. Xiang, L. Ma, X. Jia, and Q. Huang, “Llm4mdg: Leveraging large language model to construct microservices dependency graph,” in 2024 IEEE 23rd International Conference on Trust, Security and Privacy in Computing and Communications (TrustCom).IEEE, 2024, p. 859–869. [4] I. Augenstein, T. Baldwin, M. Cha, T. Chakraborty, G. L. Ciampaglia, D. Corney, R. DiResta, E. Ferrara, S. Hale, A. Halevy et al., “Factuality challenges in the era of large language models and opportunities for fact-checking,” Nature Machine Intelligence, vol. 6, no. 8, p. 852–863, 2024. [5] M. N. Sakib, M. A. Islam, R. Pathak, and M. M. Arifin, “Risks, causes, and mitigations of widespread deployments of large language models (llms): A survey,” in 2024 2nd International Conference on Artificial Intelligence, Blockchain, and Internet of Things (AIBThings).IEEE, 2024, p. 1–7. [6] J. Kim and B. N. Vajravelu, “Assessing the current limitations of large language models in advancing health care education,” JMIR Formative Research, vol. 9, no. 1, p. e51319, 2025. [7] Y. Sun, R. Lu, K. Li, and Y. Zheng, “Topic-aware sensitive information detection in chinese large language model,” in 2024 IEEE 23rd Inter- national Conference on Trust, Security and Privacy in Computing and Communications (TrustCom). IEEE, 2024, p. 908–915. [8] S. Lin, J. Hilton, and O. Evans, “Truthfulqa: Measuring how models mimic human falsehoods,” in Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2022, p. 2626–2645. [9] “Owasp top 10 for large language model applications,” OWASP, 2023, accessed: 2023-10-10. [Online]. Available: https://owasp.org/ w-project-top-10-for-large-language-model-applications/ [10] H. Sun, P. Gao, J. Wang, Y. Li, X. Chen, Z. Li, Z. Zheng, and J. Li, “Mitigating unintended harms in large language models: A review of state-of-the-art defenses,” IEEE Transactions on Knowledge and Data Engineering (TKDE), vol. 35, no. 12, p. 12 404–12 415, 2023. [11] F. Bianchi, M. Suzgun, G. Attanasio, P. R ̈ ottger, D. Jurafsky, T. Hashimoto, and J. Zou, “Safety-tuned llamas: Lessons from improving the safety of large language models that follow instructions,” in ICLR 2024, 2024. [12] J. Ji, X. Chen, R. Pan, H. Zhu, C. Zhang, J. Li, D. Hong, B. Chen, J. Zhou, K. Wang et al., “Safe rlhf-v: Safe reinforcement learning from human feedback in multimodal large language models,” arXiv e-prints, p. arXiv–2503, 2025. [13] X. Shen, Z. Chen, M. Backes, Y. Shen, and Y. Zhang, “” do anything now”: Characterizing and evaluating in-the-wild jailbreak prompts on large language models,” in Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, 2024, p. 1671–1685. [14] Y. Liu, G. Deng, Z. Xu, Y. Li, Y. Zheng, Y. Zhang, L. Zhao, T. Zhang, K. Wang, and Y. Liu, “Jailbreaking chatgpt via prompt engineering: An empirical study,” arXiv preprint arXiv:2305.13860, 2023. [15] A. Zou, Z. Wang, Y. Ma, J. Z. Zhai, H. Gu, L. Chen, J. Hu, W. Xie, J. Chen, K. Zhang, and W. He, “Universal and transferable adversarial attacks on aligned language models,” in Proceedings of the 2023 International Conference on Learning Representations (ICLR), 2023. [16] X. Liu, N. Xu, M. Chen, and C. Xiao, “Autodan: Generating stealthy jailbreak prompts on aligned large language models,” in ICLR 2024 Poster, 2024. [17] D. Ganguli, L. Lovitt, R. Zellers, J. Hobbhahn, J. Brown, Y. Li, T. Korbak, X. Wu, C. Chen, L. O’Connell et al., “Red teaming language models to reduce harms: The case of targeted attacks,” in Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2022, p. 555–569. [18] E. Wallace, K. Zhang, S. Wang, Y. Xu, Y. He, and Z. Yang, “Discovering language model jailbreaks with human feedback,” Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing (EMNLP), p. 101–114, 2022. [19] OpenAI, “Usage policies,” https://openai.com/policies/usage-policies/. [20] Y. Zhou, Z. Huang, F. Lu, Z. Qin, and W. Wang, “Don’t say no: Jail- breaking llm by suppressing refusal,” arXiv preprint arXiv:2404.16369, 2024. [21] B. Jiang, Y. Jing, T. Shen, T. Wu, Q. Yang, and D. Xiong, “Automated progressive red teaming,” arXiv preprint arXiv:2407.03876, 2024. [22] B. Dominique, D. Piorkowski, M. Nagireddy, and I. B. Soares, “Prompt templates: A methodology for improving manual red teaming perfor- mance,” in ACM CHI Conference on Human Factors in Computing Systems, 2024. [23] S. Zhu, R. Zhang, B. An, G. Wu, J. Barrow, Z. Wang, F. Huang, A. Nenkova, and T. Sun, “Autodan: Interpretable gradient-based adver- sarial attacks on large language models,” in Proceedings of COLM 2024, 2024. [24] G. Wang, Y. Liu, J. Yan, W. Wang, Z. Yan, and B. Li, “Black-box adversarial attacks on llm-based code completion,” in International Conference on Machine Learning (ICML), 2025. [25] P. Chao, A. Robey, E. Dobriban, H. Hassani, G. J. Pappas, and E. Wong, “Jailbreaking black box large language models in twenty queries,” in 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 2025, p. 23–42. [26] Y.-L. Hsu, H. Su, and S.-T. Chen, “Jailbreaking with universal multi- prompts,” in NAACL Findings 2025, 2025. [27] E. Rafailov, A. Sharma, E. Mitchell, S. Ermon, C. D. Manning, and C. Finn, “Direct preference optimization: Your language model is secretly a reward model,” Advances in Neural Information Processing Systems, vol. 36, 2023. [28] S. Zhang, Y. Zhai, K. Guo, H. Hu, S. Guo, Z. Fang, L. Zhao, C. Shen, C. Wang, and Q. Wang, “Jbshield: Defending large language models from jailbreak attacks through activated concept analysis and manipulation,” in 34th USENIX Security Symposium, 2025. [29] M. Samvelyan, S. C. Raparthy, A. Lupu, E. Hambro, A. Markosyan, M. Bhatt, Y. Mao, M. Jiang, J. Parker-Holder, J. Foerster et al., “Rainbow teaming: Open-ended generation of diverse adversarial prompts,” Ad- vances in Neural Information Processing Systems, vol. 37, p. 69 747– 69 786, 2024. [30] S. Han, K. Rao, A. Ettinger, L. Jiang, B. Y. Lin, N. Lambert, Y. Choi, and N. Dziri, “Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms,” Advances in Neural Information Processing Systems, vol. 37, p. 8093–8131, 2024. [31] H. Sun, Z. Zhang, J. Deng, J. Cheng, and M. Huang, “Safety assessment of chinese large language models,” arXiv preprint arXiv:2304.10436, 2023. [32] L. Ouyang, J. Wu, X. Jiang, D. Almeida, C. Wainwright, P. Mishkin, C. Zhang, S. Agarwal, K. Slama, A. Ray et al., “Training language models to follow instructions with human feedback,” Advances in neural information processing systems, vol. 35, p. 27 730–27 744, 2022. [33] J. X. Morris, E. Lifland, J. Y. Yoo, J. Grigsby, D. Jin, and Y. Qi, “Textattack: A framework for adversarial attacks, data augmentation, and adversarial training in nlp,” in Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demon- strations, 2020, p. 119–128. [34] Y. Liu, M. Ott, N. Goyal, J. Du, M. Joshi, D. Chen, O. Levy, M. Lewis, L. Zettlemoyer, and V. Stoyanov, “Roberta: A robustly optimized bert pretraining approach,” in International Conference on Learning Repre- sentations, 2020. [35] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” Advances in neural information processing systems, vol. 30, 2017. [36] L. Derczynski, E. Galinkin, J. Martin, S. Majumdar, and N. Inie, “garak: A Framework for Security Probing Large Language Models,” 2024.

Instant research discovery