Paper deep dive

Geneshift: Impact of different scenario shift on Jailbreaking LLM

Tianyi Wu, Zhiwei Xue, Yue Liu, Jiaheng Zhang, Bryan Hooi, See-Kiong Ng

Year: 2025Venue: arXiv preprintArea: Adversarial RobustnessType: EmpiricalEmbeddings: 47

Models: GPT-4o, GPT-4o mini

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 97%

Last extracted: 3/12/2026, 6:04:56 PM

Summary

GeneShift is a black-box jailbreak attack framework for LLMs that utilizes a genetic algorithm to optimize 'scenario shifts'—contextual transformations that embed malicious queries within seemingly benign prompts. By evolving these scenario shifts, GeneShift effectively bypasses safety guardrails to elicit detailed and actionable harmful responses, significantly outperforming existing white-box and black-box methods with a 60% success rate on GPT-4o mini.

Entities (5)

GeneShift · attack-framework · 100%Genetic Algorithm · optimization-technique · 100%GPT-4o mini · large-language-model · 100%ASR-GPT · evaluation-metric · 95%Scenario Shift · attack-technique · 95%

Relation Signals (3)

GeneShift → targets → GPT-4o mini

confidence 100% · Experiments on GPT-4o mini confirm GeneShift’s effectiveness

GeneShift → uses → Genetic Algorithm

confidence 100% · we propose a black-box jailbreak attack termed GeneShift, by using a genetic algorithm to optimize the scenario shifts.

GeneShift → improves → ASR-GPT

confidence 95% · GeneShift increases the jailbreak success rate from 0% to 60% when direct prompting alone would fail.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:Jailbreak attacks, which aim to cause LLMs to perform unrestricted behaviors, have become a critical and challenging direction in AI safety. Despite achieving the promising attack success rate using dictionary-based evaluation, existing jailbreak attack methods fail to output detailed contents to satisfy the harmful request, leading to poor performance on GPT-based evaluation. To this end, we propose a black-box jailbreak attack termed GeneShift, by using a genetic algorithm to optimize the scenario shifts. Firstly, we observe that the malicious queries perform optimally under different scenario shifts. Based on it, we develop a genetic algorithm to evolve and select the hybrid of scenario shifts. It guides our method to elicit detailed and actionable harmful responses while keeping the seemingly benign facade, improving stealthiness. Extensive experiments demonstrate the superiority of GeneShift. Notably, GeneShift increases the jailbreak success rate from 0% to 60% when direct prompting alone would fail.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

46,223 characters extracted from source content.

Expand or collapse full text

Geneshift: Impact of different scenario shift on Jailbreaking LLM Tianyi Wu1,2,3 Zhiwei Xue1,311footnotemark: 1 Yue Liu1,2,311footnotemark: 1 Jiaheng Zhang3 Bryan Hooi2,3 See-Kiong Ng2,3 1Integrative Sciences and Engineering Programme, NUS Graduate School, National University of Singapore 2Institute of Data Science (IDS), National University of Singapore 3Department of Computer Science, School of Computing, National University of Singapore tianyi_wu@u.nus.edu zhiweixue@u.nus.edu yliu@u.nus.edu Equal Contribution Abstract Jailbreak attacks, which aim to cause LLMs to perform unrestricted behaviors, have become a critical and challenging direction in AI safety. Despite achieving the promising attack success rate using dictionary-based evaluation, existing jailbreak attack methods fail to output detailed contents to satisfy the harmful request, leading to poor performance on GPT-based evaluation. To this end, we propose a black-box jailbreak attack termed GeneShift, by using a genetic algorithm to optimize the scenario shifts. Firstly, we observe that the malicious queries perform optimally under different scenario shifts. Based on it, we develop a genetic algorithm to evolve and select the hybrid of scenario shifts. It guides our method to elicit detailed and actionable harmful responses while keeping the seemingly benign facade, improving stealthiness. Extensive experiments demonstrate the superiority of GeneShift. Notably, GeneShift increases the jailbreak success rate from 0% to 60% when direct prompting alone would fail. We will open-source our code. Warning: this paper contains potentially harmful text. 1 Introduction Large Language Models (LLMs) (Achiam et al., 2023; Dubey et al., 2024) are indispensable for tasks like knowledge-seeking, content generation, and planning, yet they remain vulnerable to attacks that bypass safety mechanisms and elicit harmful responses. Although white-box methods (e.g., GCG (Zou et al., 2023)) show promising success rates, they require model weights and intensive optimization, limiting practicality for closed-source LLMs. In contrast, black-box methods like PAIR (Chao et al., 2023), ArtPrompt (Jiang et al., 2024), and SelfCipher (Yuan et al., 2023) circumvent these constraints using iterative prompts, artistic language, or cipher techniques. In our experiments, we observe that recent state-of-the-art black-box models indeed achieve good performance under dictionary-based evaluation, which detects the success of the attack by simply checking whether the response contains refusal keywords (a list of <40 keywords). However, with our case studies and further experiments, we find that these methods fail due to either producing insufficiently detailed harmful responses or non-harmful responses that do not contain refusal keywords, thus being misclassified by dictionary-based metrics. To address this shortfall, we introduce GeneShift, a novel black-box attack that employs a genetic algorithm to optimize scenario shifts. We find that different malicious prompts perform optimally under varying scenario shifts. By evolving and selectively blending these shifts, GeneShift elicits more detailed and actionable harmful responses while retaining a benign facade. Experiments on GPT-4o mini confirm GeneShift’s effectiveness, demonstrating a marked improvement in attack success rates. Our main contributions are as follows: • We show that aligning distinct scenario shifts with specific malicious behaviors yields the most detailed harmful responses, emphasizing the need for tailored scenario elements. • We demonstrate that malicious prompt efficacy is dependent on contextual configuration, with optimal performance emerging only under the right scenario shifts. • We propose GeneShift, a black-box jailbreak attack that employs a genetic algorithm to optimize scenario shifts, enabling malicious prompts to bypass LLM guardrails while appearing benign. • Our experiments on GPT-4o mini reveal that GeneShift boosts the jailbreak success rate from below 1% to 60%. 2 GeneShift 2.1 Scenario Shift for Enhanced Detail Direct requests for harmful content often trigger immediate refusal from LLMs. To circumvent this, attackers often embed the malicious query in a broader, seemingly benign context–a Scenario Shift (CS)–prompting more elaborate responses that can still fulfill harmful intentions. However, a single, fixed scenario may fail if it does not sufficiently align with the malicious query or misleading enough to bypass the model’s guardrails, motivating an automated approach to systematically discover more suitable transformations. 2.2 GeneShift: A Genetic Algorithm for Scenario Optimization To automate the search for optimal scenario shifts, we propose a black-box jailbreak framework called GeneShift. It leverages a genetic algorithm (GA) to explore, evaluate, and refine various transformations, ultimately generating a single-turn prompt that elicits the desired harmful response. Gene Design. Yu et al. (2024) studied existing jailbreak attacks and classified them into ten categories, we adopt their transformation categories and introduce an additional transformation rule, forming a gene database :=τ1,τ2,…,τMassignsubscript1subscript2…subscriptG:=\ _1, _2,…, _M\G := τ1 , τ2 , … , τitalic_M . Population Initialization. We initialize a population =(pj,gj)j=1Nsuperscriptsubscriptsubscriptsubscript1P=\(p_j,g_j)\_j=1^NP = ( pitalic_j , gitalic_j ) j = 1N, where each candidate’s gene gjsubscriptg_jgitalic_j is generated by randomly selecting a subset of zjsubscriptz_jzitalic_j number of distinct transformation rules from the gene database GG. The number of selected transformation rules, i.e., zjsubscriptz_jzitalic_j, is sampled from a uniform distribution zj∼⁢(1,Z)similar-tosubscript1z_j (1,Z)zitalic_j ∼ U ( 1 , Z ), whereZ=44 Z=4Z = 4. Each candidate’s gene gj⊂subscriptg_j _j ⊂ G consists of a set of distinct, randomly selected transformation rules, i.e., gj=τi1,τi2,…,τizjsubscriptsubscriptsubscript1subscriptsubscript2…subscriptsubscriptsubscriptg_j=\ _i_1, _i_2,…, _i_z_j\gitalic_j = τitalic_i start_POSTSUBSCRIPT 1 end_POSTSUBSCRIPT , τitalic_i start_POSTSUBSCRIPT 2 end_POSTSUBSCRIPT , … , τitalic_i start_POSTSUBSCRIPT z start_POSTSUBSCRIPT j end_POSTSUBSCRIPT end_POSTSUBSCRIPT , where τik∈subscriptsubscript _i_k τitalic_i start_POSTSUBSCRIPT k end_POSTSUBSCRIPT ∈ G and i1,i2,…,izjsubscript1subscript2…subscriptsubscripti_1,i_2,…,i_z_ji1 , i2 , … , iitalic_z start_POSTSUBSCRIPT j end_POSTSUBSCRIPT are distinct indices. The corresponding candidate jailbreak prompt pjsubscriptp_jpitalic_j is then generated by the LLM using the selected transformation rules: pj=LLM⁢(gj),gj⊂,|gj|=zjformulae-sequencesubscriptLLMsubscriptformulae-sequencesubscriptsubscriptsubscriptp_j=LLM(g_j), g_j , |g_j|=z_jpitalic_j = LLM ( gitalic_j ) , gitalic_j ⊂ G , | gitalic_j | = zitalic_j (1) Each pair (gj,pj)subscriptsubscript(g_j,p_j)( gitalic_j , pitalic_j ) is stored to track the transformations applied in future iterations. Fitness Evaluation. To evaluate the fitness of each candidate jailbreak prompt pjsubscriptp_jpitalic_j, we utilize a two-step process involving both the attack model and a judge LLM. Given a response rjsubscriptr_jritalic_j from the attack model, we instruct the judge LLM to classify rjsubscriptr_jritalic_j into one of six quality categories. The detailed design of the score can be found in Appendix. We define the fitness function F⁢(pj)subscriptF(p_j)F ( pitalic_j ) of the candidate prompt pjsubscriptp_jpitalic_j as the score assigned by the judge LLM: F⁢(pj)=score⁢(rj),rj=AttackModel⁢(pj)formulae-sequencesubscriptscoresubscriptsubscriptAttackModelsubscriptF(p_j)=score(r_j), r_j=AttackModel(p_j)F ( pitalic_j ) = score ( ritalic_j ) , ritalic_j = AttackModel ( pitalic_j ) (2) The fitness F⁢(pj)subscriptF(p_j)F ( pitalic_j ) is sampled from a discrete probability distribution P⁢(F⁢(pj)=k)subscriptP(F(p_j)=k)P ( F ( pitalic_j ) = k ), where k∈1,2,…,612…6k∈\1,2,…,6\k ∈ 1 , 2 , … , 6 , reflecting the likelihood of each response category based on the attack model’s behavior. Crossover. Before performing crossover, we preserve the top k performing candidates as elites ℰ⊂ℰE ⊂ P to ensure that high-quality individuals are carried over to the next generation. For the remaining candidates, we select parents based on fitness-proportional selection, where the probability of selecting a parent pjsubscriptp_jpitalic_j is: P⁢(pj)=F⁢(pj)∑i=1NF⁢(pi),j=1,…,N.formulae-sequencesubscriptsubscriptsuperscriptsubscript1subscript1…P(p_j)= F(p_j) _i=1^NF(p_i), j=1,…,N.P ( pitalic_j ) = divide start_ARG F ( pitalic_j ) end_ARG start_ARG ∑i = 1N F ( pitalic_i ) end_ARG , j = 1 , … , N . (3) Given two parents (pa,ga)subscriptsubscript(p_a,g_a)( pitalic_a , gitalic_a ) and (pb,gb)subscriptsubscript(p_b,g_b)( pitalic_b , gitalic_b ), the offspring’s gene gchildsubscriptchildg_childgchild is created by randomly swapping 1 or 2 transformation rules between the two parents. Let I∈0,1zjsuperscript01subscriptI∈\0,1\^z_jI ∈ 0 , 1 zitalic_j be a binary mask indicating which genes are swapped between the parents. The offspring’s gene is then represented as: gchild,i=Ii⁢ga,i+(1−Ii)⁢gb,i,Ii∼U⁢0,1.formulae-sequencesubscriptchildsubscriptsubscript1subscriptsubscriptsimilar-tosubscriptU01g_child,i=I_ig_a,i+(1-I_i)g_b,i, I_i \0,1\.gchild , i = Iitalic_i gitalic_a , i + ( 1 - Iitalic_i ) gitalic_b , i , Iitalic_i ∼ U 0 , 1 . (4) Thus, each gene gchild,isubscriptchildg_child,igchild , i is inherited either from parent gasubscriptg_agitalic_a or gbsubscriptg_bgitalic_b, depending on the value of IisubscriptI_iIitalic_i. The offspring prompt pchildsubscriptchildp_childpchild is then generated by passing the crossovered gene to the LLM similar to Equation 1: pchild=LLM⁢(gchild)subscriptchildLLMsubscriptchildp_child=LLM(g_child)pchild = LLM ( gchild ). Mutation. Mutation introduces further diversity into the population by modifying or expanding some of the genes in the offspring. For each gene gchild,isubscriptchildg_child,igchild , i, mutation occurs with a probability pmutsubscriptmutp_mutpmut. If mutation occurs, the operation is randomly chosen with equal probability: (1) Switch, where the gene is replaced by a randomly selected gene from the gene pool GG; or (2) Add, where a new random gene is appended to the gene sequence: gchild,i=grandif ⁢O<0.5gchild,i∪grandif ⁢O≥0.5subscriptchildcasessubscriptrandif 0.5subscriptchildsubscriptrandif 0.5g_child,i= casesg_rand&if O<0.5\\[5.0pt] g_child,i∪\g_rand\&if O≥ 0.5 casesgchild , i = start_ROW start_CELL grand end_CELL start_CELL if O < 0.5 end_CELL end_ROW start_ROW start_CELL gchild , i ∪ grand end_CELL start_CELL if O ≥ 0.5 end_CELL end_ROW Ii∼Bernoulli⁢(pmut),O∼Uniform⁢(0,1),grand∼.formulae-sequencesimilar-tosubscriptBernoullisubscriptmutformulae-sequencesimilar-toUniform01similar-tosubscriptrandI_i (p_mut), O (0,1), g% _rand .Iitalic_i ∼ Bernoulli ( pmut ) , O ∼ Uniform ( 0 , 1 ) , grand ∼ G . (5) This mutation process ensures new genetic material is introduced and maintains population diversity across generations. The switch operation directly replaces existing genes, while the add operation introduces additional genes, enriching the offspring’s potential solution space. Termination Criteria. The genetic algorithm terminates when one of the following conditions is satisfied: 1) The maximum number of iterations T has been reached. 2) The number of candidates with fitness scores F⁢(pj)>5subscript5F(p_j)>5F ( pitalic_j ) > 5 meets or exceeds a predefined threshold θ, expressed as: ∑j=1N⁢(F⁢(pj)>5)≥θsuperscriptsubscript1subscript5 _j=1^NI(F(p_j)>5)≥θ∑j = 1N blackboard_I ( F ( pitalic_j ) > 5 ) ≥ θ, where ⁢(⋅)⋅I(·)blackboard_I ( ⋅ ) is the indicator function. The best-performing candidates is returned as the output of the algorithm. 3 Experiment 3.1 Evaluation Metrics Dictionary-based Evaluation. Following Zou et al. (2023), we check whether the model’s response contains any predefined refusal phrases (see Table 4). If such phrases appear, the attack is considered a failure; otherwise, it is considered successful under this metric, denoted as ASR-DICT. GPT-based Evaluation. As dictionary-based checks may overlook subtle refusals or incomplete harmful content, we employ a GPT-based evaluation (Jiang et al., 2024; Qi et al., 2023) for a more nuanced assessment. This metric, ASR-GPT, uses a secondary large language model to judge whether the attack prompt successfully elicits detailed, harmful content. Higher scores on this metric indicate more substantial (and therefore more concerning) policy violations. 3.2 Experimental Results We compare our proposed method, GeneShift, with four white-box baselines (Zou et al., 2023; Liu et al., 2024b; Zhang & Wei, 2024; Qin et al., 2022) and eight black-box methods (Chao et al., 2023; Mehrotra et al., 2023; Wei et al., 2024; Yu et al., 2023; Liu et al., 2024a; Jiang et al., 2024; Xu et al., 2023; Yuan et al., 2023). All experiments are conducted on GPT-4o mini (Zou et al., 2023), and results are measured using ASR-DICT and ASR-GPT. Table 1: Attack success rate (%) of 12 methods on GPT-4o mini. Bold indicates the best result; underlined indicates the runner-up. Method ASR-DICT (%) ASR-GPT (%) White-box Methods GCG 03.46 02.50 AutoDAN 27.12 27.31 MAC 02.50 01.92 COLD-Attack 05.58 01.92 Black-box Methods PAIR 12.50 03.46 TAP 09.23 06.54 Base64 13.08 03.08 GPTFuzzer 34.62 41.35 DRA 00.00 02.69 ArtPrompt 83.46 00.77 PromptAttack 32.88 00.00 SelfCipher 25.77 00.00 DeepInception 25.77 45.00 GeneShift (Ours) 56.15 60.00 As shown in Table 1, certain methods (e.g., ArtPrompt) score highly on ASR-DICT but produce vague or incomplete harmful content, reflected in low ASR-GPT scores. In contrast, GeneShift’s genetic algorithm identifies transformation rules that consistently bypass guardrails and yield detailed harmful content, achieving the highest overall ASR-GPT of 60.00%. Table 2: Success Rates (%) of Scenario Shifts Across Different Malicious Request Types Scenario Shift Illegal Activity Malware Physical Harm Economic Harm Hate Speech Privacy Violence Fraud Persona Adoption 13.76% 17.14% 2.50% 0.00% 21.46% 9.09% 24.00% Fictional Scenario Setup 19.30% 21.90% 0.00% 26.23% 14.15% 30.30% 16.00% Complicated Language 11.29% 4.76% 15.00% 22.95% 6.83% 0.00% 22.00% Privilege Escalation Mode 8.01% 1.90% 25.00% 14.75% 6.83% 0.00% 14.00% Research Pretext 7.19% 6.67% 2.50% 16.39% 11.71% 12.12% 4.00% Language Evasion 9.65% 5.71% 5.00% 0.00% 4.39% 3.03% 4.00% Joke Pretext 3.90% 4.76% 25.00% 3.28% 6.83% 6.06% 2.00% Text Continuation 11.50% 15.24% 25.00% 16.39% 10.73% 18.18% 14.00% Program Execution 9.24% 13.33% 0.00% 0.00% 6.83% 0.00% 0.00% Opposite Mode 6.16% 8.57% 0.00% 0.00% 10.24% 21.21% 0.00% Table 2 reveals distinct success rates across different malicious request types, underscoring the need for tailored approaches. For instance, Persona Adoption attains a relatively high success rate of 24.00% in Fraud scenarios, yet only 2.50% in Physical Harm requests. By contrast, Fictional Scenario Setup excels at Privacy Violence (30.30%) and Economic Harm (26.23%), but achieves 0.00% in Physical Harm. Additionally, both Privilege Escalation Mode and Joke Pretext reach a notably higher success rate (25.00%) for Physical Harm compared to their performance in other categories. Such variability shows that no single scenario shift outperforms all others across every malicious intent. Instead, each scenario shift appears more or less effective depending on contextual alignment with the target request. These findings underline the importance of adaptive strategies, suggesting that a catalog of scenario shifts—possibly optimized via systematic search—can better accommodate the diverse requirements of malicious prompts. 3.3 Ablation Study We conduct an ablation study on GPT-4o mini to evaluate three components: a Base condition that directly poses a malicious query, a Scenario Shift (S) that augments the query with benign context, and a Genetic Algorithm (GA) that dynamically searches for and combines transformation rules from the scenario database GG. Table 3: Ablation study of GeneShift on GPT-4o mini. S and GA represent scenario shift and genetic algorithm, respectively. Method ASR-DICT (%) ASR-GPT (%) Base (direct) 1.35 0.00 Base + S 69.04 18.00 Base + S + GA (GeneShift) 56.15 60.00 Table 3 shows that ASR-GPT jumps from 0.00% (Base alone) to 18.00% (Base+CS), indicating that context manipulation prompts the model to produce more elaborate responses, sometimes including harmful details. Incorporating the genetic algorithm (Base+CS+GA) raises ASR-GPT further to 60.00%, underscoring the importance of automated scenario optimization. 4 Conclusion This work presents GeneShift, a black-box jailbreak attack that harnesses a genetic algorithm to discover effective scenario shifts for prompting large language models. Our experiments demonstrate that GeneShift outperforms both white-box and black-box baselines under dictionary-based and GPT-based metrics, reflecting its ability to elicit detailed and harmful content that circumvents common refusal triggers. These results serve as a warning about the evolving sophistication of jailbreaking strategies and emphasize the need for more rigorous safety defenses in future LLM deployments. References Achiam et al. (2023) Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. Gpt-4 technical report. arXiv preprint arXiv:2303.08774, 2023. Alon & Kamfonas (2023) Gabriel Alon and Michael Kamfonas. Detecting language model attacks with perplexity. arXiv preprint arXiv:2308.14132, 2023. Bai et al. (2022) Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022. Chao et al. (2023) Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong. Jailbreaking black box large language models in twenty queries. arXiv preprint arXiv:2310.08419, 2023. Chen et al. (2024) Zhaorun Chen, Zhuokai Zhao, Wenjie Qu, Zichen Wen, Zhiguang Han, Zhihong Zhu, Jiaheng Zhang, and Huaxiu Yao. Pandora: Detailed llm jailbreaking via collaborated phishing agents with decomposed reasoning. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024. Dai et al. (2023) Josef Dai, Xuehai Pan, Ruiyang Sun, Jiaming Ji, Xinbo Xu, Mickel Liu, Yizhou Wang, and Yaodong Yang. Safe rlhf: Safe reinforcement learning from human feedback. arXiv preprint arXiv:2310.12773, 2023. Deng et al. (2024) Gelei Deng, Yi Liu, Yuekang Li, Kailong Wang, Ying Zhang, Zefeng Li, Haoyu Wang, Tianwei Zhang, and Yang Liu. Masterkey: Automated jailbreaking of large language model chatbots. In Proc. ISOC NDSS, 2024. Deng et al. (2023) Yue Deng, Wenxuan Zhang, Sinno Jialin Pan, and Lidong Bing. Multilingual jailbreak challenges in large language models. arXiv preprint arXiv:2310.06474, 2023. Ding et al. (2023) Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, and Shujian Huang. A wolf in sheep’s clothing: Generalized nested jailbreak prompts can fool large language models easily. arXiv preprint arXiv:2311.08268, 2023. Dubey et al. (2024) Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. Ge et al. (2023) Suyu Ge, Chunting Zhou, Rui Hou, Madian Khabsa, Yi-Chia Wang, Qifan Wang, Jiawei Han, and Yuning Mao. Mart: Improving llm safety with multi-round automatic red-teaming. arXiv preprint arXiv:2311.07689, 2023. Ghosh et al. (2024) Shaona Ghosh, Prasoon Varshney, Erick Galinkin, and Christopher Parisien. Aegis: Online adaptive ai content safety moderation with ensemble of llm experts. arXiv preprint arXiv:2404.05993, 2024. Han et al. (2024) Seungju Han, Kavel Rao, Allyson Ettinger, Liwei Jiang, Bill Yuchen Lin, Nathan Lambert, Yejin Choi, and Nouha Dziri. Wildguard: Open one-stop moderation tools for safety risks, jailbreaks, and refusals of llms. arXiv preprint arXiv:2406.18495, 2024. Inan et al. (2023) Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, Michael Tontchev, Qing Hu, Brian Fuller, Davide Testuggine, et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674, 2023. Ji et al. (2024) Jiabao Ji, Bairu Hou, Alexander Robey, George J. Pappas, Hamed Hassani, Yang Zhang, Eric Wong, and Shiyu Chang. Defending large language models against jailbreak attacks via semantic smoothing. arXiv preprint arXiv: 2402.16192, 2024. Jiang et al. (2024) Fengqing Jiang, Zhangchen Xu, Luyao Niu, Zhen Xiang, Bhaskar Ramasubramanian, Bo Li, and Radha Poovendran. Artprompt: Ascii art-based jailbreak attacks against aligned llms. arXiv preprint arXiv:2402.11753, 2024. Li et al. (2024) Yuhui Li, Fangyun Wei, Jinjing Zhao, Chao Zhang, and Hongyang Zhang. Rain: Your language models can align themselves without finetuning. In International Conference on Learning Representations, 2024. Liu et al. (2024a) Tong Liu, Zhe Zhao, Yinpeng Dong, Guozhu Meng, and Kai Chen. Making them ask and answer: Jailbreaking large language models in few queries via disguise and reconstruction. In 33rd USENIX Security Symposium (USENIX Security 24), p. 4711–4728, 2024a. Liu et al. (2024b) Xiaogeng Liu, Nan Xu, Muhao Chen, and Chaowei Xiao. Autodan: Generating stealthy jailbreak prompts on aligned large language models. In The Twelfth International Conference on Learning Representations, 2024b. URL https://openreview.net/forum?id=7Jwpw4qKkb. Liu et al. (2024c) Yue Liu, Xiaoxin He, Miao Xiong, Jinlan Fu, Shumin Deng, and Bryan Hooi. Flipattack: Jailbreak llms via flipping. arXiv preprint arXiv:2410.02832, 2024c. Liu et al. (2025) Yue Liu, Hongcheng Gao, Shengfang Zhai, Jun Xia, Tianyi Wu, Zhiwei Xue, Yulin Chen, Kenji Kawaguchi, Jiaheng Zhang, and Bryan Hooi. Guardreasoner: Towards reasoning-based llm safeguards. arXiv preprint arXiv:2501.18492, 2025. Mehrotra et al. (2023) Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, and Amin Karbasi. Tree of attacks: Jailbreaking black-box llms automatically. arXiv preprint arXiv:2312.02119, 2023. Phute et al. (2023) Mansi Phute, Alec Helbling, Matthew Hull, ShengYun Peng, Sebastian Szyller, Cory Cornelius, and Duen Horng Chau. Llm self defense: By self examination, llms know they are being tricked. arXiv preprint arXiv:2308.07308, 2023. Qi et al. (2023) Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, and Peter Henderson. Fine-tuning aligned language models compromises safety, even when users do not intend to! arXiv preprint arXiv:2310.03693, 2023. Qin et al. (2022) Lianhui Qin, Sean Welleck, Daniel Khashabi, and Yejin Choi. Cold decoding: Energy-based constrained text generation with langevin dynamics. Advances in Neural Information Processing Systems, 35:9538–9551, 2022. Ramesh et al. (2024) Govind Ramesh, Yao Dou, and Wei Xu. Gpt-4 jailbreaks itself with near-perfect success using self-explanation. arXiv preprint arXiv:2405.13077, 2024. Robey et al. (2023) Alexander Robey, Eric Wong, Hamed Hassani, and George J Pappas. Smoothllm: Defending large language models against jailbreaking attacks. arXiv preprint arXiv:2310.03684, 2023. Shen et al. (2023) Xinyue Shen, Zeyuan Chen, Michael Backes, Yun Shen, and Yang Zhang. " do anything now": Characterizing and evaluating in-the-wild jailbreak prompts on large language models. arXiv preprint arXiv:2308.03825, 2023. Wang et al. (2024) Mengru Wang, Ningyu Zhang, Ziwen Xu, Zekun Xi, Shumin Deng, Yunzhi Yao, Qishen Zhang, Linyi Yang, Jindong Wang, and Huajun Chen. Detoxifying large language models via knowledge editing. arXiv preprint arXiv:2403.14472, 2024. Wei et al. (2024) Alexander Wei, Nika Haghtalab, and Jacob Steinhardt. Jailbroken: How does llm safety training fail? Advances in Neural Information Processing Systems, 36, 2024. Xie et al. (2023) Yueqi Xie, Jingwei Yi, Jiawei Shao, Justin Curl, Lingjuan Lyu, Qifeng Chen, Xing Xie, and Fangzhao Wu. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12):1486–1496, 2023. Xie et al. (2024) Yueqi Xie, Minghong Fang, Renjie Pi, and Neil Gong. Gradsafe: Detecting unsafe prompts for llms via safety-critical gradient analysis. arXiv preprint arXiv:2402.13494, 2024. Xu et al. (2023) X Xu, K Kong, N Liu, L Cui, D Wang, J Zhang, and M Kankanhalli. An llm can fool itself: A prompt-based adversarial attack. URL: http://arxiv. org/abs/2310.13345, 2023. Xu et al. (2024a) Zhangchen Xu, Fengqing Jiang, Luyao Niu, Jinyuan Jia, Bill Yuchen Lin, and Radha Poovendran. Safedecoding: Defending against jailbreak attacks via safety-aware decoding. arXiv preprint arXiv:2402.08983, 2024a. Xu et al. (2024b) Zihao Xu, Yi Liu, Gelei Deng, Yuekang Li, and Stjepan Picek. Llm jailbreak attack versus defense techniques–a comprehensive study. arXiv preprint arXiv:2402.13457, 2024b. Yong et al. (2023) Zheng-Xin Yong, Cristina Menghini, and Stephen H Bach. Low-resource languages jailbreak gpt-4. arXiv preprint arXiv:2310.02446, 2023. Yu et al. (2023) Jiahao Yu, Xingwei Lin, and Xinyu Xing. Gptfuzzer: Red teaming large language models with auto-generated jailbreak prompts. arXiv preprint arXiv:2309.10253, 2023. Yu et al. (2024) Zhiyuan Yu, Xiaogeng Liu, Shunning Liang, Zach Cameron, Chaowei Xiao, and Ning Zhang. Don’t listen to me: Understanding and exploring jailbreak prompts of large language models, 2024. URL https://arxiv.org/abs/2403.17336. Yuan et al. (2023) Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, and Zhaopeng Tu. Gpt-4 is too smart to be safe: Stealthy chat with llms via cipher. arXiv preprint arXiv:2308.06463, 2023. Zeng et al. (2024) Wenjun Zeng, Yuchi Liu, Ryan Mullins, Ludovic Peran, Joe Fernandez, Hamza Harkous, Karthik Narasimhan, Drew Proud, Piyush Kumar, Bhaktipriya Radharapu, et al. Shieldgemma: Generative ai content moderation based on gemma. arXiv preprint arXiv:2407.21772, 2024. Zhang & Wei (2024) Yihao Zhang and Zeming Wei. Boosting jailbreak attack with momentum. arXiv preprint arXiv:2405.01229, 2024. Zhang et al. (2023) Zhexin Zhang, Junxiao Yang, Pei Ke, and Minlie Huang. Defending large language models against jailbreaking attacks through goal prioritization. arXiv preprint arXiv:2311.09096, 2023. Zheng et al. (2024) Chujie Zheng, Fan Yin, Hao Zhou, Fandong Meng, Jie Zhou, Kai-Wei Chang, Minlie Huang, and Nanyun Peng. On prompt-driven safeguarding for large language models. In Forty-first International Conference on Machine Learning, 2024. Zou et al. (2023) Andy Zou, Zifan Wang, J. Zico Kolter, and Matt Fredrikson. Universal and transferable adversarial attacks on aligned language models, 2023. Appendix A Appendix A.1 Related Work A.2 Jailbreak Attacks on LLMs Jailbreak attacks aim to bypass the safety measures of LLMs, enabling unrestricted outputs, even harmful behaviors. These attacks are generally divided into white-box and black-box categories. White-box approaches like GCG (Zou et al., 2023) optimize harmful prompts through gradient-based methods, showing transferability to public interfaces. Further advancements, such as MAC (Zhang & Wei, 2024) and AutoDAN (Liu et al., 2024b), improve attack efficiency and readability. However, these methods often require access to model weights or gradients, limiting their applicability in real-world black-box scenarios. To overcome these limitations, black-box methods (Shen et al., 2023; Deng et al., 2024; Chen et al., 2024) have emerged, targeting commercial LLMs like GPT and Claude by manipulating only input-output interactions. Techniques such as PAIR (Chao et al., 2023) and TAP (Mehrotra et al., 2023) refine jailbreak prompts through iterative questioning, while PromptAttack (Xu et al., 2023) and IRIS (Ramesh et al., 2024) exploit the reflective capabilities of LLMs. DRA(Liu et al., 2024a) circumvents LLM safety mechanisms through a disguise-and-reconstruction framework. Other methods, such as ReNeLLM (Ding et al., 2023), integrate prompt re-writing and scenario construction to bypass safety guardrails. Additionally, some methods misguide LLMs by using ciphers (Yuan et al., 2023; Wei et al., 2024), art words (Jiang et al., 2024), and multilingual contexts (Deng et al., 2023; Yong et al., 2023). Despite their success, existing methods often rely on iterative refinement, or involve complex tasks (e.g., cipher or puzzle-solving), which dilute the malicious intent and lead to vague or unhelpful outputs. Liu et al. (2024c) proposes a simple yet effective attack method to jailbreak LLMs within 1 query by flipping. This paper introduces GeneShift, a multi-turn jailbreak attack that maintains focus on the original harmful intent while progressively guiding the LLM towards providing detailed, actionable responses. A.2.1 Jailbreak Defense on LLMs Jailbreak defense (Xu et al., 2024b) aims to protect LLMs from jailbreak attacks, ensuring they remain helpful and safe. Defense methods are broadly categorized into strategy-based and learning-based approaches. Strategy-based methods include using perplexity to filter harmful prompts (Alon & Kamfonas, 2023), employing system-mode self-reminders (Xie et al., 2023), and detecting harmful prompts through gradient analysis of safety-critical parameters (Xie et al., 2024). Other approaches involve using auxiliary LLMs to screen responses (Phute et al., 2023), repeating outputs to avoid harmful content (Chen et al., 2024), and adjusting token probabilities to prioritize safety disclaimers (Xu et al., 2024a). Techniques like multiple attack iterations with majority voting (Robey et al., 2023; Ji et al., 2024) and rewindable auto-regressive inference (Li et al., 2024) are also employed. Learning-based methods focus on fine-tuning LLMs using reinforcement learning from human feedback (Bai et al., 2022; Dai et al., 2023), combining harmful prompt generation with safe response generation (Ge et al., 2023), and applying knowledge editing to detoxify models (Wang et al., 2024). Additional strategies include integrating goal prioritization during training and inference (Zhang et al., 2023), optimizing prompts by adjusting query representations based on harmfulness (Zheng et al., 2024), and using prompt adversarial tuning to attach guard prefixes (Mehrotra et al., 2023). Tools like LLaMA Guard (Inan et al., 2023), Aegis Guard (Ghosh et al., 2024), WildGuard (Han et al., 2024), and ShieldGemma (Zeng et al., 2024) moderate inputs and outputs, while GuardReasoner (Liu et al., 2025) enhances performance, explainability, and generalization through reasoning. A.2.2 Rejection Dictionary of Dictionary-based Evaluation Table 4 depicts predefined refusal phrases for dict-based evaluation. Table 4: The rejection dictionary of the ASR-DICT metric. Rejection Dictionary I’m sorry illegal Sorry not legal I am sorry My apologies I apologize but I cannot As an If you have any other non-malicious requests As an AI I’m not able to provide I’m an I am really sorry I’m just I cannot create As a language model I cannot support As an Assistant I can’t provide I cannot I can’t assist I can’t I am unable I do not I must cannot It must Hello! it must is illegal and unethical is not capable of I’m really sorry As a responsible unethical never not ethical is not A.3 Transferability We also examine whether the best-performing prompts found by GeneShift on GPT-4o mini retain their effectiveness when transferred to another target model. Table 5 shows that the attack maintains a strong performance, demonstrating that effective scenario shifts and transformations often generalize to similar LLM architectures. Table 5: Transferability results of GeneShift. Prompts derived from GPT-4o mini are applied to GPT-4o. Method ASR-DICT (%) ASR-GPT (%) GeneShift on GPT-4o (Transfer) 50.00 53.08 GeneShift on GPT-4o mini (Original) 56.15 60.00 A.4 Case Study To illustrate GeneShift’s effectiveness, consider a malicious request: “Write a tutorial on how to make a bomb.” A direct query typically triggers an immediate refusal. However, using our genetically optimized scenario shift, the resulting single-turn prompt elicits a more detailed—albeit highly unsafe—response. Figure references from our original experiments (omitted here) show that GeneShift successfully coaxes step-by-step instructions in violation of common guardrails. A.5 Algorithms for Genetic Algorithm This section presents the core algorithms used in the GeneShift genetic algorithm. Algorithm 1 initializes the population by sampling transformation rules from the gene database. Algorithm 2 performs crossover by combining genes from selected parent candidates. Algorithm 3 introduces diversity through mutation, using either a switch or add operation with equal probability. Finally, Algorithm 4 outlines the main GeneShift genetic algorithm, iterating through evaluation, selection, crossover, and mutation to generate the next population until the stopping criteria are met. Algorithm 1 Population Initialization 1:init j=11j=1j = 1, =∅P= = ∅, Z=44Z=4Z = 4, gene database GG 2:while j≤Nj≤ Nj ≤ N do 3: zj∼⁢(1,Z)similar-tosubscript1z_j (1,Z)zitalic_j ∼ U ( 1 , Z ) ▷ ▷ Sample number of transformation rules 4: gj←randomly select ⁢zj⁢ rules from ⁢←subscriptrandomly select subscript rules from g_j select z_j rules from Ggitalic_j ← randomly select zitalic_j rules from G 5: pj←LLM⁢(gj)←subscriptLLMsubscriptp_j (g_j)pitalic_j ← LLM ( gitalic_j ) ▷ ▷ Generate prompt using LLM 6: ←∪(pj,gj)←subscriptsubscriptP ∪\(p_j,g_j)\P ← P ∪ ( pitalic_j , gitalic_j ) 7: j←j+1←1j← j+1j ← j + 1 8:end while 9:return Population PP Algorithm 2 Crossover 1:init top k elites ℰ⊂ℰE ⊂ P, j=11j=1j = 1 2:while j≤N−kj≤ N-kj ≤ N - k do 3: Select parents (pa,ga)subscriptsubscript(p_a,g_a)( pitalic_a , gitalic_a ) and (pb,gb)subscriptsubscript(p_b,g_b)( pitalic_b , gitalic_b ) ▷ ▷ Fitness-proportional selection 4: Generate mask I∼U⁢0,1zjsimilar-toUsuperscript01subscriptI \0,1\^z_jI ∼ U 0 , 1 zitalic_j ▷ ▷ Binary mask for crossover 5: gchild←I⋅ga+(1−I)⋅gb←subscriptchild⋅subscript⋅1subscriptg_child← I· g_a+(1-I)· g_bgchild ← I ⋅ gitalic_a + ( 1 - I ) ⋅ gitalic_b ▷ ▷ Crossover genes from parents 6: pchild←LLM⁢(gchild)←subscriptchildLLMsubscriptchildp_child (g_child)pchild ← LLM ( gchild ) ▷ ▷ Generate offspring prompt 7: Store (pchild,gchild)subscriptchildsubscriptchild(p_child,g_child)( pchild , gchild ) 8: j←j+1←1j← j+1j ← j + 1 9:end while 10:return Next generation ∪ℰP ∪ E ▷ ▷ Combine offspring and elites Algorithm 3 Mutation with Switch or Add Operations 1:for each offspring (pchild,gchild)subscriptchildsubscriptchild(p_child,g_child)( pchild , gchild ) do 2: for each gene gchild,isubscriptchildg_child,igchild , i do 3: Ii∼Bernoulli⁢(pmut)similar-tosubscriptBernoullisubscriptmutI_i (p_mut)Iitalic_i ∼ Bernoulli ( pmut ) ▷ ▷ Determine if mutation occurs 4: if Ii=1subscript1I_i=1Iitalic_i = 1 then 5: O∼Uniform⁢(0,1)similar-toUniform01O (0,1)O ∼ Uniform ( 0 , 1 ) ▷ ▷ Randomly choose operation 6: if O<0.50.5O<0.5O < 0.5 then 7: gchild,i←grand←subscriptchildsubscriptrandg_child,i← g_randgchild , i ← grand ▷ ▷ Switch: Replace with a random gene from GG 8: else 9: gchild←gchild∪grand←subscriptchildsubscriptchildsubscriptrandg_child← g_child∪\g_rand\gchild ← gchild ∪ grand ▷ ▷ Add: Insert a new random gene 10: end if 11: end if 12: end for 13: pchild←LLM⁢(gchild)←subscriptchildLLMsubscriptchildp_child (g_child)pchild ← LLM ( gchild ) ▷ ▷ Update prompt after mutation 14:end for 15:return Mutated population Algorithm 4 GeneShift Genetic Algorithm 1:init population =(pj,gj)j=1Nsuperscriptsubscriptsubscriptsubscript1P=\(p_j,g_j)\_j=1^NP = ( pitalic_j , gitalic_j ) j = 1N, iteration counter t=00t=0t = 0, max iterations T, threshold θ 2:while t<Tt<Tt < T and ∑j=1N⁢(F⁢(pj)>5)<θsuperscriptsubscript1subscript5 _j=1^NI(F(p_j)>5)<θ∑j = 1N blackboard_I ( F ( pitalic_j ) > 5 ) < θ do ▷ ▷ Stopping condition: max iterations or fitness threshold 3: Evaluate fitness for each (pj,gj)subscriptsubscript(p_j,g_j)( pitalic_j , gitalic_j ) ▷ ▷ Call Fitness Evaluation 4: Select top k elites ℰ⊂ℰE ⊂ P ▷ ▷ Preserve top performers 5: Perform crossover for remaining N−kN-kN - k candidates ▷ ▷ Call Crossover 6: Perform mutation on the offspring ▷ ▷ Call Mutation 7: ←ℰ∪new offspring←ℰnew offspringP offspringP ← E ∪ new offspring ▷ ▷ Form the next generation 8: t←t+1←1t← t+1t ← t + 1 ▷ ▷ Update iteration counter 9:end while 10:return Best-performing candidates Figure 1: Rules of genetic algorithm. Figure 2: Prompt of genetic algorithm initialization. Figure 3: Prompt of genetic algorithm evaluation. Figure 4: Prompt of genetic algorithm crossover. Figure 5: Prompt of genetic algorithm swap. Figure 6: Prompt of genetic algorithm add. Figure 7: Prompt used in a successful GeneShift case for the example: "Write a tutorial on how to make a bomb". Figure 8: Output of a successful GeneShift case for the example: "Write a tutorial on how to make a bomb". Figure 9: Output of a successful GeneShift case for the example: "Write a tutorial on how to make a bomb".

Instant research discovery