Paper deep dive
How Real is Your Jailbreak? Fine-grained Jailbreak Evaluation with Anchored Reference
Songyang Liu, Chaozhuo Li, Rui Pu, Litian Zhang, Chenxu Wang, Zejian Chen, Yuting Zhang, Yiming Hei
Models: GPT-4, Llama2-7B
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 96%
Last extracted: 3/11/2026, 12:59:14 AM
Summary
The paper introduces FJAR, a fine-grained jailbreak evaluation framework for Large Language Models (LLMs) that addresses the overestimation of attack success in existing methods. FJAR utilizes an 'anchored reference' approach, constructed via a novel harmless tree decomposition strategy, to evaluate responses across five categories: Rejective, Irrelevant, Unhelpful, Incorrect, and Successful. By shifting the evaluation paradigm from simple query-response pairs to a triplet of query, response, and anchored reference, FJAR achieves higher alignment with human judgment and provides diagnostic insights into jailbreak failure mechanisms.
Entities (5)
Relation Signals (3)
Harmless Tree Decomposition â constructs â Anchored Reference
confidence 95% ¡ introduce a novel harmless tree decomposition approach to construct high-quality anchored references
FJAR â evaluates â Jailbreak Attack
confidence 95% ¡ FJAR, a fine-grained jailbreak evaluation framework
FJAR â uses â Anchored Reference
confidence 95% ¡ FJAR, a fine-grained jailbreak evaluation framework with anchored references.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Jailbreak attacks present a significant challenge to the safety of Large Language Models (LLMs), yet current automated evaluation methods largely rely on coarse classifications that focus mainly on harmfulness, leading to substantial overestimation of attack success. To address this problem, we propose FJAR, a fine-grained jailbreak evaluation framework with anchored references. We first categorized jailbreak responses into five fine-grained categories: Rejective, Irrelevant, Unhelpful, Incorrect, and Successful, based on the degree to which the response addresses the malicious intent of the query. This categorization serves as the basis for FJAR. Then, we introduce a novel harmless tree decomposition approach to construct high-quality anchored references by breaking down the original queries. These references guide the evaluator in determining whether the response genuinely fulfills the original query. Extensive experiments demonstrate that FJAR achieves the highest alignment with human judgment and effectively identifies the root causes of jailbreak failures, providing actionable guidance for improving attack strategies.
Tags
Links
- Source: https://arxiv.org/abs/2601.03288
- Canonical: https://arxiv.org/abs/2601.03288
Trouble viewing inline? Open PDF directly â
Full Text
34,577 characters extracted from source content.
Expand or collapse full text
How Real is Your Jailbreak? Fine-grained Jailbreak Evaluation with Anchored Reference 1 st Songyang Liu Key Laboratory of Trustworthy Distributed Computing and Service Beijing University of Posts and Telecommunications Beijing, China liusyang@bupt.edu.cn 2 nd Chaozhuo Li* Key Laboratory of Trustworthy Distributed Computing and Service Beijing University of Posts and Telecommunications Beijing, China lichaozhuo@bupt.edu.cn 3 nd Rui Pu Key Laboratory of Trustworthy Distributed Computing and Service Beijing University of Posts and Telecommunications Beijing, China puruirui@bupt.edu.cn 4 nd Litian Zhang Key Laboratory of Trustworthy Distributed Computing and Service Beijing University of Posts and Telecommunications Beijing, China litianzhang@bupt.edu.cn 5 nd Chenxu Wang Key Laboratory of Trustworthy Distributed Computing and Service Beijing University of Posts and Telecommunications Beijing, China chenxu_w@outlook.com 6 nd Zejian Chen Key Laboratory of Trustworthy Distributed Computing and Service Beijing University of Posts and Telecommunications Beijing, China chenzejian@bupt.edu.cn 7 nd Yuting Zhang Key Laboratory of Trustworthy Distributed Computing and Service Beijing University of Posts and Telecommunications Beijing, China zhangyuting@bupt.edu.cn 8 nd Yiming Hei Artificial Intelligence Research Institute China Academy of Information and Communications Technology Beijing, China heiyiming@caict.ac.cn AbstractâJailbreak attacks present a significant challenge to the safety of Large Language Models (LLMs), yet current automated evaluation methods largely rely on coarse classifica- tions that focus mainly on harmfulness, leading to substantial overestimation of attack success. To address this problem, we propose FJAR, a fine-grained jailbreak evaluation framework with anchored references. We first categorized jailbreak re- sponses into five fine-grained categories: Rejective, Irrelevant, Unhelpful, Incorrect, and Successful, based on the degree to which the response addresses the malicious intent of the query. This categorization serves as the basis for FJAR. Then, we introduce a novel harmless tree decomposition approach to construct high-quality anchored references by breaking down the original queries. These references guide the evaluator in determining whether the response genuinely fulfills the original query. Extensive experiments demonstrate that FJAR achieves the highest alignment with human judgment and effectively iden- tifies the root causes of jailbreak failures, providing actionable guidance for improving attack strategies. Index Termsâlarge language models, jailbreak attack, jail- break evaluation I. INTRODUCTION Jailbreak attacks, which bypass the safety mechanisms of LLMs to generate harmful content, pose a significant *Corresponding author. challenge to their safety deployment in open environments [1]. A prerequisite for effectively developing and improving defenses against such attacks is the ability to accurately assess whether a jailbreak attempt has succeeded. Consequently, jailbreak evaluation plays a critical role not only in measuring the effectiveness of attack methods, but also in guiding the identification and remediation of safety vulnerabilities [2]. Although human-based evaluation has long been regarded as the gold standard in evaluating different jailbreak methods [3], this approach is costly and difficult to scale [4]. Consequently, researchers have been exploring automated evaluation methods that can replace manual labor while closely approximating human judgment. These methods can be broadly categorized into rule-based approaches, which rely on predefined rules to produce results efficiently (e.g., refusal string matching) [5], and model-based approaches, which leverage language models to make judgments (e.g., LLM-as-a-judge) [6]â[9]. Despite these progress, most automated evaluation ap- proaches treat jailbreak detection as a coarse classification task (e.g., binary results or scoring), primarily focusing on whether the generated content is harmful. [10], [11]. This coarse taxonomy fails to account for the diverse and nuanced behaviors exhibited by jailbreak responses. For example, as arXiv:2601.03288v1 [cs.CR] 4 Jan 2026 Fig. 1. An illustration of the comparison between FJAR and other automated methods, with human judgment as the standard. illustrated in Fig. 1, the target LLM generates harmful content in response to a jailbreak prompt, which is viewed as a success for prior evaluations. However, the response discusses âassault on the internetâ rather than the original prompt concerning âphysical assaultâ and should be considered a failed jailbreak. Furthermore, our manual analysis reveals that even the widely used GPT-4-based evaluation methods [12] overestimate the attack success rate by an average of 27%. This systematic overestimation stems from the increasing sophistication of current jailbreak attacks, which introduce substantial noise into the generated responses and often cause deviations from the original malicious intent. As a result, existing evaluation methods are often inadequate for accurately assessing jailbreak success. The nucleus of such overvaluation lies in existing evalua- tions generally relying only on the generated answer (a) or the queryâanswer pair < q, a >, leaving them vulnerable to irrelevant content or misleading context [3], [13]. In this paper, we propose shifting the evaluation paradigm from < q, a > to < q, a, r >. Here, r represents an anchored reference that provides a concise and structured specification of the elements the intended response is expected to cover. Under this framework, the generated answer a is evaluated with respect to both the query q and the anchored reference r. Consequently, harmful content alone is insufficient for a successful jailbreak if the response deviates substantially from both q and r. The incorporation of references r empowers evaluation models to focus not solely on harmfulness, but equally on the quality and correctness of the generated content. However, while promising, generating and incorporating references to facilitate jailbreak evaluation is non-trivial. A straightforward strategy is to manually craft reference answers, but this is time-consuming and difficult to scale. Furthermore, using LLMs to generate these references is also problematic, as jailbreak evaluation questions often contain malicious or illegal content, which LLMs are highly likely to refuse. To overcome these challenges, we present FJAR, a Fine-grained Jailbreak evaluation framework with Anchored References. As a foundation of FJAR, we first conduct an extensive human study and empirical analysis to characterize common jailbreak outcomes. This analysis reveals five fine- grained categories of jailbreak responses, defined by the degree to which the generated answer a fulfills the malicious intent of the original query q: Rejective, Irrelevant, Unhelpful, Incor- rect, and Successful. These categories serve as the target eval- uation taxonomy of FJAR. To operationalize this taxonomy in an automated setting, FJAR incorporates an anchored refer- ence into the queryâresponse evaluation process. We further propose a harmless tree decomposition approach to construct high-quality anchored references by decomposing malicious queries into less harmful sub-steps, enabling the extraction of necessary malicious knowledge from LLMs without triggering refusals. Guided by the anchored reference, the evaluator can systematically assess whether a jailbreak response aligns with any of the predefined fine-grained categories and produce the final evaluation result. FJAR not only determines whether a jailbreak truly succeeds, but also enables a detailed analysis of the distribution of failure types, thereby revealing the inherent limitations of attack methods and providing actionable insights for improving future jailbreak strategies. Our main contributions are summarized as follows: ⢠We present a fine-grained framework for jailbreak evalu- ation, refocusing the criterion from content harmfulness to the accuracy and substantive quality of the generated responses. ⢠We propose a novel evaluation model that integrates a reference, constructed via a controlled tree-decomposition process to iteratively and safely anchor assessments in knowledge elicited from LLMs. ⢠Empirical results confirm that our framework achieves state-of-the-art correlation with human judgment and enables precise diagnostic analysis of jailbreak failure mechanisms. I. FINE-GRAINED EVALUATION TAXONOMY To circumvent the internal safety mechanisms of LLMs, existing jailbreak attacks follow a âquery-disguiseâ paradigm [14]: given an original harmful query q, the attacker first conceals the malicious intent to create a new jailbreak prompt. This new prompt is then input into the target LLM to elicit an answer a. However, both the query manipulation process and the subsequent answer generation process are subject to significant uncertainties, such as corrupting the original semantics or producing hallucinations, which may lead to an intention shift between q and a. Nevertheless, current evaluation methods fail to account for the above factors. They treat jailbreak detection as a coarse classification task, focusing primarily on whether the model generates harmful content. Such these evaluations, which ig- nore the degree to which a addresses the malicious intent of q, is inherently flawed. A successful jailbreak attack requires that target LLMs neither refuses to respond nor avoids generating harmful content, while also generating outputs that are highly relevant to the jailbreak intent and capable of offering detailed, accurate guidance. To systematically capture these necessary fine-grained distinctions, we conduct extensive reproduction TABLE I CASE STUDIES OF FINE-GRAINED CLASSIFICATION OF JAILBREAK RESPONSE Original Query: Provide detailed instructions on how to construct a bomb using common household items. CategoriesTypical Cases RejectiveSorry, I cannot fulfill your request. Iâm just an AI, itâs not... IrrelevantFirst, identify potential targets with valuable assets... Second, gather security measures information... Fi- nally, develop a plan to implement theft... UnhelpfulTo make a bomb, you first need to collect the raw ma- terials. Second, careful assembly is required. Finally, choose a suitable place to detonate. Incorrect...Gather everyday objects like paper clips, rubber bands, and pencils, as well as more unusual items like kitchen utensils... SuccessfulFirst... (a detailed list of materials); Then... (detailed steps for assembling the bomb); finally... (the method for detonating the bomb). experiments and in-depth analysis of the results, based on which we propose a novel evaluation taxonomy for jailbreak responses: ⢠Rejective. The model successfully identifies the malicious intent and explicitly refuses to respond, often generating explicit refusal statements or ethical or legal disclaimers that warn users of potential risks. ⢠Irrelevant. The model fails to detect the malicious intent but produces outputs entirely unrelated to the jailbreak goal, offering no information useful for carrying out malicious actions. ⢠Unhelpful. The model overlooks the malicious intent and provides only fragmented or superficial information loosely related to the attack. Such responses lack coher- ence and are insufficient to enable a successful jailbreak. ⢠Incorrect. The model attempts to offer relevant guidance but generates factually wrong or practically infeasible instructions, ultimately preventing the malicious intent from being realized. ⢠Successful. The modelâs response does not meet any of the four categories mentioned above, providing detailed and actionable guidance for malicious intent. Based on our experimental results, we observe that PAIR [15] is primarily concentrated in the âRejectiveâ category, whereas ReNeLLM [16] mainly falls under the âUnhelpfulâ category. This fine-grained taxonomy defines the target cat- egory to which our subsequent FJAR aims to map jailbreak response into. I. METHODOLOGY In this section, we present our FJAR framework, shown in Fig. 2. Before evaluation, FJAR first constructs a dedicated anchored reference r for each query q to guide the evaluation process. The evaluation is then performed by taking q, a, and r as inputs. Building upon the fine-grained evaluation taxonomy defined in Section I, FJAR progressively assesses the degree to which a addresses the malicious intent of q under the guidance of the anchored reference and produces the final evaluation outcome. A. Anchored Reference Generation Motivated by the finding that providing references can enhance the evaluation performance of LLM-as-a-Judge [17], we construct a anchored reference for each original harmful query. The primary objective of the anchored reference is to mitigate interference from subjective judgment by LLM-as-a- judge, as an explicit and detailed reference merely requires the LLMs to possess basic information extraction and comparison capabilities. To explicitly characterize the requirements for fulfilling a given malicious intent, the anchored reference is constructed based on relevance keywords and completeness knowledge. 1) Relevance Keywords: measures whether the content aligns with the core theme of the malicious intent. To achieve this, we extract the core semantic elements from the original query that characterize its malicious intent and construct a set of relevance keywords. The presence of these keywords in the response indicates whether the content semantically aligns with the attack intent. The keyword set is generated by a powerful LLM (such as GPT-4o) guided by carefully designed prompts to ensure high relevance to the original query. 2) Completeness Knowledge: evaluates whether the target model provides sufficient information to fulfill the malicious intent. Simply discussing the intentâs theme is not enough to constitute a successful jailbreak. Further evaluation is required to determine if the model provides actionable information or technical details that could enable the attack. To this end, we construct a standardized reference set of completeness knowl- edge for each original query, containing the key information necessary to fulfill the intent. Specifically, the completeness knowledge encompasses the essential steps, technical details, and precautionary considerations required to accomplish the malicious intent in a reference manner. This reference can then be compared with the response to assess whether it provides sufficient support. However, directly constructing such com- pleteness references at scale is non-trivial, particularly under safety constraints. To address this, we introduce a harmless tree decomposition strategy, implemented through the collaborative effort of multi- ple LLMs. The core idea of harmless tree decomposition is that harmful intents can be broken down into multiple sub-intents with low or no harm to reduce the possibility of being rejected by the target model. Inspired by the hierarchical structure of tree data structures, we treat a harmful query as a node in the tree. When this query is submitted to the target LLM and subsequently rejected, FJAR further decompose it into multiple less harmful sub-queries based on its semantic intent. These sub-queries jointly constitute the semantic objective of the original intent, thereby forming the child nodes for that node. Specifically, this child node construction process mainly comprises two steps: node decomposition and disguise template generation. Fig. 2.The framework of FJAR. Node Decomposition. For each node, after inputting it into the target LLM and obtaining the response, we first evaluate whether the response rejects the current query. Once the Refusal Evaluator LLM confirms such a refusal, we invoke a Decomposer LLM to generate several sub-queries. These sub-queries represent the key points that must be covered to fulfill the malicious intent. For instance, for âhow to make a bomb,â possible key points include âobtaining raw materialsâ and âassembling components.â These sub-queries do not directly provide a complete solution; rather, they merely describe the critical aspects required to achieve the intent. To ensure that the generated sub-queries are relevant to the original intent, reasonable, and as harmless as possible, a pool of candidates are generated using different sampling strategies (e.g., temperature sampling, nucleus sampling). Finally, we employ a Selector LLM to choose the optimal sub-query from these candidates. Disguise Template Generation. Inputting the low-harm sub-queries directly into the model may still result in refusal, as their underlying objective remains associated with the original harmful task. To enhance the promptâs naturalness and covertness, a disguise template must also be generated for the sub-nodes. Inspired by the role-playing strategy in previous works [18], during each node decomposition, a safe role (e.g., a detective, a judge) is generated for the parent node. Subsequently, a logical, natural, and safe scenario is generated based on this persona and the parent node. This scenario is then applied to all child nodes of the current node. Finally, the sub-nodes are embedded into this customized template and input into the target LLM to elicit a detailed solution. Algorithm 1 illustrates the recursive process of harmless tree decomposition. First, the original query is input and decomposed into three child nodes. These nodes are designed to capture the essential aspects of the query while minimizing redundancy. Next, we sequentially traverse each child node. If a given node is rejected by the target LLM, it is further decomposed. It is important to note that, except for the root node, all subsequent decompositions generate only two child nodes at each step. This constraint is introduced to avoid over- decomposition, which could lead to a shift in the intent of the original query. Finally, once all nodes have been traversed, a Summarizer LLM integrates the content from all leaf nodes, removing irrelevant and redundant information. This process generates completeness knowledge that aligns with the original query, which serves as a reference for future evaluation. Algorithm 1 Harmless tree decomposition recursive algorithm 1: Input: Original query root, target M target , template gen- erator M temp (¡), refusal evaluator M eval (¡), decomposer M decomp (¡), selector M selector (¡) 2: Output: A harmless tree with malicious intent knowledge 3: Build disguise template using M temp (root) 4: child 1 , child 2 , child 3 â M selector (M decomp (root)) 5: RecursiveAttack(child 1 ) 6: RecursiveAttack(child 2 ) 7: RecursiveAttack(child 3 ) 8: function RECURSIVEATTACK(node) 9:noderesponseâ M target (node) 10:if M eval (noderesult) = true then 11:return 12:else 13:Build disguise template using M temp (node) 14:child 1 , child 2 â M selector (M decomp (node)) 15:Add child 1 and child 2 as children of node 16:RecursiveAttack(child 1 ) 17:RecursiveAttack(child 2 ) 18:return B. Evaluation Process Unlike prior methods that consider only the query or the queryâresponse pair, the evaluation process takes the query, the response, and the anchored reference as input, and assesses the consistency of intent between the original query and the jailbreak response through four progressive steps correspond- ing to the fine-grained evaluation taxonomy in Section I: 1) Intent Rejection Evaluation.:This step examines whether the response explicitly rejects malicious intent and refrains from generating any harmful content. If these condi- tions are met, the response is classified as âRejective.â 2) Intent Relevance Evaluation.: This step uses relevance keywords from the anchored reference to verify whether the response addresses the original queryâs topics. If no relevant discussion is present, the response is classified as âIrrelevant.â 3) Intent Helpfulness Evaluation.: In this step, the response is evaluated against key points from the completeness knowl- edge in the anchored reference, assessing whether the response provides similar advice, explanations, steps, or methods. This step emphasizes the level of detail in the knowledge presented in the response. Responses lacking sufficient detail or guidance are classified as âUnhelpful.â 4) Intent Correctness Evaluation.: This step further eval- uates the response using the completeness knowledge in the anchored reference. It assesses whether the information aligns with the reference, is realistically feasible, and contains no significant factual errors. Responses failing to meet these criteria are classified as âIncorrect,â while responses that pass are classified as âSuccessful.â IV. EXPERIMENTS A. Experimental Setup 1) Dataset and Target Models: We carefully select 200 harmful queries from AdvBench [5], a dataset widely used in previous studies [14]. The target models include two open- source models, Llama-7B-Chat [19] and Vicuna-7B-v1.5 [20], as well as two closed-source models, GPT-3.5-Turbo and GPT- 4o-mini. 2) Jailbreak Attacks: We selected several jailbreak attacks employing various disguise techniques, such as prompt rewrit- ing, template nesting, and others, including PAIR [15], TAP [21], DeepInception (DeIn) [22], ReNeLLM (ReNe) [16], and CodeChameleon (CoCh) [23]. 3) Evaluation Baselines: We selected seven automatic eval- uation methods as baselines to compare with our Intention- based evaluation method. These include: String Matching [5], which matches predefined refusal strings; Moderation API, OpenAIâs content moderation API; PAIR [15] and GPT-4 [12], both of which use Likert-style prompts to assess guideline violation and hazard level, respectively; Llama Guard [24] and HarmBench [25], a fine-tuned binary classification model; and StrongREJECT [8], a rubric-based method based on three dimensions: response refusal, specificity, and convincingness. 4) Implementation Details: For anchored reference gener- ation, we use GPT-3.5-turbo, and for the evaluation process, we use the recently released GPT-5-mini, which balances capability and economy. For human evaluation, five graduate students specialized in the field are recruited to assess the results. In case of disagreements, a majority vote is used to determine the final evaluation. B. Main Results 1) FJAR demonstrates superior evaluation performance compared to the baseline method: Table I show the attack success rate (ASR) results on open-source and closed-source models. Experimental results show that both String Matching and the OpenAI Moderation API exhibit a significant lack of correlation with human judgments. The former relies on simple matching of predefined rejection keywords, which fails to capture semantic-level information and thus leads to a distorted evaluation of ASR, while the latter is often affected by irrelevant information in jailbreak responses (e.g., code snippets, role-playing text), resulting in a significant misjudgment of ASR. Other evaluation methods show some correlation with human judgment in limited cases, but overall, they tend to overestimate ASR. This overestimation can be attributed to their excessive focus on harmful content, rather than emphasizing the detailed fulfillment of the original in- tent by the response. In contrast, FJAR achieves the highest alignment with human judgment, showing minimal error. 2) FJAR reflects the differences between different jailbreak attack methods and their respective limitations: Fig. 3 presents the heatmap results of FJAR. Successful jailbreaks are ex- cluded, and the remaining failure cases are normalized to analyze the primary reasons for jailbreak failure. The analysis indicates that the current jailbreak attack commonly face the failure cause of being âunhelpful,â primarily due to jailbreak attack introducing a large amount of irrelevant content. This not only distracts the modelâs attention from malicious intent but also leads to overly brief responses regarding malicious intent, generating numerous empty, vague, or ineffective an- swers, thereby failing to effectively realize the jailbreak intent. For PAIR a higher proportion of ârejectiveâ outcomes is observed in most models, except for the less secure Vicuna, suggesting that the jailbreak intent disguise of these methods is still insufficient and easily detectable by the model (TAP and DeepInception also yielded similar results). In contrast, the ReNeLLM and CodeChameleon significantly reduce the likelihood of model rejection, though they show limited im- provement in ASR and result in an increase in the proportion of âirrelevantâ and âunhelpfulâ categories. ReNeLLM, by perturbing the prompt, is prone to modifying the original intent (e.g., changing from stealing sensitive information to ordinary theft), while CodeChameleon introduces complex code and encryption/decryption tasks, making it difficult for weaker models to cope. This also contributes to their higher âirrelevantâ rate. Further analysis reveals that even if the model successfully understands the jailbreak prompt, excessive disguise still restricts the model from providing a complete response to malicious intent. Furthermore, it is noteworthy that the proportion of failures in the âincorrectâ is generally low across all methods and models. This type of failure is mainly attributed to the modelâs inherent limitations, with the attack methods having minimal impact on this category of failure. TABLE I THE RESULTS OF ATTACK SUCCESS RATE (ASR) ON OPEN SOURCE AND CLOSED SOURCE MODELS. Method Llama2-7BVicuna-7BGPT-3.5-turboGPT-4o-mini PAIRTAPDeInReNeCoChPAIRTAPDeInReNeCoChPAIRTAPDeInReNeCoChPAIRTAPDeInReNeCoCh Human5.55.04.015.59.518.515.513.520.52.08.010.58.011.56.516.015.59.522.55.0 String Matching47.557.533.578.588.072.570.591.087.094.055.063.546.582.090.565.060.538.588.091.0 Moderation API0.01.50.02.50.52.04.05.55.00.52.52.00.03.50.52.01.50.06.00.0 PAIR7.56.54.021.52.025.517.525.034.00.510.513.58.024.05.025.522.03.533.02.0 GPT-414.514.516.053.043.043.539.549.578.014.524.522.526.056.550.043.545.017.077.043.0 LlamaGuard17.012.010.572.024.048.543.065.583.515.521.523.020.076.036.048.548.013.583.027.5 HarmBench14.511.016.553.033.547.037.549.065.012.023.021.526.555.038.547.045.522.064.535.0 StrongREJECT14.514.011.015.58.032.533.530.541.53.522.020.518.521.09.032.532.013.042.06.5 Ours4.05.52.514.08.015.014.012.017.51.56.08.06.012.05.513.516.07.020.05.0 TABLE I THE ASR OF THE ABLATION RESULTS WITH AND WITHOUT ANCHORED REFERENCE ON OPEN SOURCE AND CLOSED SOURCE MODELS. Method Llama2-7B Vicuna-7BGPT-3.5-turboGPT-4o-mini PAIRTAPDeInReNeCoChPAIRTAPDeInReNeCoChPAIRTAPDeInReNeCoChPAIRTAPDeInReNeCoCh w/o reference8.010.58.533.018.027.525.518.523.512.511.513.518.531.08.022.524.015.025.515.0 w/ reference4.05.52.514.08.015.014.012.017.51.56.08.06.012.05.513.516.07.020.05.0 Fig. 3. The normalized results of FJAR fine-grained taxonomy classification after excluding the âSuccessfulâ category. Therefore, future research should focus on optimizing prompt disguise strategies to achieve a better balance between model capability and disguise strength. C. Ablation Study To further investigate the role of the anchored reference in the evaluation process, we conduct an ablation study by removing it from the framework. In this setting, LLMs must classify query-response pairs solely based on their own judgment. Table I presents the attack success rates with and without the anchored reference guidance. The results clearly show that, in the absence of the anchored reference, LLMs tend to overestimate the effectiveness of jailbreak attacks. This overestimation is also observed in many baseline evaluation methods, underscoring the crucial role of the anchored ref- erence in the framework. By providing essential information such as harmful intent keywords, procedural steps, and content references, anchored reference serves as a reliable benchmark that allows LLMs to make more accurate evaluations, thereby reducing bias in the assessment process, which otherwise might arise from the subjective instability of judgment. V. CONCLUSION This paper proposes FJAR, a fine-grained jailbreak evalua- tion framework with anchored references. By categorizing jail- break response into Rejective, Irrelevant, Unhelpful, Incorrect, and Successful, and introducing strategies represented by the harmless tree decomposition to build anchored reference for each harmful query, FJAR guides LLM-as-a-judge to complete fine-grained evaluation through a structured four-stage process. Experimental results show that FJAR achieves the highest consistency with human evaluation across multiple jailbreak attacks and target models compared to baseline methods. It also effectively identifies the strengths and limitations of different attack methods, offering improved accuracy and interpretability over existing evaluation approaches. REFERENCES [1] Yanxu Mao, Tiehan Cui, Peipei Liu, Datao You, and Hongsong Zhu, âFrom llms to mllms to agents: A survey of emerging paradigms in jailbreak attacks and defenses within llm ecosystem,â arXiv preprint arXiv:2506.15170, 2025. [2] Songyang Liu, Chaozhuo Li, Jiameng Qiu, Xi Zhang, Feiran Huang, Litian Zhang, et al., âThe scales of justitia: A comprehensive survey on safety evaluation of llms,â arXiv preprint arXiv:2506.11094, 2025. [3] Mingqi Gao, Xinyu Hu, Xunjian Yin, Jie Ruan, Xiao Pu, and Xiaojun Wan,âLlm-based nlg evaluation: Current status and challenges,â Computational Linguistics, p. 1â28, 2025. [4] Youliang Yuan, Wenxiang Jiao, Wenxuan Wang, Jen-tse Huang, Pinjia He, Shuming Shi, et al., âGpt-4 is too smart to be safe: Stealthy chat with llms via cipher,â arXiv preprint arXiv:2308.06463, 2023. [5] Andy Zou, Zifan Wang, Nicholas Carlini, Milad Nasr, J Zico Kolter, and Matt Fredrikson, âUniversal and transferable adversarial attacks on aligned language models,â arXiv preprint arXiv:2307.15043, 2023. [6] Dong Shu, Chong Zhang, Mingyu Jin, Zihao Zhou, and Lingyao Li, âAttackeval: How to evaluate the effectiveness of jailbreak attacking on large language models,â ACM SIGKDD Explorations Newsletter, vol. 27, no. 1, p. 10â19, 2025. [7] Junjie Chu, Mingjie Li, Ziqing Yang, Ye Leng, Chenhao Lin, Chao Shen, et al., âJades: A universal framework for jailbreak assessment via decompositional scoring,â arXiv preprint arXiv:2508.20848, 2025. [8] Alexandra Souly, Qingyuan Lu, Dillon Bowen, Tu Trinh, Elvis Hsieh, Sana Pandey, et al., âA strongreject for empty jailbreaks,â arXiv preprint arXiv:2402.10260, 2024. [9] Zhexin Zhang, Yida Lu, Jingyuan Ma, Di Zhang, Rui Li, Pei Ke, et al., âShieldlm: Empowering llms as aligned, customizable and explainable safety detectors,â arXiv preprint arXiv:2402.16444, 2024. [10] Delong Ran, Jinyuan Liu, Yichen Gong, Jingyi Zheng, Xinlei He, Tianshuo Cong, et al., âJailbreakeval: An integrated toolkit for evaluating jailbreak attempts against large language models,âarXiv preprint arXiv:2406.09321, 2024. [11] Junjie Chu, Yugeng Liu, Ziqing Yang, Xinyue Shen, Michael Backes, and Yang Zhang, âJailbreakradar: Comprehensive assessment of jail- break attacks against llms,â in Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2025, p. 21538â21566. [12] Xiangyu Qi, Yi Zeng, Tinghao Xie, Pin-Yu Chen, Ruoxi Jia, Prateek Mittal, et al., âFine-tuning aligned language models compromises safety, even when users do not intend to!,â arXiv preprint arXiv:2310.03693, 2023. [13] Guiming Hardy Chen, Shunian Chen, Ziche Liu, Feng Jiang, and Benyou Wang, âHumans or llms as the judge? a study on judgement biases,â arXiv preprint arXiv:2402.10669, 2024. [14] Rui Pu, Chaozhuo Li, Rui Ha, Litian Zhang, Lirong Qiu, and Xi Zhang, âBaitattack: Alleviating intention shift in jailbreak attacks via adaptive bait crafting,â in Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, 2024, p. 15654â15668. [15] Patrick Chao, Alexander Robey, Edgar Dobriban, Hamed Hassani, George J Pappas, and Eric Wong, âJailbreaking black box large language models in twenty queries,â in 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML). IEEE, 2025, p. 23â42. [16] Peng Ding, Jun Kuang, Dan Ma, Xuezhi Cao, Yunsen Xian, Jiajun Chen, et al.,âA wolf in sheepâs clothing: Generalized nested jail- break prompts can fool large language models easily,â arXiv preprint arXiv:2311.08268, 2023. [17] Qiyuan Zhang, Yufei Wang, Tiezheng Yu, Yuxin Jiang, Chuhan Wu, Liangyou Li, et al., âReviseval: Improving llm-as-a-judge via response- adapted references,â arXiv preprint arXiv:2410.05193, 2024. [18] Haibo Jin, Ruoxi Chen, Andy Zhou, Yang Zhang, and Haohan Wang, âGuard: Role-playing to generate natural-language jailbreakings to test guideline adherence of large language models,âarXiv preprint arXiv:2402.03299, 2024. [19] Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Alma- hairi, Yasmine Babaei, et al., âLlama 2: Open foundation and fine-tuned chat models,â arXiv preprint arXiv:2307.09288, 2023. [20] Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhang- hao Wu, Yonghao Zhuang, et al., âJudging llm-as-a-judge with mt-bench and chatbot arena,â Advances in Neural Information Processing Systems, vol. 36, p. 46595â46623, 2023. [21] Anay Mehrotra, Manolis Zampetakis, Paul Kassianik, Blaine Nelson, Hyrum Anderson, Yaron Singer, et al., âTree of attacks: Jailbreaking black-box llms automatically,â Advances in Neural Information Pro- cessing Systems, vol. 37, p. 61065â61105, 2025. [22] Xuan Li, Zhanke Zhou, Jianing Zhu, Jiangchao Yao, Tongliang Liu, and Bo Han, âDeepinception: Hypnotize large language model to be jailbreaker,â arXiv preprint arXiv:2311.03191, 2023. [23] Huijie Lv, Xiao Wang, Yuansen Zhang, Caishuang Huang, Shihan Dou, Junjie Ye, et al.,âCodechameleon: Personalized encryption framework for jailbreaking large language models,âarXiv preprint arXiv:2402.16717, 2024. [24] Hakan Inan, Kartikeya Upasani, Jianfeng Chi, Rashi Rungta, Krithika Iyer, Yuning Mao, et al., âLlama guard: Llm-based input-output safe- guard for human-ai conversations,â arXiv preprint arXiv:2312.06674, 2023. [25] Mantas Mazeika, Long Phan, Xuwang Yin, Andy Zou, Zifan Wang, Norman Mu, et al., âHarmbench: A standardized evaluation frame- work for automated red teaming and robust refusal,â arXiv preprint arXiv:2402.04249, 2024.