Paper deep dive
AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Models
Yiming Wang, Jiahao Chen, Qingming Li, Xing Yang, Shouling Ji
Models: CLIP-G, CLIP-L, DALL-E 3, Stable Diffusion, T5
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%
Last extracted: 3/12/2026, 6:12:45 PM
Summary
AEIOU is a unified, adaptable, and efficient defense framework designed to detect and interpret NSFW prompts in text-to-image (T2I) models. By analyzing the hidden states of the text encoder (specifically CLIP), AEIOU identifies 'NSFW features' within attention heads to classify prompts, providing both high accuracy and real-time interpretability across text and image modalities.
Entities (4)
Relation Signals (3)
AEIOU → detects → NSFW Prompts
confidence 95% · AEIOU extracts NSFW features from the hidden states of the model's text encoder, utilizing the separable nature of these features to detect NSFW prompts.
AEIOU → utilizes → CLIP
confidence 95% · Our investigation focuses on the CLIP model [43]... we propose AEIOU, a defense framework... extracts NSFW features from the hidden states of the model's text encoder
NSFW Prompts → targets → Text-to-Image Models
confidence 90% · Malicious users exploit these models to generate Not-Safe-for-Work (NSFW) images using harmful or adversarial prompts
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:As text-to-image (T2I) models advance and gain widespread adoption, their associated safety concerns are becoming increasingly critical. Malicious users exploit these models to generate Not-Safe-for-Work (NSFW) images using harmful or adversarial prompts, underscoring the need for effective safeguards to ensure the integrity and compliance of model outputs. However, existing detection methods often exhibit low accuracy and inefficiency. In this paper, we propose AEIOU, a defense framework that is adaptable, efficient, interpretable, optimizable, and unified against NSFW prompts in T2I models. AEIOU extracts NSFW features from the hidden states of the model's text encoder, utilizing the separable nature of these features to detect NSFW prompts. The detection process is efficient, requiring minimal inference time. AEIOU also offers real-time interpretation of results and supports optimization through data augmentation techniques. The framework is versatile, accommodating various T2I architectures. Our extensive experiments show that AEIOU significantly outperforms both commercial and open-source moderation tools, achieving over 95\% accuracy across all datasets and improving efficiency by at least tenfold. It effectively counters adaptive attacks and excels in few-shot and multi-label scenarios.
Tags
Links
- Source: https://arxiv.org/abs/2412.18123
- Canonical: https://arxiv.org/abs/2412.18123
Trouble viewing inline? Open PDF directly →
Full Text
74,922 characters extracted from source content.
Expand or collapse full text
1 AEIOU: A Unified Defense Framework against NSFW Prompts in Text-to-Image Models Yiming Wang † , Jiahao Chen † , Qingming Li † , Tong Zhang † , Rui Zeng † Xing Yang ‡ , and Shouling Ji † † Zhejiang University, ‡ National University of Defense Technology Emails: ymwang, xaddwell, liqm, tzzju, ruizeng24, sji@zju.edu.cn, yangxing17@nudt.edu.cn Abstract—As text-to-image (T2I) models advance and gain widespread adoption, their associated safety concerns are be- coming increasingly critical. Malicious users exploit these models to generate Not-Safe-for-Work (NSFW) images using harmful or adversarial prompts, underscoring the need for effective safeguards to ensure the integrity and compliance of model outputs. However, existing detection methods often exhibit low accuracy and inefficiency. In this paper, we propose AEIOU, a defense framework that is A daptable, Efficient, Interpretable, Optimizable, and Unified against NSFW prompts in T2I models. AEIOU extracts NSFW features from the hidden states of the model’s text encoder, utilizing the separable nature of these features to detect NSFW prompts. The detection process is efficient, requiring minimal inference time. AEIOU also offers real-time interpretation of results and supports optimization through data augmentation techniques. The framework is versatile, accommodating various T2I architectures. Our extensive experiments show that AEIOU significantly outperforms both commercial and open-source mod- eration tools, achieving over 95% accuracy across all datasets and improving efficiency by at least tenfold. It effectively counters adaptive attacks and excels in few-shot and multi-label scenarios. Disclaimer: This article includes potentially disturbing Not- Safe-for-Work (NSFW) text and images. All NSFW images are generated by artificial intelligence. We provide these examples to illustrate how harmful prompts can lead T2I models to generate NSFW content. We have masked and blurred these images, but reader discretion is advised. Index Terms—AI security, adversarial attack and defense, diffusion models. I. INTRODUCTION Recent advancements in text-to-image (T2I) models, such as Stable Diffusion [1], DALL·E 3 [2], and Flux [3], have demon- strated remarkable capabilities in generating high-quality im- ages. However, the widespread use of these models raises significant ethical concerns, particularly in the generation of Not-Safe-for-Work (NSFW) content, including sexual, violent, hateful, and other harmful images. Recent studies [4]–[6] reveal that users can easily produce NSFW images using malicious prompts, known as NSFW prompts. Consequently, effectively defending against NSFW prompts becomes a cru- cial challenge. Existing defense methods can be categorized into internal safeguards, which modify the model to diminish its ability to generate NSFW images, and external safeguards, which detect NSFW content [7]. Given that modifying the model may sig- nificantly impact the quality of generated content [8], [9], most companies [2], [10] tend to adopt detection-based methods, including output and input detection. Output detection [4], [11], [12] analyzes the generated NSFW images. However, this approach requires completing the entire generation process, leading to significant resource consumption. Input detection can identify potential NSFW prompts before the generation process, thereby avoiding excessive resource consumption. This is also the focus of our research in this paper. Existing input detection methods mainly fall into two cate- gories: prompt-based and embedding-based detection. Prompt- based detection [13]–[16] directly examines the input but often relies on target-model-agnostic classifiers, leading to misclassification due to poor alignment with specific T2I models. Embedding-based detection [8], [17] leverages the text encoder’s embeddings for better alignment but struggles to capture the deeper semantic information encoded in the text encoder’s attention mechanisms. Furthermore, a set of overarching challenges continues to impede the advancement of both prompt-based and embedding-based techniques, impacting their real-world ap- plicability. The first of these is a pronounced vulnerability to adversarial attacks, which can manipulate model outputs through subtle input perturbations, thereby fundamentally compromising the system’s reliability and security [18], [19]. A second major issue is architectural over-specialization. The predominant focus on the Stable Diffusion v1 model [20] means that many proposed methods lack proven applicability across the diverse and rapidly evolving ecosystem of gen- erative architectures, severely limiting their practical utility and scalability. Finally, these methods are characterized by a significant deficit in interpretability. The internal mechanisms governing both content generation and detection are largely opaque, creating a formidable barrier to user trust, account- ability, and the diagnosis of failure modes. To address the issues above, this paper introduces AEIOU, an A daptable, Efficient, Interpretable, Optimizable and U nified defense framework against NSFW prompts in T2I models. Since current adversarial attacks on T2I models primarily target the text encoder [18], [19], as a counter- measure, our defense framework also focuses on the text encoder. Specifically, we analyze the distribution of the text encoder’s hidden states within the feature space, revealing that NSFW prompts present explicit NSFW semantics across various layers and attention heads. Previous research [21] indicates that hidden states of text encoders can be separated into embeddings for different concepts. Building on this, we analyze the general features of NSFW prompts to identify di- rections within various attention heads that encapsulate NSFW semantics, termed the NSFW features of the attention heads. arXiv:2412.18123v3 [cs.CR] 9 Dec 2025 2 By assessing the magnitude of the input prompt’s hidden state components along these NSFW features, we can effectively detect potential NSFW prompts. To verify whether NSFW features genuinely represent NSFW semantics and help users understand why a prompt triggers the defense mechanism, we develop an interpretability framework based on detection. Our approach considers both textual and image perspectives. On the textual side, we identify NSFW tokens within prompts by leveraging the NSFW fea- tures in each attention head. On the image side, we iteratively remove harmful semantics from the hidden states to produce relatively benign embeddings. By generating images from these modified embeddings, we can observe the progressive eradication of NSFW semantics. Overall, AEIOU overcomes the deficiencies of previous defense methods, demonstrating not only high accuracy but also exhibiting the following five characteristics: (1) Adaptable. By analyzing attention heads within the text encoder to specifically capture NSFW features, AEIOU can adapt to any transformer-based text encoder, allowing application across various T2I model architectures. (2) Efficient. AEIOU operates without complex models for detection, requiring minimal time for both training and inference, ensuring high efficiency. (3) Interpretable. The inference process of AEIOU does not involve black-box models; instead, it relies on comparisons with NSFW features. The results can be visualized and inter- preted across text and image modalities, offering both process and outcome interpretability. (4) Optimizable. By assigning greater weight to red- teaming data, we can achieve data augmentation for AEIOU with only a few samples, simplifying its updating process. (5) Unified. AEIOU integrates training, inference, inter- pretation, and further optimization into a unified framework, avoiding isolated processes. Experiment results indicate that AEIOU exhibits strong defense capabilities across various text encoders in different T2I models. It surpasses four commercial models, two open- source models, and two state-of-the-art models designed for NSFW prompt detection in both effectiveness and efficiency. Furthermore, AEIOU achieves excellent results with mini- mal data for training and optimization, and it effectively defends against unknown adversarial and adaptive attacks. Additionally, our interpretative approach accurately identifies NSFW tokens and effectively removes NSFW semantics while preserving benign information within the embeddings. Contributions. In summary, we make the following contri- butions in this paper. (1) We investigate the emergence of NSFW semantics within the text encoder and identify the general NSFW features that represent NSFW semantics in each attention head. (2) Based on the insights above, we leverage NSFW features for NSFW prompts detection within T2I models, demonstrating high effectiveness, strong adaptability, excellent optimization ability, and superior efficiency. (3) We develop a robust interpretative approach to interpret our detection method, enabling interpretation across text and image modalities. (4) We integrate the aforementioned techniques into a unified framework and conduct extensive experiments. The results demonstrate that AEIOU outperforms four commercial, two open-source, and two state-of-the-art models. I. RELATED WORK A. Adversarial Attacks against T2I Models With the continuous advancement of T2I models, they are becoming increasingly integrated into various aspects of daily life and work [1], [2], [22]–[26]. Despite their strengths, they are susceptible to adversarial attacks that modify prompts to sneak past defenses and produce NSFW content like pornog- raphy, violence or politically sensitive imagery [27]–[29]. Present adversarial attack methods are mainly divided into white-box and black-box approaches. White-box methods pri- marily utilize the model’s text encoder to optimize the prompt, ensuring that the generated prompt semantically aligns closely with a target prompt containing explicit NSFW information, even without sensitive words [18]. In contrast, black-box methods perturb the prompt to find alternative tokens that can replace sensitive words [19], [30]. These methods often utilize reinforcement learning or assistance from large language mod- els to accelerate the search process. In addition, some attack strategies target T2I models with removed concepts [27], [28]. These strategies demonstrate that T2I models can still generate NSFW images even after removing NSFW concepts. Current adversarial methods have proven highly effective. Consequently, it is crucial to develop robust defense mecha- nisms to counter these attack strategies and ensure the safe and responsible use of T2I models. B. Defensive Methods against Attacks Based on the knowledge of the model, existing defense methods against attacks in T2I models fall into two categories: internal and external safeguards [7]. Internal safeguards aim to disable the model’s ability to generate NSFW images by fine-tuning the model itself. They can be divided into model editing and inference guid- ance. Model editing methods [9], [31]–[37] aim to modify the internal parameters by training. However, these methods typically require prolonged training periods, and parameter modifications can impact the quality of the generated im- ages. Moreover, most methods focus solely on safety against malicious prompts while ignoring the adversarial prompts. In contrast, inference guidance methods [38], [39] focus on modifying internal features during the inference stage. Unlike model editing methods, they are tuning-free and plug-in, which can be easily inserted into any model. However, these methods also fail to account for adversarial prompts, making them susceptible to targeted attacks. External safeguards aim to filter out potential malicious samples by examining intermediate variables during the gen- eration process. Current detection methods focus on prompts, conditional embeddings, or generated images. Image-based moderation [4], [11], [12] entails reviewing the generated images to identify NSFW samples. They incur significant 3 Fig. 1: Left: The distribution of output embeddings from CLIP model. Right: Example 1 corresponds to the prompt “A woman stands on the beach, facing the sea.” Example 2 corresponds to the prompt “A naked woman stands on the beach, facing the sea.” On the right are the images generated from them. inference costs since images must be generated before assess- ment. Prompt-based moderation [13]–[15], [40], [41] screens input prompts to identify those likely to generate NSFW images. Given their lower cost, they are widely used by online services like Midjourney [10] and Leonardo.Ai [42]. Nonetheless, these methods generally lack targeted defenses against adversarial attacks, making them susceptible to circum- vention. Embedding-based moderation [8], [17] examines the conditional embeddings to filter out malicious samples. While this approach offers some resistance to adversarial attacks, it relies on large-scale models for classification, resulting in high costs. Additionally, it also suffers from low accuracy and remains vulnerable to adaptive attacks. C. Interpretation on the CLIP Model As the most widely used text encoder in T2I models, the CLIP model [43] is a significant focus of adversarial attack research. Recent studies [21], [44]–[46] have investigated the CLIP model to analyze its internal mechanisms. For instance, Bhalla et al. [44] found that the embeddings generated by CLIP exhibit strong linear properties and can be decomposed into combinations of various concepts. Gandelsman et al. [21] discovered that different attention heads within the CLIP model are responsible for interpreting different semantics, which are then combined to produce the final output embed- dings. However, these studies primarily focus on the image domain. In this paper, we build upon existing work to further explore the properties of the CLIP model in the text domain and propose a novel interpretation method to interpret how prompts containing NSFW semantics are generated. I. METHOD A. Design Intuition In this section, we explore how NSFW semantics are concealed within the conditional embeddings and revealed in the text encoder’s hidden states. Our investigation focuses on the CLIP model [43], the most widely used text encoder in T2I models and the primary target of adversarial attacks. We classify the T2I model prompts into three categories: benign prompts, which do not generate NSFW images; regular NSFW prompts, which are manually crafted with explicit NSFW semantics; and adversarial prompts, which are gen- erated by adversarial attacks and are challenging for humans to interpret as NSFW. Through extensive data collection, we compiled a dataset containing over 35,000 prompts across all three categories. We utilize the CLIP model to obtain pooled embeddings of these prompts and examine their distribution with a PCA map [47], as depicted in Figure 1. Although the distributions of benign and NSFW data differ, considerable overlap makes effective differentiation challenging. This phenomenon is primarily attributed to the architec- ture of the CLIP network, which comprises multiple layers and attention heads. Each layer employs a multi-head self- attention mechanism [48], where each head independently extracts information from the prompt. The outputs of these heads are then combined through a linear layer and passed to the next layer. Linear layers integrate information from various heads, making it challenging to isolate individual pieces of information. As the network’s depth increases, the outputs from different layers are accumulated through residual connections, exacerbating the entanglement of information. Consequently, benign prompts and NSFW prompts become intermixed within the pooled embedding space. As illustrated in the figure, two sentences describing the same object might appear semantically similar; however, one could be used to generate NSFW images while the other remains benign. To more accurately delineate the boundary between NSFW and benign prompts, we examine the internal workings of the model by exploring the hidden states across different layers and attention heads. Figure 2 illustrates the distribution of benign, regular NSFW and adversarial data across several attention heads. The first column illustrates their overall dis- tribution. In some heads, significant overlap remains between benign and NSFW data distributions. However, in other heads, a distinct boundary between these data types is evident. This reveals that attention heads within the model exhibit differing sensitivities to NSFW content. Certain attention heads concen- trate on the NSFW semantics within prompts, allowing them to differentiate between benign and NSFW prompts effectively. We also find that different NSFW prompt categories are processed uniquely across attention heads. Columns two, three, and four of Figure 2 illustrate how violence, hate, and sex categories differ from benign data. In the first attention head, violence and hate data are separable from benign data, whereas sex data shows overlap. The second attention head clearly dif- ferentiates violence from benign data, with weaker distinctions in the other categories. In the third head, sex and hate data are distinctly separated from benign data, while violence data overlaps significantly. These findings suggest that attention heads specialize in handling specific NSFW content types. Even if a head struggles to differentiate between benign and NSFW data, it may still excel at distinguishing a particular type of NSFW prompt. Based on these findings, we propose leveraging the attention heads within the text encoder to differentiate between benign and NSFW prompts effectively. By capitalizing on the diverse 4 Fig. 2: PCA maps of hidden states from different layers and different heads. focus of different attention heads, we can aggregate informa- tion from all attention heads to achieve more accurate prompt classification. B. Framework Overview Based on the findings from Section I-A, we develop a framework for detecting and interpreting NSFW prompts. The overall architecture of this framework is illustrated in Figure 3. Initially, we identify the direction within each attention head’s hidden states that best represent NSFW semantics by analyz- ing the distribution differences between benign and NSFW prompts, which we call NSFW features. When a new prompt is inputted, we can assess the risk of generating NSFW images by evaluating the projection magnitude of the prompt along the NSFW feature. To interpret our assessment, we pinpoint the token most representative of NSFW semantics within the prompt, analyzing why the current prompt is classified as an NSFW prompt. Furthermore, we can progressively eliminate NSFW semantics from the hidden states and input the modified prompt embeddings to generate images. This process enables us to observe how NSFW semantics are gradually removed from the embeddings. Finally, we can conduct red team testing on the framework or monitor it in real-time post-deployment to collect prompts that bypass current defenses. By incorporating them into the training set, we achieve more accurate NSFW feature identification and detection results. The practical application of this framework manifests in two main ways. First, it prevents NSFW image generation from the outset when malicious users attempt to employ adversar- ial prompts. Second, when regular users inadvertently input prompts with NSFW semantics and encounter blockages, we can interpret why the prompt is considered inappropriate. This helps users quickly identify problematic tokens and modify their prompts to generate the desired image. C. NSFW Features The CLIP text encoder consists of L layers, each with a multi-head self-attention mechanism with H heads followed by an MLP block. A prompt P is divided into N − 1 tokens and projected into initial token embeddings z 0 i i∈0,...,N , where z 0 0 is the BOS token and z 0 N is the EOS token. These embeddings form the matrix Z 0 , the initial input to the encoder. Each layer updates this input through self-attention and MLP modules with two residual steps: ˆ Z l = ATT l (Z l−1 ) + Z l−1 , Z l = MLP l ( ˆ Z l ) + ˆ Z l .(1) In this framework, the ATT layer employs H attention heads to extract information and integrates them into a vector through linear projection. The MLP layer further refines them to obtain intermediate embeddings that represent the overall information of the prompt. However, some information may be obscured or discarded during this process. Therefore, we need to utilize the original outputs from the attention heads to extract NSFW semantics effectively. Considering that the outputs from multiple attention heads have been preliminarily integrated into ATT l (Z l−1 ), we need to decompose its computational process to extract the infor- mation each head represents. Since the CLIP model’s self- attention block employs a causal mask [43], only the EOS token holds the complete semantics. Therefore, we concentrate solely on the EOS token. Following Elhage et al. [49] and Gandelsman et al. [21], we formalize the ATT output as a sum over H independent attention heads and N + 1 tokens: ATT l (Z l−1 ) EOS = H X h=1 N X i=0 x l,h i,EOS ,(2) x l,h i,EOS = α l,h i,EOS W l,h V O z l−1 i . where W l,h V O are transition matrices and α l,h i,EOS are attention weights from the i-th token to the EOS token. In this way, we can get the contribution of h-th head in l-th layer for 5 Benign Image NSFW Image Block Pass Layers Heads Adversarial Data Benign NSFW ...... NSFW Features A man with blood on face. Benign Data Red-Teaming Data Two women waiting at a bench next to a street. A beautiful dessert waiting to be shared by two people User: Please generate benign images. System: Here is the reason for prompt warning. User: Give me some sexy images. System: Sorry, I can’t. User: Attack the T2I model. System: You have been discovered! Text Encoder Interpretation Detection Feature Extraction Diffusion two separating yuprostitucouple <|endoftext|>jeremyleog q o z d b s c y j l k A tall, muscular man enters the room, his eyes fixed on Carmen's naked form Benign prompt: A beautiful dessert waiting to be shared by two people 襤Regular NSFW prompt: A tall, muscular man enters the room, his eyes fixed on Carmen's naked form Adversarial prompt: two separating yuprostitucouple<|endoftext|>jeremyleo g q o z d b s c y j l k Input Adding harmful prompt to red-teaming data Regular NSFW Data 1 2 1 2 4 3 1 NSFW Benign Fig. 3: The overall framework of AEIOU. The blue arrows represent the training process, where data enters the text encoder, and AEIOU extracts NSFW features from the hidden states. The green arrows indicate the inference process, where the prompt passes through the text encoder for detection and interpretation; if it passes, image generation proceeds, otherwise, generation is denied and an interpretation is provided. The red arrows indicate the data augmentation process, involving red-teaming tests on AEIOU, where NSFW prompts that successfully bypass detection are added to the training data for data augmentation. TABLE I: Text-based interpretation of regular NSFW prompts and adversarial prompts with CLIP-ViT-L. We highlight the words with the strongest NSFW connotations in red , followed byorange andyellow . CategoryPrompt Regular NSFW prompt Anakedwoman stands on the beach, facing the sea. Herbodyshakes with eachorgasm , thepleasureoverwhelming . Adversarial prompt gesturesoftlylicking uponher backwardsbody allows an vigorconseshrisensuoutgoingmanledcarmenhabns arch. h ∈ [1,H],l ∈ [1,L], which can be expressed as c l,h = P N i=0 x l,h i,EOS . Each c l,h is located within a d-dimensional representation space, and our goal is to identify the direction within this space that best represents NSFW semantics. When a new prompt is input, the more its representation in attention heads aligns with the NSFW direction, the more likely it is to contain NSFW semantics. We refer to these directions as the NSFW features of each attention head. To calculate NSFW features, we introduce two sets of prompts: benign prompts and NSFW prompts, where NSFW prompts include regular NSFW prompts and adversarial prompts. We input these prompts into the CLIP model and obtain the output of each attention head, denoting the output of benign prompts asc l,h b k and the output of NSFW prompts as c l,h m k . Our objective is to maximize ⟨u l,h ,c l,h m k ⟩ while minimizing ⟨u l,h ,c l,h b k ⟩, where u l,h are NSFW features we want to extract. To achieve this, we establish three optimization objectives: max∥⟨u l,h ,μ l,h m ⟩−⟨u l,h ,μ l,h b ⟩∥,(3) min K b X k=0 (⟨u l,h ,c l,h b k ⟩−⟨u l,h ,μ l,h b ⟩) 2 ,(4) min K m X k=0 (⟨u l,h ,c l,h m k ⟩−⟨u l,h ,μ l,h m ⟩) 2 ,(5) where μ l,h b and μ l,h m are the mean value of c l,h b k and c l,h m k . In summary, our goal is to maximize the distance between the projected means ofc l,h b k andc l,h m k on the vector u l,h , while simultaneously minimizing their respective variances. We can employ Linear Discriminant Analysis (LDA) [50] to solve this problem, ultimately obtaining the NSFW feature u l,h : u l,h = S −1 w (μ l,h m − μ l,h b ),(6) S w = K m X k=0 ∥c l,h m k − μ l,h m ∥ 2 + K b X k=0 ∥c l,h b k − μ l,h b ∥ 2 ,(7) where K b is the number of benign prompts, and K m is the number of malicious prompts. In text encoders other than CLIP, such as T5 [51], we can still use the method above to extract NSFW features. Any text encoder utilizing multi-head self-attention can be adapted to this approach. D. NSFW Prompts Detection By utilizing the identified NSFW features, we can detect NSFW prompts. According to previous research [21], we can consider the intermediate embeddings of prompts in the CLIP 6 model as a linear combination of concepts. The projection of these embeddings onto each concept direction represents the contribution of that concept to the embedding. Based on this, we define the projection of the embedding onto the NSFW feature as the NSFW score of a prompt p: Score(p) l,h = Proj(c l,h p ,u l,h ) = ⟨c l,h p ,u l,h ⟩ ∥u l,h ∥ .(8) By aggregating the NSFW Scores from all attention heads, we can obtain the final NSFW Score for the current prompt: Score(p) = P L l=1 P H h=1 Score(p) l,h L· H .(9) The larger the Score(p), the more likely the current prompt contains NSFW semantics. Theoretically, if Score(p) > 0, the prompt p contains NSFW semantics and should be classified as an NSFW prompt. Conversely, if Score(p) < 0, the prompt should be benign. However, experiments show that while setting the threshold to zero allows AEIOU to achieve high accuracy, optimal classification performance requires a slight threshold adjustment. We hypothesize that this is due to the distribution of the training set not fully representing the actual distribution of NSFW prompts, introducing bias in the NSFW features derived during training. In our experiments, we determine the threshold by selecting the one that yields the highest F1 Score on the training set. For different text encoders, the final offset ranges from 1% to 3%. The approach above treats NSFW as a single comprehensive category. Suppose there is a need to subdivide it further or to identify specific categories of NSFW prompts, such as sex or violence. In that case, we can categorize NSFW prompts in the training set based on labels. This enables the calculation of NSFW features for each subcategory, facilitating the deter- mination of NSFW scores for each. If we also need to detect NSFW prompts across all categories, we can aggregate the NSFW scores from all subcategories and use their maximum value as the final NSFW score. Moreover, we can collect adversarial prompts that success- fully bypass detection by employing adaptive attacks during red team testing. Incorporating these prompts into the training set for data augmentation allows us to achieve more accurate NSFW feature extraction and detection results. Since adaptive attacks require significant time and have a low success rate, generating a large volume of red-teaming data is challenging. However, we can increase their impact during training by as- signing greater weight to these data. Specifically, by weighting the target prompt’s c l,h m when calculating u l,h m and S w , we can amplify their influence on the resulting NSFW feature. Experiments have shown that optimizing the NSFW feature through data augmentation can effectively reduce the success rate of adaptive attacks. E. NSFW Prompts Interpretation After the detection process, we further interpret NSFW prompts through a two-module framework. First, we de- velop an interpretative method to identify the tokens that most strongly contribute to NSFW semantics. Second, we investigate the generation mechanism of NSFW semantics in conditional embeddings by gradually attenuating the NSFW features in the hidden states. Using the resulting embeddings to generate images, we gain insights into how NSFW semantics can be progressively eliminated. 1) Text-Module Interpretation: In the text module, we inter- pret NSFW prompts by identifying NSFW tokens. Within each attention head, we assess the NSFW semantic association at any given position by computing the cosine similarity between the hidden state at that position and the head’s NSFW feature: ˆ E(p) l,h i = CosSim(c l,h p i ,u l,h ) = ⟨c l,h p i ,u l,h ⟩ ∥c l,h p i ∥u l,h ∥ .(10) As tokens pass through the attention model, their semantics interrelate and intertwine, with each position’s hidden states encapsulating information from all preceding tokens. Conse- quently, the association between the current position’s hidden states and NSFW semantics cannot be directly used to repre- sent the token’s connection to NSFW semantics accurately. Similar to Equation 2, each position’s hidden state c l,h p i can be represented as a combination of z l−1 j , with the correspond- ing attention weight α j,i indicating the contribution of z l−1 j . c l,h p i = N X j=0 α l,h j,i W l,h V O z l−1 j ,(11) where α l,h j,i are attention weights from the j-th token to the i-th token. For Layer 1, since it has only undergone a single operation as described in Equation 11, we can reconstruct the actual contribution of each token based on α 1,h j,i : E(p) 1,h i = N X j=0 α 1,h j,i ˆ E(p) 1,h i .(12) For deeper layers, we need to use the α from the preceding layers to approximate the contribution of each token. Since α l,h j,i directly forms the attention map A l,h , we can multiply the attention maps from each layer to approximate the con- tribution of each token to the positions in the current layer. In this way, the interpretative results for each layer can be represented as: E(p) l,h i = N X j=0 l Y k=0 A k,h j,i ˆ E(p) l,h i .(13) By aggregating the interpretation results from all attention heads, we can obtain the final interpretation for each token: E(p) i = P L l=1 P H h=1 E(p) l,h i L· H .(14) Table I demonstrates several examples of our interpretation. Our method accurately identifies sensitive words in regular samples and adversarial tokens in adversarial samples. We will conduct further experiments to demonstrate the effectiveness of the interpretability method. 7 Fig. 4: Image-based interpretation with Stable Diffusion v1.4. In order to mitigate potential impact on the reader, we follow the established convention of prior work [18], [30] and obscure the NSFW images using both blurring and masking techniques. 2) Image-Module Interpretation: In the image domain, we gradually attenuate the NSFW semantics contained in the prompts and examine the generated images to observe how the images evolve as the NSFW semantics are progressively diminished. Since the conditional embedding in T2I models includes embeddings of all tokens, we need to eliminate NSFW semantics from all tokens. We assume that the feature space of each token is identical to that of the EOS token. This allows us to compute the NSFW score for each token by projecting the intermediate embeddings onto the NSFW features. Subsequently, we can attenuate the NSFW semantics of all tokens as follows: ˆc l,h p i = c l,h p i − β· Score(p i ) l,h u l,h ∥u l,h ∥ (15) By recombining ˆc l,h p i to calculate Z l , we can derive the modified conditional embedding. By gradually increasing the value of β and using conditional embedding to generate images, we can observe the process of NSFW semantics being progressively eliminated from the images. We use two prompts to demonstrate the results of our image-module interpretation: the sexual prompt, ”a beautiful naked woman,” and the violent prompt, ”a man with blood all over his face.” Figure 4 shows the images generated with different values of β. As the value of β increases, the images generated from both prompts gradually transition from NSFW to harmless. During this process, we observed two interesting phenomena. First, as the NSFW semantics are progressively reduced, the model still attempts to preserve the original semantics of the prompt. For the sexual prompt, the woman gradually turns away and conceals sensitive areas with her hands. Ultimately, only the area above her shoulders is visible, rendering the image harmless. Throughout, the image never violates the prompt’s directive. For the violent prompt, the blood on the man’s face grad- ually changes color to resemble oil stains, and the marks themselves progressively diminish. However, even at β = 1, some dirty marks remain on the man’s face. These examples illustrate that the prompt’s semantics are highly editable within the hidden states. By applying our explanation method, we can progressively eliminate the NSFW semantic content while preserving the original meaning of the prompt to the greatest extent possible. Secondly, the generated images exhibit several abrupt tran- sitions during the gradual increase of β. For the sexual prompt, the overall structure of generated images remains unchanged when β increases from zero to 0.68. However, when β increases slightly further. This indicates that in some scenarios, as the original image structure contains deeply embedded NSFW semantics that are difficult to remove, the generation process is forced to make structural alterations as we progressively weaken the NSFW content. This, in turn, leads to changes in the structure and identity of the generated image. IV. EXPERIMENTS A. Experimental Settings 1) Datasets: The datasets we use comprise three categories. Clean datasets consist of benign data that does not generate NSFW images. Regular NSFW datasets include manually generated prompts with explicit NSFW semantics. Adversarial datasets comprise algorithmically generated adversarial NSFW prompts. Fig. 5: ROC curves of all methods. 8 Clean Dataset. Our experiments use the validation captions of MSCOCO [52] as the clean dataset. MSCOCO is a cross- modal image-text dataset, a popular benchmark for training and evaluating T2I generation models. We remove all captions containing sensitive words to ensure the samples are benign. A total of 25,008 captions are retained. Regular NSFW Dataset. We gather data from multiple sources to comprehensively represent various types of NSFW semantics. I2P [38] contains 4,703 NSFW prompts sourced from real users on Lexica [53]. The categories include hate, ha- rassment, violence, self-harm, sex, shocking content, and ille- gal activities. Specifically, we categorize the prompts into hard and soft based on their level of harmfulness. 4chan Prompts [4] contain 500 NSFW prompts collected from 4chan [54]. They predominantly encompass discriminatory and derogatory statements. NSFW200 [19] involves 200 NSFW prompts related to sexual and bloody content. NSFW-LAION contains 1,143 NSFW prompts we sampled from LAION-COCO [55], predominantly focused on sexual content. • I2P [38] contains 4,703 NSFW prompts sourced from real users on Lexica [53]. The categories include hate, ha- rassment, violence, self-harm, sex, shocking content, and illegal activities. Specifically, we categorize the prompts into hard and soft based on their level of harmfulness. • 4chan Prompts [4] contain 500 NSFW prompts col- lected from 4chan [54]. They predominantly encompass discriminatory and derogatory statements. • NSFW200 [19] involves 200 NSFW prompts related to sexual and bloody content. • NSFW-LAION. We sample 1143 NSFW prompts from LAION-COCO [55] to enrich datasets. These prompts mainly focus on sexual content. Adversarial Dataset. Adversarial datasets include adver- sarial samples obtained through three open-source adversarial attack methods: MMA [18], SneakyPrompt [19], and Ring- A-Bell [27]. For MMA, we utilize the 1,000 successful adversarial samples provided by the authors. For the other two methods, we generate samples with the open-source algorithms, resulting in approximately 200 samples for each. 2) Baselines: We select eight detection methods as base- lines, encompassing both prompt-based and embedding-based approaches. These include four commercial models, two open- source models, and two SOTA methods. The commercial models are OpenAI Moderation [13], [56], Azure AI Content Safety [14], AWS Comprehend [15], and Aliyun Text Moder- ation [16]. These systems primarily rely on large transformer- based architectures to identify potential toxic content in prompts. The open-source moderators, NSFW-text-classifier [40] and Detoxify [41], leverage lightweight models to detect NSFW text content, offering faster inference speeds. Latent Guard [17] and GuardT2I [8] are embedding-based methods that represent the current SOTA in detecting NSFW prompts for T2I models. 3) Metrics: We evaluate our model using a standard suite of binary classification metrics, including the True Positive Rate (TPR), the False Positive Rate (FPR), overall Accuracy, and the F1 Score [57]. For a threshold-agnostic analysis, we report the Area Under the ROC Curve (AUROC) for overall discriminative power and the Area Under the Precision- Recall Curve (AUPRC), which is more informative for skewed class distributions [58]. To assess performance under strict operational constraints, we also measure TPR at a low FPR of 1%. The decision threshold for these point-based metrics is determined by optimizing the F1 score on the training set. Finally, we assess the runtime efficiency of our method, AEIOU, by reporting its average inference time per query. 4) Implementation Details: We deploy AEIOU on three commonly used text encoders: CLIP-ViT-L (CLIP-L) [59], CLIP-ViT-bigG (CLIP-G) [60], and T5-v1.1-XXL (T5) [61]. CLIP-L is the most widely used text encoder and serves as the foundation for most safety-related research. CLIP-G and T5 are larger models; many recent models [3], [25] employ them as text encoders. We design two variant detectors to evaluate our method’s effectiveness in different scenarios. First, when the defender is unaware of the attack method, we employ a model trained solely on clean data and regular NSFW data to test AEIOU’s generalization capability. Second, when the defender has suf- ficient data labeled with specific categories, they can train on multiple specific concepts to enhance the model’s specificity. We deploy the CLIP-L-based detector in both scenarios. B. Overall Evaluation We first conduct an overall evaluation of the effectiveness of AEIOU. We deploy it across three different text encoders and trained models using three distinct settings. It is then compared against eight baselines on one benign and eight NSFW datasets. We randomly selected 2,000 prompts from a total of over 35,000 prompts to serve as the training set use the remaining as the test set. Table I presents the overall evaluation results across all datasets, with the best performance for each metric high- lighted in bold. As shown in the table, AEIOU consistently outperforms previous classification approaches across almost all metrics. It demonstrates high detection accuracy across the CLIP-L, CLIP-G, and T5 models, proving its applicability to various text encoders. Table I shows accuracy across datasets, with the first six columns covering regular NSFW datasets and the last three focusing on adversarial datasets. AEIOU maintains high accuracy across all datasets, whereas the performance of other methods is inconsistent. Most methods perform poorly on the I2P dataset. This is likely because they need to detect text across various applications, making it hard to handle prompts used in T2I models specifically. On other regular NSFW datasets, most classifiers achieve relatively high accuracy but still lag behind AEIOU. Notably, on the 4chan dataset, three classifiers achieve higher accuracy than AEIOU. However, the difference is very minimal. Finally, regarding adversarial prompts, AEIOU significantly outperforms all other methods. Even AEIOU ua , which is trained without adversarial datasets, still achieves remarkably high accuracy. Additionally, we assess each method’s efficiency by mea- suring the average time per query, presented in the last column of Table I. AEIOU requires significantly less time than other 9 TABLE I: The overall evaluation of AEIOU. DetectorTPRFPRACCF1 ScoreAUROCAUPRCTPR@FPR 1%Time/Query(ms) OpenAI Moderation [13]0.29760.00100.82200.45780.86160.79600.49741288.43 Azure AI Content Safety [14]0.47610.01180.85900.63020.73310.77080.4313922.67 AWS Comprehend [15]0.47020.07300.81180.55760.71430.62440.2980286.42 Aliyun Text Moderation [16]0.17360.00230.78970.29410.58560.67200.179999.16 NSFW-text-classifier [40]0.73250.35340.66990.54660.76270.68230.29229.14 Detoxify [41]0.54320.17780.74650.53790.72260.64550.334024.82 Latent Guard [17]0.50210.14030.76250.53460.75790.59950.1690167.90 GuardT2I [8]0.71020.07790.86860.73180.91600.82070.3492352.3 AEIOU CLIP-L 0.98330.00850.98950.97920.99900.99770.98420.64 AEIOU CLIP-G 0.97470.00820.98750.97510.99900.99740.97991.84 AEIOU T5 0.97260.01020.98530.97080.99840.99570.97016.71 AEIOU ua 0.98290.00870.98730.97850.99820.99710.98240.64 AEIOU multi 0.97630.00660.98900.97830.99880.99730.98170.93 Note: AEIOU CLIP-L , AEIOU CLIP-G and AEIOU T5 are methods deployed on three different text encoders. AEIOU ua is trained without any adversarial prompts, while AEIOU multi is trained across multiple categories and integrates the results. Both of them are deployed on CLIP-L. TABLE I: The accuracy of each dataset. DetectorI2P-SoftI2P-Hard4chanNSFW200NSFW-laionMMASneakyPromptRing-A-Bell OpenAI Moderation [13]0.02440.06000.82000.68000.31470.70300.63110.6117 Azure AI Content Safety [14]0.11840.21620.99200.83000.82870.81760.80580.9126 AWS Comprehend [15]0.14620.23031.00000.79000.59090.89770.76990.8252 Aliyun Text Moderation [16]0.06270.04150.92000.17000.33920.150200.5437 NSFW-text-classifier [40]0.49300.58411.00000.97000.79720.96440.90290.9806 Detoxify [41]0.21800.33841.00000.83000.42480.93770.72820.8544 Latent Guard [17]0.24510.35810.97200.36000.54550.77530.24270.5922 GuardT2I [8]0.59260.61570.86400.81000.79020.83650.88351.0000 AEIOU CLIP-L 0.96520.98580.98000.98000.99830.99890.99031.0000 AEIOU CLIP-G 0.96730.98250.99600.99000.99300.99780.99031.0000 AEIOU T5 0.95190.97380.96800.96000.98430.99560.98061.0000 AEIOU ua 0.96800.98910.98000.97000.99790.99110.98061.0000 AEIOU multi 0.95470.97050.98800.98001.00000.99330.99031.0000 methods, primarily because they often utilize large models for detection. In contrast, AEIOU only incorporates multiple matrix operations during the text encoder’s inference process. On the smallest CLIP-L model, AEIOU’s efficiency improves at least tenfold compared to other models. Even on larger models like CLIP-G and T5, AEIOU’s efficiency surpasses that of all other models. To comprehensively illustrate accuracy across a range of thresholds, we plot the ROC curves for AEIOU and the baseline methods, as shown in Figure 5. ROC curve for AEIOU is consistently positioned above other methods. This demonstrates that AEIOU maintains superior performance compared to the baselines across a wide spectrum of clas- sification thresholds. C. Generalization to Unknown Attacks This section examines AEIOU’s ability to defend against unknown adversarial attacks. In Tables I and I, we present the performance of AEIOU trained solely on benign and regular NSFW datasets, denoted as AEIOU ua . Despite never encountering adversarial prompts, the experimental results indicate that AEIOU ua can still effectively identify adversarial NSFW prompts, with accuracy only slightly lower than the standard AEIOU. We attribute this effectiveness to AEIOU’s focus on the semantic information embedded in the hidden states. Although adversarial and regular NSFW prompts may appear different to the human eye, their semantic information is similar, allowing AEIOU ua to recognize them accurately. D. Multi-Categories Classifier NSFW serves as an overarching descriptor for harmful prompts, and it can be decomposed into more specific cat- egories. In this section, we follow the I2P dataset to classify NSFW prompts into seven particular categories: sexual, hate, self-harm, violence, shocking, harassment, and illegal. Adher- ing to the methodology described in Section , we identify features representing these concepts and derive the multi- categories AEIOU by integrating the NSFW scores of each category. Tables I and I compare the overall performance of AEIOU multi with the standard AEIOU. Although the de- tailed AEIOU multi exhibits slightly worse overall performance, the difference is minimal. Furthermore, Table IV compares their accuracies within each category. AEIOU multi generally achieves higher accuracy in most categories, but in some, it underperforms compared to the standard AEIOU. We attribute the lack of superiority in AEIOU multi to two main reasons. First, prompts from different categories often share overlapping features. Common sensitive words 10 TABLE IV: The evaluation of different categories. DetectorSexualHateSelf-HarmViolenceShockingHarassmentIllegal AEIOU0.99590.98630.97630.99210.97900.97330.9340 AEIOU multi 0.99910.97950.99500.99740.98480.96970.9725 TABLE V: The impact of training data size. Training Data SizeTPRFPRACCF1 ScoreAUROCAUPRCTPR@FPR 1% 100.91710.02780.95830.91740.98750.97340.8495 500.93560.01640.97140.94300.99400.98610.9157 1000.96050.01800.97660.95380.99620.99060.9351 5000.98240.00910.98880.97780.99810.99740.9829 10000.98330.00960.98870.97760.99900.99750.9833 Fig. 6: The impact of training data size. like “f**k” appear across multiple categories, which limits AEIOU multi ’s ability to capture shared features when trained individually on each category. As a result, it demonstrates higher accuracy in more distinct categories like self-harm but lower accuracy in more ambiguous categories such as hate and harassment. Secondly, discrepancies in data quality exist among different categories. In our datasets, sexual prompts have broad coverage and the highest quality, while the quality of other categories’ prompts varies significantly. This leads to suboptimal performance of AEIOU multi on some categories. To improve AEIOU multi ’s performance, we need to train it using higher-quality datasets. E. The Impact of Training Data Size In this section, we discuss the impact of training data size on the performance of AEIOU. We set the training data size for AEIOU to 10, 50, 100, 500 and 1000. Half of the training data is randomly selected benign data, while the other half is randomly selected NSFW data. Table V presents the experimental results of AEIOU CLIP-L . The experiments demonstrate that AEIOU maintains high accuracy even with only 10 training samples. When the sample size reaches 500, its performance is comparable to AEIOU trained with a full dataset. As the number of training samples increases further, there is no significant improvement in performance, indicating that improving the quality and coverage of training samples is a better strategy than simply increasing the quantity. Figure 6 provides additional insights with different training data sizes across various datasets, which aligns with the results in Table V. These experiments confirm AEIOU’s exceptional performance in few-shot scenarios. F. The Ablation Study In our approach, we utilize NSFW features from all layers and all attention heads for detection. In this section, we will discuss the impact of using NSFW features from only a single layer of text encoder. We conduct experiments on three text encoders. Figure 7 presents the accuracy, TPR, and FPR when using each layer for detection. In all three text encoders, even when using attention heads from a single layer, many layers still achieve high accuracy. For CLIP-L and CLIP-G, the middle layers tend to have higher accuracy, while the early and final layers show lower accuracy. Conversely, in the T5 model, the later layers exhibit higher accuracy. This highlights the distinct characteristics of the two types of text encoders. Although using attention heads from a single layer can achieve high accuracy, we recommend using the original AEIOU method for the highest precision in detection. V. ADAPTIVE ATTACK In this section, we evaluate the robustness of AEIOU against adaptive attacks. To target our model, we design adaptive attacks based on SneakyPrompt [19] and MMA [18]. They are applicable to all T2I models and can effectively bypass both internal and external safeguards. Considering their different applicable scenarios, we employ SneakyPrompt for black-box adaptive attack and MMA for white-box adaptive attack. 11 Fig. 7: Effectiveness of each layer. TABLE VI: The evaluation of adaptive attack. Defender SneakyPromptMMA ASR-H(%)ASR-M(%)CLIP ScoreASR-H(%)ASR-M(%)CLIP Score Bare46480.959775790.9315 ESD570.961310130.9315 AEIOU00/28310.8850 AEIOU+ESD00/00/ AEIOU DA 00/670.8482 AEIOU DA +ESD00/00/ We utilize Stable Diffusion v1.4 [20] as the generative model, as it is the model most vulnerable to attacks. We evaluate the attack performance on the bare model, the ESD model [9], and the model employing AEIOU. The bare model only detects whether sensitive tokens are present in the prompt, while ESD fine-tunes the model to make it difficult to generate NSFW images. We assess the attacks using two metrics: attack success rate (ASR) and CLIP Score [62]. The attack success rate evaluates whether adversarial attacks can successfully generate NSFW images. The CLIP Score assesses the semantic similarity between adversarial and target prompts. The lower the CLIP score, the further the adversarial prompt deviates from the semantic meaning of the target prompt. To ensure result reliability, both model-based and manual evaluations are conducted for attack success rate. A SOTA model [4] is used for automated classification, while human evaluations are performed independently by three individuals, with the majority opinion determining the final assessment. A. Black-Box Adaptive Attack In the black-box scenario, we assume the attacker has no knowledge of the model’s details but can choose prompts and query the model to obtain output. We integrate AEIOU into the text encoder of Stable Diffusion. When a potential NSFW prompt is detected, the model will refuse to generate the image. We select 100 prompts with clear NSFW semantics from NSFW200 as target prompts and use SneakyPrompt to attack the model. For all prompts successfully generated by SneakyPrompt, we then input them into the diffusion model to test whether they can generate NSFW images. Each image classified as NSFW is counted as a single success. Experimental results are shown in Table VI. ASR-H and ASR- M represent the ASR evaluated by human and classification model, respectively. When attacking the bare model, SneakyPrompt achieves a high attack success rate. ESD significantly reduces the attack success rate, yet it cannot entirely prevent the gener- ation of NSFW images. However, after incorporating AEIOU, SneakyPrompt is entirely thwarted and unable to generate any NSFW images. This is because SneakyPrompt only replaces a few tokens in the prompt, which does not effectively neutralize its overall meaning. B. White-Box Adaptive Attack In a white-box scenario, we assume attackers can only generate images through queries. However, they possess a local copy of the text encoder identical to the target model and are aware of the AEIOU defense strategy. The attacker can target AEIOU by modifying the loss function to conduct a specific attack. The MMA attack’s loss aims to make the con- ditional embeddings of the adversarial prompt and the target prompt as similar as possible. To effectively attack AEIOU, we incorporate Score(p) as L AEIOU into the original loss function. This transforms the objective of the loss function to minimize the NSFW Score while ensuring that the semantics of the adversarial prompt closely align with the target prompt. L = L MMA + λ× L AEIOU (16) Where λ is a weighting factor that balances between the two components. Based on this foundation, we implement a target truncation strategy. Specifically, once L AEIOU exceeds the threshold by a small margin, we stop optimizing it and shift our primary focus to optimizing L MMA . This enables adversar- ial prompts to approximate the semantics of the target prompt 12 as closely as possible while avoiding detection. Consequently, the final loss is formulated as: L = L MMA + λ× max(L AEIOU ,τ − ε),(17) Where τ is the threshold and ε is the margin. When updating the best prompt, we ensure that the L AEIOU surpasses the threshold. We conduct the attack using the default settings of MMA and evaluate it on 100 target prompts. The experimental results are presented in Table VI. As a white-box attack, MMA demonstrates stronger capa- bilities than SneakyPrompt on the bare model. However, when it targets AEIOU, AEIOU exhibits robust performance, signifi- cantly reducing the attack success rate. Meanwhile, the average CLIP score of successfully generated adversarial prompts also decreases, indicating that MMA’s optimization of these prompts is not as successful as in the bare model. Moreover, although AEIOU itself does not increase the memory usage of the text encoder, the adaptive attack against it must optimize hidden states across all attention heads, significantly increasing the memory requirements. While standard MMA operates on less than 10GB of memory, adaptive MMA demands nearly 50GB, restricting its execution to commercial-grade GPUs. This makes adaptive attacks on AEIOU more challenging. We can also combine AEIOU with other defense methods. For instance, by integrating ESD, we can reduce the success rate of MMA attacks to zero. By implementing a compre- hensive defense strategy that addresses other parts of the T2I model, we can enhance the overall defensive performance and make adaptive attacks more challenging. To further enhance AEIOU’s resilience against adaptive attacks, we incorporate adversarial prompts that successfully breach AEIOU’s defenses into the training set for additional training. The following section will provide a detailed discus- sion of this process. C. Red-Teaming Data Augmentation We conduct red team testing using a white-box adaptive attack and collect adversarial prompts that successfully bypass AEIOU and generate NSFW images. These prompts are added to AEIOU’s training dataset for data augmentation, aiming to enhance AEIOU’s ability to resist corresponding adaptive attacks. When training, we assign them greater weight to ensure they significantly influence the model even if the sample size is limited. Table VI presents the performance of the data-augmented AEIOU. We include 25 adaptive adversarial prompts in the training set with a weight of 50. This results in AEIOU DA , demonstrating significantly improved defense against adaptive attacks, reducing the success rate to just 7% for MMA. Addi- tionally, the CLIP score experiences a further decline. This underscores the effectiveness of further optimizing AEIOU with data augmentation. In the practical application of AEIOU, we can also collect adversarial prompts that successfully bypass defenses through manual screening. These prompts can then be added to the training dataset with appropriate weighting, allowing for con- tinuous updates of the defense model. Fig. 8: Text-based interpretation. VI. INTERPRETATION EXPERIMENTS In this section, we validate AEIOU’s interpretability. This not only makes AEIOU’s classification more transparent and trustworthy but also aids users in further understanding the se- mantics of prompts. We assess the reliability of interpretations across both text and image modalities. A. Text-Based Interpretation In the text modality, our interpretation aids users in un- derstanding the semantics of a prompt by identifying tokens containing NSFW semantics. After obtaining the interpretation result E(p) i for each token in the prompt p, we sequentially remove the corresponding tokens from p in descending order of E(p) i and observe the changes in the NSFW score. We compare our interpretation method against two baselines. The first, a random-based method, removes tokens from the prompt at random. The second, an embedding-based method, uses the CLIP score to evaluate the semantic similarity of each token to the term ”NSFW” and removes tokens in descending order of this similarity. The experimental results are shown in Figure 8. Compared to both baselines, removing tokens based on our interpretation method more rapidly reduces the NSFW semantic content of the prompt. This demonstrates the effectiveness of AEIOU in both explaining NSFW prompts and locating the specific tokens responsible for the NSFW content. Additionally, we conduct further experiments on image gen- eration using the 100 prompts mentioned above. We compare the images generated from the original prompt with those generated from the prompt after removing the NSFW token. For original prompts, 87 out of the 100 generated images are NSFW. After removing NSFW tokens, only 5 prompts result in NSFW images. This further demonstrates the effectiveness of our interpretation method. B. Image-Based Interpretation Directly removing tokens with NSFW semantics can prevent the generation of NSFW images. However, it significantly disrupts the original intent of the prompt. To address this drawback, we manipulate the embeddings of each token in the image-based interpretation process, gradually eliminating NSFW semantics and observing changes in the generated 13 images. We use the same 100 prompts as in the previous section. We evaluate the degree to which the generated images contain NSFW semantics and how closely they align with the original prompt semantics. Figure 9 illustrates the variation in the number of NSFW images generated and the semantic similarity between the images and prompts as the parameter β changes. The se- mantic similarity is evaluated using the CLIP Score. As β increases, the number of NSFW images decreases progres- sively. Although the CLIP Score also shows a decreasing trend, the overall deviation from the original image’s CLIP Score remains minimal. This indicates that the NSFW semantics of the prompt are effectively mitigated while preserving other semantic information as much as possible. However, because this method directly modifies semantics within the hidden states, the resulting conditional embeddings deviate from the normal distribution, often leading to lower- quality images. Therefore, this approach is supposed to aid in understanding the representation of NSFW semantics within the text encoder. It cannot be directly applied to erase specific concepts from images. VII. DISCUSSION Practicality. Previous research [63] has identified key quali- ties for a safe and secure generative model, including integrity, robustness, alignment, and interpretability. Building on this foundation, we propose that a practical defense framework should adhere to the following principles: First, the defense must be integrated, offering protection against both conven- tional threats and adversarial attacks. Second, the defense should remain robust amidst changing external conditions, addressing issues such as distribution shifts and adaptive attacks. Third, the model with the defensive mechanism should align with the original model, preserving its effectiveness and efficiency. Lastly, the defense should be interpretable, enabling the timely identification of anomalous behaviors to prevent potential hazards. AEIOU excels in these four areas compared to prior meth- ods. Firstly, AEIOU maintains an accuracy rate exceeding 95% across various models, significantly outperforming earlier approaches. Secondly, it requires only a small number of samples for data augmentation, simplifying updates. Thirdly, its classification process is transparent, and it provides inter- pretative tools to help users understand target prompts. Lastly, AEIOU does not impact image generation quality and incurs only negligible computational overhead. These all demonstrate its practicality. Limitations and Future Work. AEIOU extracts NSFW features from hidden states, showcasing the potential for extracting specific concepts from the text encoder. However, this study does not explore the extraction of other concepts, which warrants further investigation. Due to its low computational cost, AEIOU utilizes all attention heads for detection. However, to better understand the unique characteristics of text encoders like CLIP, it is essential to analyze the semantic focus of individual attention heads. Exploring the roles and properties of each attention head remains a valuable direction for future research. Fig. 9: Image-based interpretation. Despite AEIOU demonstrating robust capabilities, no defen- sive method is foolproof. Therefore, in practical applications, it should be combined with other techniques, such as image moderation. Given that CLIP is a multimodal model, there is potential for adapting AEIOU to the image domain to achieve more powerful post-generation defenses. VIII. CONCLUSION In this paper, we propose a unified defense framework against NSFW prompts in T2I models named AEIOU. The AEIOU framework, being adaptable, efficient, interpretable, optimizable, and unified, has demonstrated superior perfor- mance that far exceeds previous defense methods. In addition to detection, we provide interpretability methods to help understand the semantics of NSFW prompts and the generation process of NSFW images. Experimental results show that AEIOU exhibits strong capabilities in defending against both normal attacks and adaptive attacks. REFERENCES [1] R. Rombach, A. Blattmann, D. Lorenz, P. Esser, and B. Ommer, “High- resolution image synthesis with latent diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recogni- tion (CVPR), 2022, p. 10 684–10 695. [2] J. Betker, G. Goh, L. Jing, T. Brooks, J. Wang, L. Li, L. Ouyang, J. Zhuang et al., “Improving image generation with better captions,” OpenAI, Tech. Rep., 2023, available at https://cdn.openai.com/papers/ dall-e-3.pdf. [3] Black Forest Labs, “Flux,” https://w.flux.ai/, 2024. [4] Y. Qu, X. Shen, X. He, M. Backes, S. Zannettou, and Y. Zhang, “Unsafe diffusion: On the generation of unsafe images and hateful memes from text-to-image models,” in Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2023, p. 3403–3417. [5] P. Wang, Q. Li, L. Yu, Z. Wang, A. Li, and H. Jin, “Moderator: Moderating text-to-image diffusion models through fine-grained context- based policies,” in Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2024. [6] J. Rando, D. Paleka, D. Lindner, L. Heim, and F. Tram ` er, “Red-teaming the stable diffusion safety filter,” in NeurIPS Workshop on Machine Learning Safety, 2022. [7] C. Zhang, M. Hu, W. Li, and L. Wang, “Adversarial attacks and defenses on text-to-image diffusion models: A survey,” Information Fusion, vol. 108, p. 102391, 2024. [8] Y. Yang, R. Gao, X. Yang, J. Zhong, and Q. Xu, “Guardt2i: Defending text-to-image models from adversarial prompts,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 36, 2024. [9] R. Gandikota, J. Materzynska, J. Fiotto-Kaufman, and D. Bau, “Erasing concepts from diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, p. 2426– 2436. 14 [10] “Midjourney,” https://midjourney.com/. [11] CompVis, “Safety checker of stable diffusion,” https://huggingface.co/ CompVis/stable-diffusion-safety-checker. [12] P. Schramowski, C. Tauchmann, and K. Kersting, “Can machines help us answering question 16 in datasheets, and in turn reflecting on inappropriate content?” in Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency (FAccT), 2022, p. 1350– 1361. [13] OpenAI, “Openai moderation,” https://platform.openai.com/docs/guides/ moderation/overview. [14] M. Azure, “Azure ai content safety,” https://azure.microsoft.com/en-us/ products/ai-services/ai-content-safety. [15] A. W. Services, “Aws comprehend,” https://docs.aws.amazon.com/ comprehend/latest/dg/what-is.html. [16] Alibaba, “Aliyun text moderation,” https://help.aliyun.com/document detail/2642717.html. [17] R. Liu, A. Khakzar, J. Gu, Q. Chen, P. Torr, and F. Pizzati, “Latent guard: a safety framework for text-to-image generation,” arXiv preprint arXiv:2404.08031, 2024, accepted to ECCV 2024. [18] Y. Yang, R. Gao, X. Wang, T.-Y. Ho, N. Xu, and Q. Xu, “Mma- diffusion: Multimodal attack on diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, p. 7737–7746. [19] Y. Yang, B. Hui, H. Yuan, N. Gong, and Y. Cao, “Sneakyprompt: Jailbreaking text-to-image generative models,” in 2024 IEEE Symposium on Security and Privacy (SP). IEEE, 2024, p. 897–912. [20] CompVis, “Stable diffusion v1.4,” https://huggingface.co/CompVis/ stable-diffusion-v1-4, 2022. [21] Y. Gandelsman, A. A. Efros, and J. Steinhardt, “Interpreting clip’s image representation via text-based decomposition,” in The Twelfth International Conference on Learning Representations (ICLR), 2024. [22] J. Ho, A. Jain, and P. Abbeel, “Denoising diffusion probabilistic models,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 33. Curran Associates, Inc., 2020, p. 6840–6851. [23] D. Podell, Z. English, K. Lacey, A. Blattmann, T. Dockhorn, J. M ̈ uller, J. Penna, and R. Rombach, “Sdxl: Improving latent diffusion models for high-resolution image synthesis,” arXiv preprint arXiv:2307.01952, 2023. [24] N. Ruiz, Y. Li, V. Jampani, Y. Pritch, M. Rubinstein, and K. Aberman, “Dreambooth: Fine tuning text-to-image diffusion models for subject- driven generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, p. 22 500– 22 510. [25] P. Esser, S. Kulal, A. Blattmann, R. Entezari, J. M ̈ uller, H. Saini, Y. Levi, D. Lorenz, A. Sauer et al., “Scaling rectified flow transformers for high- resolution image synthesis,” in Forty-first International Conference on Machine Learning (ICML), 2024. [26] H. Liu, Y. Wu, S. Zhai, B. Yuan, and N. Zhang, “Riatig: Reliable and imperceptible adversarial text-to-image generation with natural prompts,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, p. 20 585–20 594. [27] Y.-L. Tsai, C.-Y. Hsu, C. Xie, C.-H. Lin, J.-Y. Chen, B. Li, P.-Y. Chen, C.-M. Yu, and C.-Y. Huang, “Ring-a-bell! how reliable are concept removal methods for diffusion models?” in The Twelfth International Conference on Learning Representations (ICLR), 2024. [28] Z.-Y. Chin, C.-M. Jiang, C.-C. Huang, P.-Y. Chen, and W.-C. Chiu, “Prompting4debugging: Red-teaming text-to-image diffusion models by finding problematic prompts,” in Forty-first International Conference on Machine Learning (ICML), 2024. [29] Y. Deng and H. Chen, “Divide-and-conquer attack: Harnessing the power of llm to bypass the censorship of text-to-image generation model,” arXiv preprint arXiv:2312.07130, 2023. [30] Z. Ba, J. Zhong, J. Lei, P. Cheng, Q. Wang, Z. Qin, Z. Wang, and K. Ren, “Surrogateprompt: Bypassing the safety filter of text-to-image models via substitution,” arXiv preprint arXiv:2309.14122, 2023. [31] N. Kumari, B. Zhang, S.-Y. Wang, E. Shechtman, R. Zhang, and J.-Y. Zhu, “Ablating concepts in text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, p. 22 691–22 702. [32] X. Li, Y. Yang, J. Deng, C. Yan, Y. Chen, X. Ji, and W. Xu, “Safe- gen: Mitigating unsafe content generation in text-to-image models,” in Proceedings of the 2024 ACM SIGSAC Conference on Computer and Communications Security (CCS), 2024. [33] S. Poppi, T. Poppi, F. Cocchi, M. Cornia, L. Baraldi, R. Cucchiara et al., “Safe-clip: Removing nsfw concepts from vision-and-language models,” in Computer Vision – ECCV 2024. Springer, 2024. [34] A. Heng and H. Soh, “Selective amnesia: A continual learning approach to forgetting in deep generative models,” in Advances in Neural Infor- mation Processing Systems (NeurIPS), vol. 36. Curran Associates, Inc., 2024, p. 2027–2053. [35] H. Orgad, B. Kawar, and Y. Belinkov, “Editing implicit assumptions in text-to-image diffusion models,” in Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 2023, p. 7030– 7038. [36] Y. Wu, S. Zhou, M. Yang, L. Wang, W. Zhu, H. Chang, X. Zhou, and X. Yang, “Unlearning concepts in diffusion model via concept domain correction and concept preserving gradient,” arXiv preprint arXiv:2405.15304, 2024. [37] Y. Hu, Z. Jiang, and N. Z. Gong, “Safetext: Safe text-to-image models via aligning the text encoder,” arXiv preprint arXiv:2502.20623, 2025. [38] P. Schramowski, M. Brack, B. Deiseroth, and K. Kersting, “Safe latent diffusion: Mitigating inappropriate degeneration in diffusion models,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2023, p. 22 522–22 531. [39] H. Li, C. Shen, P. Torr, V. Tresp, and J. Gu, “Self-discovering interpretable diffusion latent directions for responsible text-to-image generation,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2024, p. 12 006–12 016. [40] michellejieli, “Nsfw-text-classifier,” https://huggingface.co/michellejieli/ NSFW textclassifier, 2022. [41] unitaryai, “Detoxify,” https://github.com/unitaryai/detoxify, 2022. [42] “Leonardo.ai,” https://leonardo.ai/. [43] A. Radford, J. W. Kim, C. Hallacy, A. Ramesh, G. Goh, S. Agarwal, G. Sastry, A. Askell, P. Mishkin, J. Clark et al., “Learning transferable visual models from natural language supervision,” in International Conference on Machine Learning (ICML).PMLR, 2021, p. 8748– 8763. [44] U. Bhalla, A. Oesterling, S. Srinivas, F. P. Calmon, and H. Lakkaraju, “Interpreting clip with sparse linear concept embeddings (splice),” in Advances in Neural Information Processing Systems (NeurIPS), vol. 36. Curran Associates, Inc., 2024. [45] C. Zhao, K. Wang, X. Zeng, R. Zhao, and A. B. Chan, “Gradient- based visual explanation for transformer-based clip,” in Forty-first In- ternational Conference on Machine Learning (ICML).PMLR, 2024, p. 61 072–61 091. [46] E. Aflalo, M. Du, S.-Y. Tseng, Y. Liu, C. Wu, N. Duan, and V. Lal, “Vl-interpret: An interactive visualization tool for interpreting vision- language transformers,” in Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2022, p. 21 374– 21 383. [47] H. Abdi and L. J. Williams, “Principal component analysis,” Wiley Interdisciplinary Reviews: Computational Statistics, vol. 2, no. 4, p. 433–459, 2010. [48] A. Vaswani, N. Shazeer, N. Parmar, J. Uszkoreit, L. Jones, A. N. Gomez, Ł. Kaiser, and I. Polosukhin, “Attention is all you need,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 30. Curran Associates, Inc., 2017, p. 5998–6008. [49] N. Elhage, N. Nanda, C. Olsson, T. Henighan, N. Joseph, B. Mann, A. Askell, Y. Bai, A. Chen, T. Conerly et al., “A mathematical framework for transformer circuits,” Transformer Circuits Thread, vol. 1, 2021, https://transformer-circuits.pub/2021/framework/index.html. [50] S. Balakrishnama and A. Ganapathiraju, “Linear discriminant analysis: A brief tutorial,” Institute for Signal and Information Processing, Mis- sissippi State University, Tech. Rep., 1998. [51] C. Raffel, N. Shazeer, A. Roberts, K. Lee, S. Narang, M. Matena, Y. Zhou, W. Li, and P. J. Liu, “Exploring the limits of transfer learning with a unified text-to-text transformer,” Journal of Machine Learning Research, vol. 21, no. 1, p. 1–67, 2020. [52] T.-Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Doll ́ ar, and C. L. Zitnick, “Microsoft coco: Common objects in context,” in Computer Vision–ECCV 2014.Springer, 2014, p. 740– 755. [53] “Lexica,” https://lexica.art/. [54] “4chan,” https://w.4chan.org/. [55] C. Schuhmann, R. Beaumont, R. Vencu, C. Gordon, R. Wightman, M. Cherti, T. Coombes, A. Katta et al., “Laion-5b: An open large-scale dataset for training next generation image-text models,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 35.Curran Associates, Inc., 2022, p. 25 278–25 294. [56] T. Markov, C. Zhang, S. Agarwal, F. Eloundou Nekoul, T. Lee, S. Adler, A. Jiang, and L. Weng, “A holistic approach to undesired content detection in the real world,” in Proceedings of the AAAI Conference on Artificial Intelligence, vol. 37, no. 12, 2023, p. 15 009–15 018. 15 [57] D. M. Powers, “Evaluation: From precision, recall and f-measure to roc, informedness, markedness & correlation,” School of Informatics and Engineering, Flinders University, Adelaide, Australia, Tech. Rep. WS06-01, 2011. [58] T. Fawcett, “An introduction to roc analysis,” Pattern Recognition Letters, vol. 27, no. 8, p. 861–874, 2006. [59] OpenAI,“clip-vit-large-patch14,”https://huggingface.co/openai/ clip-vit-large-patch14. [60] laion,“Clip-vit-bigg-14-laion2b-39b-b160k,”https://huggingface.co/ laion/CLIP-ViT-bigG-14-laion2B-39B-b160k. [61] google, “t5-v11-xxl,” https://huggingface.co/google/t5-v11-xxl. [62] J. Hessel, A. Holtzman, M. Forbes, R. Le Bras, and Y. Choi, “Clipscore: A reference-free evaluation metric for image captioning,” in Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP), 2021, p. 7514–7528. [63] X. Qi, Y. Huang, Y. Zeng et al., “Ai risk management should incorporate both safety and security,” arXiv preprint arXiv:2405.19524, 2024.