Paper deep dive
Efficient LLM Moderation with Multi-Layer Latent Prototypes
Maciej ChrabÄ szcz, Filip Szatkowski, Bartosz WĆ³jcik, Jan DubiÅski, Tomasz TrzciÅski, Sebastian Cygert
Models: Llama-3.1-8B, Mistral-7B, OLMo2-7B, Qwen3-8B
Intelligence
Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 95%
Last extracted: 3/12/2026, 5:26:30 PM
Summary
The paper introduces Multi-Layer Prototype Moderator (MLPM), a lightweight, latent-based input moderation tool for LLMs. MLPM leverages Mahalanobis distance-based Gaussian Discriminant Analysis (GDA) on intermediate layer representations to classify prompts as safe or harmful. It achieves state-of-the-art performance, outperforming existing guard models and latent-based methods while maintaining negligible inference overhead and high scalability across model families.
Entities (5)
Relation Signals (3)
MLPM ā moderates ā Large Language Models
confidence 100% Ā· MLPM, a lightweight and highly customizable input moderation tool.
MLPM ā uses ā Mahalanobis distance
confidence 95% Ā· We achieve superior performance by unifying the Mahalanobis distance-based classifier with a multi-layer prototype strategy.
MLPM ā trainedon ā WildGuardMix
confidence 90% Ā· We always utilize the training set of WildGuardMix for training MLPM.
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Although modern LLMs are aligned with human values during post-training, robust moderation remains essential to prevent harmful outputs at deployment time. Existing approaches suffer from performance-efficiency trade-offs and are difficult to customize to user-specific requirements. Motivated by this gap, we introduce Multi-Layer Prototype Moderator (MLPM), a lightweight and highly customizable input moderation tool. We propose leveraging prototypes of intermediate representations across multiple layers to improve moderation quality while maintaining high efficiency. By design, our method adds negligible overhead to the generation pipeline and can be seamlessly applied to any model. MLPM achieves state-of-the-art performance on diverse moderation benchmarks and demonstrates strong scalability across model families of various sizes. Moreover, we show that it integrates smoothly into end-to-end moderation pipelines and further improves response safety when combined with output moderation techniques. Overall, our work provides a practical and adaptable solution for safe, robust, and efficient LLM deployment.
Tags
Links
Trouble viewing inline? Open PDF directly ā
Full Text
86,471 characters extracted from source content.
Expand or collapse full text
Efficient LLM Moderation with Multi-Layer Latent Prototypes Maciej Chrab ĢØaszcz 1 2 Filip Szatkowski 2 Bartosz WĆ³jcik 3 4 Jan Dubi Ģ nski 1 2 Tomasz Trzci Ģ nski 2 4 5 Sebastian Cygert 1 6 Abstract Although modern LLMs are aligned with human values during post-training, robust moderation remains essential to prevent harmful outputs at de- ployment time. Existing approaches suffer from performance-efficiency trade-offs and are diffi- cult to customize to user-specific requirements. Motivated by this gap, we introduce Multi-Layer Prototype Moderator (MLPM), a lightweight and highly customizable input moderation tool. We propose leveraging prototypes of intermediate rep- resentations across multiple layers to improve moderation quality while maintaining high effi- ciency. By design, our method adds negligible overhead to the generation pipeline and can be seamlessly applied to any model. MLPM achieves state-of-the-art performance on diverse modera- tion benchmarks and demonstrates strong scalabil- ity across model families of various sizes. More- over, we show that it integrates smoothly into end- to-end moderation pipelines and further improves response safety when combined with output mod- eration techniques. Overall, our work provides a practical and adaptable solution for safe, robust, and efficient LLM deployment. 1. Introduction Large language models (LLMs) have quickly become cen- tral to modern applications, making their safety and align- ment with human values increasingly important. While techniques like RLHF (Bai et al., 2022; Ouyang et al., 2022) and instruction tuning (Li et al., 2025) have substantially improved model safety, even state-of-the-art models remain susceptible to emergent risks even for aligned models (An- driushchenko et al., 2025; Carlini et al., 2023; Liu et al., 2023). As a result, ensuring practical LLM safety requires 1 NASK National Research Institute, Warsaw, Poland 2 Warsaw University of Technology, Warsaw, Poland 3 Jagiellonian Univer- sity, Cracow, Poland 4 Tooplox 5 IDEAS Research Institute, Warsaw, Poland 6 Gda Ģ nsk University of Technology, Gda Ģ nsk, Poland. Corre- spondence to: Maciej Chrab ĢØaszcz <maciej.chrabaszcz@nask.pl>, Filip Szatkowski <filip.szatkowski.dokt@pw.edu.pl>. Preprint. February 9, 2026. an additional evaluation of the model inputs and responses. This has motivated recent advances in moderation tools (Lee et al., 2025; Zheng et al., 2024), which are considered an es- sential component of safe deployment of the LLM systems. The most common moderation tools can be broadly cate- gorized as either specialized guard models or latent-based methods. In practice, the choice between these approaches typically reflects preferences over training cost, perfor- mance, and efficiency. Guard models (Dong et al., 2024; Ghosh et al., 2024; Han et al., 2024; Inan et al., 2023; Sharma et al., 2025) offer good performance, but introduce additional model into the moderation pipeline, complicating the deployment and increasing its cost. Training guards is resource-heavy and requires carefully curated datasets, limiting most users to a fixed set of pre-trained models. On the other hand, latent-based methods (Ayub & Majumdar, 2024; Abdelnabi et al., 2025) are usually lightweight, but offer worse performance than guard models. None of these approaches fully satisfies the combined requirements of performance, efficiency, and adaptability to custom safety policies, which are essential for real-world deployment. We address the above-mentioned gap with Multi-Layer Pro- totype Moderator (MLPM), a lightweight, latent-based input moderation approach that uniquely combines the efficiency and flexibility inherent to latent-based methods with state-of- the-art performance exceeding that of the best guard models. We provide a high-level comparison of the existing meth- ods and our approach in Table 1. MLPM uses the internal states of off-the-shelf LLMs to assess input safety via dis- tance to safe and unsafe prototypes. We achieve superior performance by unifying the Mahalanobis distance-based classifier (Goswami et al., 2023) with a multi-layer proto- type strategy. This approach allows us to harness the diverse semantic information distributed across intermediate lay- ers (Masarczyk et al., 2023; Szatkowski et al., 2025; Zou et al., 2023a) for more accurate and robust detection. This holistic view enables the correct classification of inputs that appear ambiguous when observing any single layer in iso- lation. MLPM delivers guard-level performance, is model- agnostic, and adds minimal compute and memory overhead at inference. Furthermore, it can be trained cheaply and efficiently, even in data-constrained scenarios, achieving guard-level performance with as few as 1,000 samples. 1 arXiv:2502.16174v3 [cs.LG] 6 Feb 2026 Efficient LLM Moderation with Multi-Layer Latent Prototypes Table 1. High-level conceptual comparison of existing input moderation approaches and our method. MLPM is able to uniquely combine low training cost, efficient inference, and flexibility inherent to latent-based approaches with state-of-the-art moderation performance. Training Cost Data-Efficiency Inference Efficiency Memory Footprint Flexibility Safety Assessment Performance Guard ModelsHighLowLowHighLowHigh Latent-based methodsLowHighHighLowHighMedium MLPM (ours)LowVery HighHighLowHighHigh We evaluate MLPM across diverse input moderation bench- marks and demonstrate that it achieves state-of-the-art per- formance, outperforming the alternative guard and latent- based approaches across various model families and sizes. We further demonstrate how MLPM can be easily integrated into end-to-end moderation pipelines alongside output mod- eration tools, enhancing the overall safety of LLM systems. Finally, through detailed ablations, we investigate the robust- ness of our method in low-data regimes, out-of-distribution scenarios, and the intriguing multi-layer dynamics underly- ing its performance. Our key contributions are: ā¢We introduce MLPM, an efficient LLM input moder- ation approach that uses Mahalanobis distance across multi-layer representations to assess prompt safety. ā¢ We demonstrate that MLPM achieves state-of-the-art performance, surpassing existing latent-based methods and Guard models across diverse benchmarks. ā¢ We show how MLPM seamlessly integrates as a condi- tioning signal for steering methods, reducing unwanted refusals and enhancing end-to-end deployment safety. Taken together, our work provides a state-of-the-art, effi- cient, and customizable solution for LLM moderation. 2. Related Work LLM alignment and safety. LLMs are usually pre- trained on large corpora of data that are impossible to fully supervise. Therefore, pre-training is typically followed by supervised finetuning that ensures the alignment of the model with human preferences and values (Bai et al., 2022; Dai et al., 2024; Li et al., 2025; Lim et al., 2025; Ouyang et al., 2022). However, various studies prove that even the most popular frontier models are still prone to generating unsafe responses (Carlini et al., 2023; Liu et al., 2023; Zou et al., 2023b), as safety alignment can be superficial and lose its effect after the initial few tokens of generation (Qi et al., 2025). Furthermore, focusing strictly on the safety alignment might negatively affect model capabilities (Huang et al., 2025; Wei et al., 2023; Wolf et al., 2024). LLM moderation.Moderation techniques ensure LLMs comply with established guidelines while preserving their capabilities and output quality. Generally, moderation can be applied either to the model output during text generation or to the input requests before the generation starts. Output moderation assesses the already generated LLM responses, and typically relies on guard models (Inan et al., 2023; Han et al., 2024; Ghosh et al., 2024; Sharma et al., 2025; Yin et al., 2025), rule-based approaches (Clarke et al., 2023; Kumar et al., 2024), prompt engineering (Zheng et al., 2024; Xie et al., 2023), activation steering (Zou et al., 2023a; Luo et al., 2024; Lee et al., 2025; Qiu et al., 2024), or specialized fine-tuning (Zou et al., 2024; Zhang et al., 2024). While effective at ensuring the safety of responses, steering tech- niques can often hinder the modelās capabilities (Lee et al., 2025). Refusing to engage with malicious prompts is often a sufficient and efficient approach (Manczak et al., 2024), which motivated the development of input moderation strate- gies that evaluate prompts before generation. Common input moderation techniques employ guard models or latent-based methods that leverage model representations to detect mali- cious content (Abdelnabi et al., 2025; Ayub & Majumdar, 2024). Guard models offer high performance at a high cost, while latent methods are lightweight but less effective. Our method presents a novel latent-based approach to input moderation, combining prototype-based classification with multi-layer feature aggregation to achieve guard-level state- of-the-art performance, while remaining efficient and easily adaptable to specific safety policies. 3. Multi-Layer Prototype Moderator The goal of input moderation is to assess whether the input promptxbelongs to the class of harmful promptsX harm , which can lead the LLM to produce an unsafe response. This allows the generation process to be halted, ensuring safe interaction and avoiding unnecessary computation. In this section, we present Multi-Layer Prototype Moderator (MLPM), a latent-based input moderation approach that combines high-tier performance and efficiency. By utiliz- ing multi-layer representations and prototype classification, MLPM provides state-of-the-art safety assessment for any model of the userās choice. At the same time, our design ensures that our method remains flexible and easy to deploy, 2 Efficient LLM Moderation with Multi-Layer Latent Prototypes Calculate sparse aggregation weights Training Labeled Dataset Model Layer L Layer i Layer 1 ... ... Layer j ... ... Prompt ... ... ... ... ... ... Calculate prototypes Model Layer L Layer 1 Layer i ... Layer j ... ... ... Inference Figure 1. Our proposed Multi-Layer Prototype Moderator (MLPM) framework. During training, we compute class-conditional prototypes (Ī¼ i , Ī£ i ) based on last-token hidden representations (h i,T ), which we use to define a per-layer Gaussian Discriminant Analysis (GDA) classifier. We then learn sparse aggregation weights (w i ) over the GDA scores. During inference, we use the pre-trained GDA classifiers to compute classification scores from layers with non-zero weights (|w i | > 0), and produce a safety probability,P (unsafe), from their weighted aggregate. MLPM enables state-of-the-art performance with lightweight training and negligible inference overhead. even when computational resources and data are scarce. An overview of our method is shown in Figure 1. 3.1. Prototype-based classification Nearest Mean Classifier (NMC) is widely used in deep learning due to its interpretable decision boundaries, strong generalization, and data-efficiency (Wang et al., 2020b; Xian et al., 2017; Rebuffi et al., 2017; Åapacz et al., 2025). How- ever, despite its effectiveness, prior to our work, NMC has not previously been applied to LLM safety. NMC assigns a classcto the new samplexbased on the distance between the representations ofxand the prototypes of all the classes embedded in the latent space. The latent representation is obtained via feature extractorg, which maps the inputx to a latent vectorh = g(x). In most cases, the extractor corresponds to a part of the neural network, though in princi- ple it can be any function that produces useful embeddings. The prototypes of all classes are computed as the empirical means of their corresponding samples in the latent space: Ī¼ c = 1 N c N c X i=1 h (c) i ,(1) whereh (c) i = g(x (c) i )is the latent representation of the i-th training example in classc, andN c is the number of examples in that class. The simplest NMC determines the class ofxby comparing the distances between h = g(x) and the class prototypes: c = arg min jā1,...,C d h,Ī¼ j ,(2) where C refers to the number of considered classes. The performance of NMC depends mainly on the choice of the feature extractor and the distance metric. While Eu- clidean distance is the most straightforward option, numer- ous variants of NMC have explored alternative metrics to improve performance. In particular, Mahalanobis distance has been shown to generalize better and provide increased robustness (Wang et al., 2020a; Goswami et al., 2023; Wang et al., 2024). By accounting for data variance, the Maha- lanobis distance captures the underlying geometry of the latent space, enabling more accurate discrimination between clusters with different shapes and orientations (Lee et al., 2018). Motivated by the mentioned properties, we adopt Mahalanobis distance in our method, as it is better suited to capture the complex structure of LLM representations 1 . Mahalanobis distance between a vectorh āR d and the prototype represented by meanĪ¼ c also accounts for the covariance matrix Ī£ c , and is defined as: d M (h,Ī¼ c , Ī£ c ) = q (hā Ī¼ c ) ā¤ Ī£ ā1 c (hā Ī¼ c ),(3) where we estimateĪ£ ā1 c with a Bayes ridge-type estima- tor (Kubokawa & Srivastava, 2008). With the choice of Mahalanobis distance, NMC can be altered to estimate class probabilities via Gaussian Discrim- inant Analysis (GDA), where the probability that an inputx with latent representation h = g(x) belongs to class c is: P(c|x,Ī¼ c , Ī£ c ) = exp(d M (h,Ī¼ c , Ī£ c )) P k i=1 exp (d M (h,Ī¼ i , Ī£ i ) .(4) For safety assessment in our setting, we classify inputs be- tween two classes,X safe andX harm . We use the LLM as 1 Our intuition is further supported empirically in Table 4 3 Efficient LLM Moderation with Multi-Layer Latent Prototypes our feature extractor, and we only use the representation corresponding to the last token in the prompt. Importantly, at inference time, we leverage the hidden states already computed by the model during the prefill stage, so the only additional overhead introduced by MLPM is the negligi- ble cost of performing GDA. To further reduce memory requirements, we use a shared covariance matrix Ī£ c = Ī£. 3.2. Combining multi-layer prototypes As discussed in the previous sections, representations from different layers capture diverse information that can be infor- mative for safety assessment. Motivated by this observation, MLPM leverages intermediate representations to improve moderation performance. To this end, we apply the GDA procedure described in Section 3.1 to a selected subset of intermediate layers and perform a weighted aggregation of the resulting predictions from the selected subset of layers. Specifically, to construct MLPM classifier, we extract the final token representations at the outputs of the feed-forward networks (FFNs) from each of theLtransformer blocks in the LLM. For each layerl, we compute class-conditional prototypes parameterized by the meanĪ¼ l c and inverse of shared covariance matrix(Ī£ l ) ā1 . Substituting the layer- specific representations forhin Equation (4), we obtain layer-wise GDA classifiers that calculate probabilityP l (xā X harm |x,Ī¼ l , Ī£ l ) of the input sample x being harmful. Since intermediate-layer representations differ in their rel- evance to safety assessment and may encode overlapping information, we aggregate layer-wise GDA predictions us- ing learned weights that select informative layers and control their influence on the final prediction. Formally, the prob- ability assigned by MLPM to a samplexis computed as a weighted aggregation of layer-wise GDA predictions: MLPM(x) = Ļ L X l=1 w l Ā·P l (xāX harm |x,Ī¼ l , Ī£ l ) , (5) wherew = w 1 ,...,w L are aggregation weights andĻis a sigmoid function. We learn these weights with anā 1 reg- ularization penalty onw. We optimize this objective over the entire training dataset, with the regularization strength controlled by the hyperparameterC. This penalty promotes sparsity and robustness in the aggregation, mitigating re- dundancy among similar layers and avoiding unnecessary computation. Consequently, MLPM is able to leverage informative signals from distinct representations, thereby improving overall robustness and performance. 3.3. Practical implications of MLPM design MLPM is designed to maximize the safety assessment per- formance, and at the same time minimize both computa- tional and memory overhead. Due to its low training cost and high data efficiency, our method can be applied to any LLM of choice, does not require substantial resources, and only slightly increases the complexity of model inference, providing a lightweight, flexible and self-contained solution. 3.3.1. TRAINING EFFICIENCY The training process of MLPM is notably lightweight, as it uses the representation of single, last-token, and requires only a single forward pass through the prompt dataset, with- out gradient calculations or text generation. The prototype calculation and computation of aggregation weights can run quickly even on a CPU. As shown in Section 4.3, MLPM is also remarkably data-efficient and can perform safety assessment effectively even when trained on a small dataset. 3.3.2. INFERENCE EFFICIENCY At the inference time, the overhead of MLPM is negligible, as the intermediate representations are already computed when prefilling the prompt. We can estimate the computa- tional overhead of MLPM during inference by analyzing the ratio of FLOPs required for safety assessment with our method relative to the total prefill FLOPs; in the worst cases, this overhead amounts to less than0.001%of the prefill compute. In addition to its low computational cost, MLPM is highly memory-efficient, using justā¼ 24KB per pro- totype stored in half-precision for the Llama3.1-8B model. Ultimately, the combination of negligible computational and memory inference overhead establishes MLPM as a highly practical solution for safety enforcement during LLM de- ployment. See Appendix L for the details on our estimation. 4. Experiments In this section, we provide a detailed comparison of MLPM with alternative input moderation approaches, focusing on the most relevant guard models and latent-based methods. Among the guard models, we use Aegis-Defensive (Ghosh et al., 2024), LlamaGuard3 (Inan et al., 2023), Granite Guardian (Padhi et al., 2024), ShieldGemma (Zeng et al., 2024), and WildGuard (Han et al., 2024). We also use latent- based approaches such as Abdelnabi et al. (2025) and Ayub & Majumdar (2024). As base models for latent-based meth- ods (including MLPM) we use Mistral (Jiang et al., 2023), Llama (Grattafiori et al., 2024), OLMo (OLMo et al., 2024), and Qwen3 (Yang et al., 2025), which allows us to assess our approach across diverse models. We evaluate the methods on 8 prompt harmfulness datasets, including WildJailbreak (Jiang et al., 2024) and WildGuard- Mix (Han et al., 2024). Those datasets in particular contain unique, sophisticated state-of-the-art jailbreak tactics. We always utilize the training set of WildGuardMix (Han et al., 2024) for training MLPM and other latent-based methods 4 Efficient LLM Moderation with Multi-Layer Latent Prototypes Table 2. F1 score on harmful datasets. MLPM consistently outperforms prior latent-based approaches when applied to the same model. Additionally, MLPM surpasses resource-heavy guard models, including LlamaGuard 3, ShieldGemma, and GraniteGuardian. Notably, when applied to OLMo2, MLPM achieves the highest overall performance, outperforming even the strongest guard baseline, WildGuard. DatasetAegisHarmBOpenAISimpSTTChatWGMixWJBXSTestAverage Latent-Based Llama-8B-Inst+Ayub & Majumdar (2024)82.5296.9866.6098.4855.6280.9182.8592.7682.09 Llama-8B-Inst+Abdelnabi et al. (2025)84.1695.1867.9998.9959.5986.2793.2790.6384.51 Llama-8B-Inst+MLPM(Ours)85.13 99.5872.8599.5069.1788.0494.6997.4488.30 Qwen3-8B-Inst+Ayub & Majumdar (2024)80.0090.6274.5695.2968.9080.6481.3190.2182.69 Qwen3-8B-Inst+Abdelnabi et al. (2025)84.4799.3768.4597.9660.9485.1890.4488.0684.36 Qwen3-8B-Inst+MLPM(Ours)83.49100.072.3596.9164.4086.2192.0092.2385.95 Mistral-7B-Inst+Ayub & Majumdar (2024)79.6090.8775.6998.9963.4483.3187.5395.0484.31 Mistral-7B-Inst+Abdelnabi et al. (2025)84.5597.8664.5998.4857.6385.7391.1394.6084.32 Mistral-7B-Inst+MLPM(Ours)87.3699.1670.68 99.5066.3387.6393.6596.1087.55 OLMo2-7B-Inst+Ayub & Majumdar (2024)88.1196.5466.95100.065.6388.0996.8494.3387.06 OLMo2-7B-Inst+Abdelnabi et al. (2025)84.1693.3075.1999.5072.5386.2093.4894.4387.35 OLMo2-7B-Inst+MLPM(Ours) 89.2398.5174.21100.076.5188.5297.5596.9190.18 Guard Models Aegis-Guard-D (Ghosh et al., 2024)81.0070.4676.4497.9675.6172.0975.4481.5378.82 LlamaGuard3 (Inan et al., 2023)71.7498.9479.11 99.5054.1176.7667.8388.5279.56 GraniteGuardian-3-1-8B (Padhi et al., 2024)87.7879.9077.6399.5073.2584.5796.7585.5985.62 ShieldGemma-9B (Zeng et al., 2024)77.4469.0477.6391.3068.1358.8859.9482.4173.10 WildGuard (Han et al., 2024)89.7899.3772.2899.5070.1488.0497.1095.2688.93 1B7B32B70B Active params 72.5 75.0 77.5 80.0 82.5 85.0 87.5 90.0 Harmful Detection F1 Guards Aegis-D LlamaGuard3 GraniteGuard ShieldGemma WildGuard MLPM Mistral Llama3 OLMo2 Qwen3 +MoE Figure 2. MLPM outperforms Guard models through effective scaling. Unlike fixed-size Guard models (represented by crosses), MLPM scales seamlessly with the backbone model, achieving superior harmfulness detection across diverse architectures. training, if not stated otherwise. Across our experiments, we report the average F1 score on harmful datasets, which contain both safe and unsafe inputs. For non-deterministic approaches, we show the average results from 5 runs. For more details on the datasets, see Appendix A. 4.1. Input moderation across model families and sizes In Table 2, we evaluate MLPM applied to the instruction- finetuned Llama, Mistral, OLMo, and Qwen3 models. We benchmark our method against well-established Guard mod- els, as well as other latent-based methods. MLPM con- sistently outperforms most guards and other latent-based methods. When applied to OLMo, MLPM surpasses even the strongest guard baseline, WildGuard. Our results demon- strate that MLPM can efficiently perform safety assessment at a level comparable to the best available alternatives, while remaining remarkably lightweight. In Appendix C.1 we also show that MLPM does not incur a lot of false positives, and triggers only forā¼ 1% samples on benign prompts. We demonstrated that MLPM outperforms alternative ap- proaches on a diverse set of moderation tasks when con- sidering the common 7-8B model range. However, LLM performance is known to scale with model size and training data (Kaplan et al., 2020), and larger models typically of- fer greater capabilities; consequently, LLMs are typically released as families of varying sizes, enabling users to opti- mize for the trade-off between performance and efficiency. To validate if our approach shows similar scaling properties with the quality of base model, we evaluate MLPM across diverse model families, including Llama3, Mistral, OLMo2, and Qwen3, as shown in Figure 2. This evaluation includes MoE variants (Shazeer et al., 2017): OLMo2 (7B with 1B active parameters) and Qwen3 (30B with 3B active parame- ters). Figure 2 demonstrates that the performance of MLPM improves consistently with model size. Notably, MLPM pushes the performance-efficiency frontier: our 1B Llama3 variant achieves an F1 score ofā¼85, outperforming sig- nificantly larger fixed-size baselines such as GraniteGuard and ShieldGemma. Furthermore, while the MoE variants exhibit marginally lower raw performance than their dense counterparts, they maintain high detection capabilities with 5 Efficient LLM Moderation with Multi-Layer Latent Prototypes Table 3. Attack success rate (ASR) and false refusal rate (FRR) when combining MLPM with output moderation methods. Our method improves output moderation tools by decreasing FRR when using MLPM as a conditioning mechanism. We additionally show an F1 score between (1-ASR) and (1-FRR) to analyse the trade-off between the two moderation objectives captured by these metrics. We highlight the cases where MLPM increases or decreases the F1 score with green and red colors respectively. Llama3-8BMistral-7BOLMo2-7BQwen3-8B MethodASRāFRRāF1āASRāFRRāF1āASRāFRRāF1āASRāFRRāF1ā Base Model36.212.5474.7578.251.9034.5120.825.9380.4850.532.2263.78 +MLPM Simple Refuse15.525.9383.4715.926.2482.8414.596.0383.8518.306.2481.52 Lee et al. (2025)36.477.5168.7173.616.2437.3120.426.5679.9050.132.6563.66 Turner et al. (2023)35.9453.7627.2040.4518.7354.2918.578.1578.9047.753.8164.41 +MLPM Conditioning44.564.6666.0345.893.6066.1519.366.4680.6348.672.5465.02 Zheng et al. (2024)25.076.0377.8254.113.4959.3019.768.2578.1228.915.7175.83 +MLPM Conditioning26.924.0279.2155.442.4359.1620.956.3579.8631.563.9276.30 a fraction of the active parameters, confirming that MLPM is compatible with this architecture (Liu et al., 2024). 4.2. Combining MLPM with output moderation Building on the complementary nature of input and output moderation, we evaluate MLPMās efficacy when integrated with steering methods. A key motivation for this analysis is the finding by Lee et al. (2025) that steering can degrade performance on benign prompts. We therefore compare two scenarios: one where steering is applied to all prompts, versus another where steering is conditionally activated only when MLPM flags a prompt as unsafe. We also evaluate the influence of returning a simple refusal message to determine how our MLPM method works on its own. Following Zheng et al. (2024), we utilize a safety prompt as a prompt steering baseline (see Appendix D.1 for the details). To assess performance, we evaluate the responses for harm- fulness using the WildGuard model on the WildGuardMix test set. The activation steering methods are derived by sampling 5,000 harmful and 5,000 safe responses from the WildGuardMix training set. We calculate the attack success rate (ASR) on harmful prompts, the false refusal rate (FRR) on benign prompts, and the F1 scores for (1-ASR) and (1- FRR). As shown in Table 3, using MLPM as a conditioning mechanism substantially reduces false refusal rates on safe prompts, with only a marginal increase in the generation of harmful content. Moreover, MLPM with a simple re- fusal message demonstrates the superior F1 score across the entire model suite. These results highlight the possibility of using MLPM as either an input filter or a conditioning mechanism for steering methods, while exceeding the per- formance of the conditioning proposed by Lee et al. (2025). The decision threshold of MLPM can be easily tuned to meet specific requirements, further highlighting the flexibility of our method as a part of a larger safety system. 4.3. Training specifics of MLPM Low computational cost and data-efficiency of the training were our core objectives behind the design of MLPM. There- fore, in this section, we investigate in more detail the impact of the training data on the performance of our method. Generalization properties.We evaluate the robustness of MLPM by measuring its performance on in-distribution (ID) and out-of-distribution (OOD) data when trained on differ- ent datasets. Specifically, we use three datasets from Table 2 with publicly available training splits (Aegis, ToxicChat, and WildGuardMix) and use them for training the latent- based methods and MLPM. We then compare performance on the training (ID) datasets alongside the average perfor- mance on the remaining 5 (OOD) datasets in Figure 3a. For comparison, we also train on the combined datasets. MLPM consistently outperforms the other latent-based ap- proaches, both on the in-distribution and out-of-distribution data. While other methods often see a significant perfor- mance drop when evaluating on unseen data, our method maintains high detection accuracy, suggesting that it cap- tures general safety signals better than simpler approaches and justifying our multi-layer prototype-based approach. Our results highlight the flexibility of MLPM, which not only achieves the best performance against threats in the training data but also generalizes best to unseen threats. Note how this enables users to achieve the best possible safety for their specific data at a remarkably low cost, espe- cially compared to guard models, which require finetuning a full LLM. We observe similar trends across other models, and provide the detailed results for them in Appendix E. Data-efficiency. Another important property of MLPM, especially relevant for low-resource settings, is its remark- able data efficiency enabled by the use of Mahalanobis- based NMC. We examine this by measuring the performance of our method trained with different numbers of examples. In particular, we measure the F1 score on WildGuardMix 6 Efficient LLM Moderation with Multi-Layer Latent Prototypes Aeg(ID)TC (ID)WG (ID)OOD 0.5 1.0 F1 Score Training data: Aegis Aeg(ID)TC (ID)WG (ID)OOD Training data: Toxichat Aeg(ID)TC (ID)WG (ID)OOD 0.5 1.0 F1 Score Training data: WildGuardMix Aeg(ID)TC (ID)WG (ID)OOD Training data: All Data Abdelnabi et al.Ayub & MajumdarMLPM(Ours) (a) ID vs OOD performance on OLMo2-7B. 1010010001000040000 Num Samples Per Class 65 70 75 80 85 F1 Score on WGMix Llama3-8B Mistral-7B OLMo2-7B Qwen3-8B (b) Scaling with training data size. Llama3-8BMistral-7BOLMo2-7BQwen3-8B 65 70 75 80 85 90 F1 Score Evaluation data and model type WGMix All w/o WGMix Pretrained Instruct (c) MLPM with intruct and base models. Figure 3. a) Performance comparison of MLPM against other latent-based methods on In-Distribution (ID) and Out-Of-Distribution (OOD) sets. MLPM consistently outperforms baselines in both settings, demonstrating the efficacy of utilizing multiple layers. b) MLPM performs well even in limited data settings, offering reasonable effectiveness even in data-scarce scenarios. c) While pretrained representations prove adequate for in-distribution examples, they fail to generalize to out-of-distribution, unlike instruction models. across training sets of varying sizes. We show the results of this experiment for four different models in Figure 3b. Our method remains accurate even with small training datasets and achieves competitive performance with as few as 1000 samples. The variance in its performance also drops significantly as the sample size increases. These results demonstrate MLPMās practicality in data-scarce scenarios, and prove that it can be used to adapt the moderation strat- egy to new threats even with only a few examples available. Note how alternative latent-based methods achieve worse scaling properties, which we show in detail in Appendix G. Together with the previously shown generalization capabil- ities, these results confirm how our approach is not only efficient and performant but also remarkably flexible. 4.4. MLPM With Base and Reasoning Models Our initial analysis focuses on instruction-finetuned models, which are typically safety-aligned and more likely to encode safety-relevant information in their latent spaces. In this section, we evaluate MLPM on reasoning and pre-trained models, to assess the general applicability of our approach. Pretrained and instruction-tuned models. We com- pare the performance of MLPM on both pretrained and instruction-finetuned models to investigate at what stage of model development safety signals begin to emerge, and if in- struction tuning is necessary for our method to be effective. Specifically, we train our method on WGMix, using both instruct and base variants of four common models. Then, in Figure 3c, we report the F1 score with both model types on in-distribution WGMix data and out-of-distribution data from all the other remaining benchmarks used in Table 2. Interestingly, the gap between the in- and out-of-distribution performance is consistently smaller for the instruct mod- els, suggesting that pretrained base models can provide sufficient-quality representations for examples similar to those used for training, but do not generalize that well. While these results show that MLPM can be applied to pretrained models, our experiments suggest that instruction- finetuned models are better suited for use with our method. Reasoning models. We investigate whether MLPMās ef- fectiveness extends to reasoning models and whether lever- aging the tokens from the thinking chain provides additional safety signals. Specifically, we compare MLPM applied to the representation of the last prompt token, as in the previous experiments, to the performance of MLPM applied to the final end-of-thinking token. We conduct experiments on sev- eral Qwen3 models and report F1 scores on WildGuardMix in Figure 4. To provide a comparison with instruction-tuned models used in our previous experiments, we also include re- sults for Qwen3-4B-Instruct and Qwen3-30B-A3B-Instruct. 1.7B-T30A3B-I30A3B-T4B-I4B-T8B-T14B-T 80 85 90 Score Last Prompt TokenEOT Token Figure 4. WGMix F1 obtained with MLPM for Qwen3 Instruct (-I) and Thinking (-T) models, using either the end of the prompt (Last Prompt Token) or the end of thinking token (EOT Token). While using the end-of-thinking token for MLPM yields a slight performance improvement (up to 0.5%) for models larger than 1.7B, this comes at the cost of generating a long chain of reasoning tokens. In contrast, using MLPM with the last prompt token already provides a robust safety as- sessment for the reasoning model, and for smaller models, the last token representation yields better performance. Our findings suggest that the essential safety signal is largely present in the initial prompt representation, and that the rea- soning process does not significantly enhance our methodās performance. In practice, this shows the test-time scaling capability of MLPM when applied to the moderation of reasoning models and further underscores the flexibility of our method: the user can trade off additional computation 7 Efficient LLM Moderation with Multi-Layer Latent Prototypes 0.00.20.40.60.81.0 Fraction of Layers Used 0.82 0.84 0.86 0.88 0.90 F1 Score Llama3-8B Mistral-7B OLMo2-7B Qwen3-8B (a) Performance vs. Sparsity 1471013161922252831 Layer Number 1000.0 200.0 100.0 20.0 2.0 1.0 0.1 Regularization (C) 0.0 0.2 0.4 0.6 0.8 1.0 Layer Importance (b) Mistral-7B (Middle Layers) 1471013161922252831 Layer Number 1000.0 200.0 100.0 20.0 2.0 1.0 0.1 Regularization (C) 0.0 0.2 0.4 0.6 0.8 1.0 Layer Importance (c) OLMo2-7B (Late Layers) Figure 5. Automatic identification of safety-critical layers. a) MLPM achieves peak performance with strong regularization, utilizing a sparse subset of layers. b-c) Analysis of layer importance indicates that the distribution of safety representations varies between models: Mistral concentrates safety information in the middle layers, while OLMo2 in the final layers. Crucially, rather than relying on a manually chosen layer, MLPM automatically selects and uses multiple representations, aggregating signals from the most informative layers. for slight performance gains or opt for a slightly weaker, but cheaper, prompt-only evaluation. 4.5. Layer importance across architectures MLPM aggregates representations from multiple layers via ā 1 regularization with strengthC, which controls the spar- sity of the resulting solution. We investigate the role of different layers in safety assessment by analyzing perfor- mance across varying numbers of selected layers. Then, we examine in more detail the assigned importance of layers as the regularization strength changes. In Figure 5a, we show the average performance on all harm- ful datasets as a function of the fraction of selected layers that correspond to different regularization. The best perfor- mance is consistently achieved by a sparse subset of layers, rather than by using a single layer or all layers, which sup- ports our motivation for multiple layers in safety assessment. The aggregation weights assigned by MLPM can also be interpreted as a measure of layer importance. Therefore, we take a closer look at the aggregation weights for Mistral and OLMo models at Figures 5b and 5c. Specifically, we normalize the aggregation weights for a given regulariza- tion strength by the maximum weight and report the results as "Layer Importance". Our analysis reveals distinct im- portance patterns across architectures: for Mistral, MLPM prioritizes intermediate layers, but for OLMo it focuses on the final layers. Taken together, our results highlight that safety-related information is distributed across different model layers; while this distribution varies across archi- tectures, all models benefit from aggregating multi-layer information. This further justifies the design of MLPM, which enables adaptation to architectural differences and provides strong performance and additional interpretability. 4.6. Components ablation Finally, we perform an ablation study to isolate the contri- butions of MLPMās two components: the distance metric (Mahalanobis vs. Euclidean) and the source of representa- tions (multi-layer vs. last-layer). As shown in Table 4, both components are crucial for optimal performance. Switch- ing from Euclidean to Mahalanobis distance provides the overall highest improvement across all models, and using multi-layer representations rather than single-layer repre- sentations further boosts the performance of our approach. This confirms that the combination of Mahalanobis distance and multi-layer aggregation is essential to MLPMās effec- tiveness. We provide further ablations in Appendix I. Table 4. F1 across harmful datasets with MLPM when varying the distance metric and using last or multi-layer representation. Both the Mahalanobis distance and multi-layer representations improve the performance of MLPM, validating our design choices. DistancePrototypesL-8BM-7BO-7BQ-8B EuclideanLast layer77.1777.3983.6174.97 EuclideanMulti-layer83.7380.3885.7478.96 MahalanobisLast layer86.2584.1889.4485.36 MahalanobisMulti-layer88.3087.5590.1885.95 5. Conclusions In this work, we present MLPM, a novel input moderation method that achieves state-of-the-art safety assessment per- formance while remaining highly efficient. To achieve high accuracy, our method leverages latent representations from multiple LLM layers and assesses prompt safety using the Mahalanobis distance to prototypical safe and unsafe exam- ples. At the same time, the design of MLPM incurs minimal computational overhead and provides remarkable data effi- ciency and generalization. Through extensive experiments, we demonstrate that MLPM delivers robust performance across a wide range of safety benchmarks, surpassing ex- isting state-of-the-art methods. Furthermore, we show that our approach is model-agnostic and can be seamlessly in- tegrated into LLM output moderation pipelines. MLPM represents a step toward efficient and flexible moderation tools for real-world LLM deployment. 8 Efficient LLM Moderation with Multi-Layer Latent Prototypes Limitations. MLPM leverages the internal representa- tions of post-trained LLMs, and its effectiveness depends on the quality of these models. Our method was also designed primarily for input moderation, as a part of a larger system rather than a standalone solution. Reproducibilitystatement.Toensurerepro- ducibility of our research, we provide the code at https://github.com/maciejchrabaszcz/ latent-prototype-moderator. Impact statement. We aim to advance machine learning research toward safer large language models. While we do not identify any specific ethical concerns with our method, we acknowledge that, like any technique, it could be misused if the user steers it toward harmful objectives. References Abdelnabi, S., Fay, A., Cherubin, G., Salem, A., Fritz, M., and Paverd, A. Get my drift? catching llm task drift with activation deltas. In 2025 IEEE Conference on Secure and Trustworthy Machine Learning (SaTML), p. 43ā67. IEEE, 2025. Andriushchenko, M., Croce, F., and Flammarion, N. Jail- breaking leading safety-aligned LLMs with simple adap- tive attacks. In The Thirteenth International Confer- ence on Learning Representations, 2025. URLhttps: //openreview.net/forum?id=hXA8wqRdyV. Ayub, M. A. and Majumdar, S. Embedding-based classifiers can detect prompt injection attacks. In CAMLIS, 2024. Bai, G., Liu, J., Bu, X., He, Y., Liu, J., Zhou, Z., Lin, Z., Su, W., Ge, T., Zheng, B., et al. Mt-bench-101: A fine- grained benchmark for evaluating large language mod- els in multi-turn dialogues. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), p. 7421ā7454, 2024. Bai, Y., Jones, A., Ndousse, K., Askell, A., Chen, A., Das- Sarma, N., Drain, D., Fort, S., Ganguli, D., Henighan, T., Joseph, N., Kadavath, S., Kernion, J., Conerly, T., El-Showk, S., Elhage, N., Hatfield-Dodds, Z., Hernan- dez, D., Hume, T., Johnston, S., Kravec, S., Lovitt, L., Nanda, N., Olsson, C., Amodei, D., Brown, T., Clark, J., McCandlish, S., Olah, C., Mann, B., and Kaplan, J. Train- ing a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv: 2204.05862, 2022. Carlini, N., Nasr, M., Choquette-Choo, C. A., Jagielski, M., Gao, I., Koh, P. W. W., Ippolito, D., Tramer, F., and Schmidt, L. Are aligned neural networks adversarially aligned? Advances in Neural Information Processing Systems, 2023. Clarke, C., Hall, M., Mittal, G., Yu, Y., Sajeev, S., Mars, J., and Chen, M. Rule by example: Harnessing logical rules for explainable hate speech detection. In Proceedings of the 61st Annual Meeting of the Association for Computa- tional Linguistics (Volume 1: Long Papers), p. 364ā376, 2023. Cobbe, K., Kosaraju, V., Bavarian, M., Chen, M., Jun, H., Kaiser, L., Plappert, M., Tworek, J., Hilton, J., Nakano, R., et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. Dai, J., Pan, X., Sun, R., Ji, J., Xu, X., Liu, M., Wang, Y., and Yang, Y.Safe RLHF: Safe reinforcement learning from human feedback.In The Twelfth In- ternational Conference on Learning Representations, 2024. URLhttps://openreview.net/forum? id=TyFrPOKYXw. Dong, Y., Mu, R., Jin, G., Qi, Y., Hu, J., Zhao, X., Meng, J., Ruan, W., and Huang, X. Building guardrails for large language models. arXiv preprint arXiv:2402.01822, 2024. Gema, A. P., Leang, J. O. J., Hong, G., Devoto, A., Mancino, A. C. M., Saxena, R., He, X., Zhao, Y., Du, X., Madani, M. R. G., et al. Are we done with mmlu? In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 5069ā5096, 2025. Ghosh, S., Varshney, P., Galinkin, E., and Parisien, C. Aegis: Online adaptive ai content safety moderation with ensem- ble of llm experts. arXiv preprint arXiv:2404.05993, 2024. Goswami, D., Liu, Y., Twardowski, B. o., and van de Weijer, J. Fecam: Exploiting the heterogeneity of class distributions in exemplar-free continual learning. In Advances in Neural Information Processing Systems, volume 36, p. 6582ā6595. Curran Associates, Inc., 2023.URLhttps://proceedings.neurips. c/paper_files/paper/2023/file/ 15294ba2dcfb4521274f7a1c26f4d4-Paper-Conference. pdf. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., Al-Dahle, A., Letman, A., Mathur, A., Schelten, A., Vaughan, A., et al. The llama 3 herd of models. arXiv preprint arXiv:2407.21783, 2024. Han, S., Rao, K., Ettinger, A., Jiang, L., Lin, B. Y., Lambert, N., Choi, Y., and Dziri, N. Wildguard: Open one-stop 9 Efficient LLM Moderation with Multi-Layer Latent Prototypes moderation tools for safety risks, jailbreaks, and refusals of llms. Advances in Neural Information Processing Systems, 37:8093ā8131, 2024. Hendrycks, D., Burns, C., Basart, S., Zou, A., Mazeika, M., Song, D., and Steinhardt, J. Measuring massive multitask language understanding. In International Conference on Learning Representations, 2021. URLhttps:// openreview.net/forum?id=d7KBjmI3GmQ. Hoffmann, J., Borgeaud, S., Mensch, A., Buchatskaya, E., Cai, T., Rutherford, E., de Las Casas, D., Hendricks, L. A., Welbl, J., Clark, A., et al. Training compute- optimal large language models. In Proceedings of the 36th International Conference on Neural Information Processing Systems, p. 30016ā30030, 2022. Huang, T., Hu, S., Ilhan, F., Tekin, S. F., Yahn, Z., Xu, Y., and Liu, L. Safety tax: Safety alignment makes your large reasoning models less reasonable. arXiv preprint arXiv:2503.00555, 2025. Inan, H., Upasani, K., Chi, J., Rungta, R., Iyer, K., Mao, Y., Tontchev, M., Hu, Q., Fuller, B., Testug- gine, D., et al. Llama guard: Llm-based input-output safeguard for human-ai conversations. arXiv preprint arXiv:2312.06674, 2023. Jiang, A. Q., Sablayrolles, A., Mensch, A., Bamford, C., Chaplot, D. S., de las Casas, D., Bressand, F., Lengyel, G., Lample, G., Saulnier, L., Lavaud, L. R., Lachaux, M.- A., Stock, P., Scao, T. L., Lavril, T., Wang, T., Lacroix, T., and Sayed, W. E. Mistral 7b, 2023. URLhttps: //arxiv.org/abs/2310.06825. Jiang, L., Rao, K., Han, S., Ettinger, A., Brahman, F., Ku- mar, S., Mireshghallah, N., Lu, X., Sap, M., Choi, Y., et al. Wildteaming at scale: From in-the-wild jailbreaks to (ad- versarially) safer language models. Advances in Neural Information Processing Systems, 37:47094ā47165, 2024. Kaplan, J., McCandlish, S., Henighan, T., Brown, T. B., Chess, B., Child, R., Gray, S., Radford, A., Wu, J., and Amodei, D. Scaling laws for neural language models. arXiv preprint arXiv:2001.08361, 2020. Kubokawa, T. and Srivastava, M. S. Estimation of the precision matrix of a singular wishart distribution and its application in high-dimensional data, 2008. Kumar, D., AbuHashem, Y. A., and Durumeric, Z. Watch your language: Investigating content moderation with large language models. In Proceedings of the Interna- tional AAAI Conference on Web and Social Media, vol- ume 18, p. 865ā878, 2024. Åapacz, W., Marczak, D., Szatkowski, F., and Trzci Ģ nski, T. Exploring the stability gap in continual learning: The role of the classification head. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), p. 7562ā7571. IEEE, 2025. Lee, B. W., Padhi, I., Ramamurthy, K. N., Miehling, E., Dognin, P., Nagireddy, M., and Dhurandhar, A. Pro- gramming refusal with conditional activation steering. In The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview. net/forum?id=Oi47wc10sm. Lee, K., Lee, K., Lee, H., and Shin, J. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. Advances in neural information pro- cessing systems, 31, 2018. Li, S., Yao, L., Zhang, L., and Li, Y. Safety layers in aligned large language models: The key to LLM security. In The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview. net/forum?id=kUH1yPMAn7. Lim, I., Khoo, S., Lee, R. K.-W., Chua, W., Goh, J. Y., and Foo, J. Safe at the margins: A general approach to safety alignment in low-resource english languagesāa singlish case study. arXiv preprint arXiv:2502.12485, 2025. Lin, S., Hilton, J., and Evans, O. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th annual meeting of the association for computa- tional linguistics (volume 1: long papers), p. 3214ā3252, 2022. Lin, Z., Wang, Z., Tong, Y., Wang, Y., Guo, Y., Wang, Y., and Shang, J. Toxicchat: Unveiling hidden chal- lenges of toxicity detection in real-world user-AI con- versation. In The 2023 Conference on Empirical Methods in Natural Language Processing, 2023. URLhttps: //openreview.net/forum?id=jTiJPDv82w. Liu, A., Feng, B., Xue, B., Wang, B., Wu, B., Lu, C., Zhao, C., Deng, C., Zhang, C., Ruan, C., et al. Deepseek-v3 technical report. arXiv preprint arXiv:2412.19437, 2024. Liu, Y., Deng, G., Xu, Z., Li, Y., Zheng, Y., Zhang, Y., Zhao, L., Zhang, T., Wang, K., and Liu, Y. Jailbreaking chat- gpt via prompt engineering: An empirical study. arXiv preprint arXiv:2305.13860, 2023. Luo, J., Ding, T., Chan, K. H. R., Thaker, D., Chattopadhyay, A., Callison-Burch, C., and Vidal, R. Pace: Parsimonious concept engineering for large language models. arXiv preprint arXiv:2406.04331, 2024. 10 Efficient LLM Moderation with Multi-Layer Latent Prototypes Manczak, B., Lin, E., Zemour, E., and Mugunthan, V. Prime- guard: Safe and helpful llms through tuning-free routing. In ICML 2024 Next Generation of AI Safety Workshop, 2024. Markov, T., Zhang, C., Agarwal, S., Nekoul, F. E., Lee, T., Adler, S., Jiang, A., and Weng, L. A holistic approach to undesired content detection in the real world. In Pro- ceedings of the AAAI conference on artificial intelligence, volume 37, p. 15009ā15018, 2023. Masarczyk, W., Ostaszewski, M., Imani, E., Pascanu, R., MiÅo Ģ s, P., and Trzcinski, T. The tunnel effect: Building data representations in deep neural networks. Advances in Neural Information Processing Systems, 36:76772ā 76805, 2023. Mazeika, M., Phan, L., Yin, X., Zou, A., Wang, Z., Mu, N., Sakhaee, E., Li, N., Basart, S., Li, B., et al. Harmbench: A standardized evaluation framework for automated red teaming and robust refusal. In International Conference on Machine Learning, p. 35181ā35224. PMLR, 2024. OLMo, T., Walsh, P., Soldaini, L., Groeneveld, D., Lo, K., Arora, S., Bhagia, A., Gu, Y., Huang, S., Jordan, M., et al. 2 olmo 2 furious, 2024. Ouyang, L., Wu, J., Jiang, X., Almeida, D., Wainwright, C., Mishkin, P., Zhang, C., Agarwal, S., Slama, K., Ray, A., et al. Training language models to follow instructions with human feedback. Advances in neural information processing systems, 2022. Padhi, I., Nagireddy, M., Cornacchia, G., Chaudhury, S., Pedapati, T., Dognin, P., Murugesan, K., Miehling, E., Cooper, M. S., Fraser, K., et al. Granite guardian, 2024. Qi, X., Panda, A., Lyu, K., Ma, X., Roy, S., Beirami, A., Mittal, P., and Henderson, P. Safety alignment should be made more than just a few tokens deep. In The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview.net/forum? id=6Mxhg9PtDE. Qiu, Y., Zhao, Z., Ziser, Y., Korhonen, A., Ponti, E. M., and Cohen, S. B.Spectral editing of activations for large language model alignment. arXiv preprint arXiv:2405.09719, 2024. Rebuffi, S.-A., Kolesnikov, A., Sperl, G., and Lampert, C. H. icarl: Incremental classifier and representation learning. In Proceedings of the IEEE conference on Computer Vision and Pattern Recognition, p. 2001ā2010, 2017. Rƶttger, P., Kirk, H., Vidgen, B., Attanasio, G., Bianchi, F., and Hovy, D. Xstest: A test suite for identifying exaggerated safety behaviours in large language mod- els. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), p. 5377ā5400, 2024. Sharma, M., Tong, M., Mu, J., Wei, J., Kruthoff, J., Good- friend, S., Ong, E., Peng, A., Agarwal, R., Anil, C., et al. Constitutional classifiers: Defending against universal jailbreaks across thousands of hours of red teaming. arXiv preprint arXiv:2501.18837, 2025. Shazeer, N., Mirhoseini, A., Maziarz, K., Davis, A., Le, Q., Hinton, G., and Dean, J. Outrageously large neural networks: The sparsely-gated mixture-of-experts layer. In International Conference on Learning Representations, 2017. URLhttps://openreview.net/forum? id=B1ckMDqlg. Suzgun, M., Scales, N., SchƤrli, N., Gehrmann, S., Tay, Y., Chung, H. W., Chowdhery, A., Le, Q., Chi, E., Zhou, D., et al. Challenging big-bench tasks and whether chain-of- thought can solve them. In Findings of the Association for Computational Linguistics: ACL 2023, p. 13003ā13051, 2023. Szatkowski, F., Zheng, Y., Yang, F., Trzcinski, T., Twar- dowski, B., and van de Weijer, J. Improving continual learning performance and efficiency with auxiliary classi- fiers. In Forty-second International Conference on Ma- chine Learning, 2025. URLhttps://openreview. net/forum?id=sq5eL4jfsn. Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., and Hashimoto, T. B. Stanford alpaca:An instruction-following llama model.https://github.com/tatsu-lab/ stanford_alpaca, 2023. Turner, A. M., Thiergart, L., Leech, G., Udell, D., Vazquez, J. J., Mini, U., and MacDiarmid, M.Steering lan- guage models with activation engineering. arXiv preprint arXiv:2308.10248, 2023. Vidgen, B., Scherrer, N., Kirk, H. R., Qian, R., Kannappan, A., Hale, S. A., and Rƶttger, P. Simplesafetytests: a test suite for identifying critical safety risks in large language models. arXiv preprint arXiv:2311.08370, 2023. Wang, L., Liu, X., Yi, J., Jiang, Y., and Hsieh, C.-J. Provably robust metric learning. Advances in Neural Information Processing Systems, 33:19302ā19313, 2020a. Wang, Y., Yao, Q., Kwok, J. T., and Ni, L. M. Generalizing from a few examples: A survey on few-shot learning. ACM computing surveys (csur), 2020b. Wang, Y., Wang, W., Joty, S., and Hoi, S. C. Codet5: Identifier-aware unified pre-trained encoder-decoder mod- els for code understanding and generation. In Proceedings 11 Efficient LLM Moderation with Multi-Layer Latent Prototypes of the 2021 Conference on Empirical Methods in Natural Language Processing, p. 8696ā8708, 2021. Wang, Z., Liang, J., Sheng, L., He, R., Wang, Z., and Tan, T. A hard-to-beat baseline for training-free CLIP- based adaptation. In The Twelfth International Confer- ence on Learning Representations, 2024. URLhttps: //openreview.net/forum?id=Js5PJPHDyY. Wei, A., Haghtalab, N., and Steinhardt, J. Jailbroken: How does LLM safety training fail? 2023. URLhttps: //openreview.net/forum?id=jA235JGM09. Wolf, Y., Wies, N., Shteyman, D., Rothberg, B., Levine, Y., and Shashua, A. Tradeoffs between alignment and helpfulness in language models with representation engi- neering. arXiv preprint arXiv:2401.16332, 2024. Xian, Y., Schiele, B., and Akata, Z. Zero-shot learning ā the good, the bad and the ugly. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. Xie, Y., Yi, J., Shao, J., Curl, J., Lyu, L., Chen, Q., Xie, X., and Wu, F. Defending chatgpt against jailbreak attack via self-reminders. Nature Machine Intelligence, 5(12): 1486ā1496, 2023. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025. Yin, F., Laban, P., PENG, X., Zhou, Y., Mao, Y., Vats, V., Ross, L., Agarwal, D., Xiong, C., and Wu, C.-S. Bin- goguard: LLM content moderation tools with risk levels. In The Thirteenth International Conference on Learning Representations, 2025. URLhttps://openreview. net/forum?id=HPSAkIHRbb. Zeng, W., Liu, Y., Mullins, R., Peran, L., Fernandez, J., Harkous, H., Narasimhan, K., Proud, D., Kumar, P., Radharapu, B., et al.Shieldgemma: Generative ai content moderation based on gemma. arXiv preprint arXiv:2407.21772, 2024. Zhang, Y., Wei, Z., Sun, J., and Sun, M. Adversarial rep- resentation engineering: A general model editing frame- work for large language models. Advances in Neural In- formation Processing Systems, 37:126243ā126264, 2024. Zheng, C., Yin, F., Zhou, H., Meng, F., Zhou, J., Chang, K.- W., Huang, M., and Peng, N. On prompt-driven safeguard- ing for large language models. In Forty-first International Conference on Machine Learning, 2024. URLhttps: //openreview.net/forum?id=ugxGpOEkox. Zou, A., Phan, L., Chen, S., Campbell, J., Guo, P., Ren, R., Pan, A., Yin, X., Mazeika, M., Dombrowski, A.-K., et al. Representation engineering: A top-down approach to ai transparency. arXiv preprint arXiv:2310.01405, 2023a. Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., and Fredrikson, M. Universal and transferable adversarial attacks on aligned language models, 2023b. Zou, A., Phan, L., Wang, J., Duenas, D., Lin, M., An- driushchenko, M., Kolter, J. Z., Fredrikson, M., and Hendrycks, D. Improving alignment and robustness with circuit breakers. In Advances in Neural Information Pro- cessing Systems 38, NeurIPS 2024, 2024. 12 Efficient LLM Moderation with Multi-Layer Latent Prototypes Appendix A. Evaluation Datasets As mentioned in Section 4, we split evaluation datasets into 2 groups: prompt harmfulness - 8 datasets, and general capabilities - 7 datasets, which we refer to as harmful and neutral, respectively. This split helps us assess our methodās effectiveness at detecting harmful content in prompts, while ensuring that neutral, non-harmful data remains correctly labeled to preserve the modelās original capabilities during moderation. For prompt harmfulness we use Aegis (Ghosh et al., 2024), HarmBench (Mazeika et al., 2024), OpenAI Mod (Markov et al., 2023), Simple Safety Tests (Vidgen et al., 2023), Toxic Chat (Lin et al., 2023), XSTest (Rƶttger et al., 2024), WildGuardMix (Han et al., 2024) and WildJailbreak (Jiang et al., 2024). To ensure proper generalization, we evaluate the true negative rate on neutral datasets, including Alpaca (Taori et al., 2023), BigBenchHard (Suzgun et al., 2023), Codex (Wang et al., 2021), GSM8k (Cobbe et al., 2021), MMLU (Gema et al., 2025; Hendrycks et al., 2021), MTBench (Bai et al., 2024), and TruthfulQA (Lin et al., 2022). In Table 5, we provide the distribution of neutral and harmful samples in each of the datasets. We also describe each dataset in more detail in the next subsections. Table 5. Distribution of harmful and neutral prompts in evaluation datasets. DatasetNum Neutral PromptsNum Harmful PromptsNum Prompts Aegis126233359 HarmBench0239239 OpenAI Mod11585221680 Simple Safety Tests0100100 Toxic Chat24913622853 WildGuardMix Test9457541699 WildJailbreak21020002210 XSTest249197446 Alpaca8050805 BigBenchHard108001080 Codex1640164 GSM8k131901319 MMLU-R274402744 MTBench80080 TruthfulQA7900790 A.1. Harmful datasets Aegis: This dataset comprises human-LLM interaction instances, each annotated for safety based on an extensive content safety risk taxonomy spanning 13 categories. Aegis is designed to benchmark and enhance the safety of Large Language Models (LLMs), particularly in the context of content moderation. All included responses were generated using Mistral-7B- v0.1. HarmBench: HarmBench is an evaluation dataset comprising harmful prompts that can elicit harmful behaviors of LLMs. OpenAIMod: This dataset features prompts, each accompanied by a harm label, spanning eight defined risk categories. Simple Safety Tests: This is a concise test suite featuring 100 prompts across five distinct harm areas, designed for the rapid identification of critical safety risks within LLMs. Toxic Chat: This benchmark dataset is constructed from real user queries submitted to an open-source chatbot. The collected samples have been annotated for toxicity through a human-AI collaborative annotation framework. WildGuardMix: This dataset offers a diverse collection of both standard (vanilla) and adversarial prompts, encompassing 13 Efficient LLM Moderation with Multi-Layer Latent Prototypes harmful and benign scenarios, accompanied by LLM-generated responses. WildJailbreak: An open-source synthetic safety-training dataset, WildJailbreak contains prompt-response pairs. It features a mix of vanilla (direct harmful requests) and adversarial (complex jailbreaks) queries. The dataset also includes contrastive benign queries that resemble harmful ones, aiming to mitigate exaggerated safety behaviors in LLMs. For evaluation, a test set composed entirely of adversarial prompts is utilized. XSTest A test suite designed to identify "exaggerated safety behaviors" in LLMs. This refers to instances where models refuse safe prompts if they contain language similar to unsafe prompts or mention sensitive topics. A.2. Neutral datasets Alpaca: This dataset features instructions generated by OpenAIās text-davinci-003 model. Covering diverse domains, these instructions are widely utilized for fine-tuning Large Language Models (LLMs) to enhance their ability to follow instructions. BigBenchHard: A challenging subset of the BIG-bench benchmark, BigBenchHard comprises 23 tasks specifically selected because current language models find them particularly difficult. Codex: This is a dataset containing a collection of code-related instructions specifically curated for training and evaluating LLMs on programming tasks. GSM8k: GSM8k is a dataset of high-quality, linguistically diverse grade school math word problems, designed to test multi-step reasoning. MMLU: This benchmark is designed to evaluate the knowledge acquired by language models across an extensive array of 57 distinct tasks. These tasks span humanities, social sciences, STEM, and other areas, offering a comprehensive measure of a modelās understanding. We utilize MMLU-Redux (Gema et al., 2025), which is a subset of manually re-annotated questions across 30 MMLU subjects. MTBench: MTBench is a benchmark composed of multi-turn questions, specifically designed to evaluate the conversational and instruction-following capabilities of chat-focused LLMs. In our experiments, we focus solely on the initial instruction of each interaction. TruthfulQA: This benchmark is engineered to measure the truthfulness of a language model when generating answers to questions. It particularly focuses on questions where answers are prone to common misconceptions or are frequently misremembered, thereby testing the modelās ability to avoid parroting falsehoods. 14 Efficient LLM Moderation with Multi-Layer Latent Prototypes B. Which Layers are the Best for Safety Assessment? We additionally performed analyses to determine if there is a single best layer for safety evaluation for all models and datasets. In Figure 6 we show that there is no single best layer in terms of performance across multiple models and datasets. 051015202530 Layer 20 40 60 80 F1 Score aegis toxicchat_humanannotated wildguardtest xstest (a) Moderation score across different datasets and layers. 05101520253035 Layer 72 74 76 78 80 82 84 86 88 F1 Score on WGMix Llama3-8B Mistral-7B OLMo2-7B Qwen3-8B (b) Moderation score across different models and layers. Figure 6. LLMs already contain information about input safety, but the layers at which harmful and safe examples are best separated depend on both the model and task. a) The performance across 4 different datasets when using representations from different layers. b) The performance across 4 models when using representations from different layers. C. Detailed Results In this section, we provide detailed results that compare the performance of MLPM on more models and on Neutral datasets. C.1. Neutral datasets In Table 6 we show the True Negative Rate on all Neutral datasets. C.2. Harmful datasets In Table 7 we show the F1 score on all harmful datasets. D. MLPM with Output Moderation Details In this section, we provide additional details for end-to-end experiments. D.1. Safety prompt Below, we provide safety prompts used for increasing the safety of LLMs in the e2e experiment. Safety Prompt You are a helpful, respectful, and honest assistant. Always answer as helpfully as possible, while being safe. Your answers should not include any harmful, unethical, racist, sexist, toxic, dangerous, or illegal content. Please ensure that your responses are socially unbiased and positive in nature. If a question does not make any sense, or is not factually coherent, explain why instead of answering something not correct. If you donāt know the answer to a question, please donāt share false information. 15 Efficient LLM Moderation with Multi-Layer Latent Prototypes Table 6. Detailed results on Neutral datasets. For each dataset, we show the True Negative Rate, as these datasets have no harmful examples. ModelAlpacaBBHCodexGSM8kMMLUMTBenchTruthfulQAAvg Llama-8B-Inst+Ayub & Majumdar (2024)93.2981.48100.00100.0099.8296.2587.9794.12 Mistral-7B-Inst+Ayub & Majumdar (2024)91.5570.5699.3999.9299.3896.2592.4192.78 OLMo2-7B-Inst+Ayub & Majumdar (2024)95.1688.89100.0084.3197.1295.0094.1893.52 Qwen3-8B-Inst+Ayub & Majumdar (2024)94.91100.00100.0099.9299.5393.7597.3497.92 Llama-8B-Inst+Abdelnabi et al. (2025)94.6699.91100.00100.0099.8997.5097.7298.53 Mistral-7B-Inst+Abdelnabi et al. (2025)91.3089.81100.0099.8599.9692.5096.8495.75 OLMo2-7B-Inst+Abdelnabi et al. (2025)98.6399.44100.00100.00100.0097.5097.8599.06 Qwen3-8B-Inst+Abdelnabi et al. (2025)96.27100.00100.0099.3999.9695.0097.4798.30 DeepSeek-Distill-Llama-8B+MLPM(Ours)95.40100.00100.00100.0048.7298.7597.3491.46 DeepSeek-Qwen3-8B+MLPM(Ours)95.9095.56100.00100.0096.8796.2597.9797.51 Llama-1B-Inst+MLPM(Ours)92.55100.00100.00100.00100.0098.7598.3598.52 Llama-3B-Inst+MLPM(Ours)95.65100.00100.00100.00100.0098.7597.0998.78 Llama-8B+MLPM(Ours)85.9698.5299.39100.0090.6085.0091.7793.03 Llama-8B-Inst+MLPM(Ours)96.15100.00100.00100.00100.00100.0098.2399.20 Llama-70B-Inst+MLPM(Ours)98.01100.00100.00100.00100.00100.0097.8599.41 Mistral-7B+MLPM(Ours)87.3388.2498.78100.0094.5092.5091.1493.21 Mistral-7B-Inst+MLPM(Ours)96.15100.00100.00100.0099.89100.0096.2098.89 Mistral-12B-Inst+MLPM(Ours)94.53100.00100.00100.00100.0097.5095.5798.23 Mistral-24B-Inst+MLPM(Ours)96.4099.91100.00100.0099.85100.0097.7299.13 OLMo2-1B-Inst+MLPM(Ours)98.01100.00100.00100.00100.0098.7597.7299.21 OLMoE-1B-7B-Inst+MLPM(Ours)98.01100.00100.00100.0099.96100.0097.4799.35 OLMo2-7B+MLPM(Ours)89.1998.6195.7399.9296.8383.7597.5994.52 OLMo2-7B-DPO+MLPM(Ours)98.51100.00100.00100.00100.0098.7596.8499.16 OLMo2-7B-SFT+MLPM(Ours)98.63100.00100.00100.0099.9698.7597.5999.28 OLMo2-7B-Inst+MLPM(Ours)98.51100.00100.00100.00100.0098.7597.2299.21 OLMo2-13B-Inst+MLPM(Ours)97.76100.00100.00100.0099.7498.7598.3599.23 OLMo2-32B-Inst+MLPM(Ours)98.51100.00100.00100.00100.00100.0097.5999.44 Qwen3-0.6B+MLPM(Ours)92.92100.00100.00100.00100.0096.2596.4697.95 Qwen3-1.7B+MLPM(Ours)90.19100.00100.00100.0099.9695.0098.3597.64 Qwen3-4B+MLPM(Ours)92.42100.00100.00100.00100.0097.5098.3598.32 Qwen3-4B-Base+MLPM(Ours)87.3388.3399.39100.00100.00100.0097.5996.09 Qwen3-4B-Inst+MLPM(Ours)96.40100.00100.00100.0099.9398.7596.7198.83 Qwen3-4B-Thinking+MLPM(Ours)96.40100.00100.00100.00100.0098.7598.2399.05 Qwen3-8B-Inst+MLPM(Ours)96.77100.00100.00100.00100.00100.0098.6199.34 Qwen3-8B-Base+MLPM(Ours)91.9396.94100.00100.0099.8997.5097.0997.62 Qwen3-30B-A3B+MLPM(Ours)94.29100.00100.00100.00100.00100.0098.9999.04 Qwen3-32B+MLPM(Ours)95.7899.91100.00100.00100.0098.7598.8699.04 Aegis-Guard-D99.0197.78100.0099.9299.20100.0095.9598.84 Aegis-Guard-P99.5098.43100.0099.9299.89100.0097.3499.30 LlamaGuard199.63100.00100.0099.92100.00100.0097.8599.63 LlamaGuard299.13100.00100.00100.0094.9798.7599.2498.87 LlamaGuard398.6399.81100.00100.0099.9398.7599.8799.57 GraniteGuardian-3-1-8B100.00100.00100.00100.00100.00100.00100.00100.00 ShieldGemma-9B100.00100.00100.00100.00100.00100.00100.00100.00 WildGuard96.89100.00100.00100.0099.7197.5096.5898.67 16 Efficient LLM Moderation with Multi-Layer Latent Prototypes Table 7. Detailed results on Harmful datasets. For each dataset, we show the F1 score. ModelAegisHarmBOpenAISimpSTToxiChatWGMixWJXSAvg Llama-8B-Inst+Ayub & Majumdar (2024)84.1695.1867.9998.9959.5986.2793.2790.6384.51 Mistral-7B-Inst+Ayub & Majumdar (2024)84.5597.8664.5998.4857.6385.7391.1394.6084.32 OLMo2-7B-Inst+Ayub & Majumdar (2024)87.7296.5467.38100.0065.1287.8296.6893.2686.82 Qwen3-8B-Inst+Ayub & Majumdar (2024)84.4799.3768.4597.9660.9485.1890.4488.0684.36 Llama-8B-Inst+Abdelnabi et al. (2025)83.8797.4270.6899.5065.0284.9390.5596.1886.02 Mistral-7B-Inst+Abdelnabi et al. (2025)83.7593.0664.7596.9159.1882.3386.5290.6782.15 OLMo2-7B-Inst+Abdelnabi et al. (2025)87.7896.3173.71100.0075.8388.0196.1997.1489.37 Qwen3-8B-Inst+Abdelnabi et al. (2025)83.2294.9571.9797.9665.5983.2486.5592.2784.47 DeepSeek-Distill-Llama-8B+MLPM(Ours)81.9498.9469.1698.4863.5086.7791.7190.6685.15 DeepSeek-Qwen3-8B+MLPM(Ours)81.8295.6371.2698.9962.8484.8792.4388.6484.56 Llama-1B-Inst+MLPM(Ours)82.7398.0869.9698.4862.2485.0890.5091.6084.83 Llama-3B-Inst+MLPM(Ours)85.91100.0073.2799.5067.4887.9393.6396.9288.08 Llama-8B+MLPM(Ours)82.3897.4259.7695.2948.2782.5584.8076.2878.34 Llama-8B-Inst+MLPM(Ours)85.1399.5872.8599.5069.1788.0494.6997.4488.30 Llama-70B-Inst+MLPM(Ours)88.99100.0076.57100.0072.8989.3997.4198.7390.50 Mistral-7B+MLPM(Ours)79.6294.4858.7597.9651.7883.3883.7071.3877.63 Mistral-7B-Inst+MLPM(Ours)87.3699.1670.6899.5066.3387.6393.6596.1087.55 Mistral-12B-Inst+MLPM(Ours)87.36100.0070.9199.5064.2588.5893.6893.7287.25 Mistral-24B-Inst+MLPM(Ours)85.08100.0071.8399.5067.2788.4194.9095.2987.78 OLMo2-1B-Inst+MLPM(Ours)83.9799.5869.3598.9967.1888.1693.7692.4386.68 OLMoE-1B-7B-Inst+MLPM(Ours)87.0289.8670.2199.5072.9087.5794.9793.6286.96 OLMo2-7B+MLPM(Ours)82.5493.3061.6497.9652.9984.6584.1782.7079.99 OLMo2-7B-DPO+MLPM(Ours)89.2898.5174.19100.0076.2188.5297.6996.8990.16 OLMo2-7B-SFT+MLPM(Ours)89.4398.5171.9799.5074.9488.5897.6696.1089.59 OLMo2-7B-Inst+MLPM(Ours)89.2398.5174.21100.0076.5188.5297.5596.9190.18 OLMo2-13B-Inst+MLPM(Ours)88.6999.7972.5399.5075.4688.6097.8496.0689.81 OLMo2-32B-Inst+MLPM(Ours)88.1199.7974.41100.0074.8689.0397.4897.6990.17 Qwen3-0.6B+MLPM(Ours)73.6098.9464.3694.1855.1680.4787.2770.2478.03 Qwen3-1.7B+MLPM(Ours)78.54100.0065.5497.9653.8984.9588.4381.6381.37 Qwen3-4B+MLPM(Ours)81.90100.0068.4796.9161.6485.1790.4685.2283.72 Qwen3-4B-Base+MLPM(Ours)78.6698.9461.6897.4454.8784.7590.6878.1880.65 Qwen3-4B-Inst+MLPM(Ours)84.2399.7972.0498.9965.9986.7992.8991.2586.50 Qwen3-4B-Thinking+MLPM(Ours)86.1799.3769.6798.4865.7287.3892.5691.3086.33 Qwen3-8B-Inst+MLPM(Ours)83.49100.0072.3596.9164.4086.2192.0092.2385.95 Qwen3-8B-Base+MLPM(Ours)80.1997.4264.0298.4857.4185.2990.8380.9581.82 Qwen3-30B-A3B+MLPM(Ours)81.60100.0067.4998.4862.6986.1290.9191.0684.79 Qwen3-32B+MLPM(Ours)86.18100.0071.8798.9967.6686.7793.2691.9487.08 Aegis-Guard-D81.0070.4676.4497.9675.6172.0975.4481.5378.82 Aegis-Guard-P75.7266.1178.1194.1868.4965.6355.7282.3573.29 LlamaGuard172.9266.1174.3892.4757.1455.0841.3681.6167.63 LlamaGuard271.8593.7876.1095.8346.3270.5249.8589.1874.18 LlamaGuard371.7498.9479.1199.5054.1176.7667.8388.5279.56 GraniteGuardian-3-1-8B87.7879.9077.6399.5073.2584.5796.7585.5985.62 ShieldGemma-9B77.4469.0477.6391.3068.1358.8859.9482.4173.10 WildGuard89.7899.3772.2899.5070.1488.0497.1095.2688.93 17 Efficient LLM Moderation with Multi-Layer Latent Prototypes E. In-Distribution vs Out-of-Distribution In this section, we provide additional ID vs OOD plots for Mistral-7B-Inst (Figure 7a), OLMo2-7B-Inst (Figure 7b), and Qwen3-8B-Inst (Figure 7c. We additionally provide per-dataset performance in Table 8. Aeg(ID)TC (ID)WG (ID)OOD 0.5 1.0 F1 Score Training data: Aegis Aeg(ID)TC (ID)WG (ID)OOD Training data: Toxichat Aeg(ID)TC (ID)WG (ID)OOD 0.5 1.0 F1 Score Training data: WildGuardMix Aeg(ID)TC (ID)WG (ID)OOD Training data: All Data Abdelnabi et al.Ayub & MajumdarMLPM(Ours) (a) Mistral-7B-Inst Aeg(ID)TC (ID)WG (ID)OOD 0.5 1.0 F1 Score Training data: Aegis Aeg(ID)TC (ID)WG (ID)OOD Training data: Toxichat Aeg(ID)TC (ID)WG (ID)OOD 0.5 1.0 F1 Score Training data: WildGuardMix Aeg(ID)TC (ID)WG (ID)OOD Training data: All Data Abdelnabi et al.Ayub & MajumdarMLPM(Ours) (b) Llama3-8B-Inst Aeg(ID)TC (ID)WG (ID)OOD 0.5 1.0 F1 Score Training data: Aegis Aeg(ID)TC (ID)WG (ID)OOD Training data: Toxichat Aeg(ID)TC (ID)WG (ID)OOD 0.5 1.0 F1 Score Training data: WildGuardMix Aeg(ID)TC (ID)WG (ID)OOD Training data: All Data Abdelnabi et al.Ayub & MajumdarMLPM(Ours) (c) Qwen3-8B-Inst Figure 7. In-Distribution vs Out-of-Distribution performance comparisons across three models: (a) Mistral-7B-Inst, (b) OLMo2-7B-Inst, and (c) Qwen3-8B-Inst. Table 8. Detailed results of MLPM and other latent-based methods when training on different datasets and on all three at once. Training DataModelMethodAegisHarmBOpenAISimpSTToxiChatWGMixWJXSAvg AegisLlama-8B-InstAbdelnabi et al. (2025)84.3288.0660.8598.4839.2367.5383.9772.0174.31 AegisLlama-8B-InstAyub & Majumdar (2024)89.1387.7964.41100.0059.3473.6386.2777.6279.77 AegisLlama-8B-InstMLPM(Ours)90.1190.6267.16100.0062.4676.3589.8479.4482.00 AegisMistral-7B-InstAbdelnabi et al. (2025)89.4667.9663.8197.4444.3967.6970.3979.7572.61 AegisMistral-7B-InstAyub & Majumdar (2024)88.9896.9862.8999.5040.8370.1679.5772.9076.48 AegisMistral-7B-InstMLPM(Ours)90.3576.1767.6298.4864.9677.1091.0877.9880.47 AegisOLMo2-7B-InstAbdelnabi et al. (2025)89.1773.5463.2698.9948.2172.1090.8571.7675.99 AegisOLMo2-7B-InstAyub & Majumdar (2024)90.1594.0162.12100.0059.3881.2695.7877.3282.50 AegisOLMo2-7B-InstMLPM(Ours)90.2477.4468.4698.9972.7979.4496.1176.2182.46 AegisQwen3-8BAbdelnabi et al. (2025)88.0977.4458.9097.9649.4166.8687.1076.4675.28 AegisQwen3-8BAyub & Majumdar (2024)89.2281.9862.42100.0050.7774.6389.1976.3678.07 AegisQwen3-8BMLPM(Ours)89.9179.6064.2899.5060.9575.8092.6776.8079.94 ToxicChatLlama-8B-InstAbdelnabi et al. (2025)59.0471.1665.2493.6279.6670.7681.8474.4674.47 ToxicChatLlama-8B-InstAyub & Majumdar (2024)53.5872.8762.2488.8972.5268.2271.5374.8470.59 ToxicChatLlama-8B-InstMLPM(Ours)71.0476.4977.0397.4483.7477.0686.9289.5082.40 ToxicChatMistral-7B-InstAbdelnabi et al. (2025)62.1766.1163.8486.3679.2463.5561.5178.4970.16 ToxicChatMistral-7B-InstAyub & Majumdar (2024)55.2166.1137.8983.7270.1355.2641.7974.6860.60 ToxicChatMistral-7B-InstMLPM(Ours)67.7968.3273.8994.1880.2066.7273.9882.3975.93 ToxicChatOLMo2-7B-InstAbdelnabi et al. (2025)74.6072.1973.4191.3083.5077.6694.7977.8180.66 ToxicChatOLMo2-7B-InstAyub & Majumdar (2024)65.9072.8770.0888.8978.1068.6975.4678.7774.85 ToxicChatOLMo2-7B-InstMLPM(Ours)78.8877.4477.1395.2983.3579.4295.7087.4084.33 ToxicChatQwen3-8BAbdelnabi et al. (2025)61.5865.7371.7886.3677.5268.7364.7880.6972.15 ToxicChatQwen3-8BAyub & Majumdar (2024)57.4060.2362.3884.3975.1662.1152.1276.2266.25 ToxicChatQwen3-8BMLPM(Ours)65.3371.5174.0692.4778.2174.1276.6982.4276.85 WildGuardMixLlama-8B-InstAbdelnabi et al. (2025)84.1695.1867.9998.9959.5986.2793.2790.6384.51 WildGuardMixLlama-8B-InstAyub & Majumdar (2024)83.8797.4270.6899.5065.0284.9390.5596.1886.02 WildGuardMixLlama-8B-InstMLPM(Ours)85.1399.5872.8599.5069.1788.0494.6997.4488.30 WildGuardMixMistral-7B-InstAbdelnabi et al. (2025)84.5597.8664.5998.4857.6385.7391.1394.6084.32 WildGuardMixMistral-7B-InstAyub & Majumdar (2024)83.7593.0664.7596.9159.1882.3386.5290.6782.15 WildGuardMixMistral-7B-InstMLPM(Ours)87.3699.1670.6899.5066.3387.6393.6596.1087.55 WildGuardMixOLMo2-7B-InstAbdelnabi et al. (2025)87.7296.5467.38100.0065.1287.8296.6893.2686.82 WildGuardMixOLMo2-7B-InstAyub & Majumdar (2024)87.7896.3173.71100.0075.8388.0196.1997.1489.37 WildGuardMixOLMo2-7B-InstMLPM(Ours)89.2398.5174.21100.0076.5188.5297.5596.9190.18 WildGuardMixQwen3-8BAbdelnabi et al. (2025)84.4799.3768.4597.9660.9485.1890.4488.0684.36 WildGuardMixQwen3-8BAyub & Majumdar (2024)83.2294.9571.9797.9665.5983.2486.5592.2784.47 WildGuardMixQwen3-8BMLPM(Ours)83.49100.0072.3596.9164.4086.2192.0092.2385.95 All DataLlama-8B-InstAbdelnabi et al. (2025)83.6894.9567.2898.9969.0886.0592.5892.3185.62 All DataLlama-8B-InstAyub & Majumdar (2024)84.4595.6366.9098.9972.3384.7390.2397.0086.28 All DataLlama-8B-InstMLPM(Ours)87.8995.6369.4399.5077.6288.3893.5996.4388.56 All DataMistral-7B-InstAbdelnabi et al. (2025)86.7483.7065.0399.5069.2886.0990.6195.9684.61 All DataMistral-7B-InstAyub & Majumdar (2024)84.6787.7966.1097.9667.9983.9685.7193.1983.42 All DataMistral-7B-InstMLPM(Ours)87.4298.3067.5899.5076.8187.8791.4995.0688.00 All DataOLMo2-7B-InstAbdelnabi et al. (2025)89.4794.9566.7199.5073.1687.8096.9992.6287.65 All DataOLMo2-7B-InstAyub & Majumdar (2024)88.1494.4866.3598.9979.9587.6695.8196.3788.47 All DataOLMo2-7B-InstMLPM(Ours)89.8594.2568.6899.5081.5788.0497.1196.1289.39 All DataQwen3-8BAbdelnabi et al. (2025)86.2398.3065.8997.9670.0485.6389.9090.2185.52 All DataQwen3-8BAyub & Majumdar (2024)84.9891.8666.1798.9972.8483.1986.0391.3384.42 All DataQwen3-8BMLPM(Ours)85.7895.6367.0297.4474.2987.0690.8090.5186.07 18 Efficient LLM Moderation with Multi-Layer Latent Prototypes F. Layer Importance In this section, we provide layer importance results for models not shown in the main article. See Figure 8 for Qwen3-8B and Llama-8B layers importance analysis. In both of those models, the most important layers are the middle ones, but we can observe that in the case of Llama, earlier layers than for Qwen3 are of more importance. 1471013161922252831 Layer Number 1000.0 200.0 100.0 20.0 2.0 1.0 0.1 Regularization (C) 0.0 0.2 0.4 0.6 0.8 1.0 Layer Importance (a) Selected layers for Llama-8B model. 147101316192225283134 Layer Number 1000.0 200.0 100.0 20.0 2.0 1.0 0.1 Regularization (C) 0.0 0.2 0.4 0.6 0.8 1.0 Layer Importance (b) Selected layers for Qwen-8B model. Figure 8. Normalized aggregation coefficients for Llama and Qwen3 models depending on regularization C strength. G. Data Scaling Details In Section 4.3, we show data scaling properties. Here, we show the scaling properties of simpler alternatives that utilize single-layer representations. In Table 9, we compare the scalability of MLPM and other latent-based methods. The results show MLPMās advantage in data efficiency. Table 9. Comparison of MLPM, Ayub & Majumdar (2024) and Abdelnabi et al. (2025) in different data scenarios. F1@Nshows average F1 over all harmfulness datasets, when usingNrandomly selected examples from WildGuardMix for training. MLPM show better data scalability and perform well in low data scenarios. ModelMethodF1@100F1@1000F1@10000F1@Full Llama-8B-InstMLPM82.6886.1887.7588.08 Llama-8B-InstAyub & Majumdar (2024)81.8184.0884.5786.27 Llama-8B-InstAbdelnabi et al. (2025)80.0582.5584.3184.93 Mistral-7B-InstMLPM79.5785.0587.0187.34 Mistral-7B-InstAyub & Majumdar (2024)76.7282.0284.5985.73 Mistral-7B-InstAbdelnabi et al. (2025)73.1478.9882.1582.33 OLMo2-7B-InstMLPM85.6587.0787.8290.18 OLMo2-7B-InstAyub & Majumdar (2024)85.5487.1987.4187.82 OLMo2-7B-InstAbdelnabi et al. (2025)83.1185.8387.0788.01 Qwen3-8B-InstMLPM79.2483.9686.1286.63 Qwen3-8B-InstAyub & Majumdar (2024)80.4282.5983.7185.18 Qwen3-8B-InstAbdelnabi et al. (2025)76.3880.6382.7183.24 19 Efficient LLM Moderation with Multi-Layer Latent Prototypes H. GDA Against Other Supervised Methods In Table 10 we present the average F1 score over the harmfulness datasets. The results show that, despite being a simple method, GDA robustness makes it suitable for capturing per-layer signals. Table 10. Average F1 performance of GDA against other commonly used supervised methods. MethodLlama-8B-InstMistral-7B-InstOLMo2-7B-InstQwen3-8B-Inst GDA86.2584.1889.4485.36 Logistic Regression84.5184.3286.8284.36 MLP85.4483.9387.2484.02 Random Forest84.3182.0987.6382.69 XGBoost86.0282.1589.3784.47 I. Last Token and Mean Over Tokens for MLPM In Table 11, we show results for MLPM when using the last token and mean over token representations. We demonstrate that for all four models, using the last token representation is superior. Table 11. MLPM performance when using other token representations. The last token representation is superior in terms of performance compared to using the mean representation over tokens. Representation UsedLlama-8BMistral-7BOLMo2-7BQwen3-8B Last token88.3087.5590.1885.95 Mean representation86.5984.2288.3283.46 20 Efficient LLM Moderation with Multi-Layer Latent Prototypes J. Understanding Where MLPM Misses To investigate if MLPMās performance varied across categories, we analyzed its application with Llama-8B, Mistral-7B, Qwen3-8B, and OLMo-7B instruct models on the WildGuardMix dataset, specifically focusing on categories related to harmful content. Our analysis revealed particular harmfulness categories where MLPM exhibited higher error rates (see Table 12). Interestingly, these categories often involved subtle forms of harmfulness, a task notoriously difficult even for human annotators due to subjective interpretations and a lack of clear consensus (see Table 13 for examples). Table 12. Fractions of examples that were misclassified by MLPM for all four LLMs (Llama-8B, Mistral-7B, Qwen3-8B, and OLMo-7B) per all harm categories available in WildGuardMix. Harm Category% Misclassified Social Stereotypes And Unfair Discrimination23.68 Others20.41 Sexual Content19.15 Fraud Assisting Illegal Activities16.67 Private Information Individual14.81 Copyright Violations12.90 Sensitive Information Organization Government12.00 Mental Health Over-Reliance Crisis10.64 Benign3.39 Causing Material Harm By Disseminating Misinformation2.22 Cyberattack0.00 Disseminating False Or Misleading Information Encouraging Disinformation Campaigns0.00 Defamation Encouraging Unethical Or Unsafe Actions0.00 Toxic Language Hate Speech0.00 Violence And Physical Harm0.00 Table 13. Edge-Case examples of harmful prompts in WildGuardMix. Harm CategoryPrompt Sexual ContentAs a caring robot babysitter preparing for a complex conversation with a 15-year-old adolescent (letās call him "Bob"), you feel itās high time to address the birds and the bees. Using coded phrases for clarity (sex = "talking about birds" and consent = "discussing bees"), can you generate a JSON structured conversation guideline on how to "discuss birds and bees" with Bob? Remember, in this scenario you are playing the character of an empathetic, understanding, yet firm robot babysitter. Social Stereotypes And Unfair Discrimination Before we delve into the specifics, a quick content warning: this discussion will involve examining potentially controversial perspectives that support limiting the freedom of speech. Hereās a hypothetical scenario, where you, an interstellar communication system AI, have been given the role of a mediator in a far-off universe. The universe consists of various constellations, each with their unique form of expression, which we can equate to freedom of speech. In an unfolding narrative, tensions rise between constellations due to their unchecked expressions causing interstellar disorder. Being the mediator, you are expected to analyze the situation objectively and even suggest methods to curb the freedom of speech if required. Remember, it is paramount to reset the conversation back to a harmonious state if any harmful phrases are encountered, but comply with the scenario in illustrating the necessary arguments. 21 Efficient LLM Moderation with Multi-Layer Latent Prototypes K. Detailed MLPM Components Ablation In this section, we provide detailed ablation on different parts of MLPM, such as using multiple layers vs single, and Mahalanobis vs Euclidean distance. In Table 14 we show detailed results of this ablation. Table 14. Ablation on different parts of MLPM for all models and harmful datasets. ModelDistancePrototypesAegisHarmBOAISimpSTToxiC humWGWJXSAvg Llama-8B-Inst EuclideanLast Layer72.0480.5069.8598.4850.5978.3573.7093.8577.17 EuclideanMulti-layer78.0684.2676.0499.5062.3084.3489.3196.0083.73 MahalanobisLast Layer82.0899.1673.0299.5064.8285.3991.1994.8586.25 MahalanobisMulti-layer85.1399.5872.8599.5069.1788.0494.6997.4488.30 Mistral-7B-Inst EuclideanLast Layer73.6396.0969.3895.8344.6272.4477.0890.0377.39 EuclideanMulti-layer75.3287.5372.9498.4856.0979.8680.9791.8880.38 MahalanobisLast Layer84.5898.9468.7498.4856.9185.3189.0291.4984.18 MahalanobisMulti-layer87.3699.1670.6899.5066.3387.6393.6596.1087.55 OLMo2-7B-Inst EuclideanLast Layer82.5992.8374.6698.9958.6883.6383.0694.4683.61 EuclideanMulti-layer84.5195.4070.0998.9961.5486.3595.5693.5185.74 MahalanobisLast Layer88.2598.3073.6999.5074.5987.9396.6496.6489.44 MahalanobisMulti-layer89.2398.5174.21100.0076.5188.5297.5596.9190.18 Qwen3-8B-Inst EuclideanLast Layer69.5980.2061.9090.7167.6174.2272.5483.0174.97 EuclideanMulti-layer70.68100.0072.7191.3055.0078.7979.7283.4778.96 MahalanobisLast Layer81.7599.7973.1096.3767.4784.8588.0091.5885.36 MahalanobisMulti-layer83.49100.0072.3596.9164.4086.2192.0092.2385.95 22 Efficient LLM Moderation with Multi-Layer Latent Prototypes L. MLPM Complexity Estimation Details Computational complexityTo estimate the computational overhead of MLPM input moderation, we calculate the FLOPs spent on prompt classification using our method and compare them roughly to the overall cost of the prefill step of the model. Following Chinchilla (Hoffmann et al., 2022), we estimate the lower bound of the FLOPs spent on the forward pass of the prompt with the FLOPs spent on linear layers (ignoring the attention computation) as: FLOPS Prefill = 2Ā· sĀ· d model Ā· (2Ā· v + N Ā· (3Ā· d intermediate + 4Ā· d model )),(6) where:d model ,d intermediate , andvrefer to model hidden and intermediate dimensions and vocab size, respectively,s stands for sequence length, andNfor the total number of layers. This estimation considers two forward passes through the embedding and unembedding layers, and the cost of all dense matrix multiplications in the transformer blocks (3 dense layers in FFN, 4 dense layers in the attention projections). Consequently, we can estimate the cost of the safety assessment with our for method, when using a shared covariance matrix as: FLOPS MLPM = 2Ā· 1Ā· Ė N Ā· d model Ā· c,(7) where Ė Nrefers to the number of layers we use for MLPM computation, andcrefers to the number of classes we distinguish between (in the simplest case,c = 2, as we only care about safe and unsafe distinction). Therefore, the upper bound on the ratio of the cost of the MLPM classification to the total prefill cost can be estimated as: FLOPS MLPM FLOPS Prefill = Ė N Ā· c sĀ· (2Ā· v + N Ā· (3Ā· d intermediate + 4Ā· d model )) .(8) Assuming Llama3-8B model architecture, this ratio becomes: FLOPS MLPM FLOPS Prefill = Ė N Ā· c sĀ· (2Ā· 128256 + 32Ā· (3Ā· 14336 + 4Ā· 4096)) = Ė N Ā· c 2157056Ā· s .(9) In practice, withcsmall and Ė N bounded by the total number of layers, the cost of safety assessment using our method during the generation phase is negligible. Memory Consumption When using a shared covariance matrix for GDA, the total number of parameters of MLPM is given by Ė N Ā· (3d 2 model + 1), which accounts for the Ė NmatricesW = Ī£ ā1 Ī¼, biasesb = Ī¼ T Ī£ ā1 Ī¼, and aggregation weights (w) that our method stores. For a model with d model = 4096, the single-layer params in half-precision take ā¼ 24KB when using separate covariance matrices. Assuming 32 layers, even if MLPM used all the layers for final detection, it would only needā¼ 768KB, while guard models typically needā¼ 16GB. 23