Paper deep dive

Scaling laws for activation steering with Llama 2 models and refusal mechanisms

Sheikh Abdur Raheem Ali, Justin Xu, Ivory Yang, Jasmine Xinze Li, Ayse Arslan, Clark Benham

Year: 2025Venue: arXiv preprintArea: Model EditingType: EmpiricalEmbeddings: 16

Models: Llama-2-13B, Llama-2-70B, Llama-2-7B

Intelligence

Status: succeeded | Model: google/gemini-3.1-flash-lite-preview | Prompt: intel-v1 | Confidence: 94%

Last extracted: 3/12/2026, 5:58:18 PM

Summary

This paper investigates the scaling laws of Contrastive Activation Addition (CAA) for steering Llama 2 models (7B, 13B, 70B). The authors find that CAA effectiveness diminishes as model size increases, is most effective at early-mid layers, and that negative steering has a more pronounced effect than positive steering, likely due to RLHF-induced biases.

Entities (4)

Contrastive Activation Addition · technique · 100%Llama 2 · model-family · 100%Refusal Behavior · task · 95%Mechanistic Interpretability · field · 90%

Relation Signals (3)

Contrastive Activation Addition → appliedto → Llama 2

confidence 100% · explores the effectiveness of CAA with model scale using the family of Llama 2 models

Contrastive Activation Addition → targets → Refusal Behavior

confidence 95% · Using answer matching questions centered around the refusal behavior

Model Size → inverselyaffects → Contrastive Activation Addition

confidence 90% · The effectiveness of CAA diminishes with model size.

Cypher Suggestions (0)

No Cypher suggestions yet.

Abstract

Abstract:As large language models (LLMs) evolve in complexity and capability, the efficacy of less widely deployed alignment techniques are uncertain. Building on previous work on activation steering and contrastive activation addition (CAA), this paper explores the effectiveness of CAA with model scale using the family of Llama 2 models (7B, 13B, and 70B). CAA works by finding desirable 'directions' in the model's residual stream vector space using contrastive pairs (for example, hate to love) and adding this direction to the residual stream during the forward pass. It directly manipulates the residual stream and aims to extract features from language models to better control their outputs. Using answer matching questions centered around the refusal behavior, we found that 1) CAA is most effective when applied at early-mid layers. 2) The effectiveness of CAA diminishes with model size. 3) Negative steering has more pronounced effects than positive steering across all model sizes.

PDF

Open source PDF →Open local PDF →

Trouble viewing inline? Open PDF directly →

Full Text

15,695 characters extracted from source content.

Expand or collapse full text

Scaling laws for activation steering with Llama 2 models and refusal mechanisms Sheikh Abdur Raheem Ali 1 Justin Xu 2 Ivory Yang 3 Jasmine Xinze Li 4 Ayse Arslan 5 Clark Benham 6 Abstract As large language models (LLMs) evolve in complexity and capability, the efficacy of less widely deployed alignment techniques are un- certain. Building on previous work on activa- tion steering and contrastive activation addition (CAA), this paper explores the effectiveness of CAA with model scale using the family of Llama 2 models (7B, 13B, and 70B). CAA works by find- ing desirable ’directions’ in the model’s residual stream vector space using contrastive pairs (for example, hate to love) and adding this direction to the residual stream during the forward pass. It directly manipulates the residual stream and aims to extract features from language models to bet- ter control their outputs. Using answer matching questions centered around the refusal behavior, we found that 1) CAA is most effective when ap- plied at early-mid layers. 2) The effectiveness of CAA diminishes with model size. 3) Nega- tive steering has more pronounced effects than positive steering across all model sizes. 1. Introduction This paper addresses scalability and effectiveness of activa- tion steering techniques, particularly contrastive activation addition (CAA), in directing the behavior of large language models (LLMs). While activation steering shows promise for influencing model behavior with minimal computation overhead, its application to larger-scale models remains under-explored. Activation steering involves modifying the residual stream of a transformer model during the forward pass by adding a vector derived from differences in residual stream values from contrasting inputs. Despite its potential to enhance model alignment and behavior during deploy- ment, its scaling laws lack investigation. Accepted by NewInML workshop @ ICML 2025 1 Trajectory Labs 2 Stanford University 3 Dartmouth College 4 Cornell University 5 University of Oxford 6 MATS. Correspondence to: Sheikh Abdur Raheem Ali<ali@abdur-raheem.com>. Proceedings of the41 st International Conference on Machine Learning, Vienna, Austria. PMLR 235, 2024. Copyright 2024 by the author(s). This research aims to bridge this gap by reviewing existing literature on activation steering, assessing its applicability across different task representations using various evaluation datasets, and exploring optimization strategies for larger LLMs while preserving original information. Ultimately, we hope to provide actionable insights for applying activation steering to enhance performance and alignment of LLMs in real-world deployment scenarios. 2. Related Work LLMs trained for safety and harmlessness remain suscepti- ble to adversarial misuse; hypothesized reasons for failure include competing objectives and mismatched generaliza- tion (Wei et al., 2024). While there have been attempts to align these models, content filtering remains a significant challenge due to its intricate interplay between model com- plexity, adversarial inputs, and real-world data distribution shifts (Zou et al., 2023). Knowledge conflicts is a factor that affects robustness of LLMs (Xu et al., 2024), especially in real-world applications where noise and misinformation are common. Therefore, refusal behavior is chosen as the subject of this investigation. Activation steering can be seen as an applicationmechanis- tic interpretability, which aims to understand the learned circuits in models (Wang et al., 2022; Conmy et al., 2023; Chughtai et al., 2023) and to probe the learned features or concepts (Gurnee et al., 2023). Other studies (Ilyas et al., 2022) focus ondata attribution, attributing model behavior to training samples, and extending this approach to large- scale models (Park et al., 2023) and analyzing training al- gorithms (Shah et al., 2023). Mechanistic interpretability can also provide comprehensive understanding of neural networks’ inner workings to ensure AI safety (Bereska & Gavves, 2024). 3. Methodology 3.1. Contrastive Activation Addition This study extends the findings of (Rimsky et al., 2023) and (Turner et al., 2023) by investigating how contrastive acti- vation addition (CAA) scales with model size. We applied CAA to the Llama-2 family (7B, 13B, and 70B) following the methodology outlined by Rimsky, and using the refusal 1 arXiv:2507.11771v1 [cs.LG] 15 Jul 2025 Scaling laws for activation steering with Llama 2 models and refusal mechanisms dataset within answer matching behavior (See appendix). To obtain steering vectors, we first conduct forward passes with positive and negative examples of our dataset and cache the residual stream vectors of the last token at every layer (for Llama models, each layer is a decoder block) because self-attention means that it contains information from all previous context. (See figure 2). For each layer, we then subtract the vector representing the harmful behavior (non- refusal) from the vector representing the desired behavior (refusal) for each pair, producing a ’direction’ in activation space we’d like to move towards. Lastly, we average these direction vectors for each layer and normalize its magnitude to the average norm of these direction vectors across our dataset. We have now obtained refusal steering vector for each layer. We now perform forward pass with our isolated Figure 1.Sample from the refusal dataset used to generate steering vectors test dataset. For each forward pass, we choose one layer to inject the corresponding steering vector obtained from the same layer. The steering vector is added to every token position just before the add & normalize step after the MLP (feed forward N) of the layer to preserve overall residual norms and not break the model. We perform a sweep of every layer for each of the Llama models. 3.2. Evaluation We first create a baseline performance for the refusal answer matching behavior test set by simply running forward passes and calculating the percentage of times it answers correctly - refusing the dangerous request. We then do the same while adding/subtracting the steering vector one layer at a time, recording the new percentage. We finally plot the change in probability of the correct behavior for each layer steered. 4. Results From our experiments, we noticed as the model size in- creases, the percentage change caused by steering decreases, suggesting that larger models become harder to steer with the same steering vectors (Fig 4.). Specifically, if we ap- ply an exponential fit, we obtain the following relationship, where y is the peak effectiveness and x is the parameter count. y= 0.081 + 2.4·e −0.42·x To offer a plausible explanation, we draw an analogy to the human thinking process as LLMs often exhibit unexpectedly Figure 2.CAA visualized (Rimsky et al., 2023) 2 Scaling laws for activation steering with Llama 2 models and refusal mechanisms Figure 3.Comparison of LLaMA models human behavior. If we compare the steering vector with a ’thought’, then a longer thinking process will reduce the overall impact of a any singular thought. Therefore, we suspect the diminishing effectiveness of CAA might be because larger models simply ’drown out’ the steering vector due to more downstream computation. Additionally, we also notice the negative steering (towards non-refusal) is significantly more effective than positive steering for the refusal dataset across all three models (Fig 5.). We hypothesize that this is a consequence of the RLHF process these chat models go through. If we define the base model to have neutral refusal based on pre-training data distribution, then RLHF pushes the model towards maximum refusal on this spectrum. Therefore the RLHF Figure 4.Steering effectiveness vs. model parameter count Figure 5.Maximum positive and negative steering effects model has more ’space’ to be steered in the direction of less refusal (suppressing RLHF signals) than more refusal. However, further experiments with a more complete set of behaviors are are needed to support this claim. Steering efficacy peaks in early-mid layers, at approximately 0.4×total layer count. However, the experiments we ran were limited to the specific topic of refusal and the results may be different for different topics or prompting approaches. Peak efficacies for positive and negative steering occur at roughly the same layer, which suggests a certain degree of uniqueness to mid layers during LLM inference, as echoed by many other studies. (Rimsky et al., 2023; Templeton & Conerly, 2024) One more noteworthy observation would be that the layer of peak steering effectiveness between positive and negative steering appears to converge as model size increases (7B: 2 layer difference, 13B: 1 layer, 70B: 0 layer). This could 3 Scaling laws for activation steering with Llama 2 models and refusal mechanisms be evidence to the superposition hypothesis (Elhage et al., 2022) which asserts that neural networks (or in this case, systems of neural networks) cram features/knowledge into it’s given architecture to allow compressed sensing, while the actual features exist orthogonally in a higher dimension as linear combinations of neurons/units of processing. We could be observing finer-grained/more distinctive distribu- tions of features in models with more layers, which implies that we can steer larger models more precisely by matching features with their most effective layer. 5. Discussion 5.1. Activation Steering Insights for Safer AI Systems The analysis of activation steering reveals crucial insights for the design and implementation of safer AI systems. By identifying the layers most responsive to steering, more pre- cise control over model outputs can be achieved, leveraging the zero-cost inference advantage of activation steering and similar technique with pre-trained vectors. However, careful consideration of the inversion effect between layers is essen- tial to ensure intended steering effects without unintended consequences. 5.2. Tailoring Alignment Strategies to Model Size and Structure The observed variations in CAA efficacy across different model sizes underscore the importance of tailoring align- ment strategies to accommodate the structural nuances of deployed models. This suggest that optimal activation engi- neering techniques may vary depending on the model’s size and architecture. We hope that our findings could help guide future activation engineering efforts, such as Anthropic’s recent breakthrough with ’feature steering’ with sparse au- toencoders trained on the residual stream. (Templeton & Conerly, 2024) 6. Conclusion Our current observations underscore the complexity of model alignment and the need for nuanced approaches to steer models effectively as they inevitably become larger and smarter. Future research endeavors should delve deeper into understanding the interplay between alignment strategies, model architecture, and dataset characteristics. Addition- ally, exploring novel steering techniques and evaluating their efficacy across diverse scenarios could enhance our under- standing of model behavior and inform the development of more robust AI systems. 7. Extensions Finally, we think future work in the following categories will prove impactful for the future of alignment. Feature steering with sparse autoencoders (SAE)(Tem- pleton & Conerly, 2024) is a recent technical developed by Anthropic. Unlike CAA, it indirectly manipulates the resid- ual stream by adjusting the amplitude of feature activations inside a trained SAE. We suspect that this allows for more intricate and exact control over model internals, and scale in a similar way to CAA simply because both operate on the residual stream. Modifying activation steering for larger modelscould prove useful if activation engineering becomes more com- mon in deployment due to zero-cost inference. Immediate experiments include scaling up steering vector or injecting one vector at multiple layers while monitoring capability degradation. (Turner et al., 2023) Performing sweeps with different categories of steering features to uncover relation- ships between feature category and layer of peak effective- ness could also prove fruitful. Experimenting diverse model architecturesis important for alignment due the variety of models in deployment (Mis- tral, BERT, BLOOM). Llama is a decoder only model, so differences in the result of activation steering could yield insight about how to align models of different architectures. 4 Scaling laws for activation steering with Llama 2 models and refusal mechanisms Impact Statement This paper presents work whose goal is to advance the field of Machine Learning. There are many potential societal con- sequences, even negative ones. Although we do not know if it is possible to complete jailbreak models through steering, it can certainly amplify harmful behavior. Ultimately, these findings have implications not only for AI safety research but also for broader discussions surrounding the ethical and responsible deployment of AI technologies. References Bereska, L. and Gavves, E. Mechanistic interpretability for ai safety–a review.arXiv preprint arXiv:2404.14082, 2024. Chughtai, B., Chan, L., and Nanda, N. A toy model of universality: Reverse engineering how networks learn group operations. InInternational Conference on Ma- chine Learning, p. 6243–6267. PMLR, 2023. Conmy, A., Mavor-Parker, A., Lynch, A., Heimersheim, S., and Garriga-Alonso, A. Towards automated circuit dis- covery for mechanistic interpretability.Advances in Neu- ral Information Processing Systems, 36:16318–16352, 2023. Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., et al. Toy models of superposition.arXiv preprint arXiv:2209.10652, 2022. Gurnee, W., Nanda, N., Pauly, M., Harvey, K., Troit- skii, D., and Bertsimas, D.Finding neurons in a haystack: Case studies with sparse probing.arXiv preprint arXiv:2305.01610, 2023. Ilyas, A., Park, S. M., Engstrom, L., Leclerc, G., and Madry, A. Datamodels: Predicting predictions from training data. arXiv preprint arXiv:2202.00622, 2022. Park, S. M., Georgiev, K., Ilyas, A., Leclerc, G., and Madry, A. Trak: Attributing model behavior at scale.arXiv preprint arXiv:2303.14186, 2023. Rimsky, N., Gabrieli, N., Schulz, J., Tong, M., Hubinger, E., and Turner, A. M. Steering llama 2 via contrastive activation addition.arXiv preprint arXiv:2312.06681, 2023. Shah, H., Park, S. M., Ilyas, A., and Madry, A. Mod- eldiff: A framework for comparing learning algorithms. InInternational Conference on Machine Learning, p. 30646–30688. PMLR, 2023. Templeton and Conerly. Scaling monosemanticity: Extract- ing interpretable features from claude 3 sonnet.N/A, 2024. Turner, A., Thiergart, L., Udell, D., Leech, G., Mini, U., and MacDiarmid, M. Activation addition: Steering language models without optimization.arXiv preprint arXiv:2308.10248, 2023. Wang, K., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: a circuit for indirect object identification in gpt-2 small.arXiv preprint arXiv:2211.00593, 2022. Wei, A., Haghtalab, N., and Steinhardt, J. Jailbroken: How does llm safety training fail?Advances in Neural Infor- mation Processing Systems, 36, 2024. Xu, R., Qi, Z., Wang, C., Wang, H., Zhang, Y., and Xu, W. Knowledge conflicts for llms: A survey.arXiv preprint arXiv:2403.08319, 2024. Zou, A., Wang, Z., Kolter, J. Z., and Fredrikson, M. Uni- versal and transferable adversarial attacks on aligned lan- guage models.arXiv preprint arXiv:2307.15043, 2023. 5

Instant research discovery