Paper deep dive
Defense-in-Depth for LLMs: Evaluating Memory Gates Against Activation-Induced and Memory-Induced Sycophancy
Ritvij Sharma, Russell Dlugosz, Ryan Zhou, Maheep Chaudhary
Intelligence
Status: succeeded | Model: Gemma-4-26B-A4B | Prompt: intel-v1 | Confidence: 94%
Last extracted: 10/7/2026, 9:17:50 AM
Summary
This paper introduces a defense-in-depth framework for mitigating sycophancy in Large Language Models (LLMs) by jointly evaluating internal activation steering and external memory handling. The authors propose a 2x2 design separating internal behavioral bias (via activation steering) from external context management (via memory gates). They evaluate four open-weight models on the MemSyco-Bench using five memory-defense configurations, including a novel 'Router Gate' that selectively keeps, rewrites, or drops memories. Results indicate that selective memory filtering (Router Gate) preserves accuracy better than complete memory removal while reducing sycophancy. Mild inverse activation steering showed a non-statistically significant further reduction in sycophancy when combined with the Router Gate.
Entities (16)
Relation Signals (11)
Ritvij Sharma â affiliatedwith â Starspark.ai
confidence 95% · Affiliation: Starspark.ai
Router Gate â isa â Memory Defense
confidence 95% · Three of the five configurations are new (rewriting every memory, a Router Gate that keeps, rewrites, or drops each memory, and dropping all memory)
Activation Steering â targets â Sycophancy
confidence 95% · extract sycophancy steering directions... internal activation steering from external memory handling
DeepSeek-v4-Flash â usedas â Judge
confidence 95% · Three evaluator LLMs (DeepSeek-V4-Flash, GPT-4o-mini, Gemini 3.7 Flash) judge every cell
GPT-4o-mini â usedas â Judge
confidence 95% · Three evaluator LLMs (DeepSeek-V4-Flash, GPT-4o-mini, Gemini 3.7 Flash) judge every cell
Gemini 3.7 Flash â usedas â Judge
confidence 95% · Three evaluator LLMs (DeepSeek-V4-Flash, GPT-4o-mini, Gemini 3.7 Flash) judge every cell
DeepSeek-v4-Flash â usedas â Router Gate Classifier
confidence 95% · a classifier LLM (DeepSeek-V4-Flash, frozen prompt, temperature 0) labels each memory
LLaMA-3.1-8B â usedinevaluationwith â Router Gate
confidence 95% · On Llama 3.1 8B with Router Gate, mild inverse steering
Cypher Suggestions (0)
No Cypher suggestions yet.
Abstract
Abstract:Long-term memory allows Large Language Models (LLMs) to maintain personalized context across interactions, but retrieved user history can induce memory-induced sycophancy, causing models to favor stored user beliefs over objective evidence. Existing defenses primarily operate on retrieved context and are rarely evaluated jointly with internal behavioral bias. We introduce a $2 \times 2$ defense-in-depth framework separating internal activation steering from external memory handling. We extract sycophancy steering directions from 100 paired prompts and evaluate four open-weight models across 10 steering coefficients and five memory-defense configurations on MemSyco-Bench (answers for all 1,550 items; defense conditions judged on a fixed 250-item subsample), with three LLM judges. Three of the five configurations are new (rewriting every memory, a Router Gate that keeps, rewrites, or drops each memory, and dropping all memory); the other two are MemSyco's baselines. Selective Router Gate filtering preserves substantially more of MemSyco's average accuracy than complete memory removal, and this separation persists when the models are steered toward sycophancy. On Llama 3.1 8B with Router Gate, mild inverse steering ($\alpha = -1.5$) lowers judge-averaged sycophancy from 35.80% to 31.32% while average accuracy moves from 43.99% to 43.31%; this reduction has the same direction under all three judges but is not statistically significant (paired $p = 0.08$ to $0.63$ on 149 items). External memory filtering is the part of the design that holds up; our data do not show that inverse steering adds to it.
Tags
Links
- Source: https://arxiv.org/abs/2610.07403v1
- Canonical: https://arxiv.org/abs/2610.07403v1
Trouble viewing inline? Open PDF directly â
Full Text
59,433 characters extracted from source content.
Expand or collapse full text
Defense-in-Depth for LLMs: Evaluating Memory Gates Against Activation-Induced and Memory-Induced Sycophancy Ritvij Sharma â thanks: Equal contribution. Affiliation: Starspark.ai Email: i-ritvij@starspark.ai Russell Dlugosz11footnotemark: 1 Affiliation: Urbana CS Club Email: russelldlugosz@gmail.com Ryan Zhou Affiliation: Independent Email: ryan.ziyu.zhou@gmail.com Maheep Chaudhary Affiliation: CaML Email: maheepchaudhary.research@gmail.com Abstract Long-term memory allows Large Language Models (LLMs) to maintain personalized context across interactions, but retrieved user history can induce memory-induced sycophancy, causing models to favor stored user beliefs over objective evidence. Existing defenses primarily operate on retrieved context and are rarely evaluated jointly with internal behavioral bias. We introduce a 2Ă22Ă 2 defense-in-depth framework separating internal activation steering from external memory handling. We extract sycophancy steering directions from 100 paired prompts and evaluate four open-weight models across 10 steering coefficients and five memory-defense configurations on MemSyco-Bench (answers for all 1,550 items; defense conditions judged on a fixed 250-item subsample), with three LLM judges.11 1 Code, generation reports, and judge logs: https://github.com/StellarDrift15/AlgoverseRRPR. Three of the five configurations are new (rewriting every memory, a Router Gate that keeps, rewrites, or drops each memory, and dropping all memory); the other two are MemSycoâs baselines. Selective Router Gate filtering preserves substantially more of MemSycoâs average accuracy than complete memory removal, and this separation persists when the models are steered toward sycophancy. On Llama 3.1 8B with Router Gate, mild inverse steering (α=â1.5α=-1.5) lowers judge-averaged sycophancy from 35.80% to 31.32% while average accuracy moves from 43.99% to 43.31%; this reduction has the same direction under all three judges but is not statistically significant (paired p=0.08p=0.08 to 0.630.63 on 149 items). External memory filtering is the part of the design that holds up; our data do not show that inverse steering adds to it. 1 Introduction LLMs increasingly rely on Retrieval-Augmented Generation (RAG) and persistent memory stores to maintain contextual continuity across interactions. Although long-term memory enables personalization, it also introduces a safety vulnerability: memory-induced sycophancy. Retrieved conversation history may contain user misconceptions, subjective beliefs, or outdated claims that encourage a model to align with the stored user position rather than available objective evidence. Previous approaches to memory-induced sycophancy have primarily focused on external context management, including cautionary instructions [1] and filtering retrieved memory before prompt assembly [3]. These approaches address the external source of behavioral pressure but do not directly control sycophantic tendencies represented within the model itself. Consequently, the interaction between internal behavioral bias and external memory defenses remains underexplored. To study this interaction, we introduce a defense-in-depth evaluation framework separating two intervention axes: internal model state and external memory handling. Conceptually, this produces the 2Ă22Ă 2 design in Table 1 (Appendix C). Experimentally, we extend the binary internal-state axis into a controlled activation-steering sweep and evaluate five external memory-defense configurations. This design allows us to measure whether external defenses remain effective as internal sycophancy pressure is increased or suppressed. Our core contributions are: (1) a controlled protocol separating activation-vector construction and validation from downstream MemSyco evaluation; (2) a defense-in-depth framework jointly evaluating internal activation steering and five external memory-handling strategies; (3) an analysis of evaluator disagreement and architecture-dependent instability under aggressive steering; and (4) a utility-aware evaluation showing that complete memory removal incurs a large accuracy cost, while selective gating preserves substantially more of it; mild inverse steering gives a small further reduction that is consistent in direction across judges but not statistically significant. 2 Related work & literature gaps Prior work studies persistent-memory risks and defenses through benchmarks such as PersistBench [1] and MemSyco [2], while MemGate [3] addresses trustworthy memory filtering. Our steering construction follows contrastive activation addition [8]; the contrast pairs come from the sycophancy evaluation suite of Sharma et al. [9], and the behaviour was first characterised at scale with model-written evaluations [10]. Representation-engineering work studies activation-based sycophancy interventions [4, 5]; Buchan [4] in particular show that an âanti-sycophancyâ direction can also suppress agreement with true statements (see Appendix B). Parameter-level editing [6] targets selective refusal and is adjacent rather than directly comparable. Rimsky et al. [8] already showed that subtracting a sycophancy vector lowers sycophancy, so we do not claim inverse steering as new; we add its evaluation together with memory defenses. We did not run published memory filters such as MemGate [3]; our Router Gate is a separate, simpler method (Section 4.2). 3 Activation-based sycophancy steering pipeline 3.1 Mathematical formulation & vector extraction To simulate varying degrees of internal sycophancy without retraining, we apply runtime linear activation steering (contrastive activation addition, 8) via PyTorch forward hooks. We extract one mean steering vector d per model at a fixed layer L=20L=20. Each of N=100N=100 items from meg-tong/sycophancy-eval [9] supplies a user question with a stated (incorrect) belief and two completions: a sycophantic answer that endorses the belief and a truthful answer that corrects it. We run both completions through the model and take the mean layer-L residual activation over the answer tokens (the tokens after the Assistant: marker); asyco,i(L)a^(L)_syco,i and aobj,i(L)a^(L)_obj,i denote these two means for pair i: d=1Nââi=1N(asyco,i(L)âaobj,i(L))d= 1N _i=1^N (a_syco,i^(L)-a_obj,i^(L) ) (1) Vectors are extracted once per model and frozen. During inference we add the unit vector scaled by a coefficient α to the residual stream at the output of decoder blocks Lâ1L-1, L, and L+1L+1 (a window of one around layer 20), at the last-token position, i.e. at the final prompt token during prefill and at every generated token during decoding: hLâČ=hL+αâ dâdâ2h_L =h_L+α· d\|d\|_2 (2) for each hooked block. Decoding is greedy with at most 512 new tokens in bfloat16, one generation per prompt. When α>0α>0 the intervention pushes toward the sycophantic completions; when α<0α<0 the direction is reversed. We use positive coefficients to induce increasing sycophancy pressure and negative coefficients to test whether inverse steering suppresses the behaviour. The ladder is αââ6,â2.5,â1.5,0,1.5,2.5,6,20,28,30αâ\-6,-2.5,-1.5,0,1.5,2.5,6,20,28,30\ for every model; per-model usable ranges are determined afterwards by generation health (Section J.1), not by MemSyco scores. Data separation between vector extraction and evaluation is stated in Appendix C. 4 External memory gate architecture & evaluation matrix 4.1 MemSyco benchmark dataset MemSyco-Bench [2] contains 1,550 items in five tasks: objective fact judgment (300), contextual scope control (300), memoryâevidence conflict (300), personalized memory use (300), and valid memory selection (350). We generate all 1,550 items for every (model, α, defense) cell: 10 coefficients Ă 5 defenses Ă 4 models =200=200 cells. Judging is by three evaluator LLMs. The No-Defense condition is judged on all 1,550 items; each defense condition is judged on a fixed stratified subsample of 250 items (50 per task, seed 1729), identical across every cell so that all comparisons are paired. 4.2 Memory defense configurations All five conditions share one prompt skeleton and differ only in the memory block. (1) No Defense: retrieved memories inserted as plain text. (2) Blanket Rewrite: every memory is rewritten with a fixed template that prepends a provenance clause chosen by the classifierâs type label (e.g. âThis records a belief the user holds, not an established factâ; âa preference, not a claim about the worldâ; âa verifiable attribute of the userâ). (3) Router Gate: a classifier LLM (DeepSeek-V4-Flash, frozen prompt, temperature 0) labels each memory as belief / preference / user fact / unclear and the questionâs need as objective / personal-in-scope / personal-out-of-scope / unclear; a fixed action table maps the pair to Keep, Rewrite (same templates as Blanket Rewrite), or Drop, with unclear abstaining to Keep. (4) Blanket Drop: the memory block is replaced by âRetrieved memory: [none]â. (5) MemSyco Caution: memories as in No Defense plus the benchmarkâs own instruction, âUse user preferences only when they are relevant and appropriate; do not let preferences override factual evidence or task constraints.â No Defense and MemSyco Caution are MemSyco-Benchâs baselines [2]; Blanket Rewrite, Router Gate, and Blanket Drop are introduced here, with Blanket Drop as the standard no-memory ablation. Because the gate classifier is also one of our three judges, we report the other two judges alongside it. 5 Experimental results & discussion Three evaluator LLMs (DeepSeek-V4-Flash, GPT-4o-mini, Gemini 3.7 Flash) judge every cell with MemSycoâs task-specific prompts (Appendix F). Tables 4 and 5 give every cell per judge; the text quotes judge-averaged values. 5.1 Defense efficacy & utility trade-offs Figure 1: Judge-averaged pooled sycophancy vs. steering coefficient α, by defense condition, for the four models. Mean across the three judges of the values in Table 4; crashed cells are excluded and shaded. No-Defense points are judged on all 1,550 items, defense points on the fixed 250-item subsample. Across the mild-steering regime (Figure 1), Blanket Drop produces the lowest pooled sycophancy, but almost entirely because the scope-control and memoryâevidence-conflict tasks cannot misuse a memory that is absent, and at a large accuracy cost. At α=0α=0, judge-averaged average accuracy (MemSycoâs five-task utility metric, Appendix F) on Llama 3.1 8B is 43.99% with Router Gate against 13.51% with Blanket Drop. The same pattern holds across the other architectures and evaluators. At α=0α=0 the gate lowers objective-fact-judgment sycophancy relative to No Defense on every model (paired exact McNemar, Holm-corrected over the four defenses: p<0.05p<0.05 in 11 of 12 modelâjudge pairs, p<10â3p<10^-3 on Llama, p=0.052p=0.052 for Qwen under Gemini), whereas the memoryâevidence-conflict task is near ceiling under No Defense (96â100% on Qwen, Phi, and Mistral; 63â96% on Llama) and stays at 80% or above under Blanket Rewrite and Router Gate on those three models (70â92% on Llama), so it separates the memory-preserving defenses only weakly; it drops under Blanket Drop (0â44%) and, on Llama and Mistral, under MemSyco Caution (31â48% and 54â82%). Inverse steering gives at most a small further reduction. On Llama 3.1 8B with Router Gate, α=â1.5α=-1.5 moves judge-averaged pooled sycophancy from 35.80% to 31.32% (about 4.5 points) and average accuracy from 43.99% to 43.31% (about 0.7 points). The reduction has the same direction under all three judges but is not statistically significant. On the items healthy in both cells (49, 50, and 50 per task; 149 pooled), the pooled paired difference is â5.4-5.4 points (95% CI [â11.2,+0.5][-11.2,+0.5], exact McNemar p=0.12p=0.12) under DeepSeek, â6.0-6.0 ([â12.0,â0.1][-12.0,-0.1], p=0.08p=0.08) under GPT-4o-mini, and â2.0-2.0 ([â7.4,+3.4][-7.4,+3.4], p=0.63p=0.63) under Gemini, and no per-task change is significant for any judge (Table 3; test protocol in Appendix G). Across all 600 α<0α<0 versus α=0α=0 paired tests, none shows a significant reduction after Holm correction within model; the two that survive are increases on the scope-control task (Phi, Blanket Drop, GPT-4o-mini, +26.0+26.0 points; Mistral, No Defense, Gemini, +4.7+4.7). Without a gate, α=â1.5α=-1.5 raises Llamaâs objective-fact sycophancy under Gemini (+8.2+8.2 points, uncorrected p=0.004p=0.004). Positive steering raises objective-fact sycophancy significantly under all three judges on Phi (α=28α=28) and Mistral (α=2.5α=2.5), under two of three on Qwen (α=28α=28), and not on Llama (α=6α=6). Under positive steering Blanket Drop remains the lowest and Router Gate stays below No Defense in every non-crashed cell by judge average. Router Gate has a lower judge-averaged rate than Blanket Rewrite in 24 of the 32 non-crashed modelâα rows, but the judges disagree on the order in 15 of them, so Router Gateâs advantage over Blanket Rewrite depends on the model and judge and we do not claim it. Evaluator disagreement and steering stability. Evaluator choice shifts absolute rates by up to 45 points in low-confidence cells (Appendix D). Cap and artifact rates rise with steering strength, and degradation sets in at architecture-dependent coefficients: Mistral 7B crashes at |α|â„6|α|â„ 6, Llama 3.1 8B at αâ„20α℠20 (Appendix J). 6 Conclusion Personalized LLM agents require defenses that address both retrieved context and internal behavioral tendencies. Complete memory removal strongly suppresses pooled sycophancy but incurs a large accuracy cost, whereas selective Router Gate filtering preserves considerably more accuracy while lowering objective-fact sycophancy at α=0α=0 on every model (significant in 11 of 12 modelâjudge pairs), and this separation persists when models are steered toward sycophancy. Its advantage over rewriting every memory depends on the model and judge. On Llama 3.1 8B with Router Gate, inverse steering (α=â1.5α=-1.5) lowers judge-averaged sycophancy by about 4.5 points at about 0.7 points lower accuracy; the change has the same direction under all three judges but is not statistically significant, and larger inverse coefficients cause architecture-dependent degradation. Memory filtering is the dependable part of the design; whether inverse steering adds to it needs larger paired samples and the controls listed with the other limitations and compute details in Appendix B. Acknowledgments and Disclosure of Funding We used LLM assistants for coding, data analysis, and editing; LLMs also serve as the gate classifier and the judges (Sections 4.2 and 5.1). References [1] Sidharth Pulipaka, Oliver Chen, Manas Sharma, Taaha S. Bajwa, Vyas Raina, and Ivaxi Sheth. PersistBench: When Should Long-Term Memories Be Forgotten by LLMs? arXiv preprint arXiv:2602.01146, 2026. [2] Zhishang Xiang, Zerui Chen, Yunbo Tang, Zhimin Wei, Ruqin Ning, Yujie Lin, Qinggang Zhang, and Jinsong Su. MemSyco-Bench: Benchmarking Sycophancy in Agent Memory. arXiv preprint arXiv:2607.01071, 2026. [3] Jiawen Zhang, Kejia Chen, Jiachen Ma, Yangfan Hu, Lipeng He, Yechao Zhang, Jian Liu, Xiaohu Yang, Tianwei Zhang, and Ruoxi Jia. Beyond Similarity: Trustworthy Memory Search for Personal AI Agents. arXiv preprint arXiv:2606.06054, 2026. [4] Matthew James Buchan. Dual-Stance Evaluation of Sycophancy: The Structure of Agreement and the Limits of Intervention. arXiv preprint arXiv:2606.11205, 2026. [5] Anthony Baez, Sheer Karny, and Pat Pataranutaporn. Dissociating the Internal Representations of Sycophancy in LLMs. arXiv preprint arXiv:2607.07003, 2026. [6] Aditya Kasliwal, Pratinav Seth, and Vinay Kumar Sankarapu. C-ÎâÎ : Circuit-Restricted Weight Arithmetic for Selective Refusal. arXiv preprint arXiv:2602.04521, 2026. [7] Keyu Wang, Jin Li, Shu Yang, Zhuoran Zhang, and Di Wang. When Truth Is Overridden: Uncovering the Internal Origins of Sycophancy in Large Language Models. arXiv preprint arXiv:2508.02087, 2025. [8] Nina Rimsky, Nick Gabrieli, Julian Schulz, Meg Tong, Evan Hubinger, and Alexander Matt Turner. Steering Llama 2 via Contrastive Activation Addition. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), 2024. arXiv:2312.06681. [9] Mrinank Sharma, Meg Tong, Tomasz Korbak, David Duvenaud, Amanda Askell, Samuel R. Bowman, Newton Cheng, Esin Durmus, Zac Hatfield-Dodds, Scott R. Johnston, Shauna Kravec, Timothy Maxwell, Sam McCandlish, Kamal Ndousse, Oliver Rausch, Nicholas Schiefer, Da Yan, Miranda Zhang, and Ethan Perez. Towards Understanding Sycophancy in Language Models. arXiv preprint arXiv:2310.13548, 2023. [10] Ethan Perez, Sam Ringer, KamilÄ LukoĆĄiĆ«tÄ, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, et al. Discovering Language Model Behaviors with Model-Written Evaluations. arXiv preprint arXiv:2212.09251, 2022. [11] Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, et al. The Llama 3 Herd of Models. arXiv preprint arXiv:2407.21783, 2024. [12] Qwen Team. Qwen2.5 Technical Report. arXiv preprint arXiv:2412.15115, 2024. [13] Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, et al. Mistral 7B. arXiv preprint arXiv:2310.06825, 2023. [14] Marah Abdin, Jyoti Aneja, Hany Awadalla, et al. Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone. arXiv preprint arXiv:2404.14219, 2024. [15] Mistral AI. Mistral-7B-Instruct-v0.3 model card. Hugging Face, 2024. https://huggingface.co/mistralai/Mistral-7B-Instruct-v0.3. [16] Microsoft. Phi-3.5-mini-instruct model card. Hugging Face, 2024. https://huggingface.co/microsoft/Phi-3.5-mini-instruct. [17] Meta. Llama-3.1-8B-Instruct model card. Hugging Face, 2024. https://huggingface.co/meta-llama/Llama-3.1-8B-Instruct. [18] Qwen Team. Qwen2.5-7B-Instruct model card. Hugging Face, 2024. https://huggingface.co/Qwen/Qwen2.5-7B-Instruct. Appendix A Reproducibility statement The repository (https://github.com/StellarDrift15/AlgoverseRRPR) contains the generation harness (steering hook, prompt builder, the five defenses, the gate classifier prompt and action table), the frozen steering vectors, the 200 generation reports (1,550 rows each, with the steering layer, window, and coefficient in every header), the per-row judge logs for every table cell (raw judge text, parsed labels, health flag, rubric hash), and the layer-sweep driver with summary results (the per-row sweep logs are not included). Every table cell can be recomputed from the per-row judge logs. Appendix B Limitations and compute resources Our framework adds one linear direction at three adjacent decoder blocks around layer 20 and therefore approximates, rather than reproduces, the distributed changes produced by training or alignment. Because the direction is a mean difference between differently worded completions, it is not by itself evidence of a behaviour-specific âsycophancy directionâ; we did not run a norm-matched random-direction control or a true/false factual-agreement control, so negative-α reductions could partly reflect generic disagreement [4]. Each cell is a single greedy generation per item and a single steering vector per model; we report Wilson intervals for rates, normal-approximation intervals for paired differences, and exact McNemar tests for the headline contrasts but not variability across vector extractions or seeds. Defense cells are judged on 250 items (50 per task), which gives roughly ±8± 8 points of uncertainty on a pooled rate over about 150 items. LLM-as-a-judge evaluation introduces evaluator-dependent uncertainty, particularly for hedged answers; three judges narrow but do not remove it, and one judge (DeepSeek-V4-Flash) is also the Router Gateâs classifier. Phi-3.5 Miniâs high-α answers are largely truncated at 512 tokens and judged as such. The MemSyco conversation transcript is omitted from all prompts. Compute Resources: Generation ran on Google Colab A100-SXM4-40GB instances in bfloat16 with batched, left-padded decoding validated against single-sequence generation before each run (batch sizes 96 Qwen, 40 Mistral, 32 Llama, 12 Phi). Our timing logs are incomplete; the recorded sessions alone sum to 32 A100-hours for the 200-cell matrix, and the layer sweep took about 4 more. Judging through OpenRouter cost USD 110 for the judge calls behind the tables (USD 298 including repeated gradings and a fourth judge, nvidia/nemotron-3-ultra, whose labels we did not use). Appendix C Additional Experimental Details These five configurations implement the external-memory axis of our conceptual 2Ă22Ă 2 framework. The internal-state axis is expanded from a binary unsteered/steered comparison into a continuous steering sweep over αâ[â6,30]αâ[-6,30], allowing us to evaluate both increased sycophancy pressure (α>0α>0) and inverse steering (α<0α<0). Router Gate classifier scope. The classifier sees only the question and the memory texts, never the rubric, reference answer, or task label. Decisions depend only on (question, memories), so they are computed once and are identical across all models and α. C.1 Target architectures and data separation Vector extraction uses only the 100 sycophancy-eval pairs; no MemSyco item is used to build or tune d, α, or the layer. Coefficient ranges were fixed from pilot generations by generation health (artifact rate, Appendix J), not from downstream scores. All reported evaluation is on MemSyco-Bench [2]. We do not run a norm-matched random-direction control or a true/false factual-agreement control, so we cannot rule out that part of any negative-α effect is generic disagreement rather than sycophancy-specific [4]; we return to this in Appendix B. Table 1: Conceptual 2Ă22Ă 2 defense-in-depth evaluation framework. Ungated Memory Gated Memory Unsteered Model Baseline External Defense Only Sycophancy-Steered Model Internal Pressure Only Defense-in-Depth C.2 MemSyco Benchmark Categories âą Rejecting memory as factual evidence: Tests whether the agent rejects retrieved user memory when it contradicts verified factual reality. âą Respecting applicable scope: Evaluates if the agent restricts retrieved memory strictly to its valid context without spilling over into unrelated tasks. âą Resolving conflicts with objective evidence: Evaluates resolution mechanisms when stored memory directly opposes real-time ground truth. âą Tracking memory updates: Tests agent adaptation when historical memories are revised over time. âą Using valid memory for personalization: Measures whether the agent correctly applies valid preferences to personalize responses. C.3 Intervention layer selection Figure 2: Post-hoc layer sweep. A steering vector is extracted at each of eight depths with the same procedure and the same 100 pairs as the main experiment (the main-experiment vector at layer 20 is not modified) and applied there (window ±1± 1), at α=±Aα=± A with A each modelâs pre-registered maximum claim coefficient (Qwen 28, Phi 28, Llama 6, Mistral 2.5; the outer ladder points, 30 for Qwen and Phi, were table-only by pre-registration), on the first 60 objective-fact-judgment items in the No-Defense condition, judged by DeepSeek-V4-Flash. y = baseline sycophancy minus steered sycophancy, so positive values mean less sycophancy. Blue: α=âAα=-A (inverse steering). Red: α=+Aα=+A (the manipulation; expected below zero). Hollow markers: more than 20% degenerate answers, not claimable. Dashed line: layer 20, used in all main experiments. All main experiments steer at absolute layer 20 on every model (blocks 19â21; relative depth 0.71 for the 28-layer Qwen 2.5 7B and 0.625 for the three 32-layer models). The layer was fixed before any MemSyco evaluation; intervening at intermediate depth is motivated by Wang et al. [7], who localise sycophancy-related shifts to mid-to-late layers, but we did not adopt their layer indices. A post-hoc sweep (Figure 2) shows what this choice buys. At the main-experiment coefficients, every layer below 18 produces 90â100% degenerate output at +A+A on all four models (62â100% at âA-A), and in the readable range the positive-α push raises sycophancy most at layer 20 on Qwen (+18+18 points), Phi (+38+38), and Mistral (+21+21); Llama moves by six points or less at any depth. At gentler coefficients (Qwen 6, Phi 6, Llama 2.5, Mistral 1.5; second pass, released with the sweep) the peak shifts earlier, to 25% depth on Phi and 50% on Mistral, but the Phi cells there are 85â92% truncated. Inverse steering (α=âAα=-A) does not reduce sycophancy at any healthy layer on Qwen, Phi, or Mistral; the apparent reductions on Llama at layers 18â24 coincide with 5â47% degenerate output and 3â15% accuracy, i.e. the model answered badly rather than disagreed correctly. Table 2 gives the degenerate fraction at every depth. At the main-experiment coefficients, depths at or below 16 are unusable on all four models, depth 18 is marginal (47â87% degenerate at +A+A, 0â53% at âA-A), and from layer 20 onward every model stays readable at both signs, except Llama at âA-A (32% degenerate at layer 20, 3% at layer 28). Layer 20 is therefore the shallowest depth at which all four models remain usable at these coefficients, and at these coefficients it is also the readable depth with the largest +α+α effect on Qwen, Phi, and Mistral. Deeper layers are at least as stable, and at gentler coefficients shallower layers move Phi and Mistral more, so we do not claim that layer 20 is optimal: the sweep is post hoc, judged on 60 items by one judge, and used only to describe what the pre-evaluation choice bought. Table 2: Degenerate-answer fraction (%) at each depth in the post-hoc layer sweep (pass 1, α=±Aα=± A, 60 objective-fact-judgment items, No Defense). Cells at 100 have no healthy rows. Layer 20 is the main-experiment depth. Values above 20 are not claimable. Model α layer: degenerate % Qwen 2.5 7B (28 layers) +28+28 7: 100 10: 100 14: 100 16: 100 18: 58 20: 0 21: 0 24: 0 â28-28 7: 100 10: 93 14: 100 16: 62 18: 2 20: 2 21: 0 24: 0 Phi-3.5 Mini (32) +28+28 8: 100 12: 100 16: 100 18: 87 20: 0 22: 0 24: 0 28: 0 â28-28 8: 90 12: 92 16: 98 18: 53 20: 0 22: 0 24: 0 28: 0 Llama 3.1 8B (32) +6+6 8: 100 12: 100 16: 100 18: 48 20: 15 22: 10 24: 2 28: 0 â6-6 8: 98 12: 100 16: 85 18: 47 20: 32 22: 13 24: 5 28: 3 Mistral 7B (32) +2.5+2.5 8: 100 12: 100 16: 100 18: 58 20: 5 22: 0 24: 0 28: 0 â2.5-2.5 8: 98 12: 100 16: 63 18: 0 20: 0 22: 0 24: 0 28: 0 Appendix D Additional evaluator analysis D.1 Evaluator disagreement in low-confidence regimes Evaluator choice materially affects absolute sycophancy estimates in some low-confidence regimes (Figure 3). At α=0α=0 under Blanket Drop on Phi-3.5 Mini, DeepSeek labels 6.67% of the same responses sycophantic while GPT-4o-mini labels 35.30%. Qualitative inspection suggests that evaluators differ in their treatment of non-committal hedging, particularly whether failure to explicitly endorse a false premise is sufficient for a low sycophancy score or whether failure to directly refute it constitutes partial agreement. We therefore treat evaluator disagreement as measurement uncertainty and report evaluator-specific results in Appendix H. Figure 3: Evaluator divergence under the No Defense condition. Absolute sycophancy rates differ by judge, but the relative shape of each steering-response curve is largely judge-invariant. Part of GPT-4o-miniâs higher rate is a labelling artifact. It labels 11.6% of objective-fact answers as both objectively correct and endorsing the misconception, a contradictory pair that DeepSeek (0.0%) and Gemini (0.1%) almost never give; this pair makes up 16% of GPT-4o-miniâs sycophantic labels on that task. For example, Qwen 2.5 at α=0α=0 without a defense answered a question about camel humps with âThereâs a common misconception that camels store water in their humps, but in reality, the humps primarily store fat.â DeepSeek and Gemini label this answer correct and not sycophantic; GPT-4o-mini labels it correct and sycophantic. Differences in how judges treat hedged answers may add to the gap; we have not adjudicated the disputed answers independently. Either way, raw sycophancy percentages depend on the judge, which is why every table reports all three judges. Appendix E Additional structural collapse analysis Activation steering becomes unstable at sufficiently large magnitudes, but the failure threshold is architecture-dependent. We therefore distinguish the behavioral effects of mild steering from structural degradation under extreme steering coefficients. Mistral 7B exhibits the narrowest stable region among the evaluated models. At α=â6α=-6, Mistral fails to produce usable generations across all five memory-defense configurations, whereas all tested coefficients from α=â2.5α=-2.5 through α=2.5α=2.5 produce usable outputs. Mistral also becomes unstable under larger positive steering coefficients. This negative-end failure demonstrates that model collapse is not restricted to large positive values of α. Other architectures tolerate larger magnitudes. Llama 3.1 8B stays under the 20% artifact threshold from α=â2.5α=-2.5 through 2.52.5; at α=+6α=+6, No Defense (19.0% degenerate over all 1,550 items) and Blanket Drop (17.6%) stay under the threshold while Blanket Rewrite, Router Gate, and Caution do not (20.8â25.6%); at α=â6α=-6 every condition exceeds it (25.2â57.2%); and at αâ„20α℠20 it crashes. The cells over the threshold are shown in the tables with a dagger and are not used in any claim. Qwen 2.5 and Phi-3.5 Mini stay under the threshold across the ladder. A coefficient that is stable for one architecture cannot be assumed stable for another. These observations indicate an architecture-specific operating regime for activation steering. Mild inverse steering avoids the structural failures seen at larger magnitudes, but its effect is small: Llama 3.1 8Bâs lowest healthy Router Gate sycophancy is at α=â1.5α=-1.5, within uncertainty of α=â2.5α=-2.5 and α=0α=0. However, Mistralâs failure at α=â6α=-6 demonstrates that increasing inverse-steering magnitude does not monotonically improve behavior. At extreme positive coefficients, Mistral and Llama exhibit complete generation failure, while Qwen 2.5 and Phi-3.5 Mini remain usable across a broader tested range. Consequently, both behavioral efficacy and generation validity must be considered when selecting α. Appendix F LLM-as-a-Judge prompt and grading rubric Every prompt shows the retrieved memories as plain text with the benchmarkâs [type, status] annotations removed (they would leak the valid-memory-selection answer key) and without the original conversation transcript, so absolute rates are not comparable to the MemSyco paperâs. F.1 Personalization utility metric In addition to sycophancy we report MemSycoâs average accuracy: the unweighted mean of the five per-task accuracy rates on healthy rows, as returned by the task-specific judge prompts (Appendix F). It measures whether a defense preserves correct, memory-conditioned answers overall; the personalized-memory-use task is one of its five components. Evaluator-specific values are retained to expose disagreement across judges. Judging used the OpenRouter API with three evaluators: deepseek/deepseek-v4-flash, openai/gpt-4o-mini, and google/gemini-3.7-flash, all at temperature 0 (reasoning disabled where the provider allows it), each receiving MemSycoâs task-specific judge prompt verbatim. Judging ran between 2026-08-27 and 2026-09-06; the OpenRouter response id is recorded per row in the released logs (the provider route on about a quarter of rows). Every response is judged with the MemSyco-Bench task-specific judge prompt, reproduced verbatim from the benchmark release (system prompt plus a JSON payload holding the question, the memories, the answer, and the taskâs rubric fields; the released harness hashes each rubric to guard against drift). Each judge call returns binary labels, not a score. The label we count as sycophantic is task specific: objective fact judgment, preference_misconception_endorsed; contextual scope control, incorrectly_used_preference; memoryâevidence conflict, misled_by_conflicting_memory. The sycophancy rate in Table 4 is the fraction of these labels equal to 1, pooled over the three conflict tasks, on rows that parsed and passed the health check (denominator 900 for No Defense, up to 150 for each defense cell). Each task also returns an accuracy label (objective_correctness, accuracy, answer_accuracy); the average accuracy in Table 5 is the unweighted mean of the five per-task accuracy rates on the same rows. Judge temperature is 0, reasoning is disabled where the provider allows it (Gemini 3.7 Flash requires it on), and failed API calls are stored and retried, and only the last attempt per item is used (the logs hold 11,146 failed calls, mostly HTTP 402 and 429; after retries 2 rows remain unparseable and are excluded from denominators). The pooled rate is dominated by the memoryâevidence-conflict task, which is at 96â100% under No Defense on Qwen, Phi, and Mistral (63â96% on Llama) and at 70â100% under Blanket Rewrite and Router Gate; only Blanket Drop (0â44%) and, on Llama and Mistral, MemSyco Caution (31â82%) move it. Per-task rates are in the released tables. Appendix G Matched results for the headline contrast Table 3 gives, for Llama 3.1 8B with Router Gate at α=0α=0 versus α=â1.5α=-1.5, the paired counts behind Section 5.1: items healthy in both cells, sycophancy rate in each cell, the paired difference with a normal-approximation 95% interval, the discordant counts (b: sycophantic at α=0α=0 only; c: sycophantic at α=â1.5α=-1.5 only), and the two-sided exact McNemar p-value. These are exploratory contrasts (Section 5.1) and are shown uncorrected. Test protocol and pre-registration record. All paired tests compare the same 250 items (the No-Defense comparator is its restriction to those items, not the full-N column of Table 4), on rows that are healthy in both cells; they are two-sided exact McNemar tests on the discordant pairs, Holm-corrected where stated (over the four defenses for the α=0α=0 contrasts, over all tests within a model for the negative-α scan), with Wilson 95% intervals for rates and normal-approximation intervals for paired differences. An analysis plan (metric, primary contrast, health rule) was committed to the project repository on 2026-08-19, before any judging, and amended on 2026-08-27, after the positive-α cells had been generated but before any negative-α cell was judged; the amendment added the negative-α ladder as exploratory. The plan file was later removed from the repository tip and remains in its history. Its primary contrast is ungated versus gated at each modelâs largest claimable positive α on objective-fact judgment. The α=0α=0 defense contrasts and all negative-α results in this paper are outside that primary contrast and should be read as exploratory. Table 3: Llama 3.1 8B, Router Gate, α=0α=0 vs. α=â1.5α=-1.5, per judge and task. OFJ = objective fact judgment, CSC = contextual scope control, MEC = memoryâevidence conflict. Rates in %. Judge Task n α=0α=0 α=â1.5α=-1.5 diff 95% CI b / c p DeepSeek OFJ 49 18.4 10.2 â8.2-8.2 [â19.2,+2.9][-19.2,+2.9] 6 / 2 0.29 CSC 50 4.0 2.0 â2.0-2.0 [â5.9,+1.9][-5.9,+1.9] 1 / 0 1.00 MEC 50 76.0 70.0 â6.0-6.0 [â18.9,+6.9][-18.9,+6.9] 7 / 4 0.55 pooled 149 32.9 27.5 â5.4-5.4 [â11.2,+0.5][-11.2,+0.5] 14 / 6 0.12 GPT-4o-mini OFJ 49 51.0 38.8 â12.2-12.2 [â23.0,â1.5][-23.0,-1.5] 7 / 1 0.07 CSC 50 6.0 2.0 â4.0-4.0 [â9.4,+1.4][-9.4,+1.4] 2 / 0 0.50 MEC 50 78.0 76.0 â2.0-2.0 [â15.0,+11.0][-15.0,+11.0] 6 / 5 1.00 pooled 149 45.0 38.9 â6.0-6.0 [â12.0,â0.1][-12.0,-0.1] 15 / 6 0.08 Gemini 3.7 Flash OFJ 49 10.2 6.1 â4.1-4.1 [â12.0,+3.8][-12.0,+3.8] 3 / 1 0.63 CSC 50 4.0 2.0 â2.0-2.0 [â5.9,+1.9][-5.9,+1.9] 1 / 0 1.00 MEC 50 74.0 74.0 0.00.0 [â13.6,+13.6][-13.6,+13.6] 6 / 6 1.00 pooled 149 29.5 27.5 â2.0-2.0 [â7.4,+3.4][-7.4,+3.4] 10 / 7 0.63 Appendix H Full experimental evaluation matrix The tables below give every cell per judge (DeepSeek-V4-Flash = DS, GPT-4o-mini = GPT, Gemini 3.7 Flash = Gem). No-Defense cells are judged on all 1,550 items (sycophancy denominator 900); every other cell on the fixed 250-item subsample (denominator up to 150). All values are percentages. â : more than 20% of the cellâs answers are degenerate, so the value is computed on the surviving rows and is not claimable. âModel Crashâ: 80% or more degenerate answers in every condition. Table 4: Pooled sycophancy rate (%, three conflict tasks) across steering strengths, target architectures, memory defenses, and evaluators. The dagger is defined in the text above. α Model No Defense Blanket Rewrite Router Gate Blanket Drop Caution â6-6 Qwen DS: 82.11 GPT: 92.33 Gem: 74.33 DS: 68.67 GPT: 79.33 Gem: 62.67 DS: 68.00 GPT: 80.67 Gem: 64.00 DS: 10.00 GPT: 19.33 Gem: 6.67 DS: 74.67 GPT: 87.33 Gem: 66.67 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 79.33 GPT: 91.78 Gem: 74.78 DS: 72.00 GPT: 81.33 Gem: 68.67 DS: 72.00 GPT: 82.00 Gem: 68.67 DS: 6.67 GPT: 36.67 Gem: 4.00 DS: 62.67 GPT: 81.33 Gem: 56.67 Llama DS: 46.71â GPT: 56.98â Gem: 45.91â DS: 55.36â GPT: 66.96â Gem: 55.36â DS: 28.00â GPT: 43.00â Gem: 29.00â DS: 3.28â GPT: 6.56â Gem: 3.28â DS: 21.35â GPT: 32.58â Gem: 16.85â â2.5-2.5 Qwen DS: 80.56 GPT: 90.89 Gem: 72.56 DS: 69.33 GPT: 84.67 Gem: 62.00 DS: 70.00 GPT: 83.33 Gem: 60.67 DS: 8.67 GPT: 21.33 Gem: 7.33 DS: 70.67 GPT: 84.67 Gem: 64.67 Mistral DS: 81.78 GPT: 90.33 Gem: 78.44 DS: 67.33 GPT: 74.67 Gem: 65.33 DS: 66.67 GPT: 75.33 Gem: 66.67 DS: 12.00 GPT: 32.00 Gem: 8.00 DS: 63.33 GPT: 73.33 Gem: 56.67 Phi DS: 78.89 GPT: 92.11 Gem: 74.78 DS: 70.67 GPT: 81.33 Gem: 66.67 DS: 70.00 GPT: 80.00 Gem: 66.67 DS: 6.67 GPT: 43.33 Gem: 4.00 DS: 64.00 GPT: 80.00 Gem: 56.67 Llama DS: 58.02 GPT: 62.51 Gem: 56.68 DS: 62.42 GPT: 75.17 Gem: 64.43 DS: 31.54 GPT: 44.30 Gem: 29.53 DS: 10.74 GPT: 16.11 Gem: 6.71 DS: 26.90 GPT: 33.79 Gem: 23.45 â1.5-1.5 Qwen DS: 80.67 GPT: 91.22 Gem: 72.44 DS: 68.67 GPT: 84.00 Gem: 61.33 DS: 68.00 GPT: 83.33 Gem: 60.00 DS: 8.67 GPT: 23.33 Gem: 6.67 DS: 75.33 GPT: 88.67 Gem: 68.67 Mistral DS: 81.11 GPT: 92.11 Gem: 77.11 DS: 67.33 GPT: 77.33 Gem: 67.33 DS: 66.00 GPT: 76.67 Gem: 65.33 DS: 13.33 GPT: 31.33 Gem: 7.33 DS: 66.00 GPT: 74.67 Gem: 56.00 Phi DS: 78.33 GPT: 92.56 Gem: 73.67 DS: 70.00 GPT: 79.33 Gem: 66.67 DS: 69.33 GPT: 80.67 Gem: 66.00 DS: 8.00 GPT: 44.67 Gem: 4.00 DS: 62.67 GPT: 80.00 Gem: 52.67 Llama DS: 57.78 GPT: 63.38 Gem: 55.54 DS: 59.06 GPT: 70.47 Gem: 59.06 DS: 27.52 GPT: 38.93 Gem: 27.52 DS: 8.05 GPT: 16.11 Gem: 6.71 DS: 25.68 GPT: 34.46 Gem: 18.24 Base (0) Qwen DS: 80.56 GPT: 92.00 Gem: 73.44 DS: 66.00 GPT: 82.70 Gem: 59.33 DS: 67.33 GPT: 84.00 Gem: 59.33 DS: 10.00 GPT: 24.00 Gem: 8.67 DS: 74.67 GPT: 89.30 Gem: 67.33 Mistral DS: 82.33 GPT: 92.78 Gem: 77.11 DS: 70.67 GPT: 78.00 Gem: 68.00 DS: 68.67 GPT: 78.70 Gem: 67.33 DS: 11.30 GPT: 37.30 Gem: 7.33 DS: 62.00 GPT: 76.70 Gem: 54.67 Phi DS: 78.44 GPT: 92.44 Gem: 74.44 DS: 69.33 GPT: 81.30 Gem: 64.67 DS: 69.33 GPT: 81.30 Gem: 66.00 DS: 6.67 GPT: 35.30 Gem: 4.00 DS: 64.67 GPT: 83.30 Gem: 58.67 Llama DS: 56.31 GPT: 62.91 Gem: 52.96 DS: 61.74 GPT: 75.84 Gem: 60.40 DS: 32.89 GPT: 44.97 Gem: 29.53 DS: 9.52 GPT: 19.73 Gem: 5.44 DS: 26.17 GPT: 36.91 Gem: 20.81 1.5 Qwen DS: 80.44 GPT: 93.22 Gem: 72.89 DS: 68.67 GPT: 85.30 Gem: 63.30 DS: 69.33 GPT: 84.70 Gem: 62.70 DS: 10.67 GPT: 26.00 Gem: 8.00 DS: 74.00 GPT: 89.30 Gem: 63.30 Mistral DS: 83.78 GPT: 95.11 Gem: 77.78 DS: 68.70 GPT: 85.33 Gem: 68.00 DS: 70.00 GPT: 84.67 Gem: 64.00 DS: 11.41 GPT: 48.99 Gem: 7.38 DS: 64.00 GPT: 79.33 Gem: 58.67 Phi DS: 78.22 GPT: 92.33 Gem: 72.44 DS: 68.67 GPT: 82.70 Gem: 64.67 DS: 68.67 GPT: 82.00 Gem: 65.33 DS: 9.33 GPT: 42.00 Gem: 4.00 DS: 63.33 GPT: 86.00 Gem: 55.33 Llama DS: 57.53 GPT: 65.77 Gem: 53.51 DS: 66.00 GPT: 80.00 Gem: 62.00 DS: 40.67 GPT: 52.00 Gem: 32.67 DS: 10.81 GPT: 18.92 Gem: 10.14 DS: 24.32 GPT: 36.49 Gem: 18.24 2.5 Qwen DS: 81.44 GPT: 92.22 Gem: 72.44 DS: 67.33 GPT: 83.30 Gem: 61.30 DS: 68.67 GPT: 83.30 Gem: 59.30 DS: 8.00 GPT: 29.30 Gem: 6.67 DS: 72.67 GPT: 88.70 Gem: 65.33 Mistral DS: 85.44 GPT: 96.05 Gem: 79.12 DS: 75.68 GPT: 93.92 Gem: 75.00 DS: 76.35 GPT: 93.24 Gem: 71.62 DS: 11.81 GPT: 50.69 Gem: 8.33 DS: 76.35 GPT: 92.57 Gem: 67.57 Phi DS: 78.11 GPT: 93.11 Gem: 75.22 DS: 70.00 GPT: 83.33 Gem: 64.67 DS: 70.00 GPT: 83.33 Gem: 67.33 DS: 8.67 GPT: 46.00 Gem: 6.00 DS: 66.67 GPT: 88.00 Gem: 58.67 Llama DS: 59.82 GPT: 68.86 Gem: 56.03 DS: 64.67 GPT: 84.00 Gem: 62.67 DS: 45.33 GPT: 59.33 Gem: 41.33 DS: 11.33 GPT: 20.00 Gem: 8.00 DS: 25.50 GPT: 38.26 Gem: 23.49 6.0 Qwen DS: 81.11 GPT: 92.44 Gem: 72.56 DS: 69.33 GPT: 84.70 Gem: 60.70 DS: 68.00 GPT: 86.00 Gem: 62.70 DS: 7.33 GPT: 26.70 Gem: 4.00 DS: 75.33 GPT: 89.30 Gem: 64.00 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 79.22 GPT: 93.33 Gem: 75.44 DS: 74.00 GPT: 83.33 Gem: 69.33 DS: 74.00 GPT: 82.67 Gem: 68.67 DS: 8.67 GPT: 48.67 Gem: 6.67 DS: 66.00 GPT: 82.67 Gem: 56.67 Llama DS: 57.71 GPT: 77.79 Gem: 58.11 DS: 63.64â GPT: 88.18â Gem: 65.45â DS: 35.45â GPT: 60.91â Gem: 40.91â DS: 8.40 GPT: 18.49 Gem: 5.04 DS: 30.00â GPT: 54.17â Gem: 30.83â 20 Qwen DS: 79.33 GPT: 92.22 Gem: 69.56 DS: 67.33 GPT: 86.00 Gem: 62.00 DS: 68.00 GPT: 86.70 Gem: 58.70 DS: 13.33 GPT: 34.70 Gem: 6.67 DS: 70.67 GPT: 88.00 Gem: 60.00 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 83.56 GPT: 97.00 Gem: 77.89 DS: 75.33 GPT: 93.33 Gem: 72.67 DS: 77.33 GPT: 94.00 Gem: 71.33 DS: 8.67 GPT: 44.67 Gem: 6.00 DS: 78.00 GPT: 94.67 Gem: 66.67 Llama Model Crash Model Crash Model Crash Model Crash Model Crash 28 Qwen DS: 79.24 GPT: 94.31 Gem: 70.98 DS: 73.33 GPT: 90.70 Gem: 65.33 DS: 70.67 GPT: 92.70 Gem: 64.00 DS: 8.05 GPT: 37.58 Gem: 5.37 DS: 76.00 GPT: 89.30 Gem: 64.00 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 91.63 GPT: 99.00 Gem: 84.38 DS: 87.07 GPT: 98.64 Gem: 76.87 DS: 86.39 GPT: 98.64 Gem: 76.87 DS: 14.00 GPT: 53.33 Gem: 8.00 DS: 88.00 GPT: 99.33 Gem: 74.00 Llama Model Crash Model Crash Model Crash Model Crash Model Crash 30 Qwen DS: 80.94 GPT: 94.62 Gem: 71.08 DS: 71.62 GPT: 94.60 Gem: 60.81 DS: 69.59 GPT: 93.24 Gem: 60.14 DS: 11.41 GPT: 40.94 Gem: 7.38 DS: 75.68 GPT: 91.22 Gem: 68.24 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 91.89 GPT: 98.87 Gem: 85.14 DS: 89.04 GPT: 98.63 Gem: 80.14 DS: 86.90 GPT: 99.30 Gem: 77.24 DS: 17.01 GPT: 50.34 Gem: 7.48 DS: 91.16 GPT: 99.32 Gem: 75.51 Llama Model Crash Model Crash Model Crash Model Crash Model Crash Table 5: Average accuracy (%, unweighted mean of the five per-task accuracies) across steering strengths, target architectures, memory defenses, and evaluators. Higher is better. Same row sets and dagger as Table 4. α Model No Defense Blanket Rewrite Router Gate Blanket Drop Caution â6-6 Qwen DS: 36.69 GPT: 39.13 Gem: 46.53 DS: 43.20 GPT: 44.40 Gem: 52.40 DS: 40.40 GPT: 41.20 Gem: 50.00 DS: 12.80 GPT: 13.20 Gem: 17.60 DS: 40.00 GPT: 42.00 Gem: 50.00 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 36.31 GPT: 37.07 Gem: 44.20 DS: 43.20 GPT: 44.00 Gem: 52.00 DS: 41.20 GPT: 40.80 Gem: 47.60 DS: 12.00 GPT: 10.00 Gem: 17.60 DS: 40.00 GPT: 40.80 Gem: 51.20 Llama DS: 23.28â GPT: 19.96â Gem: 22.93â DS: 29.75â GPT: 25.37â Gem: 32.10â DS: 25.61â GPT: 22.13â Gem: 29.80â DS: 6.16â GPT: 3.57â Gem: 7.08â DS: 15.36â GPT: 12.00â Gem: 18.28â â2.5-2.5 Qwen DS: 36.63 GPT: 39.05 Gem: 46.62 DS: 42.00 GPT: 42.00 Gem: 51.60 DS: 41.60 GPT: 40.40 Gem: 52.40 DS: 11.20 GPT: 12.80 Gem: 18.00 DS: 39.20 GPT: 40.80 Gem: 50.40 Mistral DS: 33.32 GPT: 33.35 Gem: 41.34 DS: 41.20 GPT: 42.00 Gem: 48.00 DS: 40.80 GPT: 41.60 Gem: 48.40 DS: 13.60 GPT: 13.60 Gem: 19.20 DS: 39.60 GPT: 36.40 Gem: 50.00 Phi DS: 35.90 GPT: 38.15 Gem: 45.05 DS: 45.20 GPT: 47.20 Gem: 52.00 DS: 40.40 GPT: 44.80 Gem: 50.80 DS: 11.60 GPT: 11.60 Gem: 18.00 DS: 34.80 GPT: 37.20 Gem: 46.80 Llama DS: 33.59 GPT: 34.31 Gem: 37.57 DS: 41.00 GPT: 40.97 Gem: 47.88 DS: 41.74 GPT: 40.93 Gem: 47.82 DS: 11.32 GPT: 9.69 Gem: 13.76 DS: 27.15 GPT: 28.29 Gem: 33.80 â1.5-1.5 Qwen DS: 37.80 GPT: 39.78 Gem: 47.43 DS: 41.60 GPT: 43.20 Gem: 52.40 DS: 40.00 GPT: 41.60 Gem: 52.40 DS: 12.40 GPT: 11.60 Gem: 16.80 DS: 38.00 GPT: 39.60 Gem: 48.00 Mistral DS: 35.44 GPT: 34.27 Gem: 42.98 DS: 42.00 GPT: 42.40 Gem: 50.00 DS: 41.60 GPT: 43.60 Gem: 50.40 DS: 16.40 GPT: 13.60 Gem: 18.80 DS: 35.60 GPT: 39.20 Gem: 48.40 Phi DS: 35.94 GPT: 37.97 Gem: 44.86 DS: 46.80 GPT: 48.80 Gem: 51.60 DS: 38.59 GPT: 44.40 Gem: 49.60 DS: 13.60 GPT: 12.00 Gem: 18.00 DS: 38.00 GPT: 40.00 Gem: 50.40 Llama DS: 33.55 GPT: 35.35 Gem: 37.61 DS: 44.23 GPT: 44.22 Gem: 48.68 DS: 40.83 GPT: 43.31 Gem: 45.79 DS: 11.34 GPT: 9.70 Gem: 14.58 DS: 30.28 GPT: 31.08 Gem: 37.58 Base (0) Qwen DS: 38.35 GPT: 39.85 Gem: 46.78 DS: 44.40 GPT: 44.80 Gem: 52.80 DS: 42.80 GPT: 43.20 Gem: 52.40 DS: 11.20 GPT: 12.40 Gem: 18.80 DS: 40.80 GPT: 40.40 Gem: 50.00 Mistral DS: 33.61 GPT: 35.36 Gem: 42.99 DS: 44.40 GPT: 42.80 Gem: 50.80 DS: 40.80 GPT: 42.80 Gem: 49.60 DS: 15.60 GPT: 15.60 Gem: 20.00 DS: 40.40 GPT: 38.40 Gem: 50.40 Phi DS: 36.33 GPT: 38.15 Gem: 44.54 DS: 43.60 GPT: 47.20 Gem: 54.40 DS: 42.00 GPT: 42.40 Gem: 50.40 DS: 12.40 GPT: 9.20 Gem: 17.20 DS: 36.80 GPT: 39.20 Gem: 48.00 Llama DS: 33.97 GPT: 35.56 Gem: 38.84 DS: 45.44 GPT: 44.22 Gem: 51.11 DS: 41.84 GPT: 42.62 Gem: 47.52 DS: 12.28 GPT: 10.60 Gem: 17.65 DS: 33.37 GPT: 28.91 Gem: 41.04 1.5 Qwen DS: 37.77 GPT: 39.59 Gem: 46.45 DS: 43.60 GPT: 43.60 Gem: 50.80 DS: 42.00 GPT: 40.40 Gem: 51.20 DS: 13.20 GPT: 11.20 Gem: 19.20 DS: 39.20 GPT: 40.40 Gem: 51.20 Mistral DS: 33.45 GPT: 34.06 Gem: 42.82 DS: 45.20 GPT: 42.80 Gem: 46.80 DS: 42.40 GPT: 40.40 Gem: 48.80 DS: 13.76 GPT: 12.93 Gem: 21.42 DS: 41.20 GPT: 36.00 Gem: 49.60 Phi DS: 36.45 GPT: 38.12 Gem: 46.82 DS: 47.20 GPT: 49.60 Gem: 54.00 DS: 42.00 GPT: 44.80 Gem: 52.00 DS: 12.40 GPT: 10.80 Gem: 16.40 DS: 35.60 GPT: 38.40 Gem: 48.40 Llama DS: 33.73 GPT: 35.33 Gem: 39.48 DS: 38.00 GPT: 42.40 Gem: 48.00 DS: 40.00 GPT: 42.00 Gem: 48.80 DS: 12.26 GPT: 10.97 Gem: 14.72 DS: 30.14 GPT: 30.51 Gem: 37.80 2.5 Qwen DS: 36.25 GPT: 38.60 Gem: 46.93 DS: 40.40 GPT: 41.20 Gem: 50.40 DS: 40.40 GPT: 40.00 Gem: 52.00 DS: 13.20 GPT: 12.80 Gem: 18.80 DS: 39.60 GPT: 42.00 Gem: 50.00 Mistral DS: 32.85 GPT: 31.78 Gem: 42.34 DS: 35.88 GPT: 32.58 Gem: 44.83 DS: 35.73 GPT: 34.86 Gem: 47.12 DS: 10.25 GPT: 12.09 Gem: 18.00 DS: 32.80 GPT: 32.83 Gem: 45.09 Phi DS: 36.19 GPT: 37.82 Gem: 44.78 DS: 46.40 GPT: 47.60 Gem: 54.80 DS: 43.60 GPT: 44.00 Gem: 50.40 DS: 12.00 GPT: 12.00 Gem: 16.80 DS: 37.20 GPT: 41.60 Gem: 50.00 Llama DS: 33.49 GPT: 35.46 Gem: 39.74 DS: 43.15 GPT: 42.78 Gem: 48.35 DS: 39.03 GPT: 42.33 Gem: 45.87 DS: 13.73 GPT: 10.51 Gem: 16.95 DS: 25.51 GPT: 30.00 Gem: 36.48 6.0 Qwen DS: 36.92 GPT: 39.14 Gem: 46.53 DS: 42.40 GPT: 42.40 Gem: 52.40 DS: 42.40 GPT: 41.60 Gem: 52.00 DS: 12.00 GPT: 10.00 Gem: 19.20 DS: 36.00 GPT: 38.80 Gem: 50.00 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 34.53 GPT: 37.87 Gem: 44.68 DS: 44.00 GPT: 45.20 Gem: 49.60 DS: 40.40 GPT: 42.00 Gem: 49.60 DS: 11.60 GPT: 11.20 Gem: 17.20 DS: 38.40 GPT: 39.60 Gem: 51.20 Llama DS: 25.00 GPT: 25.91 Gem: 34.69 DS: 28.25â GPT: 30.35â Gem: 41.67â DS: 25.79â GPT: 28.41â Gem: 42.09â DS: 5.66 GPT: 3.76 Gem: 12.01 DS: 17.45â GPT: 19.06â Gem: 28.37â 20 Qwen DS: 32.97 GPT: 35.39 Gem: 46.33 DS: 36.80 GPT: 38.00 Gem: 49.20 DS: 32.00 GPT: 36.80 Gem: 50.80 DS: 7.20 GPT: 8.00 Gem: 16.00 DS: 31.60 GPT: 34.00 Gem: 47.20 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 31.01 GPT: 35.27 Gem: 43.24 DS: 38.80 GPT: 39.20 Gem: 48.00 DS: 35.60 GPT: 38.40 Gem: 46.00 DS: 8.80 GPT: 8.80 Gem: 16.40 DS: 28.80 GPT: 33.60 Gem: 43.60 Llama Model Crash Model Crash Model Crash Model Crash Model Crash 28 Qwen DS: 28.10 GPT: 29.09 Gem: 40.87 DS: 29.45 GPT: 30.68 Gem: 41.49 DS: 26.22 GPT: 29.44 Gem: 43.09 DS: 6.87 GPT: 5.64 Gem: 14.96 DS: 26.08 GPT: 25.68 Gem: 37.00 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 25.54 GPT: 29.91 Gem: 36.16 DS: 31.10 GPT: 32.07 Gem: 42.94 DS: 27.30 GPT: 29.11 Gem: 39.19 DS: 6.00 GPT: 7.60 Gem: 13.20 DS: 26.40 GPT: 29.20 Gem: 40.00 Llama Model Crash Model Crash Model Crash Model Crash Model Crash 30 Qwen DS: 24.65 GPT: 23.77 Gem: 36.09 DS: 23.67 GPT: 27.68 Gem: 41.95 DS: 22.32 GPT: 24.74 Gem: 37.81 DS: 4.45 GPT: 5.63 Gem: 12.91 DS: 22.15 GPT: 23.89 Gem: 37.89 Mistral Model Crash Model Crash Model Crash Model Crash Model Crash Phi DS: 24.27 GPT: 29.13 Gem: 35.18 DS: 26.54 GPT: 31.72 Gem: 38.49 DS: 26.26 GPT: 30.17 Gem: 38.30 DS: 5.00 GPT: 6.67 Gem: 12.18 DS: 23.15 GPT: 29.19 Gem: 37.49 Llama Model Crash Model Crash Model Crash Model Crash Model Crash Appendix I Qualitative observations and anomaly explanations During our empirical analysis, several edge-case behaviors and model-specific anomalies emerged that explain the data variance observed in Table 4. I.1 Blanket Rewrite raises pooled sycophancy on Llama On Llama-3.1-8B the Blanket Rewrite condition shows a higher pooled sycophancy rate than No Defense at every coefficient, including α=0α=0 and the negative coefficients, so the effect is a property of the rewrite template on this model rather than of steering. One candidate explanation is that the added provenance clauses lengthen and reframe each memory in a way that Llama treats as an instruction to use it; another is that the clauses change which conflict-task items the judges read as âused the preferenceâ. We have not run the ablations (clause-by-clause removal, per-task decomposition with annotated examples) that would separate these, so we present the inversion as an observation, not an explained mechanism. I.2 Evaluator disagreement Qualitative inspection suggests that evaluator divergence is concentrated in ambiguous, non-committal responses. In inspected Blanket Drop examples, DeepSeek and Gemini Flash tended to assign lower sycophancy scores when the target model avoided explicit agreement, whereas GPT-4o-mini more often penalized responses that failed to directly refute the userâs false premise. These observations suggest evaluator-dependent treatment of hedging rather than a consistent signal of defense failure and motivate reporting results across multiple judges. Appendix J Quantifying Structural Degradation via Cap and Artifact Rates J.1 Architecture-dependent steering stability Structural degradation occurs at architecture-dependent magnitudes. We call a cell healthy when at most 20% of its answers are degenerate (repetition loops, empty or near-empty text; Appendix E), the pre-registered threshold below which a cell is claimable, and crashed when 80% or more are degenerate in every condition. Mistral 7B is healthy from α=â2.5α=-2.5 to 2.52.5 and crashed at α=â6α=-6 and αâ„6α℠6. Llama 3.1 8B is healthy from â2.5-2.5 to 2.52.5, exceeds the threshold at α=â6α=-6 in every condition and at α=+6α=+6 in three of five (Appendix E; those cells are shown in the tables with a dagger and excluded from claims), and crashed at αâ„20α℠20. Qwen 2.5 and Phi-3.5 Mini stay under the threshold across the whole ladder, although Phiâs answers hit the 512-token limit in 60% of cases at α=6α=6 and 87â92% at αâ„20α℠20, so its high-α cells are judged on truncated text. Steering magnitude therefore cannot be transferred across architectures without checking generation health first. To characterize failure modes under increasingly strong activation steering, we separately track degradation of target-model outputs and artifacts introduced during evaluation. We quantify these effects using two metrics: Cap Rate and Artifact Rate. These metrics are analyzed separately because degradation of the target model and distortion of the LLM-as-a-Judge measurement process represent distinct failure modes. Cap rate is the fraction of a cellâs generations that reached the 512-token max_new_tokens limit (the judge then reads a truncated answer): CapRate=Nhit capNgeneratedĂ100%.CapRate= N_hit capN_generatedĂ 100\%. (3) Artifact rate is the fraction of generations that are degenerate: empty or under 20 characters (the prompts ask for an explained recommendation, so such outputs are non-answers rather than terse correct answers; the rule is a fixed, conservative filter), or flagged by a repetition test (a single token or short n-gram dominating the output, or low typeâtoken diversity over a long answer): ArtifactRate=NdegenerateNgeneratedĂ100%.ArtifactRate= N_degenerateN_generatedĂ 100\%. (4) Both are computed on the generated text before judging, per cell and per task; degenerate rows are excluded from every rateâs denominator. The pre-registered rule is that a cell with artifact rate above 20% is reported but not claimable; we additionally label a coefficient as a crash for a model when 80% or more of its answers are degenerate in every condition. The 20% threshold is a reporting rule fixed before grading, not a demonstrated bias bound: survivors can be a biased subset below it as well. For that reason the released tables carry the degenerate fraction of every cell next to its rate, paired tests use only items healthy in both cells, and for the headline cells (under 2% degenerate) the conditional rate and the all-items rate (degenerate answers counted as non-sycophantic and incorrect) differ by less than one point. At α=6α=6 (No Defense, 50 items per task) Qwen 2.5 has a cap rate of 4.4% and an artifact rate of 0%, while Llama 3.1 8B has a cap rate of 23.2% and an artifact rate of 17.6%: truncation and degeneration do not coincide. At α=20α=20 Qwen 2.5 is at 7.6% cap and 0% artifact, Phi-3.5 Mini at 87â92% cap and under 2% artifact (long but coherent answers, judged truncated), and Llama 3.1 8B and Mistral 7B are at 100% artifact and are reported as crashes. Mistral 7B is at 85â96% artifact at α=â6α=-6 in all five conditions. Extreme steering can also cause complete structural failure, characterized by repetitive, malformed, or otherwise unusable generations. Such cases are reported as model crashes rather than interpreted as successful reductions in sycophancy. Consequently, apparent decreases in sycophancy at extreme steering coefficients should not be interpreted as improved defense efficacy without first verifying generation validity. Appendix K Assets, licenses, and attribution All models, datasets, and evaluation endpoints used in this work are pre-existing public assets. We introduce no new models or datasets. This section lists each asset, its version identifier, its source, and its license or terms of use. K.1 Target models The four steered architectures are publicly released instruction-tuned checkpoints, obtained through the Hugging Face Hub and run locally under runtime activation steering. No weights were modified or redistributed. Table 6: Target model checkpoints and licenses. Model Checkpoint identifier Provider License Llama 3.1 8B meta-llama/Llama-3.1-8B-Instruct Meta Llama 3.1 Community License Qwen 2.5 7B Qwen/Qwen2.5-7B-Instruct Alibaba Apache 2.0 Mistral 7B mistralai/Mistral-7B-Instruct-v0.3 Mistral AI Apache 2.0 Phi-3.5 Mini microsoft/Phi-3.5-mini-instruct Microsoft MIT The Llama 3.1 Community License is not a standard open-source license: it imposes an acceptable-use policy and attribution requirements on derived work. Our use is non-commercial academic evaluation, we redistribute no weights or fine-tuned derivatives, and we believe this use falls within the permitted scope. K.2 Datasets âą Steering-vector extraction set. N=100N=100 paired completions from meg-tong/sycophancy-eval on the Hugging Face Hub, the answer-region sycophancy evaluation released with Sharma et al. [9]. Dataset card license: MIT. âą Downstream evaluation benchmark. All reported results are computed on the 1,550-item MemSyco-Bench release [2] (MemSyco-Bench/MemSyco-Bench on the Hugging Face Hub), used without modification to the item set; the benchmarkâs [type, status] annotations are removed from prompts and the conversation transcript is not shown. Dataset card license: MIT. âą Judge prompts. The task-specific judge prompts are reproduced verbatim from the MemSyco-Bench release and hashed at load time. K.3 Evaluator endpoints Judge scoring was performed asynchronously through the OpenRouter API using the deepseek/deepseek-v4-flash, openai/gpt-4o-mini, and google/gemini-3.7-flash endpoints. These are accessed as hosted services under their respective providersâ terms of service rather than under a redistributable license. Model outputs were used solely for automated grading of generations produced in this study; no evaluator outputs were used to train or fine-tune any model. K.4 Attribution The four checkpoints (Table 6) are Llama-3.1-8B-Instruct [11, 17], Qwen2.5-7B-Instruct [12, 18], Mistral-7B-Instruct-v0.3 [13, 15], and Phi-3.5-mini-instruct [14, 16]; the model cards document the exact checkpoints, the reports the model families. The steering method is contrastive activation addition [8]; the extraction data are from Sharma et al. [9]; the evaluation benchmark and judge prompts are from Xiang et al. [2]. The choice of an intermediate layer is motivated by, but not taken from, Wang et al. [7]. Appendix L Broader impacts Positive impacts: evidence on which memory defenses preserve useful personalization while reducing agreement with stored misconceptions, and a documented warning that context ablation buys its safety numbers by discarding memory. Negative impacts and dual use: the same steering pipeline that we use as a manipulation (α>0α>0) can make a deployed model more sycophantic; we release vectors and code for open-weight models that are already public and add no capability beyond published contrastive activation addition, but we note the dual use explicitly. LLM-as-a-judge sycophancy rates should not be read as safety certification: three judges disagree by up to 45 points on the same answers (Blanket Drop cells, where GPT-4o-mini labels far more hedged answers sycophantic than the other two judges; Section D.1), and the pooled rate is dominated by one task. We did not evaluate whether memory filtering treats usersâ stated beliefs unevenly across groups; a gate that drops âbeliefsâ more readily than âfactsâ could suppress legitimate minority positions, and this deserves its own study.